AZEN-SGG/Kavita
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Performance, Scan Loop, Specials, and cleanup (#150)

Browse source 
* More cases for parsing regex

* Fixed a bug where chapter cover images weren't being updated due to a missed not.

* Removed a piece of code that was needed for upgrading, since all beta users agreed to wipe db.

* Fixed InProgress to properly respect order and show more recent activity first. Issue is with IEntityDate LastModified not updating in DataContext.

* Updated dependencies to lastest stable.

* LastModified on Volumes wasn't updating, validated it does update when data is changed.

* Rewrote a check to avoid a small heap object warning.

* Ensure UpdateSeries checks all libraries for unique name.

* Took care of some todos, removed unused imports, on dev go ahead and schedule reoocuring jobs since LiteDB caused the locking issue.

* No Tracking when we aren't using entities.

* Added code to remove abandoned progress rows after a chapter gets deleted.

* RefreshMetadata uses one large query rather than many trips to DB for updating metadata. Significantly faster.

* Fixed a bug where UpdateSeries would always complain about a unique name even when we weren't updating name.

* Files that are linked to a series but can't parse out Vol/Chapter information are properly grouped like other Specials.

* Refresh metadata on UI should call the task directly

* Fixed a bug on updating series to make sure we don't complain if we aren't trying to update the name to an existing name.

* Fixed #142 - Library cards should be sorted.

* Refactored the name of some variables to be more agnostic to comics.

* Implemented ScanLibrary but abandoning it.

* Code Cleanup & removing ScanSeries code.

* Some more tests and new Comparators for natural sorting.

* Fixed #137 - When performing I/O on archives, ignore __MACOSX folders completely.

* Fixed #137 - When performing I/O on archives, ignore __MACOSX folders completely.

* All entities that will show under specials tab should be marked special, rather than just what has a special keyword.

* Don't let specials generate cover images

* Don't let specials generate cover images

* SearchResults should send LocalizedName back since we are searching against it.

* Added some tests around macosx folders found from my actual server.

* Put extra notes about a case where duplicates come about, logger will now tell user about this issue.

* Missed a build issue somehow...

* Some code smells
This commit is contained in:
Joseph Milazzo 2021-04-05 08:37:45 -05:00 • committed by GitHub
parent 7790cf31fd
commit d3c14863d6
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
39 changed files with 401 additions and 184 deletions
Download patch file Download diff file Expand all files Collapse all files

36
API/Services/MetadataService.cs
View file

@ -4,6 +4,7 @@ using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using API.Comparators;
using API.Entities;
using API.Extensions;
using API.Interfaces;
@ -45,9 +46,9 @@ namespace API.Services
{
if (volume != null && ShouldFindCoverImage(volume.CoverImage, forceUpdate))
{
// TODO: Create a custom sorter for Chapters so it's consistent across the application
// TODO: Replace this with ChapterSortComparator
volume.Chapters ??= new List<Chapter>();
var firstChapter = volume.Chapters.OrderBy(x => Double.Parse(x.Number)).FirstOrDefault();
var firstChapter = volume.Chapters.OrderBy(x => double.Parse(x.Number)).FirstOrDefault();
var firstFile = firstChapter?.Files.OrderBy(x => x.Chapter).FirstOrDefault();
// Skip calculating Cover Image (I/O) if the chapter already has it set
@ -67,16 +68,29 @@ namespace API.Services
public void UpdateMetadata(Series series, bool forceUpdate)
{
// TODO: Use new ChapterSortComparer() here instead
if (series == null) return;
if (ShouldFindCoverImage(series.CoverImage, forceUpdate))
{
series.Volumes ??= new List<Volume>();
var firstCover = series.Volumes.OrderBy(x => x.Number).FirstOrDefault(x => x.Number != 0);
byte[] coverImage = null;
if (firstCover == null && series.Volumes.Any())
{
firstCover = series.Volumes.FirstOrDefault(x => x.Number == 0);
// If firstCover is null and one volume, the whole series is Chapters under Vol 0.
if (series.Volumes.Count == 1)
{
coverImage = series.Volumes[0].Chapters.OrderBy(c => double.Parse(c.Number))
.FirstOrDefault(c => !c.IsSpecial)?.CoverImage;
}
if (coverImage == null)
{
coverImage = series.Volumes[0].Chapters.OrderBy(c => double.Parse(c.Number))
.FirstOrDefault()?.CoverImage;
}
}
series.CoverImage = firstCover?.CoverImage;
series.CoverImage = firstCover?.CoverImage ?? coverImage;
}
if (!string.IsNullOrEmpty(series.Summary) && !forceUpdate) return;
@ -88,22 +102,20 @@ namespace API.Services
if (firstFile != null && !new FileInfo(firstFile.FilePath).DoesLastWriteMatch(firstFile.LastModified))
{
series.Summary = _archiveService.GetSummaryInfo(firstFile.FilePath);
firstFile.LastModified = DateTime.Now;
}
}
public void RefreshMetadata(int libraryId, bool forceUpdate = false)
{
var sw = Stopwatch.StartNew();
var library = Task.Run(() => _unitOfWork.LibraryRepository.GetLibraryForIdAsync(libraryId)).Result;
var allSeries = Task.Run(() => _unitOfWork.SeriesRepository.GetSeriesForLibraryIdAsync(libraryId)).Result.ToList();
var library = Task.Run(() => _unitOfWork.LibraryRepository.GetFullLibraryForIdAsync(libraryId)).Result;
_logger.LogInformation("Beginning metadata refresh of {LibraryName}", library.Name);
foreach (var series in allSeries)
foreach (var series in library.Series)
{
series.NormalizedName = Parser.Parser.Normalize(series.Name);
var volumes = Task.Run(() => _unitOfWork.SeriesRepository.GetVolumes(series.Id)).Result.ToList();
foreach (var volume in volumes)
foreach (var volume in series.Volumes)
{
foreach (var chapter in volume.Chapters)
{