AZEN-SGG/Kavita
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Infinite Scroll + List View + Cover Upload Redesign (#1319)

Browse source 
* Started with the redesign of the cover image chooser redesign to be less click intensive for volume/chapter images.

Made some headings bold in card detail drawer.

* Tweaked the styles

* Moved where the info cards show

* Added an ability to open a page settings drawer

* Cleaned up some old code that isn't needed anymore.

* Started implementing a list view. Refactored some title code to a dedicated component

* List view implemented but way too many API calls. Either need caching or adjusting the SeriesDetail api.

* Fixed a bug where if the progress bar didn't render on a card item while a download was in progress, the download indicator would be removed.

* Large refactor to move a lot of the needed fields to the chapter and volume dtos for series detail. All fields are noted when only used in series detail.

* Implemented cards for other tabs (except related)

* Fixed the unit test which needed a mocked reader service call.

* More cleanup around age rating and removing old code from the refactor. Commented out sorting till i feel motivated to work on that.

* Some cleanup and restored cards as initial layout. Time to test this out and see if there is value add.

* Added ability for Chapters tab to show the volume chapters belong to (if applicable)

* Adding style fixes

* Cover image updates, don't allow the first image (which is what is currently set) to respond to cover changes.

Hide the ID field on list item for series detail.

* Refactored the title for list item to be injectable

* Cleaned up the selection code to make it less finicky on mobile when tap scrolling.

* Refactored chapter tab to show volume as well on list view.

* Ensure word count shows for Volumes

* Started adding virtual scrolling, pushing up so Robbie can mess around

* Started adding virtual scrolling, pushing up so Robbie can mess around

* Fixed a bug where all chapters would come under specials

* Show title data as accent if set.

* Style fixes for virtual scroller

* Restyling scroll

* Implemented a way to show storyline with virtual scrolling

* Show Word Count for chapters and cleaned up some logics.

* I might have card layout working with virtual scroll code.

* Some cleanup to hide more system like properties from info bar on series detail page. Fixed some missing time estimate info on storyline chapters.

* Fixed a regression on series service when I integrated VolumeTitle.

* Refactored read time to the backend. Added WordCount to the volume itself so we don't need to calculate on frontend. When asking to analyze files from a series, force the calculation.

* Fixed SeriesDetail api code

* Fixed up the code in the drawer to better update list/card mode

* Basic infinite scroll implemented, however due to how we are updating the list to render, we are re-rending cards that haven't been touched.

* Updated how we render and layout data for infinite scroll on library detail. It's almost there.

* Started laying foundation for loading pages backwards.

Removed lazy loading of images since we are now using virtual paging.

* Hooked in some basic code to allow user to load a prev page with infinite scroll.

* Fixed up series detail api and undid the non-lazy loaded images.

Changed the router to help with this infinite loading on Firefox issue.

* Fixed up some naming issues with Series Detail and added a new test.

* This is an infinite scroll without pagination implementation. It is not fully done, but off to a good start. Virtual scroller with jump bar is working pretty well, def needs more polishing and tweaking. There are hacks in this implementation that need to be revisited.

* Refactored code so that we don't use any pagination and load all results by default.

* Misc code cleanup from build warnings.

* Cleaned up some logic for how to display titles in list view.

* More title cleanup for specials

* Hooked up page layout to user preferences and renamed an existing user pref name to match the dto.

* Swapped out everything but storyline with virtual-scroller over CDK

* Removed CDK from series detail.

* Default value for migration on page layout

* Updating card layout for library detail page

* fixing height for mobile

* Moved scrollbar

* Tweaked some styling for layouts when there is no data

* Refactored the series cards into their own component to make it re-usable.

* More tweaks on series info cards layout and enhanced a few pages with trackby functions.

* Removed some dead code

* Added download on series detail to actionables to fit in with new scroll strategy.

* Fixed language not being updated and sent to the backend for series update.

* Fixed a bad migration (if you ran any prior migration in this branch, you need to undo before you use this commit)

* Adding sticky tabs

* fixed mobile gap on sticky tab

* Enhanced the card title for books to show number up front.

* Adjusted the gutters on admin dashboard

* Removed debug code

* Removing duplicate book title

* Cleaned up old references to cdk scroller

* Implemented a basic jump bar scaling algorithm. Not perfect, but works pretty well.

* Code smells

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
This commit is contained in:
Joseph Milazzo 2022-06-13 16:37:49 -05:00 • committed by GitHub
parent f0f0e23e88
commit bbc48a5f5b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
122 changed files with 7863 additions and 2097 deletions
Download patch file Download diff file Expand all files Collapse all files

2
API/Services/ArchiveService.cs
View file

@ -249,7 +249,7 @@ namespace API.Services
/// <summary>
/// Given an archive stream, will assess whether directory needs to be flattened so that the extracted archive files are directly
/// under extract path and not nested in subfolders. See <see cref="DirectoryInfoExtensions"/> Flatten method.
/// under extract path and not nested in subfolders. See <see cref="DirectoryService"/> Flatten method.
/// </summary>
/// <param name="archive">An opened archive stream</param>
/// <returns></returns>

2
API/Services/CacheService.cs
View file

@ -32,7 +32,7 @@ namespace API.Services
string GetCachedEpubFile(int chapterId, Chapter chapter);
public void ExtractChapterFiles(string extractPath, IReadOnlyList<MangaFile> files);
Task<int> CacheBookmarkForSeries(int userId, int seriesId);
void CleanupBookmarkCache(int bookmarkDtoSeriesId);
void CleanupBookmarkCache(int seriesId);
}
public class CacheService : ICacheService
{

2
API/Services/DirectoryService.cs
View file

@ -724,7 +724,7 @@ namespace API.Services
FileSystem.Path.Join(directoryName, "test.txt"),
string.Empty);
}
catch (Exception ex)
catch (Exception)
{
ClearAndDeleteDirectory(directoryName);
return false;

2
API/Services/ImageService.cs
View file

@ -50,7 +50,7 @@ public class ImageService : IImageService
_directoryService = directoryService;
}
public void ExtractImages(string fileFilePath, string targetDirectory, int fileCount)
public void ExtractImages(string fileFilePath, string targetDirectory, int fileCount = 1)
{
_directoryService.ExistOrCreate(targetDirectory);
if (fileCount == 1)

2
API/Services/MetadataService.cs
View file

@ -35,7 +35,7 @@ public interface IMetadataService
/// </summary>
/// <param name="libraryId"></param>
/// <param name="seriesId"></param>
Task RefreshMetadataForSeries(int libraryId, int seriesId, bool forceUpdate = false);
Task RefreshMetadataForSeries(int libraryId, int seriesId, bool forceUpdate = true);
}
public class MetadataService : IMetadataService

35
API/Services/ReaderService.cs
View file

@ -29,7 +29,7 @@ public interface IReaderService
Task<ChapterDto> GetContinuePoint(int seriesId, int userId);
Task MarkChaptersUntilAsRead(AppUser user, int seriesId, float chapterNumber);
Task MarkVolumesUntilAsRead(AppUser user, int seriesId, int volumeNumber);
HourEstimateRangeDto GetTimeEstimate(long wordCount, int pageCount, bool isEpub, bool hasProgress = false);
HourEstimateRangeDto GetTimeEstimate(long wordCount, int pageCount, bool isEpub);
}
public class ReaderService : IReaderService
@ -330,7 +330,7 @@ public class ReaderService : IReaderService
{
var chapterVolume = volumes.FirstOrDefault();
if (chapterVolume?.Number != 0) return -1;
var firstChapter = chapterVolume.Chapters.OrderBy(x => double.Parse(x.Number), _chapterSortComparer).FirstOrDefault();
var firstChapter = chapterVolume.Chapters.MinBy(x => double.Parse(x.Number), _chapterSortComparer);
if (firstChapter == null) return -1;
return firstChapter.Id;
}
@ -372,17 +372,16 @@ public class ReaderService : IReaderService
if (volume.Number == currentVolume.Number - 1)
{
if (currentVolume.Number - 1 == 0) break; // If we have walked all the way to chapter volume, then we should break so logic outside can work
var lastChapter = volume.Chapters
.OrderBy(x => double.Parse(x.Number), _chapterSortComparerForInChapterSorting).LastOrDefault();
var lastChapter = volume.Chapters.MaxBy(x => double.Parse(x.Number), _chapterSortComparerForInChapterSorting);
if (lastChapter == null) return -1;
return lastChapter.Id;
}
}
var lastVolume = volumes.OrderBy(v => v.Number).LastOrDefault();
var lastVolume = volumes.MaxBy(v => v.Number);
if (currentVolume.Number == 0 && currentVolume.Number != lastVolume?.Number && lastVolume?.Chapters.Count > 1)
{
var lastChapter = lastVolume.Chapters.OrderBy(x => double.Parse(x.Number), _chapterSortComparerForInChapterSorting).LastOrDefault();
var lastChapter = lastVolume.Chapters.MaxBy(x => double.Parse(x.Number), _chapterSortComparerForInChapterSorting);
if (lastChapter == null) return -1;
return lastChapter.Id;
}
@ -406,7 +405,7 @@ public class ReaderService : IReaderService
if (progress.Count == 0)
{
// I think i need a way to sort volumes last
return volumes.OrderBy(v => double.Parse(v.Number + ""), _chapterSortComparer).First().Chapters
return volumes.OrderBy(v => double.Parse(v.Number + string.Empty), _chapterSortComparer).First().Chapters
.OrderBy(c => float.Parse(c.Number)).First();
}
@ -499,41 +498,38 @@ public class ReaderService : IReaderService
}
}
public HourEstimateRangeDto GetTimeEstimate(long wordCount, int pageCount, bool isEpub, bool hasProgress = false)
public HourEstimateRangeDto GetTimeEstimate(long wordCount, int pageCount, bool isEpub)
{
if (isEpub)
{
var minHours = Math.Max((int) Math.Round((wordCount / MinWordsPerHour)), 1);
var maxHours = Math.Max((int) Math.Round((wordCount / MaxWordsPerHour)), 1);
var minHours = Math.Max((int) Math.Round((wordCount / MinWordsPerHour)), 0);
var maxHours = Math.Max((int) Math.Round((wordCount / MaxWordsPerHour)), 0);
if (maxHours < minHours)
{
return new HourEstimateRangeDto
{
MinHours = maxHours,
MaxHours = minHours,
AvgHours = (int) Math.Round((wordCount / AvgWordsPerHour)),
HasProgress = hasProgress
AvgHours = (int) Math.Round((wordCount / AvgWordsPerHour))
};
}
return new HourEstimateRangeDto
{
MinHours = minHours,
MaxHours = maxHours,
AvgHours = (int) Math.Round((wordCount / AvgWordsPerHour)),
HasProgress = hasProgress
AvgHours = (int) Math.Round((wordCount / AvgWordsPerHour))
};
}
var minHoursPages = Math.Max((int) Math.Round((pageCount / MinPagesPerMinute / 60F)), 1);
var maxHoursPages = Math.Max((int) Math.Round((pageCount / MaxPagesPerMinute / 60F)), 1);
var minHoursPages = Math.Max((int) Math.Round((pageCount / MinPagesPerMinute / 60F)), 0);
var maxHoursPages = Math.Max((int) Math.Round((pageCount / MaxPagesPerMinute / 60F)), 0);
if (maxHoursPages < minHoursPages)
{
return new HourEstimateRangeDto
{
MinHours = maxHoursPages,
MaxHours = minHoursPages,
AvgHours = (int) Math.Round((pageCount / AvgPagesPerMinute / 60F)),
HasProgress = hasProgress
AvgHours = (int) Math.Round((pageCount / AvgPagesPerMinute / 60F))
};
}
@ -541,8 +537,7 @@ public class ReaderService : IReaderService
{
MinHours = minHoursPages,
MaxHours = maxHoursPages,
AvgHours = (int) Math.Round((pageCount / AvgPagesPerMinute / 60F)),
HasProgress = hasProgress
AvgHours = (int) Math.Round((pageCount / AvgPagesPerMinute / 60F))
};
}
}

23
API/Services/SeriesService.cs
View file

@ -8,6 +8,7 @@ using API.Data;
using API.DTOs;
using API.DTOs.CollectionTags;
using API.DTOs.Metadata;
using API.DTOs.Reader;
using API.DTOs.SeriesDetail;
using API.Entities;
using API.Entities.Enums;
@ -458,7 +459,6 @@ public class SeriesService : ISeriesService
var volumes = (await _unitOfWork.VolumeRepository.GetVolumesDtoAsync(seriesId, userId))
.OrderBy(v => Parser.Parser.MinNumberFromRange(v.Name))
.ToList();
var chapters = volumes.SelectMany(v => v.Chapters).ToList();
// For books, the Name of the Volume is remapped to the actual name of the book, rather than Volume number.
var processedVolumes = new List<VolumeDto>();
@ -479,8 +479,15 @@ public class SeriesService : ISeriesService
processedVolumes.ForEach(v => v.Name = $"Volume {v.Name}");
}
var specials = new List<ChapterDto>();
var chapters = volumes.SelectMany(v => v.Chapters.Select(c =>
{
if (v.Number == 0) return c;
c.VolumeTitle = v.Name;
return c;
})).ToList();
foreach (var chapter in chapters)
{
chapter.Title = FormatChapterTitle(chapter, libraryType);
@ -490,7 +497,6 @@ public class SeriesService : ISeriesService
specials.Add(chapter);
}
// Don't show chapter 0 (aka single volume chapters) in the Chapters tab or books that are just single numbers (they show as volumes)
IEnumerable<ChapterDto> retChapters;
if (libraryType == LibraryType.Book)
@ -503,18 +509,17 @@ public class SeriesService : ISeriesService
.OrderBy(c => float.Parse(c.Number), new ChapterSortComparer());
}
var storylineChapters = volumes
.Where(v => v.Number == 0)
.SelectMany(v => v.Chapters.Where(c => !c.IsSpecial))
.OrderBy(c => float.Parse(c.Number), new ChapterSortComparer());
return new SeriesDetailDto()
{
Specials = specials,
Chapters = retChapters,
Volumes = processedVolumes,
StorylineChapters = volumes
.Where(v => v.Number == 0)
.SelectMany(v => v.Chapters.Where(c => !c.IsSpecial))
.OrderBy(c => float.Parse(c.Number), new ChapterSortComparer())
StorylineChapters = storylineChapters
};
}

2
API/Services/TaskScheduler.cs
View file

@ -180,7 +180,7 @@ public class TaskScheduler : ITaskScheduler
BackgroundJob.Enqueue(() => _metadataService.RefreshMetadata(libraryId, forceUpdate));
}
public void RefreshSeriesMetadata(int libraryId, int seriesId, bool forceUpdate = true)
public void RefreshSeriesMetadata(int libraryId, int seriesId, bool forceUpdate = false)
{
_logger.LogInformation("Enqueuing series metadata refresh for: {SeriesId}", seriesId);
BackgroundJob.Enqueue(() => _metadataService.RefreshMetadataForSeries(libraryId, seriesId, forceUpdate));

10
API/Services/Tasks/BackupService.cs
View file

@ -45,11 +45,6 @@ public class BackupService : IBackupService
_config = config;
_eventHub = eventHub;
// var maxRollingFiles = config.GetMaxRollingFiles();
// var loggingSection = config.GetLoggingFileName();
// var files = GetLogFiles(maxRollingFiles, loggingSection);
_backupFiles = new List<string>()
{
"appsettings.json",
@ -59,11 +54,6 @@ public class BackupService : IBackupService
"kavita.db-shm", // This wont always be there
"kavita.db-wal" // This wont always be there
};
// foreach (var file in files.Select(f => (_directoryService.FileSystem.FileInfo.FromFileName(f)).Name))
// {
// _backupFiles.Add(file);
// }
}
public IEnumerable<string> GetLogFiles(int maxRollingFiles, string logFileName)

103
API/Services/Tasks/Metadata/WordCountAnalyzerService.cs
View file

@ -32,14 +32,16 @@ public class WordCountAnalyzerService : IWordCountAnalyzerService
private readonly IUnitOfWork _unitOfWork;
private readonly IEventHub _eventHub;
private readonly ICacheHelper _cacheHelper;
private readonly IReaderService _readerService;
public WordCountAnalyzerService(ILogger<WordCountAnalyzerService> logger, IUnitOfWork unitOfWork, IEventHub eventHub,
ICacheHelper cacheHelper)
ICacheHelper cacheHelper, IReaderService readerService)
{
_logger = logger;
_unitOfWork = unitOfWork;
_eventHub = eventHub;
_cacheHelper = cacheHelper;
_readerService = readerService;
}
@ -142,58 +144,78 @@ public class WordCountAnalyzerService : IWordCountAnalyzerService
private async Task ProcessSeries(Series series, bool forceUpdate = false, bool useFileName = true)
{
if (series.Format != MangaFormat.Epub) return;
var isEpub = series.Format == MangaFormat.Epub;
long totalSum = 0;
foreach (var chapter in series.Volumes.SelectMany(v => v.Chapters))
foreach (var volume in series.Volumes)
{
// This compares if it's changed since a file scan only
if (!_cacheHelper.HasFileNotChangedSinceCreationOrLastScan(chapter, false,
chapter.Files.FirstOrDefault()) && chapter.WordCount != 0)
continue;
long sum = 0;
var fileCounter = 1;
foreach (var file in chapter.Files.Select(file => file.FilePath))
foreach (var chapter in volume.Chapters)
{
var pageCounter = 1;
try
// This compares if it's changed since a file scan only
if (!_cacheHelper.HasFileNotChangedSinceCreationOrLastScan(chapter, forceUpdate,
chapter.Files.FirstOrDefault()) && chapter.WordCount != 0)
continue;
if (series.Format == MangaFormat.Epub)
{
using var book = await EpubReader.OpenBookAsync(file, BookService.BookReaderOptions);
var totalPages = book.Content.Html.Values;
foreach (var bookPage in totalPages)
long sum = 0;
var fileCounter = 1;
foreach (var file in chapter.Files.Select(file => file.FilePath))
{
var progress = Math.Max(0F,
Math.Min(1F, (fileCounter * pageCounter) * 1F / (chapter.Files.Count * totalPages.Count)));
var pageCounter = 1;
try
{
using var book = await EpubReader.OpenBookAsync(file, BookService.BookReaderOptions);
var totalPages = book.Content.Html.Values;
foreach (var bookPage in totalPages)
{
var progress = Math.Max(0F,
Math.Min(1F, (fileCounter * pageCounter) * 1F / (chapter.Files.Count * totalPages.Count)));
await _eventHub.SendMessageAsync(MessageFactory.NotificationProgress,
MessageFactory.WordCountAnalyzerProgressEvent(series.LibraryId, progress,
ProgressEventType.Updated, useFileName ? file : series.Name));
sum += await GetWordCountFromHtml(bookPage);
pageCounter++;
}
fileCounter++;
}
catch (Exception ex)
{
_logger.LogError(ex, "There was an error reading an epub file for word count, series skipped");
await _eventHub.SendMessageAsync(MessageFactory.Error,
MessageFactory.ErrorEvent("There was an issue counting words on an epub",
$"{series.Name} - {file}"));
return;
}
await _eventHub.SendMessageAsync(MessageFactory.NotificationProgress,
MessageFactory.WordCountAnalyzerProgressEvent(series.LibraryId, progress,
ProgressEventType.Updated, useFileName ? file : series.Name));
sum += await GetWordCountFromHtml(bookPage);
pageCounter++;
}
fileCounter++;
}
catch (Exception ex)
{
_logger.LogError(ex, "There was an error reading an epub file for word count, series skipped");
await _eventHub.SendMessageAsync(MessageFactory.Error,
MessageFactory.ErrorEvent("There was an issue counting words on an epub",
$"{series.Name} - {file}"));
return;
chapter.WordCount = sum;
series.WordCount += sum;
volume.WordCount += sum;
}
var est = _readerService.GetTimeEstimate(chapter.WordCount, chapter.Pages, isEpub);
chapter.MinHoursToRead = est.MinHours;
chapter.MaxHoursToRead = est.MaxHours;
chapter.AvgHoursToRead = est.AvgHours;
_unitOfWork.ChapterRepository.Update(chapter);
}
chapter.WordCount = sum;
_unitOfWork.ChapterRepository.Update(chapter);
totalSum += sum;
var volumeEst = _readerService.GetTimeEstimate(volume.WordCount, volume.Pages, isEpub);
volume.MinHoursToRead = volumeEst.MinHours;
volume.MaxHoursToRead = volumeEst.MaxHours;
volume.AvgHoursToRead = volumeEst.AvgHours;
_unitOfWork.VolumeRepository.Update(volume);
}
series.WordCount = totalSum;
var seriesEstimate = _readerService.GetTimeEstimate(series.WordCount, series.Pages, isEpub);
series.MinHoursToRead = seriesEstimate.MinHours;
series.MaxHoursToRead = seriesEstimate.MaxHours;
series.AvgHoursToRead = seriesEstimate.AvgHours;
_unitOfWork.SeriesRepository.Update(series);
}
@ -207,8 +229,7 @@ public class WordCountAnalyzerService : IWordCountAnalyzerService
if (textNodes == null) return 0;
return textNodes
.Select(node => node.InnerText)
.Select(text => text.Split(' ', StringSplitOptions.RemoveEmptyEntries)
.Select(node => node.InnerText.Split(' ', StringSplitOptions.RemoveEmptyEntries)
.Where(s => char.IsLetter(s[0])))
.Select(words => words.Count())
.Where(wordCount => wordCount > 0)

1
API/Services/Tasks/ScannerService.cs
View file

@ -772,7 +772,6 @@ public class ScannerService : IScannerService
case PersonRole.Translator:
if (!series.Metadata.TranslatorLocked) series.Metadata.People.Remove(person);
break;
case PersonRole.Other:
default:
series.Metadata.People.Remove(person);
break;