EPUB Support (#178)
* Added book filetype detection and reorganized tests due to size of file * Added ability to get basic Parse Info from Book and Pages. * We can now scan books and get them in a library with cover images. * Take the first image in the epub if the cover isn't set. * Implemented the ability to unzip the ebup to cache. Implemented a test api to load html files. * Just some test code to figure out how to approach this. * Fixed some merge conflicts * Removed some dead code from merge * Snapshot: I can now load everything properly into the UI by rewriting the urls before I send them back. I don't notice any lag from this method. It can be optimized further. * Implemented a way to load the content in the browser not via an iframe. * Added a note * Anchor mappings is complete. New anchors are updated so references now resolve to javascript:void() for UI to take care of internally loading and the appropriate page is mapped to it. Anchors that are external have target="_blank" added so they don't force you out of the app and styles are of course inlined. * Oops i need this * Table of contents api implemented (rough) and some small enhancements to codebase for books. * GetBookPageResources now only loads files from within the book. Nested chapter list support and images now use html parsing instead of string parsing. * Fonts now are remapped to load from endpoint. * book-resources now uses a key, ensuring the file is in proper format for lookup. Changed chapter list based on structure with one HEADER and nested chapters. * Properly handle svg resource requests and when there are part anchors that are clickable, make sure we handle them in the UI by adding a kavita-page handler. * Add Chapter group page even if one isn't set by using first page (without part) from nestedChildren. * Added extra debug code for issue #163. * Added new user preferences for books and updated the css so we scope it to our reading section. * Cleaned up style code * Implemented ability to save book preferences and some cleanup on existing apis. * Added an api for checking if a user has read something in a library type before. * Forgot to make sure the has reading progress is against a user lol. * Remove cacheservice code for books, sine we use an in-memory method * Handle svg images as well * Enhanced cover image extraction to check for a "cover" image if the cover image wasn't set in OPF before falling back to the first image. * Fixed an issue with special books not properly generating metadata due to not having filename set. * Cleanup, removed warmup task code from statup/program and changed taskscheduler to schedule tasks on startup only (or if tasks are changed from UI). * Code cleanup * Code cleanup * So much code. Lots of refactors to try to test scanner service. Moved a lot of the queries into Extensions to allow to easier test, even though it's hacky. Support @font-face src:url swaps with ' and ". Source summary information from epubs. * Well...baseURL needs to come from BE and not from UI lol. * Adjusted migrations so default values match Entity * Removed comment * I think I finally fixed #163! The issue was that when i checked if it had a parserInfo, i wasn't considering that the chapter range might have a - in it (0-6) and so when the code to check if range could parse out a number failed, it treated it like a special and checked range against info's filename. * Some bugfixes * Lots of testing, extracting code to make it easier to test. This code is buggy, but fixed a bug where 1) If we changed the normalization code, we would remove the whole db during a scan and 2) We weren't actually removing series properly. Other than that, code is being extracted to remove duplication and centralize logic. * More code cleanup and test cleanup to ensure scan loop is working as expected and matches expectaions from tests. * Cleaned up the code and made it so if I change normalization, which I do in this branch, it wont break existing DBs. * Some comic parser changes for partial chapter support. * Added some code for directory service and scanner service along with python code to generate test files (not used yet). Fixed up all the tests. * Code smells
This commit is contained in:
Joseph Milazzo
GitHub
parent
2b99c8abfa
commit
a01613f80f
103 changed files with 5017 additions and 2480 deletions
86
API.Tests/Extensions/ChapterListExtensionsTests.cs
Normal file
86
API.Tests/Extensions/ChapterListExtensionsTests.cs
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
using System.Collections.Generic;
|
||||
using API.Entities;
|
||||
using API.Entities.Enums;
|
||||
using API.Extensions;
|
||||
using API.Parser;
|
||||
using Xunit;
|
||||
|
||||
namespace API.Tests.Extensions
|
||||
{
|
||||
public class ChapterListExtensionsTests
|
||||
{
|
||||
private Chapter CreateChapter(string range, string number, MangaFile file, bool isSpecial)
|
||||
{
|
||||
return new Chapter()
|
||||
{
|
||||
Range = range,
|
||||
Number = number,
|
||||
Files = new List<MangaFile>() {file},
|
||||
IsSpecial = isSpecial
|
||||
};
|
||||
}
|
||||
|
||||
private MangaFile CreateFile(string file, MangaFormat format)
|
||||
{
|
||||
return new MangaFile()
|
||||
{
|
||||
FilePath = file,
|
||||
Format = format
|
||||
};
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void GetAnyChapterByRange_Test_ShouldBeNull()
|
||||
{
|
||||
var info = new ParserInfo()
|
||||
{
|
||||
Chapters = "0",
|
||||
Edition = "",
|
||||
Format = MangaFormat.Archive,
|
||||
FullFilePath = "/manga/darker than black.cbz",
|
||||
Filename = "darker than black.cbz",
|
||||
IsSpecial = false,
|
||||
Series = "darker than black",
|
||||
Title = "darker than black",
|
||||
Volumes = "0"
|
||||
};
|
||||
|
||||
var chapterList = new List<Chapter>()
|
||||
{
|
||||
CreateChapter("darker than black - Some special", "0", CreateFile("/manga/darker than black - special.cbz", MangaFormat.Archive), true)
|
||||
};
|
||||
|
||||
var actualChapter = chapterList.GetChapterByRange(info);
|
||||
|
||||
Assert.NotEqual(chapterList[0], actualChapter);
|
||||
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void GetAnyChapterByRange_Test_ShouldBeNotNull()
|
||||
{
|
||||
var info = new ParserInfo()
|
||||
{
|
||||
Chapters = "0",
|
||||
Edition = "",
|
||||
Format = MangaFormat.Archive,
|
||||
FullFilePath = "/manga/darker than black.cbz",
|
||||
Filename = "darker than black.cbz",
|
||||
IsSpecial = true,
|
||||
Series = "darker than black",
|
||||
Title = "darker than black",
|
||||
Volumes = "0"
|
||||
};
|
||||
|
||||
var chapterList = new List<Chapter>()
|
||||
{
|
||||
CreateChapter("darker than black", "0", CreateFile("/manga/darker than black.cbz", MangaFormat.Archive), true)
|
||||
};
|
||||
|
||||
var actualChapter = chapterList.GetChapterByRange(info);
|
||||
|
||||
Assert.Equal(chapterList[0], actualChapter);
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
27
API.Tests/Extensions/FileInfoExtensionsTests.cs
Normal file
27
API.Tests/Extensions/FileInfoExtensionsTests.cs
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
using System;
|
||||
using System.IO;
|
||||
using API.Extensions;
|
||||
using NSubstitute;
|
||||
using Xunit;
|
||||
|
||||
namespace API.Tests.Extensions
|
||||
{
|
||||
public class FileInfoExtensionsTests
|
||||
{
|
||||
// [Fact]
|
||||
// public void DoesLastWriteMatchTest()
|
||||
// {
|
||||
// var fi = Substitute.For<FileInfo>();
|
||||
// fi.LastWriteTime = DateTime.Now;
|
||||
//
|
||||
// var deltaTime = DateTime.Today.Subtract(TimeSpan.FromDays(1));
|
||||
// Assert.False(fi.DoesLastWriteMatch(deltaTime));
|
||||
// }
|
||||
//
|
||||
// [Fact]
|
||||
// public void IsLastWriteLessThanTest()
|
||||
// {
|
||||
//
|
||||
// }
|
||||
}
|
||||
}
|
||||
42
API.Tests/Extensions/ParserInfoListExtensionsTests.cs
Normal file
42
API.Tests/Extensions/ParserInfoListExtensionsTests.cs
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using API.Entities;
|
||||
using API.Entities.Enums;
|
||||
using API.Extensions;
|
||||
using API.Parser;
|
||||
using API.Tests.Helpers;
|
||||
using Xunit;
|
||||
|
||||
namespace API.Tests.Extensions
|
||||
{
|
||||
public class ParserInfoListExtensions
|
||||
{
|
||||
[Theory]
|
||||
[InlineData(new string[] {"1", "1", "3-5", "5", "8", "0", "0"}, new string[] {"1", "3-5", "5", "8", "0"})]
|
||||
public void DistinctVolumesTest(string[] volumeNumbers, string[] expectedNumbers)
|
||||
{
|
||||
var infos = volumeNumbers.Select(n => new ParserInfo() {Volumes = n}).ToList();
|
||||
Assert.Equal(expectedNumbers, infos.DistinctVolumes());
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(new string[] {@"Cynthia The Mission - c000-006 (v06) [Desudesu&Brolen].zip"}, new string[] {@"E:\Manga\Cynthia the Mission\Cynthia The Mission - c000-006 (v06) [Desudesu&Brolen].zip"}, true)]
|
||||
[InlineData(new string[] {@"Cynthia The Mission - c000-006 (v06-07) [Desudesu&Brolen].zip"}, new string[] {@"E:\Manga\Cynthia the Mission\Cynthia The Mission - c000-006 (v06) [Desudesu&Brolen].zip"}, true)]
|
||||
[InlineData(new string[] {@"Cynthia The Mission v20 c12-20 [Desudesu&Brolen].zip"}, new string[] {@"E:\Manga\Cynthia the Mission\Cynthia The Mission - c000-006 (v06) [Desudesu&Brolen].zip"}, false)]
|
||||
public void HasInfoTest(string[] inputInfos, string[] inputChapters, bool expectedHasInfo)
|
||||
{
|
||||
var infos = new List<ParserInfo>();
|
||||
foreach (var filename in inputInfos)
|
||||
{
|
||||
infos.Add(API.Parser.Parser.Parse(
|
||||
filename,
|
||||
string.Empty));
|
||||
}
|
||||
|
||||
var files = inputChapters.Select(s => EntityFactory.CreateMangaFile(s, MangaFormat.Archive, 199)).ToList();
|
||||
var chapter = EntityFactory.CreateChapter("0-6", false, files);
|
||||
|
||||
Assert.Equal(expectedHasInfo, infos.HasInfo(chapter));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1,4 +1,5 @@
|
|||
using API.Entities;
|
||||
using System;
|
||||
using API.Entities;
|
||||
using API.Extensions;
|
||||
using Xunit;
|
||||
|
||||
|
|
@ -10,6 +11,11 @@ namespace API.Tests.Extensions
|
|||
[InlineData(new [] {"Darker than Black", "Darker Than Black", "Darker than Black"}, new [] {"Darker than Black"}, true)]
|
||||
[InlineData(new [] {"Darker than Black", "Darker Than Black", "Darker than Black"}, new [] {"Darker_than_Black"}, true)]
|
||||
[InlineData(new [] {"Darker than Black", "Darker Than Black", "Darker than Black"}, new [] {"Darker then Black!"}, false)]
|
||||
[InlineData(new [] {"Salem's Lot", "Salem's Lot", "Salem's Lot"}, new [] {"Salem's Lot"}, true)]
|
||||
[InlineData(new [] {"Salem's Lot", "Salem's Lot", "Salem's Lot"}, new [] {"salems lot"}, true)]
|
||||
[InlineData(new [] {"Salem's Lot", "Salem's Lot", "Salem's Lot"}, new [] {"salem's lot"}, true)]
|
||||
// Different normalizations pass as we check normalization against an on-the-fly calculation so we don't delete series just because we change how normalization works
|
||||
[InlineData(new [] {"Salem's Lot", "Salem's Lot", "Salem's Lot", "salems lot"}, new [] {"salem's lot"}, true)]
|
||||
public void NameInListTest(string[] seriesInput, string[] list, bool expected)
|
||||
{
|
||||
var series = new Series()
|
||||
|
|
@ -17,7 +23,7 @@ namespace API.Tests.Extensions
|
|||
Name = seriesInput[0],
|
||||
LocalizedName = seriesInput[1],
|
||||
OriginalName = seriesInput[2],
|
||||
NormalizedName = Parser.Parser.Normalize(seriesInput[0])
|
||||
NormalizedName = seriesInput.Length == 4 ? seriesInput[3] : API.Parser.Parser.Normalize(seriesInput[0])
|
||||
};
|
||||
|
||||
Assert.Equal(expected, series.NameInList(list));
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue