EPUB Support (#178)
* Added book filetype detection and reorganized tests due to size of file * Added ability to get basic Parse Info from Book and Pages. * We can now scan books and get them in a library with cover images. * Take the first image in the epub if the cover isn't set. * Implemented the ability to unzip the ebup to cache. Implemented a test api to load html files. * Just some test code to figure out how to approach this. * Fixed some merge conflicts * Removed some dead code from merge * Snapshot: I can now load everything properly into the UI by rewriting the urls before I send them back. I don't notice any lag from this method. It can be optimized further. * Implemented a way to load the content in the browser not via an iframe. * Added a note * Anchor mappings is complete. New anchors are updated so references now resolve to javascript:void() for UI to take care of internally loading and the appropriate page is mapped to it. Anchors that are external have target="_blank" added so they don't force you out of the app and styles are of course inlined. * Oops i need this * Table of contents api implemented (rough) and some small enhancements to codebase for books. * GetBookPageResources now only loads files from within the book. Nested chapter list support and images now use html parsing instead of string parsing. * Fonts now are remapped to load from endpoint. * book-resources now uses a key, ensuring the file is in proper format for lookup. Changed chapter list based on structure with one HEADER and nested chapters. * Properly handle svg resource requests and when there are part anchors that are clickable, make sure we handle them in the UI by adding a kavita-page handler. * Add Chapter group page even if one isn't set by using first page (without part) from nestedChildren. * Added extra debug code for issue #163. * Added new user preferences for books and updated the css so we scope it to our reading section. * Cleaned up style code * Implemented ability to save book preferences and some cleanup on existing apis. * Added an api for checking if a user has read something in a library type before. * Forgot to make sure the has reading progress is against a user lol. * Remove cacheservice code for books, sine we use an in-memory method * Handle svg images as well * Enhanced cover image extraction to check for a "cover" image if the cover image wasn't set in OPF before falling back to the first image. * Fixed an issue with special books not properly generating metadata due to not having filename set. * Cleanup, removed warmup task code from statup/program and changed taskscheduler to schedule tasks on startup only (or if tasks are changed from UI). * Code cleanup * Code cleanup * So much code. Lots of refactors to try to test scanner service. Moved a lot of the queries into Extensions to allow to easier test, even though it's hacky. Support @font-face src:url swaps with ' and ". Source summary information from epubs. * Well...baseURL needs to come from BE and not from UI lol. * Adjusted migrations so default values match Entity * Removed comment * I think I finally fixed #163! The issue was that when i checked if it had a parserInfo, i wasn't considering that the chapter range might have a - in it (0-6) and so when the code to check if range could parse out a number failed, it treated it like a special and checked range against info's filename. * Some bugfixes * Lots of testing, extracting code to make it easier to test. This code is buggy, but fixed a bug where 1) If we changed the normalization code, we would remove the whole db during a scan and 2) We weren't actually removing series properly. Other than that, code is being extracted to remove duplication and centralize logic. * More code cleanup and test cleanup to ensure scan loop is working as expected and matches expectaions from tests. * Cleaned up the code and made it so if I change normalization, which I do in this branch, it wont break existing DBs. * Some comic parser changes for partial chapter support. * Added some code for directory service and scanner service along with python code to generate test files (not used yet). Fixed up all the tests. * Code smells
This commit is contained in:
Joseph Milazzo
GitHub
parent
2b99c8abfa
commit
a01613f80f
103 changed files with 5017 additions and 2480 deletions
57
API.Tests/Helpers/EntityFactory.cs
Normal file
57
API.Tests/Helpers/EntityFactory.cs
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
using System.Collections.Generic;
|
||||
using API.Entities;
|
||||
using API.Entities.Enums;
|
||||
|
||||
namespace API.Tests.Helpers
|
||||
{
|
||||
/// <summary>
|
||||
/// Used to help quickly create DB entities for Unit Testing
|
||||
/// </summary>
|
||||
public static class EntityFactory
|
||||
{
|
||||
public static Series CreateSeries(string name)
|
||||
{
|
||||
return new Series()
|
||||
{
|
||||
Name = name,
|
||||
SortName = name,
|
||||
LocalizedName = name,
|
||||
NormalizedName = API.Parser.Parser.Normalize(name),
|
||||
Volumes = new List<Volume>()
|
||||
};
|
||||
}
|
||||
|
||||
public static Volume CreateVolume(string volumeNumber, List<Chapter> chapters = null)
|
||||
{
|
||||
return new Volume()
|
||||
{
|
||||
Name = volumeNumber,
|
||||
Pages = 0,
|
||||
Chapters = chapters ?? new List<Chapter>()
|
||||
};
|
||||
}
|
||||
|
||||
public static Chapter CreateChapter(string range, bool isSpecial, List<MangaFile> files = null)
|
||||
{
|
||||
return new Chapter()
|
||||
{
|
||||
IsSpecial = isSpecial,
|
||||
Range = range,
|
||||
Number = API.Parser.Parser.MinimumNumberFromRange(range) + string.Empty,
|
||||
Files = files ?? new List<MangaFile>(),
|
||||
Pages = 0,
|
||||
|
||||
};
|
||||
}
|
||||
|
||||
public static MangaFile CreateMangaFile(string filename, MangaFormat format, int pages)
|
||||
{
|
||||
return new MangaFile()
|
||||
{
|
||||
FilePath = filename,
|
||||
Format = format,
|
||||
Pages = pages
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
25
API.Tests/Helpers/ParserInfoFactory.cs
Normal file
25
API.Tests/Helpers/ParserInfoFactory.cs
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
using System.IO;
|
||||
using API.Entities.Enums;
|
||||
using API.Parser;
|
||||
|
||||
namespace API.Tests.Helpers
|
||||
{
|
||||
public static class ParserInfoFactory
|
||||
{
|
||||
public static ParserInfo CreateParsedInfo(string series, string volumes, string chapters, string filename, bool isSpecial)
|
||||
{
|
||||
return new ParserInfo()
|
||||
{
|
||||
Chapters = chapters,
|
||||
Edition = "",
|
||||
Format = MangaFormat.Archive,
|
||||
FullFilePath = Path.Join(@"/manga/", filename),
|
||||
Filename = filename,
|
||||
IsSpecial = isSpecial,
|
||||
Title = Path.GetFileNameWithoutExtension(filename),
|
||||
Series = series,
|
||||
Volumes = volumes
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
55
API.Tests/Helpers/TestCaseGenerator.cs
Normal file
55
API.Tests/Helpers/TestCaseGenerator.cs
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using API.Services;
|
||||
|
||||
namespace API.Tests.Helpers
|
||||
{
|
||||
/// <summary>
|
||||
/// Given a -testcase.txt file, will generate a folder with fake archive or book files. These files are just renamed txt files.
|
||||
/// <remarks>This currently is broken - you cannot create files from a unit test it seems</remarks>
|
||||
/// </summary>
|
||||
public static class TestCaseGenerator
|
||||
{
|
||||
public static string GenerateFiles(string directory, string fileToExpand)
|
||||
{
|
||||
//var files = Directory.GetFiles(directory, fileToExpand);
|
||||
var file = new FileInfo(fileToExpand);
|
||||
if (!file.Exists && file.Name.EndsWith("-testcase.txt")) return string.Empty;
|
||||
|
||||
var baseDirectory = TestCaseGenerator.CreateTestBase(fileToExpand, directory);
|
||||
var filesToCreate = File.ReadLines(file.FullName);
|
||||
foreach (var fileToCreate in filesToCreate)
|
||||
{
|
||||
// var folders = DirectoryService.GetFoldersTillRoot(directory, fileToCreate);
|
||||
// foreach (var VARIABLE in COLLECTION)
|
||||
// {
|
||||
//
|
||||
// }
|
||||
File.Create(fileToCreate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
return baseDirectory;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates and returns a new base directory for data creation for a given testcase
|
||||
/// </summary>
|
||||
/// <param name="file"></param>
|
||||
/// <param name="rootDirectory"></param>
|
||||
/// <returns></returns>
|
||||
private static string CreateTestBase(string file, string rootDirectory)
|
||||
{
|
||||
var baseDir = file.Split("-testcase.txt")[0];
|
||||
var newDirectory = Path.Join(rootDirectory, baseDir);
|
||||
if (!Directory.Exists(newDirectory))
|
||||
{
|
||||
new DirectoryInfo(newDirectory).Create();
|
||||
}
|
||||
|
||||
return newDirectory;
|
||||
}
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue