AZEN-SGG/Kavita
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

EPUB Support (#178)

Browse source 
* Added book filetype detection and reorganized tests due to size of file

* Added ability to get basic Parse Info from Book and Pages.

* We can now scan books and get them in a library with cover images.

* Take the first image in the epub if the cover isn't set.

* Implemented the ability to unzip the ebup to cache. Implemented a test api to load html files.

* Just some test code to figure out how to approach this.

* Fixed some merge conflicts

* Removed some dead code from merge

* Snapshot: I can now load everything properly into the UI by rewriting the urls before I send them back. I don't notice any lag from this method. It can be optimized further.

* Implemented a way to load the content in the browser not via an iframe.

* Added a note

* Anchor mappings is complete. New anchors are updated so references now resolve to javascript:void() for UI to take care of internally loading and the appropriate page is mapped to it. Anchors that are external have target="_blank" added so they don't force you out of the app and styles are of course inlined.

* Oops i need this

* Table of contents api implemented (rough) and some small enhancements to codebase for books.

* GetBookPageResources now only loads files from within the book. Nested chapter list support and images now use html parsing instead of string parsing.

* Fonts now are remapped to load from endpoint.

* book-resources now uses a key, ensuring the file is in proper format for lookup. Changed chapter list based on structure with one HEADER and nested chapters.

* Properly handle svg resource requests and when there are part anchors that are clickable, make sure we handle them in the UI by adding a kavita-page handler.

* Add Chapter group page even if one isn't set by using first page (without part) from nestedChildren.

* Added extra debug code for issue #163.

* Added new user preferences for books and updated the css so we scope it to our reading section.

* Cleaned up style code

* Implemented ability to save book preferences and some cleanup on existing apis.

* Added an api for checking if a user has read something in a library type before.

* Forgot to make sure the has reading progress is against a user lol.

* Remove cacheservice code for books, sine we use an in-memory method

* Handle svg images as well

* Enhanced cover image extraction to check for a "cover" image if the cover image wasn't set in OPF before falling back to the first image.

* Fixed an issue with special books not properly generating metadata due to not having filename set.

* Cleanup, removed warmup task code from statup/program and changed taskscheduler to schedule tasks on startup only (or if tasks are changed from UI).

* Code cleanup

* Code cleanup

* So much code. Lots of refactors to try to test scanner service. Moved a lot of the queries into Extensions to allow to easier test, even though it's hacky. Support @font-face src:url swaps with ' and ". Source summary information from epubs.

* Well...baseURL needs to come from BE and not from UI lol.

* Adjusted migrations so default values match Entity

* Removed comment

* I think I finally fixed #163! The issue was that when i checked if it had a parserInfo, i wasn't considering that the chapter range might have a - in it (0-6) and so when the code to check if range could parse out a number failed, it treated it like a special and checked range against info's filename.

* Some bugfixes

* Lots of testing, extracting code to make it easier to test. This code is buggy, but fixed a bug where 1) If we changed the normalization code, we would remove the whole db during a scan and 2) We weren't actually removing series properly.

Other than that, code is being extracted to remove duplication and centralize logic.

* More code cleanup and test cleanup to ensure scan loop is working as expected and matches expectaions from tests.

* Cleaned up the code and made it so if I change normalization, which I do in this branch, it wont break existing DBs.

* Some comic parser changes for partial chapter support.

* Added some code for directory service and scanner service along with python code to generate test files (not used yet). Fixed up all the tests.

* Code smells
This commit is contained in:
Joseph Milazzo 2021-04-28 16:16:22 -05:00 • committed by GitHub
parent 2b99c8abfa
commit a01613f80f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
103 changed files with 5017 additions and 2480 deletions
Download patch file Download diff file Expand all files Collapse all files

15
API.Tests/Parser/BookParserTests.cs Normal file
View file

@ -0,0 +1,15 @@
using API.Services;
using Xunit;
namespace API.Tests.Parser
{
public class BookParserTests
{
[Theory]
[InlineData("Gifting The Wonderful World With Blessings! - 3 Side Stories [yuNS][Unknown]", "Gifting The Wonderful World With Blessings!")]
public void ParseSeriesTest(string filename, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseSeries(filename));
}
}
}

69
API.Tests/Parser/ComicParserTests.cs Normal file
View file

@ -0,0 +1,69 @@
using Xunit;
namespace API.Tests.Parser
{
public class ComicParserTests
{
[Theory]
[InlineData("01 Spider-Man & Wolverine 01.cbr", "Spider-Man & Wolverine")]
[InlineData("04 - Asterix the Gladiator (1964) (Digital-Empire) (WebP by Doc MaKS)", "Asterix the Gladiator")]
[InlineData("The First Asterix Frieze (WebP by Doc MaKS)", "The First Asterix Frieze")]
[InlineData("Batman & Catwoman - Trail of the Gun 01", "Batman & Catwoman - Trail of the Gun")]
[InlineData("Batman & Daredevil - King of New York", "Batman & Daredevil - King of New York")]
[InlineData("Batman & Grendel (1996) 01 - Devil's Bones", "Batman & Grendel")]
[InlineData("Batman & Robin the Teen Wonder #0", "Batman & Robin the Teen Wonder")]
[InlineData("Batman & Wildcat (1 of 3)", "Batman & Wildcat")]
[InlineData("Batman And Superman World's Finest #01", "Batman And Superman World's Finest")]
[InlineData("Babe 01", "Babe")]
[InlineData("Scott Pilgrim 01 - Scott Pilgrim's Precious Little Life (2004)", "Scott Pilgrim")]
[InlineData("Teen Titans v1 001 (1966-02) (digital) (OkC.O.M.P.U.T.O.-Novus)", "Teen Titans")]
[InlineData("Scott Pilgrim 02 - Scott Pilgrim vs. The World (2005)", "Scott Pilgrim")]
[InlineData("Wolverine - Origins 003 (2006) (digital) (Minutemen-PhD)", "Wolverine - Origins")]
[InlineData("Invincible Vol 01 Family matters (2005) (Digital).cbr", "Invincible")]
public void ParseComicSeriesTest(string filename, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseComicSeries(filename));
}
[Theory]
[InlineData("01 Spider-Man & Wolverine 01.cbr", "1")]
[InlineData("04 - Asterix the Gladiator (1964) (Digital-Empire) (WebP by Doc MaKS)", "4")]
[InlineData("The First Asterix Frieze (WebP by Doc MaKS)", "0")]
[InlineData("Batman & Catwoman - Trail of the Gun 01", "1")]
[InlineData("Batman & Daredevil - King of New York", "0")]
[InlineData("Batman & Grendel (1996) 01 - Devil's Bones", "1")]
[InlineData("Batman & Robin the Teen Wonder #0", "0")]
[InlineData("Batman & Wildcat (1 of 3)", "0")]
[InlineData("Batman And Superman World's Finest #01", "1")]
[InlineData("Babe 01", "1")]
[InlineData("Scott Pilgrim 01 - Scott Pilgrim's Precious Little Life (2004)", "1")]
[InlineData("Teen Titans v1 001 (1966-02) (digital) (OkC.O.M.P.U.T.O.-Novus)", "1")]
[InlineData("Scott Pilgrim 02 - Scott Pilgrim vs. The World (2005)", "2")]
[InlineData("Superman v1 024 (09-10 1943)", "1")]
public void ParseComicVolumeTest(string filename, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseComicVolume(filename));
}
[Theory]
[InlineData("01 Spider-Man & Wolverine 01.cbr", "0")]
[InlineData("04 - Asterix the Gladiator (1964) (Digital-Empire) (WebP by Doc MaKS)", "0")]
[InlineData("The First Asterix Frieze (WebP by Doc MaKS)", "0")]
[InlineData("Batman & Catwoman - Trail of the Gun 01", "0")]
[InlineData("Batman & Daredevil - King of New York", "0")]
[InlineData("Batman & Grendel (1996) 01 - Devil's Bones", "1")]
[InlineData("Batman & Robin the Teen Wonder #0", "0")]
[InlineData("Batman & Wildcat (1 of 3)", "1")]
[InlineData("Batman & Wildcat (2 of 3)", "2")]
[InlineData("Batman And Superman World's Finest #01", "0")]
[InlineData("Babe 01", "0")]
[InlineData("Scott Pilgrim 01 - Scott Pilgrim's Precious Little Life (2004)", "1")]
[InlineData("Teen Titans v1 001 (1966-02) (digital) (OkC.O.M.P.U.T.O.-Novus)", "1")]
[InlineData("Superman v1 024 (09-10 1943)", "24")]
[InlineData("Invincible 070.5 - Invincible Returns 1 (2010) (digital) (Minutemen-InnerDemons).cbr", "70.5")]
public void ParseComicChapterTest(string filename, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseComicChapter(filename));
}
}
}

357
API.Tests/Parser/MangaParserTests.cs Normal file
View file

@ -0,0 +1,357 @@
using System.Collections.Generic;
using API.Entities.Enums;
using API.Parser;
using Xunit;
using Xunit.Abstractions;
namespace API.Tests.Parser
{
public class MangaParserTests
{
private readonly ITestOutputHelper _testOutputHelper;
public MangaParserTests(ITestOutputHelper testOutputHelper)
{
_testOutputHelper = testOutputHelper;
}
[Theory]
[InlineData("Killing Bites Vol. 0001 Ch. 0001 - Galactica Scanlations (gb)", "1")]
[InlineData("My Girlfriend Is Shobitch v01 - ch. 09 - pg. 008.png", "1")]
[InlineData("Historys Strongest Disciple Kenichi_v11_c90-98.zip", "11")]
[InlineData("B_Gata_H_Kei_v01[SlowManga&OverloadScans]", "1")]
[InlineData("BTOOOM! v01 (2013) (Digital) (Shadowcat-Empire)", "1")]
[InlineData("Gokukoku no Brynhildr - c001-008 (v01) [TrinityBAKumA]", "1")]
[InlineData("Dance in the Vampire Bund v16-17 (Digital) (NiceDragon)", "16-17")]
[InlineData("Akame ga KILL! ZERO v01 (2016) (Digital) (LuCaZ).cbz", "1")]
[InlineData("v001", "1")]
[InlineData("Vol 1", "1")]
[InlineData("No Volume", "0")]
[InlineData("U12 (Under 12) Vol. 0001 Ch. 0001 - Reiwa Scans (gb)", "1")]
[InlineData("[Suihei Kiki]_Kasumi_Otoko_no_Ko_[Taruby]_v1.1.zip", "1")]
[InlineData("Tonikaku Cawaii [Volume 11].cbz", "11")]
[InlineData("[WS]_Ichiban_Ushiro_no_Daimaou_v02_ch10.zip", "2")]
[InlineData("[xPearse] Kyochuu Rettou Volume 1 [English] [Manga] [Volume Scans]", "1")]
[InlineData("Tower Of God S01 014 (CBT) (digital).cbz", "1")]
[InlineData("Tenjou_Tenge_v17_c100[MT].zip", "17")]
[InlineData("Shimoneta - Manmaru Hen - c001-006 (v01) [Various].zip", "1")]
[InlineData("Future Diary v02 (2009) (Digital) (Viz).cbz", "2")]
[InlineData("Mujaki no Rakuen Vol12 ch76", "12")]
[InlineData("Ichinensei_ni_Nacchattara_v02_ch11_[Taruby]_v1.3.zip", "2")]
[InlineData("Dorohedoro v01 (2010) (Digital) (LostNerevarine-Empire).cbz", "1")]
[InlineData("Dorohedoro v11 (2013) (Digital) (LostNerevarine-Empire).cbz", "11")]
[InlineData("Dorohedoro v12 (2013) (Digital) (LostNerevarine-Empire).cbz", "12")]
[InlineData("Yumekui_Merry_v01_c01[Bakayarou-Kuu].rar", "1")]
[InlineData("Yumekui-Merry_DKThias_Chapter11v2.zip", "0")]
[InlineData("Itoshi no Karin - c001-006x1 (v01) [Renzokusei Scans]", "1")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 12", "0")]
[InlineData("VanDread-v01-c001[MD].zip", "1")]
[InlineData("Ichiban_Ushiro_no_Daimaou_v04_ch27_[VISCANS].zip", "4")]
[InlineData("Mob Psycho 100 v02 (2019) (Digital) (Shizu).cbz", "2")]
[InlineData("Kodomo no Jikan vol. 1.cbz", "1")]
[InlineData("Kodomo no Jikan vol. 10.cbz", "10")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 12 [Dametrans][v2]", "0")]
[InlineData("Vagabond_v03", "3")]
[InlineData("Mujaki No Rakune Volume 10.cbz", "10")]
[InlineData("Umineko no Naku Koro ni - Episode 3 - Banquet of the Golden Witch #02.cbz", "3")]
[InlineData("Volume 12 - Janken Boy is Coming!.cbz", "12")]
[InlineData("[dmntsf.net] One Piece - Digital Colored Comics Vol. 20 Ch. 177 - 30 Million vs 81 Million.cbz", "20")]
[InlineData("Gantz.V26.cbz", "26")]
[InlineData("NEEDLESS_Vol.4_-Simeon_6_v2[SugoiSugoi].rar", "4")]
[InlineData("[Hidoi]_Amaenaideyo_MS_vol01_chp02.rar", "1")]
[InlineData("NEEDLESS_Vol.4_-_Simeon_6_v2_[SugoiSugoi].rar", "4")]
public void ParseVolumeTest(string filename, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseVolume(filename));
}
[Theory]
[InlineData("Killing Bites Vol. 0001 Ch. 0001 - Galactica Scanlations (gb)", "Killing Bites")]
[InlineData("My Girlfriend Is Shobitch v01 - ch. 09 - pg. 008.png", "My Girlfriend Is Shobitch")]
[InlineData("Historys Strongest Disciple Kenichi_v11_c90-98.zip", "Historys Strongest Disciple Kenichi")]
[InlineData("B_Gata_H_Kei_v01[SlowManga&OverloadScans]", "B Gata H Kei")]
[InlineData("BTOOOM! v01 (2013) (Digital) (Shadowcat-Empire)", "BTOOOM!")]
[InlineData("Gokukoku no Brynhildr - c001-008 (v01) [TrinityBAKumA]", "Gokukoku no Brynhildr")]
[InlineData("Dance in the Vampire Bund v16-17 (Digital) (NiceDragon)", "Dance in the Vampire Bund")]
[InlineData("v001", "")]
[InlineData("U12 (Under 12) Vol. 0001 Ch. 0001 - Reiwa Scans (gb)", "U12")]
[InlineData("Akame ga KILL! ZERO (2016-2019) (Digital) (LuCaZ)", "Akame ga KILL! ZERO")]
[InlineData("APOSIMZ 017 (2018) (Digital) (danke-Empire).cbz", "APOSIMZ")]
[InlineData("Akiiro Bousou Biyori - 01.jpg", "Akiiro Bousou Biyori")]
[InlineData("Beelzebub_172_RHS.zip", "Beelzebub")]
[InlineData("Dr. STONE 136 (2020) (Digital) (LuCaZ).cbz", "Dr. STONE")]
[InlineData("Cynthia the Mission 29.rar", "Cynthia the Mission")]
[InlineData("Darling in the FranXX - Volume 01.cbz", "Darling in the FranXX")]
[InlineData("Darwin's Game - Volume 14 (F).cbz", "Darwin's Game")]
[InlineData("[BAA]_Darker_than_Black_c7.zip", "Darker than Black")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 19 [Dametrans].zip", "Kedouin Makoto - Corpse Party Musume")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 01", "Kedouin Makoto - Corpse Party Musume")]
[InlineData("[WS]_Ichiban_Ushiro_no_Daimaou_v02_ch10.zip", "Ichiban Ushiro no Daimaou")]
[InlineData("[xPearse] Kyochuu Rettou Volume 1 [English] [Manga] [Volume Scans]", "Kyochuu Rettou")]
[InlineData("Loose_Relation_Between_Wizard_and_Apprentice_c07[AN].zip", "Loose Relation Between Wizard and Apprentice")]
[InlineData("Tower Of God S01 014 (CBT) (digital).cbz", "Tower Of God")]
[InlineData("Tenjou_Tenge_c106[MT].zip", "Tenjou Tenge")]
[InlineData("Tenjou_Tenge_v17_c100[MT].zip", "Tenjou Tenge")]
[InlineData("Shimoneta - Manmaru Hen - c001-006 (v01) [Various].zip", "Shimoneta - Manmaru Hen")]
[InlineData("Future Diary v02 (2009) (Digital) (Viz).cbz", "Future Diary")]
[InlineData("Tonikaku Cawaii [Volume 11].cbz", "Tonikaku Cawaii")]
[InlineData("Mujaki no Rakuen Vol12 ch76", "Mujaki no Rakuen")]
[InlineData("Knights of Sidonia c000 (S2 LE BD Omake - BLAME!) [Habanero Scans]", "Knights of Sidonia")]
[InlineData("Vol 1.cbz", "")]
[InlineData("Ichinensei_ni_Nacchattara_v01_ch01_[Taruby]_v1.1.zip", "Ichinensei ni Nacchattara")]
[InlineData("Chrno_Crusade_Dragon_Age_All_Stars[AS].zip", "")]
[InlineData("Ichiban_Ushiro_no_Daimaou_v04_ch34_[VISCANS].zip", "Ichiban Ushiro no Daimaou")]
[InlineData("Rent a Girlfriend v01.cbr", "Rent a Girlfriend")]
[InlineData("Yumekui_Merry_v01_c01[Bakayarou-Kuu].rar", "Yumekui Merry")]
[InlineData("Itoshi no Karin - c001-006x1 (v01) [Renzokusei Scans]", "Itoshi no Karin")]
[InlineData("Tonikaku Kawaii Vol-1 (Ch 01-08)", "Tonikaku Kawaii")]
[InlineData("Tonikaku Kawaii (Ch 59-67) (Ongoing)", "Tonikaku Kawaii")]
[InlineData("7thGARDEN v01 (2016) (Digital) (danke).cbz", "7thGARDEN")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 12", "Kedouin Makoto - Corpse Party Musume")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 09", "Kedouin Makoto - Corpse Party Musume")]
[InlineData("Goblin Slayer Side Story - Year One 025.5", "Goblin Slayer Side Story - Year One")]
[InlineData("Goblin Slayer - Brand New Day 006.5 (2019) (Digital) (danke-Empire)", "Goblin Slayer - Brand New Day")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 01 [Dametrans][v2]", "Kedouin Makoto - Corpse Party Musume")]
[InlineData("Vagabond_v03", "Vagabond")]
[InlineData("[AN] Mahoutsukai to Deshi no Futekisetsu na Kankei Chp. 1", "Mahoutsukai to Deshi no Futekisetsu na Kankei")]
[InlineData("Beelzebub_Side_Story_02_RHS.zip", "Beelzebub Side Story")]
[InlineData("[BAA]_Darker_than_Black_Omake-1.zip", "Darker than Black")]
[InlineData("Baketeriya ch01-05.zip", "Baketeriya")]
[InlineData("[PROzess]Kimi_ha_midara_na_Boku_no_Joou_-_Ch01", "Kimi ha midara na Boku no Joou")]
[InlineData("[SugoiSugoi]_NEEDLESS_Vol.2_-_Disk_The_Informant_5_[ENG].rar", "NEEDLESS")]
[InlineData("Fullmetal Alchemist chapters 101-108.cbz", "Fullmetal Alchemist")]
[InlineData("To Love Ru v09 Uncensored (Ch.071-079).cbz", "To Love Ru")]
[InlineData("[dmntsf.net] One Piece - Digital Colored Comics Vol. 20 Ch. 177 - 30 Million vs 81 Million.cbz", "One Piece - Digital Colored Comics")]
//[InlineData("Corpse Party -The Anthology- Sachikos game of love Hysteric Birthday 2U Extra Chapter", "Corpse Party -The Anthology- Sachikos game of love Hysteric Birthday 2U")]
[InlineData("Corpse Party -The Anthology- Sachikos game of love Hysteric Birthday 2U Chapter 01", "Corpse Party -The Anthology- Sachikos game of love Hysteric Birthday 2U")]
[InlineData("Vol03_ch15-22.rar", "")]
[InlineData("Love Hina - Special.cbz", "")] // This has to be a fallback case
[InlineData("Ani-Hina Art Collection.cbz", "")] // This has to be a fallback case
[InlineData("Magi - Ch.252-005.cbz", "Magi")]
[InlineData("Umineko no Naku Koro ni - Episode 1 - Legend of the Golden Witch #1", "Umineko no Naku Koro ni")]
[InlineData("Kimetsu no Yaiba - Digital Colored Comics c162 Three Victorious Stars.cbz", "Kimetsu no Yaiba - Digital Colored Comics")]
[InlineData("[Hidoi]_Amaenaideyo_MS_vol01_chp02.rar", "Amaenaideyo MS")]
[InlineData("NEEDLESS_Vol.4_-_Simeon_6_v2_[SugoiSugoi].rar", "NEEDLESS")]
public void ParseSeriesTest(string filename, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseSeries(filename));
}
[Theory]
[InlineData("Killing Bites Vol. 0001 Ch. 0001 - Galactica Scanlations (gb)", "1")]
[InlineData("My Girlfriend Is Shobitch v01 - ch. 09 - pg. 008.png", "9")]
[InlineData("Historys Strongest Disciple Kenichi_v11_c90-98.zip", "90-98")]
[InlineData("B_Gata_H_Kei_v01[SlowManga&OverloadScans]", "0")]
[InlineData("BTOOOM! v01 (2013) (Digital) (Shadowcat-Empire)", "0")]
[InlineData("Gokukoku no Brynhildr - c001-008 (v01) [TrinityBAKumA]", "1-8")]
[InlineData("Dance in the Vampire Bund v16-17 (Digital) (NiceDragon)", "0")]
[InlineData("c001", "1")]
[InlineData("[Suihei Kiki]_Kasumi_Otoko_no_Ko_[Taruby]_v1.12.zip", "12")]
[InlineData("Adding volume 1 with File: Ana Satsujin Vol. 1 Ch. 5 - Manga Box (gb).cbz", "5")]
[InlineData("Hinowa ga CRUSH! 018 (2019) (Digital) (LuCaZ).cbz", "18")]
[InlineData("Cynthia The Mission - c000-006 (v06) [Desudesu&Brolen].zip", "0-6")]
[InlineData("[WS]_Ichiban_Ushiro_no_Daimaou_v02_ch10.zip", "10")]
[InlineData("Loose_Relation_Between_Wizard_and_Apprentice_c07[AN].zip", "7")]
[InlineData("Tower Of God S01 014 (CBT) (digital).cbz", "14")]
[InlineData("Tenjou_Tenge_c106[MT].zip", "106")]
[InlineData("Tenjou_Tenge_v17_c100[MT].zip", "100")]
[InlineData("Shimoneta - Manmaru Hen - c001-006 (v01) [Various].zip", "1-6")]
[InlineData("Mujaki no Rakuen Vol12 ch76", "76")]
[InlineData("Beelzebub_01_[Noodles].zip", "1")]
[InlineData("Yumekui-Merry_DKThias_Chapter21.zip", "21")]
[InlineData("Yumekui_Merry_v01_c01[Bakayarou-Kuu].rar", "1")]
[InlineData("Yumekui-Merry_DKThias_Chapter11v2.zip", "11")]
[InlineData("Yumekui-Merry DKThiasScanlations Chapter51v2", "51")]
[InlineData("Yumekui-Merry_DKThiasScanlations&RenzokuseiScans_Chapter61", "61")]
[InlineData("Goblin Slayer Side Story - Year One 017.5", "17.5")]
[InlineData("Beelzebub_53[KSH].zip", "53")]
[InlineData("Black Bullet - v4 c20.5 [batoto]", "20.5")]
[InlineData("Itoshi no Karin - c001-006x1 (v01) [Renzokusei Scans]", "1-6")]
[InlineData("APOSIMZ 040 (2020) (Digital) (danke-Empire).cbz", "40")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 12", "12")]
[InlineData("Vol 1", "0")]
[InlineData("VanDread-v01-c001[MD].zip", "1")]
[InlineData("Goblin Slayer Side Story - Year One 025.5", "25.5")]
[InlineData("Kedouin Makoto - Corpse Party Musume, Chapter 01", "1")]
[InlineData("To Love Ru v11 Uncensored (Ch.089-097+Omake)", "89-97")]
[InlineData("To Love Ru v18 Uncensored (Ch.153-162.5)", "153-162.5")]
[InlineData("[AN] Mahoutsukai to Deshi no Futekisetsu na Kankei Chp. 1", "1")]
[InlineData("Beelzebub_Side_Story_02_RHS.zip", "2")]
[InlineData("[PROzess]Kimi_ha_midara_na_Boku_no_Joou_-_Ch01", "1")]
[InlineData("Fullmetal Alchemist chapters 101-108.cbz", "101-108")]
[InlineData("Umineko no Naku Koro ni - Episode 3 - Banquet of the Golden Witch #02.cbz", "2")]
[InlineData("To Love Ru v09 Uncensored (Ch.071-079).cbz", "71-79")]
[InlineData("Corpse Party -The Anthology- Sachikos game of love Hysteric Birthday 2U Extra Chapter.rar", "0")]
[InlineData("Beelzebub_153b_RHS.zip", "153.5")]
[InlineData("Beelzebub_150-153b_RHS.zip", "150-153.5")]
[InlineData("Transferred to another world magical swordsman v1.1", "1")]
[InlineData("Transferred to another world magical swordsman v1.2", "2")]
[InlineData("Kiss x Sis - Ch.15 - The Angst of a 15 Year Old Boy.cbz", "15")]
[InlineData("Kiss x Sis - Ch.12 - 1 , 2 , 3P!.cbz", "12")]
[InlineData("Umineko no Naku Koro ni - Episode 1 - Legend of the Golden Witch #1", "1")]
[InlineData("Kiss x Sis - Ch.00 - Let's Start from 0.cbz", "0")]
[InlineData("[Hidoi]_Amaenaideyo_MS_vol01_chp02.rar", "2")]
public void ParseChaptersTest(string filename, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseChapter(filename));
}
[Theory]
[InlineData("Tenjou Tenge Omnibus", "Omnibus")]
[InlineData("Tenjou Tenge {Full Contact Edition}", "Full Contact Edition")]
[InlineData("Tenjo Tenge {Full Contact Edition} v01 (2011) (Digital) (ASTC).cbz", "Full Contact Edition")]
[InlineData("Wotakoi - Love is Hard for Otaku Omnibus v01 (2018) (Digital) (danke-Empire)", "Omnibus")]
[InlineData("To Love Ru v01 Uncensored (Ch.001-007)", "Uncensored")]
[InlineData("Chobits Omnibus Edition v01 [Dark Horse]", "Omnibus Edition")]
[InlineData("[dmntsf.net] One Piece - Digital Colored Comics Vol. 20 Ch. 177 - 30 Million vs 81 Million.cbz", "")]
[InlineData("AKIRA - c003 (v01) [Full Color] [Darkhorse].cbz", "Full Color")]
public void ParseEditionTest(string input, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseEdition(input));
}
[Theory]
[InlineData("Beelzebub Special OneShot - Minna no Kochikame x Beelzebub (2016) [Mangastream].cbz", true)]
[InlineData("Beelzebub_Omake_June_2012_RHS", true)]
[InlineData("Beelzebub_Side_Story_02_RHS.zip", false)]
[InlineData("Darker than Black Shikkoku no Hana Special [Simple Scans].zip", true)]
[InlineData("Darker than Black Shikkoku no Hana Fanbook Extra [Simple Scans].zip", true)]
[InlineData("Corpse Party -The Anthology- Sachikos game of love Hysteric Birthday 2U Extra Chapter", true)]
[InlineData("Ani-Hina Art Collection.cbz", true)]
[InlineData("Gifting The Wonderful World With Blessings! - 3 Side Stories [yuNS][Unknown]", true)]
public void ParseMangaSpecialTest(string input, bool expected)
{
Assert.Equal(expected, !string.IsNullOrEmpty(API.Parser.Parser.ParseMangaSpecial(input)));
}
[Theory]
[InlineData("image.png", MangaFormat.Image)]
[InlineData("image.cbz", MangaFormat.Archive)]
[InlineData("image.txt", MangaFormat.Unknown)]
public void ParseFormatTest(string inputFile, MangaFormat expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseFormat(inputFile));
}
[Theory]
[InlineData("Gifting The Wonderful World With Blessings! - 3 Side Stories [yuNS][Unknown].epub", "Side Stories")]
public void ParseSpecialTest(string inputFile, string expected)
{
Assert.Equal(expected, API.Parser.Parser.ParseMangaSpecial(inputFile));
}
[Fact]
public void ParseInfoTest()
{
const string rootPath = @"E:/Manga/";
var expected = new Dictionary<string, ParserInfo>();
var filepath = @"E:/Manga/Mujaki no Rakuen/Mujaki no Rakuen Vol12 ch76.cbz";
expected.Add(filepath, new ParserInfo
{
Series = "Mujaki no Rakuen", Volumes = "12",
Chapters = "76", Filename = "Mujaki no Rakuen Vol12 ch76.cbz", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:/Manga/Shimoneta to Iu Gainen ga Sonzai Shinai Taikutsu na Sekai Man-hen/Vol 1.cbz";
expected.Add(filepath, new ParserInfo
{
Series = "Shimoneta to Iu Gainen ga Sonzai Shinai Taikutsu na Sekai Man-hen", Volumes = "1",
Chapters = "0", Filename = "Vol 1.cbz", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:\Manga\Beelzebub\Beelzebub_01_[Noodles].zip";
expected.Add(filepath, new ParserInfo
{
Series = "Beelzebub", Volumes = "0",
Chapters = "1", Filename = "Beelzebub_01_[Noodles].zip", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:\Manga\Ichinensei ni Nacchattara\Ichinensei_ni_Nacchattara_v01_ch01_[Taruby]_v1.1.zip";
expected.Add(filepath, new ParserInfo
{
Series = "Ichinensei ni Nacchattara", Volumes = "1",
Chapters = "1", Filename = "Ichinensei_ni_Nacchattara_v01_ch01_[Taruby]_v1.1.zip", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:\Manga\Tenjo Tenge (Color)\Tenjo Tenge {Full Contact Edition} v01 (2011) (Digital) (ASTC).cbz";
expected.Add(filepath, new ParserInfo
{
Series = "Tenjo Tenge", Volumes = "1", Edition = "Full Contact Edition",
Chapters = "0", Filename = "Tenjo Tenge {Full Contact Edition} v01 (2011) (Digital) (ASTC).cbz", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:\Manga\Akame ga KILL! ZERO (2016-2019) (Digital) (LuCaZ)\Akame ga KILL! ZERO v01 (2016) (Digital) (LuCaZ).cbz";
expected.Add(filepath, new ParserInfo
{
Series = "Akame ga KILL! ZERO", Volumes = "1", Edition = "",
Chapters = "0", Filename = "Akame ga KILL! ZERO v01 (2016) (Digital) (LuCaZ).cbz", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:\Manga\Dorohedoro\Dorohedoro v01 (2010) (Digital) (LostNerevarine-Empire).cbz";
expected.Add(filepath, new ParserInfo
{
Series = "Dorohedoro", Volumes = "1", Edition = "",
Chapters = "0", Filename = "Dorohedoro v01 (2010) (Digital) (LostNerevarine-Empire).cbz", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:\Manga\APOSIMZ\APOSIMZ 040 (2020) (Digital) (danke-Empire).cbz";
expected.Add(filepath, new ParserInfo
{
Series = "APOSIMZ", Volumes = "0", Edition = "",
Chapters = "40", Filename = "APOSIMZ 040 (2020) (Digital) (danke-Empire).cbz", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:\Manga\Corpse Party Musume\Kedouin Makoto - Corpse Party Musume, Chapter 09.cbz";
expected.Add(filepath, new ParserInfo
{
Series = "Kedouin Makoto - Corpse Party Musume", Volumes = "0", Edition = "",
Chapters = "9", Filename = "Kedouin Makoto - Corpse Party Musume, Chapter 09.cbz", Format = MangaFormat.Archive,
FullFilePath = filepath
});
filepath = @"E:\Manga\Goblin Slayer\Goblin Slayer - Brand New Day 006.5 (2019) (Digital) (danke-Empire).cbz";
expected.Add(filepath, new ParserInfo
{
Series = "Goblin Slayer - Brand New Day", Volumes = "0", Edition = "",
Chapters = "6.5", Filename = "Goblin Slayer - Brand New Day 006.5 (2019) (Digital) (danke-Empire).cbz", Format = MangaFormat.Archive,
FullFilePath = filepath
});
foreach (var file in expected.Keys)
{
var expectedInfo = expected[file];
var actual = API.Parser.Parser.Parse(file, rootPath);
if (expectedInfo == null)
{
Assert.Null(actual);
return;
}
Assert.NotNull(actual);
_testOutputHelper.WriteLine($"Validating {file}");
_testOutputHelper.WriteLine("Format");
Assert.Equal(expectedInfo.Format, actual.Format);
_testOutputHelper.WriteLine("Series");
Assert.Equal(expectedInfo.Series, actual.Series);
_testOutputHelper.WriteLine("Chapters");
Assert.Equal(expectedInfo.Chapters, actual.Chapters);
_testOutputHelper.WriteLine("Volumes");
Assert.Equal(expectedInfo.Volumes, actual.Volumes);
_testOutputHelper.WriteLine("Edition");
Assert.Equal(expectedInfo.Edition, actual.Edition);
_testOutputHelper.WriteLine("Filename");
Assert.Equal(expectedInfo.Filename, actual.Filename);
_testOutputHelper.WriteLine("FullFilePath");
Assert.Equal(expectedInfo.FullFilePath, actual.FullFilePath);
}
}
}
}

110
API.Tests/Parser/ParserInfoTests.cs Normal file
View file

@ -0,0 +1,110 @@
using API.Entities.Enums;
using API.Parser;
using Xunit;
namespace API.Tests.Parser
{
public class ParserInfoTests
{
[Fact]
public void MergeFromTest()
{
var p1 = new ParserInfo()
{
Chapters = "0",
Edition = "",
Format = MangaFormat.Archive,
FullFilePath = "/manga/darker than black.cbz",
IsSpecial = false,
Series = "darker than black",
Title = "darker than black",
Volumes = "0"
};
var p2 = new ParserInfo()
{
Chapters = "1",
Edition = "",
Format = MangaFormat.Archive,
FullFilePath = "/manga/darker than black.cbz",
IsSpecial = false,
Series = "darker than black",
Title = "Darker Than Black",
Volumes = "0"
};
var expected = new ParserInfo()
{
Chapters = "1",
Edition = "",
Format = MangaFormat.Archive,
FullFilePath = "/manga/darker than black.cbz",
IsSpecial = false,
Series = "darker than black",
Title = "darker than black",
Volumes = "0"
};
p1.Merge(p2);
AssertSame(expected, p1);
}
[Fact]
public void MergeFromTest2()
{
var p1 = new ParserInfo()
{
Chapters = "1",
Edition = "",
Format = MangaFormat.Archive,
FullFilePath = "/manga/darker than black.cbz",
IsSpecial = true,
Series = "darker than black",
Title = "darker than black",
Volumes = "0"
};
var p2 = new ParserInfo()
{
Chapters = "0",
Edition = "",
Format = MangaFormat.Archive,
FullFilePath = "/manga/darker than black.cbz",
IsSpecial = false,
Series = "darker than black",
Title = "Darker Than Black",
Volumes = "1"
};
var expected = new ParserInfo()
{
Chapters = "1",
Edition = "",
Format = MangaFormat.Archive,
FullFilePath = "/manga/darker than black.cbz",
IsSpecial = true,
Series = "darker than black",
Title = "darker than black",
Volumes = "1"
};
p1.Merge(p2);
AssertSame(expected, p1);
}
private void AssertSame(ParserInfo expected, ParserInfo actual)
{
Assert.Equal(expected.Chapters, actual.Chapters);
Assert.Equal(expected.Volumes, actual.Volumes);
Assert.Equal(expected.Edition, actual.Edition);
Assert.Equal(expected.Filename, actual.Filename);
Assert.Equal(expected.Format, actual.Format);
Assert.Equal(expected.Series, actual.Series);
Assert.Equal(expected.IsSpecial, actual.IsSpecial);
Assert.Equal(expected.FullFilePath, actual.FullFilePath);
}
}
}

192
API.Tests/Parser/ParserTest.cs Normal file
View file

@ -0,0 +1,192 @@
using Xunit;
using static API.Parser.Parser;
namespace API.Tests.Parser
{
public class ParserTests
{
[Theory]
[InlineData("0001", "1")]
[InlineData("1", "1")]
[InlineData("0013", "13")]
public void RemoveLeadingZeroesTest(string input, string expected)
{
Assert.Equal(expected, RemoveLeadingZeroes(input));
}
[Theory]
[InlineData("1", "001")]
[InlineData("10", "010")]
[InlineData("100", "100")]
public void PadZerosTest(string input, string expected)
{
Assert.Equal(expected, PadZeros(input));
}
[Theory]
[InlineData("Hello_I_am_here", "Hello I am here")]
[InlineData("Hello_I_am_here ", "Hello I am here")]
[InlineData("[ReleaseGroup] The Title", "The Title")]
[InlineData("[ReleaseGroup]_The_Title", "The Title")]
[InlineData("[Suihei Kiki]_Kasumi_Otoko_no_Ko_[Taruby]_v1.1", "Kasumi Otoko no Ko v1.1")]
public void CleanTitleTest(string input, string expected)
{
Assert.Equal(expected, CleanTitle(input));
}
// [Theory]
// //[InlineData("@font-face{font-family:\"PaytoneOne\";src:url(\"..\\/Fonts\\/PaytoneOne.ttf\")}", "@font-face{font-family:\"PaytoneOne\";src:url(\"PaytoneOne.ttf\")}")]
// [InlineData("@font-face{font-family:\"PaytoneOne\";src:url(\"..\\/Fonts\\/PaytoneOne.ttf\")}", "..\\/Fonts\\/PaytoneOne.ttf")]
// //[InlineData("@font-face{font-family:'PaytoneOne';src:url('..\\/Fonts\\/PaytoneOne.ttf')}", "@font-face{font-family:'PaytoneOne';src:url('PaytoneOne.ttf')}")]
// //[InlineData("@font-face{\r\nfont-family:'PaytoneOne';\r\nsrc:url('..\\/Fonts\\/PaytoneOne.ttf')\r\n}", "@font-face{font-family:'PaytoneOne';src:url('PaytoneOne.ttf')}")]
// public void ReplaceStyleUrlTest(string input, string expected)
// {
// var replacementStr = "PaytoneOne.ttf";
// // TODO: Use Match to validate since replace is weird
// //Assert.Equal(expected, FontSrcUrlRegex.Replace(input, "$1" + replacementStr + "$2" + "$3"));
// var match = FontSrcUrlRegex.Match(input);
// Assert.Equal(!string.IsNullOrEmpty(expected), FontSrcUrlRegex.Match(input).Success);
// }
[Theory]
[InlineData("test.cbz", true)]
[InlineData("test.cbr", true)]
[InlineData("test.zip", true)]
[InlineData("test.rar", true)]
[InlineData("test.rar.!qb", false)]
[InlineData("[shf-ma-khs-aqs]negi_pa_vol15007.jpg", false)]
public void IsArchiveTest(string input, bool expected)
{
Assert.Equal(expected, IsArchive(input));
}
[Theory]
[InlineData("test.epub", true)]
[InlineData("test.pdf", false)]
[InlineData("test.mobi", false)]
[InlineData("test.djvu", false)]
[InlineData("test.zip", false)]
[InlineData("test.rar", false)]
[InlineData("test.epub.!qb", false)]
[InlineData("[shf-ma-khs-aqs]negi_pa_vol15007.ebub", false)]
public void IsBookTest(string input, bool expected)
{
Assert.Equal(expected, IsBook(input));
}
[Theory]
[InlineData("test.epub", true)]
[InlineData("test.EPUB", true)]
[InlineData("test.mobi", false)]
[InlineData("test.epub.!qb", false)]
[InlineData("[shf-ma-khs-aqs]negi_pa_vol15007.ebub", false)]
public void IsEpubTest(string input, bool expected)
{
Assert.Equal(expected, IsEpub(input));
}
// [Theory]
// [InlineData("Tenjou Tenge Omnibus", "Omnibus")]
// [InlineData("Tenjou Tenge {Full Contact Edition}", "Full Contact Edition")]
// [InlineData("Tenjo Tenge {Full Contact Edition} v01 (2011) (Digital) (ASTC).cbz", "Full Contact Edition")]
// [InlineData("Wotakoi - Love is Hard for Otaku Omnibus v01 (2018) (Digital) (danke-Empire)", "Omnibus")]
// [InlineData("To Love Ru v01 Uncensored (Ch.001-007)", "Uncensored")]
// [InlineData("Chobits Omnibus Edition v01 [Dark Horse]", "Omnibus Edition")]
// [InlineData("[dmntsf.net] One Piece - Digital Colored Comics Vol. 20 Ch. 177 - 30 Million vs 81 Million.cbz", "Digital Colored Comics")]
// [InlineData("AKIRA - c003 (v01) [Full Color] [Darkhorse].cbz", "Full Color")]
// public void ParseEditionTest(string input, string expected)
// {
// Assert.Equal(expected, ParseEdition(input));
// }
// [Theory]
// [InlineData("Beelzebub Special OneShot - Minna no Kochikame x Beelzebub (2016) [Mangastream].cbz", true)]
// [InlineData("Beelzebub_Omake_June_2012_RHS", true)]
// [InlineData("Beelzebub_Side_Story_02_RHS.zip", false)]
// [InlineData("Darker than Black Shikkoku no Hana Special [Simple Scans].zip", true)]
// [InlineData("Darker than Black Shikkoku no Hana Fanbook Extra [Simple Scans].zip", true)]
// [InlineData("Corpse Party -The Anthology- Sachikos game of love Hysteric Birthday 2U Extra Chapter", true)]
// [InlineData("Ani-Hina Art Collection.cbz", true)]
// public void ParseMangaSpecialTest(string input, bool expected)
// {
// Assert.Equal(expected, ParseMangaSpecial(input) != "");
// }
[Theory]
[InlineData("12-14", 12)]
[InlineData("24", 24)]
[InlineData("18-04", 4)]
[InlineData("18-04.5", 4.5)]
[InlineData("40", 40)]
public void MinimumNumberFromRangeTest(string input, float expected)
{
Assert.Equal(expected, MinimumNumberFromRange(input));
}
[Theory]
[InlineData("Darker Than Black", "darkerthanblack")]
[InlineData("Darker Than Black - Something", "darkerthanblacksomething")]
[InlineData("Darker Than_Black", "darkerthanblack")]
[InlineData("", "")]
public void NormalizeTest(string input, string expected)
{
Assert.Equal(expected, Normalize(input));
}
[Theory]
[InlineData("test.jpg", true)]
[InlineData("test.jpeg", true)]
[InlineData("test.png", true)]
[InlineData(".test.jpg", false)]
[InlineData("!test.jpg", false)]
public void IsImageTest(string filename, bool expected)
{
Assert.Equal(expected, IsImage(filename));
}
[Theory]
[InlineData("C:/", "C:/Love Hina/Love Hina - Special.cbz", "Love Hina")]
[InlineData("C:/", "C:/Love Hina/Specials/Ani-Hina Art Collection.cbz", "Love Hina")]
[InlineData("C:/", "C:/Mujaki no Rakuen Something/Mujaki no Rakuen Vol12 ch76.cbz", "Mujaki no Rakuen")]
public void FallbackTest(string rootDir, string inputPath, string expectedSeries)
{
var actual = Parse(inputPath, rootDir);
if (actual == null)
{
Assert.NotNull(actual);
return;
}
Assert.Equal(expectedSeries, actual.Series);
}
[Theory]
[InlineData("Love Hina - Special.jpg", false)]
[InlineData("folder.jpg", true)]
[InlineData("DearS_v01_cover.jpg", true)]
[InlineData("DearS_v01_covers.jpg", false)]
[InlineData("!cover.jpg", true)]
[InlineData("cover.jpg", true)]
[InlineData("cover.png", true)]
[InlineData("ch1/cover.png", true)]
public void IsCoverImageTest(string inputPath, bool expected)
{
Assert.Equal(expected, IsCoverImage(inputPath));
}
[Theory]
[InlineData("__MACOSX/Love Hina - Special.jpg", true)]
[InlineData("TEST/Love Hina - Special.jpg", false)]
[InlineData("__macosx/Love Hina/", false)]
[InlineData("MACOSX/Love Hina/", false)]
public void HasBlacklistedFolderInPathTest(string inputPath, bool expected)
{
Assert.Equal(expected, HasBlacklistedFolderInPath(inputPath));
}
}
}