Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Scanner + People Pages #3286

Open
wants to merge 15 commits into
base: develop
Choose a base branch
from
Open

New Scanner + People Pages #3286

wants to merge 15 commits into from

Conversation

majora2007
Copy link
Member

This PR brings a brand new scanner to Kavita with a focus on reliability and tuned to help support our larger comic hoarders. The scanner now works on a system of partial information and has some areas parallelized when there is enough data for it to make sense.

In addition, special detection has been slimmed down to just files with SP marker and specials can be flat in the series folder or ideally in Specials/ folder. This is something that I have had an open issue for many months and it was finally time to make the change. While I will miss it for the cases it did work, I'm sure many users will enjoy not having some series claimed as a special due to false positives.

Lastly, a major rework of People is landing with this PR. All people will be removed from your series on updating and a force scan will be needed to get them added back. People are now entity based and that means we can have more rich metadata for each person available. This is the beginning of richer metadata around people (including characters) and plans are currently being architected to bring people images to servers (you can manually upload them via UI/API currently).

image
image
image

Added

  • Added: Ability to see individual chapters/issues (limited to 20) a person has.
  • Added: Ability to add images to the person via UI/API. A more automated method is planned in the future.
  • Added: Added the ability to browse all Writers and Artists (cover artist) in one page. This should help book users that like to browse by author.

Changed

  • Changed: Optimized a number of methods within the Scanner to reduce memory and CPU time
  • Changed: Scanner can now choose to parallel parse files when there are over 100 in a directory
  • Changed: Changed how detection and scanning of dirty directories works. The scanner will now parse bottom-up to reduce any potential misses and avoid different layouts working differently. This has extra I/O checks but much greater reliability and should reduce the amount of work needed to ingest changes after the first scan.
  • Changed: LocalizedSeries merging with Series is now done at a higher level and performs much better (in terms of reliability)
  • Changed: People are now entity based which allows for a richer experience in Kavita.
  • Changed: Clicking on any person in Kavita will now open their dedicated page which showcases their different works (feedback needed).
  • Changed: (Performance) Some large refactors to the scanner to help with the FK bug. The scanner should be slower but consume much less memory, especially for well tagged systems.
  • Changed: Kavita no longer parses keywords from filenames and groups as a Special. The way to mark a special going forward will be explicitly with SP01, SP02, etc. Specials can be flat in the series folder or ideally in Specials/ folder.
  • Changed: Changed logging level on some of the loggers that could become noisy

Fixed

  • Fixed: Fixed a bug where Series Cover image could choose a Volume 0 instead of Volume 1
  • Fixed: Fixed a bug where calculating the Lowest Folder for a Series could be incorrect when there are nested folders. This includes a migration to clear out existing entries to avoid scan series not seeing files.
    Example:
    / <- Lib root assume
    /love hina/love hina/v01.cbz
    /love hina/specials/sp01.cbz
    The lowest series folder was /love hina/love hina/ for some reason, meaning series scan wasn't getting the full series (all folders) scanned in.
  • Fixed: Fixed up a check for using lowest folder path to lookup from a Folder Watcher task. This should find series much better from a lower file path.

API

  • Search now returns a list of PersonDto instead of strings for People results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

1 participant