Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fictionlive ignores "Home" when it has story content #36

Open
thegrinner opened this issue Mar 14, 2020 · 0 comments
Open

Fictionlive ignores "Home" when it has story content #36

thegrinner opened this issue Mar 14, 2020 · 0 comments

Comments

@thegrinner
Copy link
Contributor

Looks like if the author doesn't explicitly rename/bookmark the initial chapter, it doesn't show up in the bookmark list (bm node) and thus is undetected.

One chapter example (with only special bookmarks):
https://fiction.live/stories/Primal-Ties/2mbdgmNkNY2gAE9xo
API is https://fiction.live/api/node/2mbdgmNkNY2gAE9xo

Finished story example (with content on Home):
https://fiction.live/stories/Six-Sinful-Succubi-A-Corruption-Quest-/5xa4YDkog9jBxkFFw/home
API is https://fiction.live/api/node/5xa4YDkog9jBxkFFw

Example run log of the second one:

python leech.py https://fiction.live/stories/Six-Sinful-Succubi-A-Corruption-Quest/5xa4YDkog9jBxkFFw/home
[sites] Handler: <class 'sites.fictionlive.FictionLive'> (https://fiction.live/stories/Six-Sinful-Succubi-A-Corruption-Quest/5xa4YDkog9jBxkFFw)
[sites.fictionlive] Extracting chapter "Ankee - Cap of the Southern Reaches" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1518999824068/1519337407952
[sites.fictionlive] Extracting chapter "Path of Corruption" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1519337407953/1520126691748
[sites.fictionlive] Extracting chapter "Sister Candidates" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1520126691749/1521840365620
[sites.fictionlive] Extracting chapter "Enter Metaltron" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1521840365621/1523744191655
[sites.fictionlive] Extracting chapter "The Coven Grows" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1523744191656/1524949302414
[sites.fictionlive] Extracting chapter "Eri" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1524949302415/1528064244126
[sites.fictionlive] Extracting chapter "Yuri, Angels, Maidens and other assorted deviancy." @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1528064244127/1530478643632
[sites.fictionlive] Extracting chapter "Breakout" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1530478643633/1533936628579
[sites.fictionlive] Extracting chapter "Purity...?" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1533936628580/1536442612256
[sites.fictionlive] Extracting chapter "Expedition" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1536442612257/1539456464547
[sites.fictionlive] Extracting chapter "New Roomies" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1539456464548/1542396450971
[sites.fictionlive] Extracting chapter "A New Day Dawns" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1542396450972/1546043076212
[sites.fictionlive] Extracting chapter "The Third Expedition To The Same Place" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1546043076213/1548452718548
[sites.fictionlive] Extracting chapter "Bandit Fortress" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1548452718549/1550352182086
[sites.fictionlive] Extracting chapter "Scenario: Metaltron" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1550352182087/1550531275487
[sites.fictionlive] Extracting chapter "The Bit before the epilogue" @ https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1550531275488/9999999999999998
[__main__] File created: Six_Sinful_Succubi_A_Corruption_Quest..epub

In this case, this is a working chapter url the story content on the home page: https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/0/1518999824068

Although https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1518907500406/1518999824068 is probably "more correct", I don't see a way to extract that timestamp without actually looking at the first chapter.

For that particular story, it looks like getting https://fiction.live/api/anonkun/chapters/5xa4YDkog9jBxkFFw/1518906796120/1518999824068 where the first value is the ct kvp from the initial api/node url seems to work. I don't know if that'll hold up (like if someone filled a special bookmark in first maybe it would break?)

I thought a missing isFirst kvp in the bookmarks array meant this was the case, but looking at https://fiction.live/api/node/XgpsT3k8s5qXEkJAe I don't see that value set and the Home page is empty.

For anyone else encountering this, I hacked in a short term fix by converting line 37 of fictionlive.py to chapters = ({'ct': 0},) + ({'ct': 0, "title": response['t']},) + tuple(c for c in response['bm'] if not c['title'].startswith('#special')) + ({'ct': 9999999999999999},)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant