69shuba Bug - crawling Error and Stopped Working

[steve30990]

Hi, i 69shuba Crawling is Error
Ex Link: https://69shuba.cx/book/46781.htm

When Crawling on Chrome = It Directly shows
WebToEpub
WARNING: Site '69shuba.cx' has sent an Access Denied (403) error. You may need to logon to site, or browse site normally until you get a Cloudflare "Are you a human" page or satisfy some other CAPTCHA before WebToEpub can continue. Open Page?

When Crawling on 69shuba on Firefox
After a few minutes
WebToEpub
WARNING: Site '69shuba.cx' has sent an Access Denied (403) error. You may need to logon to site, or browse site normally until you get a Cloudflare "Are you a human" page or satisfy some other CAPTCHA before WebToEpub can continue. Open Page?

Tried slowing down to 5 Sec per page =
WebToEpub
WARNING: Site '69shuba.cx' has sent an Access Denied (403) error. You may need to logon to site, or browse site normally until you get a Cloudflare "Are you a human" page or satisfy some other CAPTCHA before WebToEpub can continue. Open Page?

Slow Down to 10 Sec per Page = Shows
Error: Fetch of URL 'https://69shuba.cx/txt/46781/31216326' failed with network error 403. This is an intermittent error. If you retry in a few minutes, it may succeed. promptUserForRetry@moz-extension://699d03a2-177f-48d3-a47e-e62fca5f93b4/js/HttpClient.js:55:19
onResponseError@moz-extension://699d03a2-177f-48d3-a47e-e62fca5f93b4/js/HttpClient.js:48:25
checkResponseAndGetData@moz-extension://699d03a2-177f-48d3-a47e-e62fca5f93b4/js/HttpClient.js:201:45
wrapFetchImpl@moz-extension://699d03a2-177f-48d3-a47e-e62fca5f93b4/js/HttpClient.js:191:31

Macbook Pro M3
MacOS Sonoma Version 14.6.1
Brower: Chrome, Firefox

[gamebeaker]

@steve30990 unable to reproduce. I was able to download 50 Chapter in Chrome with WebToEpub 0.0.0.191.
Did you solve the Captcha after you got the error?
What is your WebToEpub version?
update:
https://69shuba.cx/book/46781.htm
completed download of 1297 Chapter with 1 Max web pages to fetch simultaneously and 3 Secs/Chapter without a single 403 error

[dteviot]

@steve30990

Other thing to try, when error appears, try going to page with browser normally and see if get 403 error. The page may give you more details. Or, if you have dev skills, open Browser's Dev Tools on WebToEpub and look at the network traffic, in particular the full 403 response.

[steve30990]

I see this problem gone, after checking today, i think it was that day, 69shu updated their site, and cloudflare was relogin nonstop.

[steve30990]

It seem like its a issue just pops up again. Same at Chrome
Firefox i suppose will have issue like i mention above, but i think VPN can help Firefox crawl Temporarily.

Below image is Chrome WebtoEpub Version: 0.0.0.167

[gamebeaker]

@steve30990 unable to reproduce i have no problem.
Can you browse the page normally? (Open chapter etc.)
Do you have other extensions installed?

[gamebeaker]

@steve30990 i found the error it is the same as #1439. I tested with the latest test version while you have version 0.0.0.167
@dteviot as it is a cookie issue maybe release a new version to the chrome store as all user should be impacted. (The new version should be 1.0.0.0 or something)
@steve30990 you can use the test versions for Firefox and Chrome.
They have been uploaded to https://github.com/dteviot/WebToEpub/releases/tag/developer-build. Pick the one suitable for you, follow the "How to install from Source (for people who are not developers)" instructions at https://github.com/dteviot/WebToEpub/tree/ExperimentalTabMode#user-content-how-to-install-from-source-for-people-who-are-not-developers and let me know how it goes.

[dteviot]

@gamebeaker

maybe release a new version to the chrome store

Yes, it's probably about time for a new release. Before I do that, can you please check if it's still working on Firefox Android? Site asks me to confirm I've checked, so I can tick the "Works on Android" box in the submission. And I'd rather not lie about it.

[gamebeaker]

@dteviot i tested version 0.0.0.202 in Android. Looks good.
There are obviously problems i am going to create issues for the ones i just ran into. I would still check the box as the core functionality works.

[dteviot]

@steve30990
Updated version (1.0.0.0) has been submitted to Firefox and Chrome stores.
Firefox version is available now.
Chrome might be available in a few hours to 21 days.