Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MangaDex support broken #73

Open
kattjevfel opened this issue Apr 19, 2019 · 6 comments
Open

MangaDex support broken #73

kattjevfel opened this issue Apr 19, 2019 · 6 comments

Comments

@kattjevfel
Copy link

Whenever I try either cum follow or cum get on a mangadex.org url I get the following throw right in my face:

~ » cum follow https://mangadex.org/title/30170
Traceback (most recent call last):
  File "/usr/bin/cum", line 11, in <module>
    load_entry_point('cum==0.9.1', 'console_scripts', 'cum')()
  File "/usr/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/cum/cum.py", line 15, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/cum/cum.py", line 209, in follow
    series = utility.series_by_url(url)
  File "/usr/lib/python3.7/site-packages/cum/utility.py", line 71, in series_by_url
    return Series(url)
  File "/usr/lib/python3.7/site-packages/cum/scrapers/mangadex.py", line 24, in __init__
    self._get_page(self.url)
  File "/usr/lib/python3.7/site-packages/cum/scrapers/mangadex.py", line 49, in _get_page
    self.json = json.loads(r.text)
  File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

..which I'm going to assume is not how it's supposed to be.

I am using cum version 0.9.1 on Arch Linux.

@kattjevfel
Copy link
Author

With #68 applied it first gave me different error, but after I manually opened one of the images in my browser it started working just fine and happily downloaded all my chinese cartoons.

error in question (hella long)

@kattjevfel
Copy link
Author

Whatever, that was short lived, now nothing works and it gives me Access denied | mangadex.org used Cloudflare to restrict access

@Hamuko
Copy link
Owner

Hamuko commented Apr 21, 2019

Looks like DDoS protections.

https://mangadex.org/thread/74209

Not really sure what to do about it at this moment in time.

@mxnemu
Copy link

mxnemu commented Jul 22, 2019

I didn't have any cloudflare problems since May. I guess mangadex fixed their setup and this issue can be closed?

@kattjevfel
Copy link
Author

Still happening on my end, only happens when downloading a lot of images though.

@BrainDamage
Copy link

there's a lib to bypass cloudflare's DDoS protection which has moderate chances of success, however, it requires node.js in order to fool their protection.
https://github.com/Anorov/cloudflare-scrape
It might be useful to have as optional dependency since many sites run under cloudflare and it'd allow supporting more than just mangadex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants