Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 3 weeks 6 days ago #48570

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
fieldhouse wrote:
solidus0079 wrote:
How can I get this guy some logs? I'm pretty technical so I can follow whatever instructions or ideas you might have.

I had tio dig to find it...

when you get the error message, press Ctrl-Shift-L. This will save an application log.

Yup, this.

The part that ComicVine might be interested in is the stack trace(s) at the very end of the log, which may tell them something about the error you are receiving. If you want me to help you identify the stack trace in your log, post it here or PM it to me.
The administrator has disabled public write access.
The following user(s) said Thank You: solidus0079

Comic Vine Scraper 3 weeks 6 days ago #48575

  • Matdotnet
  • Matdotnet's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 4
  • Karma: 0
Hello, I have the same problem. I went ahead and accessed ComicVines from my browsers. It prompt me to perform a CAPCHA validation. Once it is done, it works from the browser. If ComicVines start doing this validation, we might need a way to perform the CAPCHA validation from ComicRack and ComicVines Scraper for the extension to continue working. Any idea if this could be implemented?
Thank you very much in advance for your help,

Mat
Last Edit: 3 weeks 6 days ago by Matdotnet.
The administrator has disabled public write access.

Comic Vine Scraper 3 weeks 6 days ago #48576

  • Kiljoy McCoy
  • Kiljoy McCoy's Avatar
  • Offline
  • Senior Boarder
  • Posts: 74
  • Thank you received: 1
  • Karma: 0
I'm able to use comic vine scrapper at the moment . Not seeing any capcha on comicvine using a browser either.

Spoke to soon I was able to scrape all new DC Comics this week except titles with super in it. Rest of DC and 2 tpb scraped with no problems though.
Last Edit: 3 weeks 6 days ago by Kiljoy McCoy.
The administrator has disabled public write access.

Comic Vine Scraper 3 weeks 6 days ago #48577

  • Matdotnet
  • Matdotnet's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 4
  • Karma: 0
It happens when you scaped a large number of comics lately, or someone from your ISP did as well, and ComicVine black list your IP or IP range. Once you are blacklisted, when you access ComicVine from a browser, you get the CAPTCHA prompt. Once you complete the verification, a cookie is saved in the browser cache to suppress the prompt for a while. I don’t know how long the blacklist stands once in effect. In my case, I changed my external IP and it did not resolved the issue, which means it is my ISP that is banned at the moment.
The administrator has disabled public write access.

Comic Vine Scraper 3 weeks 6 days ago #48578

  • Kiljoy McCoy
  • Kiljoy McCoy's Avatar
  • Offline
  • Senior Boarder
  • Posts: 74
  • Thank you received: 1
  • Karma: 0
Yeah ive been getting banned through out the week week on and off but never had a captcha pop up in browser. Just started working after awhile. I use Edge so maybe that has to something to do with it.
The administrator has disabled public write access.

Comic Vine Scraper 3 weeks 6 days ago #48579

  • Matdotnet
  • Matdotnet's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 4
  • Karma: 0
So far I tried Edge, Chrome, Firefox and IE, all latest version, with the same CAPTCHA prompt
The administrator has disabled public write access.

Comic Vine Scraper 3 weeks 6 days ago #48581

  • duckpuppy
  • duckpuppy's Avatar
  • Offline
  • Junior Boarder
  • Posts: 39
  • Thank you received: 3
  • Karma: 1
I'm having an issue scraping Superman, but everything else I've scraped today worked fine.

Here's the relevant bit from the logs:
Caught DatabaseConnectionError: Comic Vine database could not be reached
url: http://comicvine.gamespot.com/api/search/?api_key=...&client=cvscraper&format=xml&limit=100&resources=volume&field_list=name,start_year,publisher,id,image,count_of_issues&query=superman
CAUSE: System.Net.WebException: The remote server returned an error: (502) Bad Gateway.
   at System.Net.HttpWebRequest.GetResponse()
   at CallSite.Target(Closure , CallSite , CodeContext , Object )
   at get_html_string$351(Closure , PythonFunction , Object )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2)
   at __get_page$349(Closure , PythonFunction , Object )

If I use the query URL from the logs (and put my API key where it should go), I get a "502 Bad Gateway" page with this image (posted mostly for the amusement factor):



Querying for something else works just fine... I tried the same URL replacing "superman" with "flash" and got results almost instantly.
Last Edit: 3 weeks 6 days ago by duckpuppy.
The administrator has disabled public write access.

Comic Vine Scraper 3 weeks 6 days ago #48582

  • Kiljoy McCoy
  • Kiljoy McCoy's Avatar
  • Offline
  • Senior Boarder
  • Posts: 74
  • Thank you received: 1
  • Karma: 0
yeah I got that also but with any title with super in it. Like super sons. Had to do the rescraping manual trick
The administrator has disabled public write access.

Comic Vine Scraper 3 weeks 6 days ago #48583

  • beardyandy
  • beardyandy's Avatar
  • Offline
  • Junior Boarder
  • Posts: 20
  • Thank you received: 1
  • Karma: 0
rescraping manual trick ??
The administrator has disabled public write access.

Comic Vine Scraper 3 weeks 6 days ago #48585

  • duckpuppy
  • duckpuppy's Avatar
  • Offline
  • Junior Boarder
  • Posts: 39
  • Thank you received: 3
  • Karma: 1
I was able to scrape Super Sons, but the manual trick didn't work for Superman either.

EDIT: Scratch that - manual did work. I had forgotten I had unchecked the "When rescraping..." option in the CVS settings.
Last Edit: 3 weeks 6 days ago by duckpuppy.
The administrator has disabled public write access.
Time to create page: 0.381 seconds

Who's Online

We have 242 guests and 2 members online