Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 11 months 1 week ago #48549

  • solidus0079
  • solidus0079's Avatar
  • Offline
  • Senior Boarder
  • Posts: 78
  • Thank you received: 3
  • Karma: 1
I'm also having some trouble with the "try to choose the series automatically" option. I'm trying to scrape over 500 books, and I'm getting the "Comicvine database could not be reached" error. Originally it scanned 12 and left the rest unscanned. I thought maybe they cracked down on the rate allowed again, so I slowed it to 50 seconds and it still craps out at the same place.

I'm now trying to figure out which issue(s) it is, there's lot in the queue and the issue isn't shown when the error pops up so I'll try breaking it up and doing it in chunks. Hopefully I can figure it out as it's a needle in the haystack.
Last Edit: 11 months 1 week ago by solidus0079.
The administrator has disabled public write access.

Comic Vine Scraper 11 months 1 week ago #48550

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
@solidus0079: It's probably these two:

Mummraah wrote:
Same issue here with Cyborg #18 and Superman #34. Everything else in my weekly scrape is working fine.
The administrator has disabled public write access.
The following user(s) said Thank You: solidus0079

Comic Vine Scraper 11 months 1 week ago #48551

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
Ok, so I've looked into this latest problem more closely. It is definitely an issue on the Comic Vine side. When you search for any "cyborg" or "superman" comics, the Comic Vine servers simply never respond with any information, so the scraper just sits there waiting until it eventually gives up and shows you an error message.

I've raised a bug report describing the issue over on the Comic Vine forums. The ball is now in their court; when they fix the problem I've described, you will once again be able to scrape those two comics (and whichever other ones are broken that we haven't discovered yet!)
Last Edit: 11 months 1 week ago by cbanack.
The administrator has disabled public write access.
The following user(s) said Thank You: solidus0079, Scuttle

Comic Vine Scraper 11 months 1 week ago #48552

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 313
  • Thank you received: 34
  • Karma: 5
cbanack wrote:
Ok, so I've looked into this latest problem more closely. It is definitely an issue on the Comic Vine side. When you search for any "cyborg" or "superman" comics, the Comic Vine servers simply never respond with any information, so the scraper just sits there waiting until it eventually gives up and shows you an error message.

I've raised a bug report describing the issue over on the Comic Vine forums. The ball is now in their court; when they fix the problem I've described, you will once again be able to scrape those two comics (and whichever other ones are broken that we haven't discovered yet!)

One workaround I found was to search on a subset. If I search on super it works even though I get more results. Only the specific names have an issue. So if you hit this remove some letters and try again.
The administrator has disabled public write access.

Comic Vine Scraper 11 months 1 week ago #48553

  • solidus0079
  • solidus0079's Avatar
  • Offline
  • Senior Boarder
  • Posts: 78
  • Thank you received: 3
  • Karma: 1
cbanack wrote:
@solidus0079: It's probably these two:

Mummraah wrote:
Same issue here with Cyborg #18 and Superman #34. Everything else in my weekly scrape is working fine.
Hehe. I should have checked this thread before I resumed my troubleshooting this morning. I identified Cyborg 18 (and probably would have found Supes 34). Went to report Cyborg and you had already found it. XD

Thanks so much for the help. I'll try and scrape these later.

Here's a link to the Cyborg 18 page, which barfs.
comicvine.gamespot.com/cyborg-18-singula...the-end/4000-634508/

415 left to scrape today, I'll report if something other than this week's Superman and Cyborg error out. Thanks for linking your bug report, I'll mirror it if that's the case with another issue.

Some of my issues were corrupted a few months ago, so I've been on a kick mass replacing issues.
Last Edit: 11 months 1 week ago by solidus0079.
The administrator has disabled public write access.

Comic Vine Scraper 11 months 1 week ago #48554

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
solidus0079 wrote:
Thanks for linking your bug report, I'll mirror it if that's the case with another issue.

If you find another search term that breaks like Cyborg or Superman, it's probably worthwhile to add a comment about it to my bug report/forum topic at the Comic Vine website. That'll show them that more people are interested in having them fix that bug...
The administrator has disabled public write access.
The following user(s) said Thank You: solidus0079

Comic Vine Scraper 11 months 1 week ago #48555

  • solidus0079
  • solidus0079's Avatar
  • Offline
  • Senior Boarder
  • Posts: 78
  • Thank you received: 3
  • Karma: 1
cbanack wrote:
solidus0079 wrote:
Thanks for linking your bug report, I'll mirror it if that's the case with another issue.

If you find another search term that breaks like Cyborg or Superman, it's probably worthwhile to add a comment about it to my bug report/forum topic at the Comic Vine website. That'll show them that more people are interested in having them fix that bug...
Well I noticed the website wouldn't even display those issues so I brought that up because that's probably a more critical issue for them if the website is glitched.

I did get through the rest of that 500+ stack to scrape with no problems once I took Cyborg and Superman out of the lot. I'm doing another huge scrape job and I'm in the "phase 2" where I have to specify issues it couldn't automatically figure out - so I think this batch is good. The Supes/Cyborg problem would break the scrape job during phase 1.

The entire run of Guardians of the Galaxy and Black Panther incoming to the scraper.

Like I said, I had a big library corruption problem recently so I've been mass replacing volumes. If I happen to notice any other problems I'll crosspost it there and here.

Thanks again :)
Last Edit: 11 months 1 week ago by solidus0079.
The administrator has disabled public write access.

Comic Vine Scraper 11 months 6 days ago #48556

  • Kiljoy McCoy
  • Kiljoy McCoy's Avatar
  • Offline
  • Expert Boarder
  • Posts: 80
  • Thank you received: 1
  • Karma: 0
Seems to be back to banning IP again
I was scraping 2 tpbs and it blocked me for about 15 mins than I traied again. Now been blocked for 30 mins
The administrator has disabled public write access.

Comic Vine Scraper 11 months 6 days ago #48557

It's the same for me. I made it through about 75 scrapes this morning and then it was unavailable. Even browsing to the website failed. After a while I'll be able to get to the website again. As soon as I try to do another scrape, I'm immediately unable to access the website again.
I logged into two different remote sites and was able to access the website just fine from there. Clearly the reason I can't access the site is a result of something perceived as wrong from my IP.
The administrator has disabled public write access.

Comic Vine Scraper 11 months 6 days ago #48558

  • Kiljoy McCoy
  • Kiljoy McCoy's Avatar
  • Offline
  • Expert Boarder
  • Posts: 80
  • Thank you received: 1
  • Karma: 0
k about an hour and unbanned and I tried to scrape one title and banned again
The administrator has disabled public write access.
Time to create page: 0.333 seconds

Who's Online

We have 130 guests and 2 members online