Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 1 week 4 days ago #48535

  • pweasel
  • pweasel's Avatar
  • Offline
  • Expert Boarder
  • Posts: 123
  • Thank you received: 18
  • Karma: 8
what's the URL to get those stats?
FPMPlay wrote:
I got 9 out of 44 scraped and then it errored out. Still not working for me. :(

I checked my usage and it was changing so it's definitely connecting:
Current API Usage

You have used 26 requests in the last hour for API Path '/search' (reset in 53 minutes)
You have used 4 requests in the last hour for API Path '/issues' (reset in 53 minutes)
Your request rate is fine

I'll try again later today.
CRW 0.9.178 x64 on Win10
CRA 1.80 on Nexus 10
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48536

  • boshuda
  • boshuda's Avatar
  • Offline
  • Gold Boarder
  • Posts: 296
  • Thank you received: 65
  • Karma: 8
pweasel wrote:
what's the URL to get those stats?
FPMPlay wrote:
I got 9 out of 44 scraped and then it errored out. Still not working for me. :(

I checked my usage and it was changing so it's definitely connecting:
Current API Usage

You have used 26 requests in the last hour for API Path '/search' (reset in 53 minutes)
You have used 4 requests in the last hour for API Path '/issues' (reset in 53 minutes)
Your request rate is fine

I'll try again later today.

It requires your comcvine login: https://comicvine.gamespot.com/api/
Last Edit: 1 week 4 days ago by boshuda.
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48537

  • FPMPlay
  • FPMPlay's Avatar
  • Offline
  • Senior Boarder
  • Posts: 58
  • Thank you received: 12
  • Karma: 3
So I tried again. I had a stack of 35 books and it failed. When I tried a single issue it worked. Then I tried selecting more issues but not all and it worked. Long story short it was failing on Cyborg #18 only. So weird.

BTW - What is "Xelloss auto-complete volume script"? What does it do?
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48538

  • pweasel
  • pweasel's Avatar
  • Offline
  • Expert Boarder
  • Posts: 123
  • Thank you received: 18
  • Karma: 8
Current API Usage

You have no current rate limits. Go forth and query.

:)

boshuda wrote:
pweasel wrote:
what's the URL to get those stats?
FPMPlay wrote:
I got 9 out of 44 scraped and then it errored out. Still not working for me. :(

I checked my usage and it was changing so it's definitely connecting:
Current API Usage

You have used 26 requests in the last hour for API Path '/search' (reset in 53 minutes)
You have used 4 requests in the last hour for API Path '/issues' (reset in 53 minutes)
Your request rate is fine

I'll try again later today.

It requires your comcvine login: https://comicvine.gamespot.com/api/
CRW 0.9.178 x64 on Win10
CRA 1.80 on Nexus 10
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48539

Interesting, I did a quick test this morning on some old Journey Into Mystery issues from the 60's. The 10 books that I scraped worked fine. I'll have to do some more in-depth testing when I get home.
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48540

  • FPMPlay
  • FPMPlay's Avatar
  • Offline
  • Senior Boarder
  • Posts: 58
  • Thank you received: 12
  • Karma: 3
Update: So out of 44 books all scraped fine except for Cyborg #18 and Superman #34. Very weird. Wish I could see the error.
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48542

  • boshuda
  • boshuda's Avatar
  • Offline
  • Gold Boarder
  • Posts: 296
  • Thank you received: 65
  • Karma: 8
The Xelloss auto-complete volume script is one of the scripts he posted in his thread. It's usually near the top of Recent Topics.

found it:
comicrack.cyolito.com/forum/13-scripts/3...-beta?start=10#48108
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48543

  • Scuttle
  • Scuttle's Avatar
  • Offline
  • Junior Boarder
  • Posts: 24
  • Thank you received: 14
  • Karma: 6
Update: So out of 44 books all scraped fine except for Cyborg #18 and Superman #34. Very weird. Wish I could see the error.

Interesting... Cyborg 17 won't scrape here... basically hangs all of ComicRack...
Last Edit: 1 week 4 days ago by Scuttle.
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48545

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
Hmmm...this all reminds me of when ComicVine first introduced API keys and rate limits on the number of API accesses that you can do. That was such a mess.

Anyhow, this time they probably just got new hardware (like a new load balancer or something) or else they've made some recent changes without remembering that the API website needs to have different access rules that allow repeated, rapid visits. Whatever the issue was, it sounds like they've resolved it now, so we should be back to regular scraping.

FPMPlay wrote:
Update: So out of 44 books all scraped fine except for Cyborg #18 and Superman #34. Very weird. Wish I could see the error.

This is probably a separate, unrelated issue. The ComicVine API can sometimes get out of sync with their database, and it takes a day or two to sort itself out. So the problem *might* go away on its own. I will try scraping these two comics on the weekend and see if they are still a problem. If they are, I'll try to debug it and see if there's some way to fix things by updating the scraper.
Last Edit: 1 week 4 days ago by cbanack.
The administrator has disabled public write access.

Comic Vine Scraper 1 week 4 days ago #48548

  • Mummraah
  • Mummraah's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 4
  • Karma: 0
Same issue here with Cyborg #18 and Superman #34. Everything else in my weekly scrape is working fine.

Manually adding the Comicvine issue and volume refs and scraping for the rest of the info works:

Cyborg #18 issue 634508 volume 94266
Superman #34 issue 634520 volume 91298
The administrator has disabled public write access.
Time to create page: 0.230 seconds

Who's Online

We have 178 guests and 2 members online