Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 2 weeks 3 days ago #48488

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
Yup, search for the comic manually, and enter the URL of the comic's webpage on ComicVine. Or enter the code that you mentioned. I think either might work, but at least one of those ways definitely does. I can't remember which off the top of my head.
The administrator has disabled public write access.

Comic Vine Scraper 2 weeks 3 days ago #48489

  • Pachilles
  • Pachilles's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 12
  • Karma: 0
Enter it where?

I entered the webpage into the Scraper search, and nothing, no matter how I cut it down.
I entered the full webpage into the Info page on both Scan Information and Web textboxes, and it tied up ComicRack... I'm still waiting on it.

EDIT: No joy on that.

Pachilles
Last Edit: 2 weeks 3 days ago by Pachilles.
The administrator has disabled public write access.

Comic Vine Scraper 2 weeks 3 days ago #48490

  • ksandersbna
  • ksandersbna's Avatar
  • Offline
  • Junior Boarder
  • Posts: 29
  • Thank you received: 8
  • Karma: 2
I’ve had issues with the Primer Pages in that it takes an extra week or two before the CVDB scrapes it. I just figured the data hadn’t been added to the database. I notice it takes about an extra week (I scrape every Wednesday, occasionally in between but always on Wednesday) that it’ll pick the primer pages up.
The administrator has disabled public write access.

Comic Vine Scraper 2 weeks 3 days ago #48491

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 204
  • Thank you received: 21
  • Karma: 4
Pachilles wrote:
Enter it where?

I entered the webpage into the Scraper search, and nothing, no matter how I cut it down.
I entered the full webpage into the Info page on both Scan Information and Web textboxes, and it tied up ComicRack... I'm still waiting on it.

EDIT: No joy on that.

One thing I have done... if you look at stuff previously scraped you'll see the comicvine URL in the notes field. I've pasted the URL there and then rescrapped. As an alterntive you could take the number and add a the CVDB#### number in the tag field and then rescrape. If you have "use values from previous scape when rescrapping" (or something like that) checked which it should be by default then it will then populate all the rest of the values for the comic.
The administrator has disabled public write access.

Comic Vine Scraper 2 weeks 2 days ago #48492

  • Pachilles
  • Pachilles's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 12
  • Karma: 0
Thanks! I finally got it.

You guys are great!
:cheer: :cheer:

Pachilles
Last Edit: 2 weeks 2 days ago by Pachilles.
The administrator has disabled public write access.

Comic Vine Scraper 2 weeks 2 days ago #48493

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 204
  • Thank you received: 21
  • Karma: 4
Pachilles wrote:
Thanks! I finally got it.

You guys are great!
:cheer: :cheer:

For the benefit of others what procedure finally worked for you?
The administrator has disabled public write access.

Comic Vine Scraper 1 week 5 days ago #48509

Is anyone else having trouble scraping from ComicVine at the moment?

I've been working organizing on a small collection for the last few months, scraping around 200 comics each evening with my scraper set to pause 5 seconds between lookups.

As of today ComicRack wasn't able to connect to the API. After several failed attempts I tried pointing my browser manually at comicvine.gamespot.com/api/ to see if the API was live and I got a timeout in my browser. Trying to go to comicvine.gamespot.com also gave me a timeout. Even a ping sent to comicvine.gamespot.com failed. I presumed that they were having site issues or maybe suffering from a Denial of Service attack. I remoted into a computer at another site and found that I had no issues at all going to comicvine.gamespot.com from there.
About 15 minutes later I was able to access comicvine.gamespot.com/api/ again so I fired up ComicRack and tried to scrape a few comics. The scrape failed so I tried going to comicvine.gamespot.com/api/ again and got a timeout.I was still able to access the site from another location without issue.

This really looks to me like an intrusion prevention system is blocking my IP when I try to access the API. I'm wondering if they've taken offense to me scraping a couple of hundred comics per day or if they're messing with the IP's of anyone who is trying to use the API.
The administrator has disabled public write access.

Comic Vine Scraper 1 week 5 days ago #48510

  • boshuda
  • boshuda's Avatar
  • Offline
  • Gold Boarder
  • Posts: 296
  • Thank you received: 65
  • Karma: 8
I'm having the same issue. The API server or database behind it might be down. It usually takes them some time to fully restart stuff when it goes down.
The administrator has disabled public write access.

Comic Vine Scraper 1 week 5 days ago #48511

What seems really suspicious to me is that I still can't access the website from my house although it still works fine from my customers network.
The administrator has disabled public write access.

Comic Vine Scraper 1 week 5 days ago #48512

  • Gemris
  • Gemris's Avatar
  • Offline
  • Junior Boarder
  • Posts: 38
  • Thank you received: 12
  • Karma: 6
I’ve had this issue for a couple days, but only when doing specific searches. Searching for Cyborg or Superman, for example, gave me that error regarding not being able to connect to the database — every time. All other titles scraped fine. It’s super odd.
The administrator has disabled public write access.
Time to create page: 0.310 seconds

Who's Online

We have 243 guests and 3 members online