Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 4 months 5 days ago #49247

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1363
  • Thank you received: 534
  • Karma: 186
Kiljoy McCoy wrote:
thanks, just tried the ones I haven't been able to scrape recently and it got them all =)

When you have a chance, you should re-download and install the 1.0.95 plugin. I accidentally released it with a minor bug, and which I just fixed and quickly slipped in without creating a new version of the scraper. If you downloaded the plugin prior to me posting this, you should download it again to make sure you get the file without the bug. (Sorry!)
The administrator has disabled public write access.
The following user(s) said Thank You: ghoti

Comic Vine Scraper 4 months 5 days ago #49248

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1363
  • Thank you received: 534
  • Karma: 186
Drybonz wrote:
So, I have updated to .94 from .93 and it does find the All-Star Comics... however, it's now not finding the Iron Man & Sub-Mariner one shot from the 60's. Could it be some kind of problem with titles with hyphens?

*edit* Marvel Super Special (70s-80s series) is another one I just found that used to show in a search, but now it is not.

*edit* It also doesn't like Psycho-path (Greater Mercury)

Problems with hyphens (at least the ones caused by the scraper) should be fixed in the 1.0.95 release.

I mentioned the problem finding 'Marvel Super Special' on the Giantbomb search thread, because it's a problem with their search API.

I can't even find a series called 'Psycho-path (Greater Mercury)' on the Comic Vine website.
Last Edit: 4 months 5 days ago by cbanack.
The administrator has disabled public write access.
The following user(s) said Thank You: Drybonz

Comic Vine Scraper 4 months 2 days ago #49268

  • ghoti
  • ghoti's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 19
  • Thank you received: 2
  • Karma: 0
not sure whether this is an issue with the scraper or the website, but there are some comic runs that show up as 0 issues when the option box comes up.
For example, scrape "mighty Thor 410" , all series runs for "Thor" show up no problem but looking for the 1989 volume and should be 84 issues, instead its "0"
I had another 2-3 titles do this (can't remember now) - Its not a big deal but thought I'd mention it in case its part of a bigger problem.

Thanks for fixing it anyway. :woohoo:
The administrator has disabled public write access.

Comic Vine Scraper 4 months 2 days ago #49269

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1363
  • Thank you received: 534
  • Karma: 186
ghoti wrote:
not sure whether this is an issue with the scraper or the website, but there are some comic runs that show up as 0 issues when the option box comes up.
For example, scrape "mighty Thor 410" , all series runs for "Thor" show up no problem but looking for the 1989 volume and should be 84 issues, instead its "0"
I had another 2-3 titles do this (can't remember now) - Its not a big deal but thought I'd mention it in case its part of a bigger problem.

Thanks for fixing it anyway. :woohoo:

Yeah, this is a problem with their database. It's been around for a long time, but it seems to have become worse lately.

github.com/cbanack/comic-vine-scraper/issues/46
Last Edit: 4 months 2 days ago by cbanack.
The administrator has disabled public write access.
The following user(s) said Thank You: ghoti

Comic Vine Scraper 4 months 2 days ago #49270

  • ghoti
  • ghoti's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 19
  • Thank you received: 2
  • Karma: 0
fair enough, just thought I'd mention it in case it was a symptom of something larger.

Thanks for the quick reply too! ;)
The administrator has disabled public write access.

Comic Vine Scraper 4 months 1 day ago #49276

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1128
  • Thank you received: 64
  • Karma: 51
cbanack wrote:
Yeah, this is a problem with their database. It's been around for a long time, but it seems to have become worse lately.
github.com/cbanack/comic-vine-scraper/issues/46

I guess this might be fixed, but I am sure it is quite involving and not a minor maintenance issue that Cory is tackling right now... anyone up to the task?

I guess that this might be done as follows (or something like this): when the API returns 0 issues for a series, we know it is likely to be wrong, so we query that series for a list of all the issues and then count them, and use the result as the real number of issues, and then let the scrapped do its magic... I am sure it is not as easy as I say, otherwise, it would have been done already... but as of today, I think this the mayor source of issues missing tags... usually happens in new series with 1 or 2 books, but also in others with any number of issues

Cheers!
The administrator has disabled public write access.

Comic Vine Scraper 3 months 3 weeks ago #49299

  • Kiljoy McCoy
  • Kiljoy McCoy's Avatar
  • Offline
  • Expert Boarder
  • Posts: 80
  • Thank you received: 1
  • Karma: 0
This week when doing new releases it would show only to 2 months ago.

examples, if an issue was number 10 the scrapper would only find the series to issue 8.
If the issue was 22 it would show series as only having 20 issues.....
Made it so auto wasn't working correctly.

Scrapping any other issue older than 2 months was no problem.

I reinstalled the CVS files between working DC and Marvel. Same results.
The administrator has disabled public write access.

Comic Vine Scraper 3 months 3 weeks ago #49300

  • Drybonz
  • Drybonz's Avatar
  • Offline
  • Gold Boarder
  • Posts: 314
  • Thank you received: 3
  • Karma: 10
cbanack wrote:
Problems with hyphens (at least the ones caused by the scraper) should be fixed in the 1.0.95 release.

I mentioned the problem finding 'Marvel Super Special' on the Giantbomb search thread, because it's a problem with their search API.

I can't even find a series called 'Psycho-path (Greater Mercury)' on the Comic Vine website.

Thanks for the hyphen fix, and reporting the MSS volume/comic issue... most appreciated.

Greater Mercury is a more obscure publisher (although there are more obscure ones on Comic Vine) so that is probably just not listed, as you said.

Thanks again.
The administrator has disabled public write access.

Comic Vine Scraper 3 months 3 weeks ago #49301

  • boshuda
  • boshuda's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 334
  • Thank you received: 85
  • Karma: 10
Kiljoy McCoy wrote:
This week when doing new releases it would show only to 2 months ago.

examples, if an issue was number 10 the scrapper would only find the series to issue 8.
If the issue was 22 it would show series as only having 20 issues.....
Made it so auto wasn't working correctly.

Scrapping any other issue older than 2 months was no problem.

I reinstalled the CVS files between working DC and Marvel. Same results.

I've done this before - in your settings file (little dropdown next to the main scraper icon), do you have the 'ignore_before' or 'ignore_after' options set? I don't remember the exact names for things, but you should be able to figure it out enough to check this. If it's not set I think there won't be anything listed, and I think you can comment out any lines with a '#' symbol and CVS will ignore those lines.

Otherwise that sounds a lot like an issue on the ComicVine (server) side. Could be some weird database balancing issue, lost data, etc.
The administrator has disabled public write access.

Comic Vine Scraper 2 months 6 days ago #49413

Is anyone else having near-constant "The Comic Vine online database could not be reached" messages? I seem to have that pop up more often than not when I try to search my issues.

It is happening with such a frequency that it seems odd that it wouldn't be mentioned on the CV forums, and I saw nothing there so I decided to check here. Am using Scraper v. 1.0.95.
The administrator has disabled public write access.
Time to create page: 0.228 seconds

Who's Online

We have 200 guests and 2 members online