Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 1.0.65-72 4 years 9 months ago #38206

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
@ClayM, pweasel, and actioncomics:

Looks like the guys at ComicVine are still struggling to sort out their database issues. The issues you've mentioned are all problems with the data that their database sends back to the scraper via their web API, so I cannot directly fix the problems myself.

However, I have reported the issues that each of you noticed (see here), so hopefully they'll be able to do something soon.
The administrator has disabled public write access.
The following user(s) said Thank You: TriOpticon, vc4u

Comic Vine Scraper 1.0.65-72 4 years 9 months ago #38207

  • vc4u
  • vc4u's Avatar
  • Offline
  • Junior Boarder
  • Posts: 22
  • Thank you received: 5
  • Karma: 0
Also, I couldn't find "Green Lantern Corps" Volume 2011 in the search list, only V2006 was displayed while scrapping. And when you search only "Corps" it then show and return "Green Lantern Corps" V2011 as well as V2006.
Last Edit: 4 years 9 months ago by vc4u.
The administrator has disabled public write access.

Comic Vine Scraper 1.0.65-71 4 years 9 months ago #38208

  • RevQuixo
  • RevQuixo's Avatar
  • Offline
  • Gold Boarder
  • Posts: 282
  • Thank you received: 27
  • Karma: 12
Yeah there is definitely an issue with longer names not showing up without first truncating the search criteria. i found it happening with tons of series where the exact name was in CV, but it wouldn't be searchable until you shorten the search string.
The administrator has disabled public write access.

Comic Vine Scraper 1.0.65-71 4 years 9 months ago #38210

  • actioncomics
  • actioncomics's Avatar
  • Offline
  • Senior Boarder
  • Posts: 43
  • Thank you received: 6
  • Karma: 2
thanks cbanack, wasn't sure if it was a search engine issue or an issue with the script.
The administrator has disabled public write access.

Comic Vine Scraper 1.0.65-71 4 years 9 months ago #38212

  • actioncomics
  • actioncomics's Avatar
  • Offline
  • Senior Boarder
  • Posts: 43
  • Thank you received: 6
  • Karma: 2
Don't know if this has been requested in the past, if so just ignore me :-)

the CVINFO file that the scraper can use, could an option be added to create this file in top folder? or maybe create one file that saves all directory/comicvine cross references. that way we don't have to manually create a file for each title.

just a request not a show stopper.
The administrator has disabled public write access.

Comic Vine Scraper 1.0.65-71 4 years 9 months ago #38214

  • deathcry
  • deathcry's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 6
  • Karma: 1
ClayM wrote:
I'm seeing some searching shenanigans

All New X-Factor won't bring up the new series, however X-Factor will.

X-Men Legacy will only bring up the TPB. You can find it via "Legacy" though.

Single word named comics come back find. Coffin Hill returned fine.

It's kinda weird.

to sorta build off of this...
x-factor search for me ignores the original (i think) series.
uncanny x-men ignores the original series (post issue 141) the uncanny x-men

in other news, I've noticed a lot of odd behaviors such as ...
"x-men second coming" doesn't work... "second coming" works
i thought that had to do with the title being "x-men: second coming", but..
"x-men unlimited" search didn't work. searching on "unlimited" did. (no colon in the comicvine name for that series)

edit: sorry I see others have already reported similar behavior above. i didn't read every post.
edit2: the uncanny x-men issue didn't seem to be about the length of the search string as i couldn't find the series with the search x-men. i had to find an issue on comicvine, then navigate to the series page and then use the url number. the search on comicvine is definitely messed up trying to find stuff
Last Edit: 4 years 9 months ago by deathcry.
The administrator has disabled public write access.

Comic Vine Scraper 1.0.65-71 4 years 9 months ago #38215

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
There are a number of problems that they're still trying to sort out with the ComicVine database--as you guys have noticed, it can be hard to find the series you're looking for even if you search for it directly on the ComicVine website.

Here's what one of the ComicVine guys said about all these search problems we're having:
the search engine is actually brand new, whole new engine implemented with the update so it is taking some time for it to get up to full strength as it is constantly being tweaked for relevancy and accuracy and some aspects are still missing such as alias searching and commonly named pages are disappearing. There are also some smaller oddities such as newly created/indexed pages not showing up unless you delete the last character or last two characters from the name, also if you paste in an exact search string often times no results will show up or it will default to only searching the first word and occasionally when something can't be found it just returns a bunch of random results. Lots of tweaking still goin on.

So that's that--we're stuck waiting for them to sort things out. It's annoying, but we'll have to be patient I guess. In the meantime, there's a couple ways to work around these problems:

1) Sometimes if you shorten the search expression by removing the first word or two, it will help. So if "x-men second coming" doesn't work, try searching for "second coming". In particular, if you can remove a word with a hyphen in it ("x-men" or "x-factor"), that works best.

2) You can force the scraper to find any series by entering the ComicVine URL of the series in the search field of the scraper. So if you want to find All New X-Factor, use google and search for "comicvine all new x-factor". Go down the list of results and find the "volume"...that is, find "All New X-Factor (Volume)" and click on it. It takes you here. If you search for that URL in the scraper (copy it from your browser into the search field) this will "force" the scraper to find that series.

3) You can also take that same url I described above and paste it into an otherwise empty text file. Rename that text file to "CVINFO" and copy it into the folder that contains all your comics from that series. From now on, whenever you scrape a new comic file in that folder, the scraper will automatically use that series without even searching for it.
Last Edit: 4 years 9 months ago by cbanack.
The administrator has disabled public write access.

Comic Vine Scraper 1.0.65-71 4 years 9 months ago #38216

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
actioncomics wrote:
the CVINFO file that the scraper can use, could an option be added to create this file in top folder? or maybe create one file that saves all directory/comicvine cross references. that way we don't have to manually create a file for each title.

I'm not really interested in building something like this directly into the scraper, but it does seem like a great idea for a new (and fairly straightforward) plugin. If you can find someone who knows who to write ComicRack plugins and wants to do this, I'd be happy to offer a few tips on how to go about getting the CVINFO data out of each comic file.
The administrator has disabled public write access.

Comic Vine Scraper 1.0.65-72 4 years 9 months ago #38222

  • pcguru30
  • pcguru30's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 14
  • Thank you received: 1
  • Karma: 0
vc4u wrote:
Also, I couldn't find "Green Lantern Corps" Volume 2011 in the search list, only V2006 was displayed while scrapping. And when you search only "Corps" it then show and return "Green Lantern Corps" V2011 as well as V2006.

Thank you for that tip! I came on this forum to see if there were any issues and this was the problem that was causing me to see if something was up. Glad to see it's nothing I broke :D and i'm cool with this workaround until they get the sarch issues fixed.
The administrator has disabled public write access.

Comic Vine Scraper 1.0.65-71 4 years 8 months ago #38250

  • fieldhouse
  • fieldhouse's Avatar
  • Offline
  • Expert Boarder
  • Posts: 96
  • Thank you received: 10
  • Karma: 1
cbanack wrote:
actioncomics wrote:
the CVINFO file that the scraper can use, could an option be added to create this file in top folder? or maybe create one file that saves all directory/comicvine cross references. that way we don't have to manually create a file for each title.

I'm not really interested in building something like this directly into the scraper, but it does seem like a great idea for a new (and fairly straightforward) plugin. If you can find someone who knows who to write ComicRack plugins and wants to do this, I'd be happy to offer a few tips on how to go about getting the CVINFO data out of each comic file.

Something that would be really helpful is the ability to add additional filtering data to a re- search. For example, you know Green Lantern Corps should exist but if you just search on Corps you get a big long list of series. If it were possible to add a Publisher and/or year as additional criteria it should significantly reduce the list returned. I guess I've gotten spoiled by the image match feature because now if I get a list of 100+ series back I'll maybe page through it a little and then skip to the next comic.
The administrator has disabled public write access.
Time to create page: 0.244 seconds

Who's Online

We have 131 guests and one member online