Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 1 year 1 month ago #49657

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 318
  • Thank you received: 35
  • Karma: 5
Mummraah wrote:
Longtime weekly user of Comic Vine Scraper. Noticed of late it won't auto scrape series with 'or' in the title. Recently 'Kill or be killed' and 'Death or Glory'

Manually searching on the first word in each will bring it up and can add it manually then.

Cheers for the plug-in!

That was due to changes in how comicvine.com did their searches.
The administrator has disabled public write access.

Comic Vine Scraper 1 year 1 month ago #49662

  • jkthemac
  • jkthemac's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 785
  • Thank you received: 302
  • Karma: 56
Could be a file permissions issue.

BTW you often repeat the notion that your app only uses the filename, but that is plainly not true. I often edit Series and Volume in order to allow a problem search a clearer stab at finding the right entry and it works pretty consistently. Is it just that you are keeping your instructions simple or have you forgotten this is a feature?
Last Edit: 1 year 1 month ago by jkthemac.
The administrator has disabled public write access.

Comic Vine Scraper 1 year 1 month ago #49665

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1371
  • Thank you received: 566
  • Karma: 188
Mummraah wrote:
Longtime weekly user of Comic Vine Scraper. Noticed of late it won't auto scrape series with 'or' in the title. Recently 'Kill or be killed' and 'Death or Glory'

Ah yes. Yet another weird little quirk with the Comic Vine search API. Looks like I could work around this by making an change in the Scraper, which I'll do when I get a chance.

In the meantime, has anyone else noticed this kind of weird behaviour with other search terms? I'm suspicious of other small words like "the, an, and, it, if, of" etc.
Last Edit: 1 year 1 month ago by cbanack.
The administrator has disabled public write access.

Comic Vine Scraper 1 year 1 month ago #49666

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1371
  • Thank you received: 566
  • Karma: 188
jkthemac wrote:
BTW you often repeat the notion that your app only uses the filename, but that is plainly not true. I often edit Series and Volume in order to allow a problem search a clearer stab at finding the right entry and it works pretty consistently. Is it just that you are keeping your instructions simple or have you forgotten this is a feature?

Well, when you manually set the series/volume in ComicRack, you are really changing the metadata about the comic file, not the comic file itself (unless you have a specific setting turned on, but that's another story.) The metadata about a comic is normally stored in the ComicRack database.

When trying to automatically search for info about a comic, the scraper will always use any metadata info that you've set (as you've noticed), and then it will fall back to trying to 'guess' about that metadata by looking at the comic file's filename. But most of the time for a new comic, there is no metadata, so the filename is what gets used.

But the actual contents of your comic file? They don't matter to the scraper. Mostly. If you take a file called Detective.Comics.980.cbz and rename it to My.Little.Pony.Deadly.Adventures.51.cbz. the scraper will search for "my little pony deadly adventures" when you scrape that file.

(Caveat: the actual contents of the comic do matter in one case. When trying to automatically match an issue on Comic Vine to your comic book file, the scraper will look at the first page of the file to get it's cover image, and if that cover image matches the cover of the first series that the scraper found in its search, then it is uses that as "proof" to convince itself that it found the right comic!)
The administrator has disabled public write access.
The following user(s) said Thank You: jkthemac

Comic Vine Scraper 1 year 1 month ago #49667

  • EricS1980
  • EricS1980's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 10
  • Thank you received: 4
  • Karma: 0
cbanack wrote:
Mummraah wrote:
Longtime weekly user of Comic Vine Scraper. Noticed of late it won't auto scrape series with 'or' in the title. Recently 'Kill or be killed' and 'Death or Glory'

Ah yes. Yet another weird little quirk with the Comic Vine search API. Looks like I could work around this by making an change in the Scraper, which I'll do when I get a chance.

In the meantime, has anyone else noticed this kind of weird behaviour with other search terms? I'm suspicious of other small words like "the, an, and, it, if, of" etc.

Back when Comic Vine updated their search API I did notice this issue. I ended up commenting out lines 293 & 294 in the cvdb.py file to get around it and it's worked pretty good since.

Since they updated the search API I also noticed that it seems to only return exact matches for the search term now. For example if you're trying to scrape an issue of Old Man Logan and you search for "Old Logan" or "Man Old" then you won't get any results, it only finds it if you search for 2 or more words in the correct order, such as "Old Man", "Man Logan", etc.

I'm assuming this is also a result of the updated API search
.
Last Edit: 1 year 1 month ago by EricS1980. Reason: correction
The administrator has disabled public write access.

Comic Vine Scraper 1 year 1 month ago #49668

  • Targg
  • Targg's Avatar
  • Offline
  • Senior Boarder
  • Posts: 55
  • Thank you received: 11
  • Karma: 5
I have had to completely get rid of most articles and prepositions to get matches since the API change. A few basic ones:

Anything with "of" such as Justice League "of" America

Betty "&" Veronica

"&, and, to" also sometimes give glitches. If I just pick the most prominent word in the title it usually finds it right away.

Sometimes they work, but often enough they don't. It's just a matter of adjusting the search.
The administrator has disabled public write access.

Comic Vine Scraper 1 year 4 weeks ago #49671

  • Mummraah
  • Mummraah's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 11
  • Karma: 0
Was looking at ways to find missing issue #1's in my collection and came across the comicrack.cyolito.com/forum/13-scripts/3...-updated-11-sep-2017 plugin.

I'll be first to admit I don't get how the underlying parts of the plugins all work but I was wondering.

Would it be possible for ComicVine Scraper to download an MCL file with all the ComicVine info and then scrape metadata from the MCL file? Would that potentially not be faster to scrape a collection and put less of a load on the ComicVine server as only a single 'scrape' would be done to it?

As I said I'm ignorant to how these things work so apologies if this isn't feasible.
The administrator has disabled public write access.

Comic Vine Scraper 1 year 4 weeks ago #49672

  • Rikostan
  • Rikostan's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 3
  • Karma: 0
cbanack wrote:
Hmmm...sounds like you've been pretty thorough already.

One thing to try (and it's a bit of a pain) is to set up a separate, fresh version of ComicRack somewhere else and just try adding one of the files that doesn't work to that ComicRack (as the only file in the library) and see if it scrapes then. If you can find a way to send me one of the files that doesn't work, I could give it a try on my own ComicRack installation.

The thing is, Comic Vine Scraper doesn't actually use the contents of your comic file, for the most part. It mainly just looks at the name of the file for information...so it's hard to see how anything about the contents of an individual comic file would be the cause of your problems. But the only way to be sure is to try it and see if you've got a cbr or cbz file that also causes problems on other systems...

Sorry for the late response, I honestly just kept forgetting to come back and post...

This cleared itself up. After weeks of those certain series not working, it just started working again. I am, down to just a couple hundred that need their names fixed and that's it.

So thank for the response, not sure what it was, but it's all good now.
The administrator has disabled public write access.

Comic Vine Scraper 1 year 1 week ago #49869

  • Drybonz
  • Drybonz's Avatar
  • Offline
  • Gold Boarder
  • Posts: 318
  • Thank you received: 3
  • Karma: 11
Here's a minor search issue I found today. If you search "Chamber of Chills" it does not show "Chamber of Chills Magazine" (sometimes this is called Chamber of Chills v2) as a search result. If I search just "Chamber" It does show "Chamber of Chills Magazine" as a result.

*edit* Same problem for the title "This Magazine is Haunted". Looks like the word "magazine" is screwing things up.
Last Edit: 1 year 1 week ago by Drybonz.
The administrator has disabled public write access.

Comic Vine Scraper 1 year 1 week ago #49873

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1371
  • Thank you received: 566
  • Karma: 188
Drybonz wrote:
Here's a minor search issue I found today. If you search "Chamber of Chills" it does not show "Chamber of Chills Magazine" (sometimes this is called Chamber of Chills v2) as a search result. If I search just "Chamber" It does show "Chamber of Chills Magazine" as a result.

*edit* Same problem for the title "This Magazine is Haunted". Looks like the word "magazine" is screwing things up.

Perfect, this is exactly the info I need. I'm traveling right now, but when I get home this should be pretty straightforward to fix. The scraper tries to 'fix' queries before sending them to Comic Vine, but that's now causing more problems than it solves, so I just have to take that bit out.
The administrator has disabled public write access.
The following user(s) said Thank You: Drybonz, Dereck
Time to create page: 0.322 seconds

Who's Online

We have 121 guests and one member online