Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33575

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 871
  • Thank you received: 109
  • Karma: 37
RevQuixo wrote:
...a few pesky marvel titles that fail because of Panini equivalents...

Add this to the scraper advanced settings:
IGNORE_PUBLISHER=panini
IGNORE_PUBLISHER=panini comics

and problem solved (unless you actually need to scrape panini comics :) )

fK
The administrator has disabled public write access.
The following user(s) said Thank You: RevQuixo

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33578

  • wojosama
  • wojosama's Avatar
  • Offline
  • Gold Boarder
  • Posts: 180
  • Thank you received: 45
  • Karma: 11
IGNORE_PUBLISHER=panini
IGNORE_PUBLISHER=panini comics
IGNORE_PUBLISHER=planeta deagostini
IGNORE_PUBLISHER=marvel italia
IGNORE_PUBLISHER=marvel uk
IGNORE_PUBLISHER=semic as
IGNORE_PUBLISHER=abril

Those are what I use. Takes care of most of the non-US publishers I've come across when scraping.
The administrator has disabled public write access.
The following user(s) said Thank You: 600WPMPO, RevQuixo

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33596

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
I'm glad the scraper and new autoscrape seems to be working for you guys...and thanks for making a video, 600!

Yeah, the IGNORE_PUBLISHER trick is a good way to help the scraper find the right series. If you know you're only scraping a batch of recent comics, you could throw in "IGNORE_BEFORE_YEAR=2005" and "NEVER_IGNORE_THRESHOLD=100"...that would help a lot too.

Of course, the automatic scrape feature still has problems with the same comics that often cause trouble when scraping manually: comics that have the issue title (as well as the series name) in their filename can cause problems, and 2000AD is often a problem. (That's a hard one to fix, because ComicVine contains series called "2000AD" AND series called "2000 AD"--so I can't automatically map one to the other without making it impossible to find certain series!)

I didn't realize Buffy Season 9 was a problem, though. I'll have to look into that more.

@T3KN0Gh057: nothing to worry about, CVDB isn't going anywhere. I do hope that cYo eventually decides to store more details in the comic files--the Custom Book Fields in particular would be really nice for the scraper!
Last Edit: 4 years 6 months ago by cbanack.
The administrator has disabled public write access.
The following user(s) said Thank You: T3KN0Gh057

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33600

  • Freakeao
  • Freakeao's Avatar
  • Offline
  • Gold Boarder
  • Posts: 199
  • Thank you received: 25
  • Karma: 6
cbanack wrote:
I didn't realize Buffy Season 9 was a problem, though. I'll have to look into that more.

I think some Buffy files are named using the number "9" and not the word "Nine".

Another problem is some include "the Vampire Slayer" and some do not. Not an easy generic fix. Maybe something best left to the CR Data Manager before a scrape?
The administrator has disabled public write access.

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33603

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 871
  • Thank you received: 109
  • Karma: 37
I think i found a bug.

With automated scraping on, try to scrape a series that shows up as having 0 issues (my case was arak son of thunder annual). It shows up as 0 issues, but if you manually check it, there is one issue in it.

The scraper will get caught in an infinite loop complaining that this series cannot be displayed because it does not contain any issues in the comic vine database (it does, it even shows the proper thumbnail).

fK

MOVED TO Scraper thread :)
Last Edit: 4 years 6 months ago by 600WPMPO. Reason: Corrected Subject
The administrator has disabled public write access.

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33605

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 871
  • Thank you received: 109
  • Karma: 37
Thanks for moving this, posted it on the wrong thread by mistake (followed the wrong email link).

fK
Last Edit: 4 years 6 months ago by 600WPMPO. Reason: Corrected Subject
The administrator has disabled public write access.

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33606

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1114
  • Thank you received: 64
  • Karma: 51
Cory.... I LOVE YA!!!!! :blush: :blush: :blush: :blush: :blush: :blush: :woohoo: :woohoo: :woohoo: :woohoo: :woohoo: :woohoo:
Last Edit: 4 years 6 months ago by 600WPMPO. Reason: Corrected Subject
The administrator has disabled public write access.

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33607

  • RevQuixo
  • RevQuixo's Avatar
  • Offline
  • Gold Boarder
  • Posts: 280
  • Thank you received: 26
  • Karma: 12
Found a bug..if you click on the cover in the preview window, where you previously had the right left toggle image function you now do not..and if you click on the image it breaks the thumbnail and gives a "page failed to load" screen.

got another one, not really a bug, but distracts from the convenience:

Let's say you have auto match on. If you have multiple series with the same name, it selects that one and starts the scrape, When it gets to an issue that numerically doesn't belong in that run, it prompts you for input. this is well and good in the "old way" of doing things, but it really should either skip that entry till the end or be smart enough to realize if an issue doesn't exist with say the first run of fantastic four, it must not be from that run and go and find another one.
Last Edit: 4 years 6 months ago by 600WPMPO. Reason: Corrected Subject
The administrator has disabled public write access.

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33617

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
@RevQuixo, @forkicks, @freakeao: thanks for the bug reports and your thoughts guys, I'll look into these soon.

@perezmu: right back at ya, buddy :laugh:
Last Edit: 4 years 6 months ago by cbanack.
The administrator has disabled public write access.

Re: Comic Vine Scraper 1.0.65 4 years 6 months ago #33624

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1114
  • Thank you received: 64
  • Karma: 51
Strange bug here...

Trying the automatic feature, works like a charm in general. However, I have found that if a series is wrongly guessed as a TPB, clicking "Go Back" that should take me back to the possibility of changing the search string, either skips the comic or simply does nothing, and then I have to click "skip" to keep going.

In particular this happens for "Captain America and the Hawkeye 629", or "Captain America and Iron Man 635" for which CVS suggests the TPB... I need to actually skip it to keep going, "Go Back" doesn't work.

Then, all subsequent comics in "Captain America and Hawkeye" series are mapped to the TPB and

One comment to improve the TPB/comic automatic detection: still failing on a few ones (for example, Captain America and the Hawkeye 629. You might (if you don't already) try factoring in the number of pages in the "equation"...
The administrator has disabled public write access.
Time to create page: 0.785 seconds

Who's Online

We have 209 guests and 5 members online