Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper 1.0.23/24

Comic Vine Scraper 1.0.23/24 7 years 4 months ago #7992

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Comic Vine Scraper 1.0.24 is now available. It is exactly the same as 1.0.23, but it also includes two minor bug fixes (issues 98 and 100)


Comic Vine Scraper 1.0.23 is now available.

There are only two bug fixes this time (issues 94 and 95), and two new features (issues 17, and 93.)

Of particular interest:

- the scraper now scrapes Teams, Locations, and Webpage details from Comic Vine.
- the scraper will now obtain volume numbers by detecting them in your comic's series name.
- you can also configure the scraper to strip the volume number out of each series name.

If you preferred the old way that the scraper handled volume numbers (by using the year that the comic series was first published), then you'll want to turn on the new checkbox called Use first publication year as volume number.

Also, issue 95 probably caused a lot of the Summary text for your scraped comics look to look pretty bad. Now that this problem has been fixed, you might want to try rescraping your comics, or at least rescraping their Summary data.

As always, if you find any bugs please report them in the issue tracker (you can log into it with your google/gmail account). If you prefer, you can also post a description of your problem (or any other feedback) in this forum thread.

Have fun!
Last Edit: 7 years 1 month ago by cbanack. Reason: Added icon
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8008

  • RevQuixo
  • RevQuixo's Avatar
  • Offline
  • Gold Boarder
  • Posts: 279
  • Thank you received: 25
  • Karma: 12
Loving the new volume handling...I get to have my cake and eat it too. (retain the old year as volume scheme AND strip out the volume from the series).

So far everything else appears to work just fine. Will put it through its paces when I have the time.

This script keeps on getting better and better. :woohoo:
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8010

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
Yes.. the volume field now looks much better with all those ugly v1964, v1996 gone :-)

While most comics scrape fine with v1, v2 ,v3 etc in the volume fields. However, some comics I scraped just now had the volume field empty.. like "Invincible iron man".. Earlier it was giving me Volume 2008, 2009, 2010, but now its just blank. There has to be some volume number, I mean..
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8017

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Well, I've taken a cue from ComicRack itself--when it is computing the "shadow values" for each comic, and it gets a filename with no volume information in it, it just leaves the volume field blank (rather than calling nearly every comic that it meets "Volume 1".) So I made the scraper do the same, since that scheme makes the most sense to me.

If the majority of people prefer it the other way (putting volume 1 in for every comic that has no volume number), I could change the scraper to do that, instead. But I'm hesitant to make another configuration option out of this, unless there are a sizable number of people (especially regular users like yourself) whose can't agree on which way it should be.

At least for now, if you want "volume 1" in all those blank fields, you can make a smart set that looks for blank volume fields, and then anytime you see something in there, just set them all at once.
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8021

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
cbanack wrote:
If the majority of people prefer it the other way (putting volume 1 in for every comic that has no volume number), I could change the scraper to do that, instead. But I'm hesitant to make another configuration option out of this, unless there are a sizable number of people (especially regular users like yourself) whose can't agree on which way it should be.
I feel that putting volume 1 in for every comic that has no volume number would not be a good thing. My opinion is that you keep it the way it is..
In my post above, I was just wondering if there was a volume and the scraper had missed it.. B)
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8022

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
RevQuixo wrote:
Loving the new volume handling...I get to have my cake and eat it too. (retain the old year as volume scheme AND strip out the volume from the series).
Glad you like it. :laugh:

You or others may run into a couple cases (very few, actually) where the volume number is actually part of the name of a series. For example: "Kabuki Volume 3: Metamorphosis". In cases like that, the Scraper will NOT remove the Volume number, since the series is not called "Kabuki: Metamorphosis". In that case, the volume number is really part of the title, not just tacked onto the end of the name because there's nowhere else to put it (which is what happens 99% of the rest of the time).

Even in cases like that, the fact that it is volume 3 will still be detected and stored in the volume field.
So far everything else appears to work just fine. Will put it through its paces when I have the time.
Your testing efforts are, as always, appreciated!
This script keeps on getting better and better. :woohoo:
I think my development efforts are best described as slow but steady.

I am amazed at how many changes cYo manages to do to ComicRack every month! He must drink a lot of coffee or something. :)
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8026

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
600WPMPO wrote:
I feel that putting volume 1 in for every comic that has no volume number would not be a good thing. My opinion is that you keep it the way it is.
Ah I see. :laugh: There was someone else who asked me about that recently, so I just jumped to the wrong conclusion.
In my post above, I was just wondering if there was a volume and the scraper had missed it.. B)
Well, looking at the Comic Vine listing for "Invincible Iron Man", I don't see any volume numbers in the titles, so that must be why it's not finding anything. Comic Vine forces you to store the volume number as part of the series name, which is kind of silly. It means that if the series name doesn't have a volume number, then there is no volume number for that series at all...
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8027

  • cYo
  • cYo's Avatar
  • Offline
  • Moderator
  • Posts: 3476
  • Thank you received: 675
  • Karma: 181
coffee, energy drinks and hormone pills :)
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8028

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1114
  • Thank you received: 64
  • Karma: 51
Thanks!!!!!

Again :P
The administrator has disabled public write access.

Re:Comic Vine Scraper 1.0.23 7 years 4 months ago #8029

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
cYo wrote:
coffee, energy drinks and hormone pills :)
Maybe I should try that...
The administrator has disabled public write access.
Time to create page: 0.254 seconds

Who's Online

We have 168 guests and 8 members online