Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper 1.0.44-47

Re: Comic Vine Scraper 1.0.44-47 5 years 11 months ago #19568

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
Actually, that sounds pretty interesting...I've not looked closely at that script before...I take it it just tries to pull the scanner details from the filename and add that into the comicbook metadata...

I've been slow to work on the scraper lately, but I've got some time in the next few weeks, so I'll look at this (no promises though!) as I work through my backlog of other scraper related changes and fixes. :)
Last Edit: 5 years 11 months ago by cbanack.
The topic has been locked.
The following user(s) said Thank You: 600WPMPO

Re: Comic Vine Scraper 1.0.44-47 5 years 11 months ago #19571

  • oraclexview
  • oraclexview's Avatar
  • Offline
  • Moderator
  • aka SoundWave
  • Posts: 906
  • Thank you received: 182
  • Karma: 37
cbanack wrote:
lanzajr26 wrote:
Not sure if this is an issue or intended, but with version 1.0.47 if you clear the data for a book and then rescrape it, the large preview window does not show any cover image. The small preview in the main window shows images for the search results but it's difficult to scrape multiple volumes of a title when you can't tell exactly which issue it's pulling up.

Hmmm...so it does.

I just looked at this more closely, and I think it's a problem with ComicRack, actually. When you clear data on a comic book, this seems to also make ComicRack think that the comic book has zero pages, even though it doesn't. This inconsistency is temporary--if you open the info dialog on the cleared book, or open the book itself, or (at least for me) if you even just restart ComicRack entirely, the problem goes away, and the book once again has the right number of pages.

But since the Comic Vine Scraper can't display the first page on a book that has zero pages, you'll see that empty image in the preview window if you try to scrape while ComicRack is confused about the number of pages (i.e. right after clearing data.) I imagine this won't be very hard for cYo to fix...maybe you could report this over in the ComicRack bugs forum?

I've read this comment before when it was 1st written, and after reading it again today, I had a thought. I'm curious as to the reason for clearing out ALL the data from a book just to re-scrape it again. Usually, I just clear out the "Tags", "Notes" and "Web" fields and then do a re-scrape. The fields get overwritten with the updated metadata anyway. So is there something I'm missing here? Just wondering is all. Thanks. :)
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-47 5 years 10 months ago #20295

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 233
Now that we have a dedicated metadata field for Reviews, would it be possible to scrape the reviews from ComicVine? :unsure:

p.s. This is just a thought. Nothing extremely important.
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-47 5 years 10 months ago #20305

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
If they had hyperlinks to each review available from their API, I'd consider trying to do scrape those in somehow--I'm not really interested in trying to copy the entire html text of the reviews in, though.

It doesn't matter either way, though, because right now ComicVine doesn't actually provide any way to access any Reviews from the webs API. :unsure:
The topic has been locked.
The following user(s) said Thank You: 600WPMPO

Re: Comic Vine Scraper 1.0.44-47 5 years 9 months ago #20990

  • Der Internet
  • Der Internet's Avatar
  • Offline
  • Gold Boarder
  • Posts: 176
  • Thank you received: 5
  • Karma: 1
any news about the new version?
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-47 5 years 9 months ago #20992

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
I'm still working on it, a few hours here, a few hours there. Extracting the story arc name out of the the issue title in a consistent way turned out to be very challenging (I've been at it for two weeks now), but I think I got it working decently last night.

There's still a few other (easier) changes to make and then it should be ready to go. I don't really have a solid ETA yet, though, because I don't know how much time I'll have to spare over the next month. If all goes well, a beta version could be available in around 2 weeks.
The topic has been locked.
The following user(s) said Thank You: 600WPMPO, Der Internet

Re: Comic Vine Scraper 1.0.44-47 5 years 9 months ago #21002

  • oraclexview
  • oraclexview's Avatar
  • Offline
  • Moderator
  • aka SoundWave
  • Posts: 906
  • Thank you received: 182
  • Karma: 37
cbanack wrote:
I don't really have a solid ETA yet, though, because I don't know how much time I'll have to spare over the next month. If all goes well, a beta version could be available in around 2 weeks.
I'm all for you taking your time with it because even though I'm really looking forward to the new release with the add features you mentioned, I'm still currently using version 1.0.46 and I have had very very very little problems with it. It is mos def a GREAT add-on for ComicRack and is the only reason that I eventually created a user account at ComicVine. Keep up the great work!

Another +1 karma for you friend! :woohoo:
Last Edit: 5 years 9 months ago by oraclexview.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-47 5 years 9 months ago #21003

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1114
  • Thank you received: 64
  • Karma: 51
oraclexview wrote:
Another +1 karma for you friend! :woohoo:

Yep. Also, we are fortunate that the new api did not break the previous scrapper :)
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-47 5 years 9 months ago #21004

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
oraclexview wrote:
I'm all for you taking your time with it because even though I'm really looking forward to the new release with the add features you mentioned, I'm still currently using version 1.0.46 and I have had very very very little problems with it.
Yup, the last thing I want to do is replace the current version (which is so nice and stable) with something that's broken! I'm definitely not going to do a rush job. :lol:
perezmu wrote:
Yep. Also, we are fortunate that the new api did not break the previous scrapper
Yeah, cYo has been good about adding to his API in a way that doesn't break the scraper. That said, if a new release of ComicRack ever DOES break the scraper, I will definitely try to get a fix out very quickly!
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-47 5 years 9 months ago #21005

  • lanzajr26
  • lanzajr26's Avatar
  • Offline
  • Gold Boarder
  • Posts: 165
  • Thank you received: 1
  • Karma: 2
oraclexview wrote:
I've read this comment before when it was 1st written, and after reading it again today, I had a thought. I'm curious as to the reason for clearing out ALL the data from a book just to re-scrape it again. Usually, I just clear out the "Tags", "Notes" and "Web" fields and then do a re-scrape. The fields get overwritten with the updated metadata anyway. So is there something I'm missing here? Just wondering is all. Thanks. :)

Good question, but when I posted my earlier comment I was just using the right-click "Clear Data" functionality in CR, it's easier than manually clearing out only the fields that matter :) Maybe cYo could refine that function to resolve the missing cover image issue (which is still present).
The topic has been locked.
Time to create page: 0.379 seconds

Who's Online

We have 236 guests and 3 members online