Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper 1.0.33 - 1.0.38

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 9 months ago #11782

  • Shinrai
  • Shinrai's Avatar
  • Offline
  • Platinum Boarder
  • With great power comes great W/T.
  • Posts: 885
  • Thank you received: 81
  • Karma: 33
Why are you rescraping your entire library weekly?
The topic has been locked.

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 9 months ago #11783

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 871
  • Thank you received: 109
  • Karma: 37
Because i was test-driving new cvs versions :). Before that, just to update the comic ratings, many older ones are only getting sporadic updates to ratings.
The topic has been locked.

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 9 months ago #11785

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
I'd be surprised if they were throttling you in particular--if only because it would take a lot of effort and resources to find and track which IP addresses were using "too much" bandwidth.

More likely their servers (or your ISP) are being kinda flakey, and you're just getting unlucky. Sometimes I find ComicVine really slow too. :unsure:

As an aside, "rescraping" comics is actually a lot less bandwidth-intensive than scraping them for the first time, because most of our traffic from ComicVine's servers comes from all the image files that get downloaded and displayed on that first scrape.

I also rescrape my entire collection on a semi-regular basis.
The topic has been locked.

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 9 months ago #12073

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
Hey guys, I've released version 1.0.38 of Comic Vine Scraper. It contains an important bug fix that affects new users, but doesn't really impact anyone who isn't running the scraper for the first time.
The topic has been locked.
The following user(s) said Thank You: 600WPMPO

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 9 months ago #12074

  • Samael69
  • Samael69's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 381
  • Thank you received: 47
  • Karma: 21
600WPMPO wrote:
No, I meant that the scraper automatically adds a CVDBSKIP tag when we skip a comic (this may be given as an option in settings), so that the next time the skipped file doesn't come up again & we don' have to manually add the CVDBSKIP tag.
Like others, I do this manually. I add a tag of -CVDF- (ComicVine Download Failed) when a book is not found. I then set up a smart list to populate based on this tag so I can easily go back and check them periodically without having to scrape the entire collection. The "-" signs in the tag are simply so that it sorts to the top of all the "CV" tags.
The topic has been locked.

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 9 months ago #12122

  • Samael69
  • Samael69's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 381
  • Thank you received: 47
  • Karma: 21
What's the 3rd number in the comicvine data file? Here is the data file for "3 Little Kittens"

1,123805,20647
2,123807,20647
3,123808,20647

What is the 20647? A number to indicate all three books belong to the same series?
Last Edit: 6 years 9 months ago by Samael69.
The topic has been locked.

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 9 months ago #12123

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
Samael69 wrote:
What's the 3rd number in the comicvine data file? Here is the data file for "3 Little Kittens"

1,123805,20647
2,123807,20647
3,123808,20647

What is the 20647? A number to indicate all three books belong to the same series?
Yes, the first number is the comic issue number, the second number is the ComicVine issue ID, and the third number is the ComicVine series ID.

The file you are looking at one of several kinds of "cache" files that the scraper uses so that it doesn't have to download so much information when you scrape a series that you've scraped before. The scraper only needs to do this caching because there is something fundamentally wrong with the ComicVine web API. One day, when ComicVine fixes that problem, I'll release a version of the scraper that doesn't create these files...
The topic has been locked.

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 8 months ago #12583

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
This has been on my mind for a long time, so here goes:

Everytime I want to scrape, my hand accidentally hits the settings button. I feel that if the places of the 'settings' and 'start scrapings' buttons are exchanged, it would be more ergonomic.

So, from:



to:



So, what is the general consensus? and most importantly, what do you feel, cbanack?
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
Last Edit: 6 years 8 months ago by 600WPMPO.
The topic has been locked.

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 8 months ago #12584

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
I'm not too picky about it, to be honest...the only "standard" button on that dialog is the cancel button, which should stay on the far right.

The only reason not to change the current layout would be because there are lots of people who are used to it the way it is...but if there are other people who have the same problem as you (i.e. the current button placement conflicts with their instincts) then I will change it for the next release.

(Yes, I'm still working on the scraper--it's coming along, just a bit slowly right now...I struggle to find a few hours to work on it every week. Right now I'm working on multi-language support, so at some point in the not too distant future, I'll ask the people who do the translations for comicrack to do some translations for the scraper.)
The topic has been locked.

Re: Comic Vine Scraper 1.0.33 - 1.0.37 6 years 8 months ago #12585

  • mizio66
  • mizio66's Avatar
  • Offline
  • Platinum Boarder
  • Started reading comics at 4... and still counting!
  • Posts: 451
  • Thank you received: 143
  • Karma: 67
... put the position of the buttons in the settings :-)
The topic has been locked.
Time to create page: 0.237 seconds

Who's Online

We have 287 guests and 5 members online