Welcome, Guest
Python Scripts for ComicRack
  • Page:
  • 1
  • 2

TOPIC: ComicVine Scraper - speed up scraping by using directory as strict series

ComicVine Scraper - speed up scraping by using directory as strict series 5 years 9 months ago #19731

Is it possible to speed up ComicVine scraping by forcing the scraper to trust the directory name as single series? If it is possible to trust the directory, the scraper could auto guess the issue number and then apply information automatically without the (annoying!) manual choice.

This won't work as a global behavior, so maybe it is possible for Comic Scraper to offer option that once a series is chosen in a folder, the user can assign this folder to be "single series" and then all issues found in it are auto-applied to that series.

For example:

User has a directory called /Batman DCNU and has several issues in it. First time he scans the directory, he manually assigns the first issue to be Batman 2011. There is a checkbox "Apply series to all issues in this directory" and user checks it. Now the scraper properly applies info to this issue. Month later, the user adds a new issue to the directory. He scrapes and instead being asked to choose series, the proper series is auto-scraped and applied.

Of course there can be precautions. If the issue was wrongly put in the directory with file name that differs from the remaining scraped ones, the scraper could issue a warning before applying data. This may be tricky to implement and work 100% fool proof but then again the benefit from auto-scraping without manual choice of series every month, is a great book to current issue collectors.
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 9 months ago #19737

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Thanks for the suggestion. I'll add it to the feature request list so it doesn't get forgotten.
Last Edit: 5 years 9 months ago by cbanack.
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 9 months ago #19742

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 869
  • Thank you received: 108
  • Karma: 37
This would not be very good for the 0day folder, so special care when handling this, please.

fK
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 9 months ago #19745

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Yeah, I can see a number of pitfalls with this feature idea, along with the benefits. When and if I get to implementing this, I'll try very hard (as always!) not to wreck things for users who aren't interested in the new feature.
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 9 months ago #19750

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
I agree with old forkicks! :-)
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 9 months ago #19752

Definitely such feature should be off by default and should carry a warning in the Settings of the plugin. It should be intentionally turned on in the setting for the ability to set Strict Series Folders to become available.
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 9 months ago #19753

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
One way to implement this feature is to make the scraper check for a special file called "cvinfo". The contents of this file would just be a series ID number from Comic Vine.

So anytime the scraper is scraping a comic, it would look for a "cvinfo" file in that comic's directory, and if it finds one, it will automatically use the file's contents to define the series for that comic (rather than asking you to specify it manually.)

This is exactly how the XBMC media player's TVDB.com scraper works...it's nice way to implement this feature, because novice users (or those who don't want "strict series" directories) will never accidentally get into trouble with it.
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 9 months ago #19754

I like this idea because if a folder is mistakenly labeled as strict series, then the user can delete this cvinfo file and the folder will revert to standard scraping.
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 8 months ago #20189

  • superjoe
  • superjoe's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 12
  • Karma: 0
I just want to say that this would be an absolutely killer feature. It would save so much time. Since it's often quite a while between sorting a scraping my new comics, this would keep me from having to manually identify the series each time. Personally, I have a hard time keeping them all straight when there are multiple series with the same name, sometimes running concurrently.


Thanks!
The administrator has disabled public write access.

Re: ComicVine Scraper - speed up scraping by using directory as strict series 5 years 8 months ago #20191

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
I quite like this feature suggestion too, and it will definitely be in the next release of Comic Vine Scraper.

Things have been very busy in my life lately, though--I'm lucky if I get 3 or 4 hours a week to work on the next release, so it's taking a while. But I AM still working on it, and you will see this (and a few other) features sooner or later. ;)
The administrator has disabled public write access.
  • Page:
  • 1
  • 2
Time to create page: 0.218 seconds

Who's Online

We have 234 guests and one member online