Welcome, Guest
Python Scripts for ComicRack
  • Page:
  • 1
  • 2

TOPIC: Hack: Fill missing issues in Series with ComicVineScraper

Hack: Fill missing issues in Series with ComicVineScraper 4 years 2 months ago #35996

  • Caliph
  • Caliph's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 10
  • Thank you received: 13
  • Karma: 8
Pre-Alpha - Proof of Concept - Backup before Using
So not being happy with the various alternatives to discovering missing issues I hacked together this proof of concept Fill Series 'plugin' to the ComicVineScraper. For every series you select in ComicRack it will create Fileless entries for the missing issues with covers and full metadata. Then it's cake to see what you are missing with a smart list for issues missing a file.

Basic usage is to select one or more issues from each series you want and right click 'Automation->Fill Series' and it will do the rest, assuming your comic rack library has been fully scraped with CVS. If it's not you are going to get a lot more Fileless entries than you wanted.

This requires at least every issue in the series to be scraped with a newer version of the CVS, if every issue has the custom values comicvine_issue and comicvine_volume then you are good.

To install download the attached zip and copy the three files so they replace those in the "%userprofile%\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper" folder.

File Attachment:

File Name: Fill Series.zip
File Size:14 KB


You need ComicVineScraper 1.0.69, currently the latest version.

Short explanation of the files:
Fill Series.py - new file that adds the hook into comic vine to select the series you want and figure out what's missing.
scrapeengine.py - replaced CVS file that adds in a public call to scrape a book.
cvconnection.py - replaced CVS file that changes the API call to mine instead of the default CVS key. Obviously the polite thing to do.

This has no GUI elements short of selecting the books to search the series for.
No way to cancel and you can't see any progress other than watching the entries scroll by in the Script Output window.
Not sure it respects any of the CVS settings you might have made.

Not sure this is the best way to go about it, open to ideas and suggestions! Also don't want to step on any of the CVS team's toes - that's a seriously nice code base to work with - so I couldn't pass up the opportunity to just implement it as a plug in to ComicVineScraper.
The administrator has disabled public write access.
The following user(s) said Thank You: perezmu, 600WPMPO, actioncomics, Shinrai

Re: Hack: Fill missing issues in Series with ComicVineScraper 4 years 2 months ago #35998

  • kenjio
  • kenjio's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 597
  • Thank you received: 127
  • Karma: 32
Just tried this on an almost empty library, on the series "Chew".
I had the latest 4 issues, all scraped, and the script worked like a charm.

It's a great concept, and, with a few improvements, could become one of the most useul scripts here.

I think (speaking as someone who has less than zero knowledge about developing scripts) that the priority would be to create a .crplugin that would not interfere with (overwrite) the Scraper's files (obviously this is presented as a hack at the moment)

Given that, +1 karma for the work!

I'm looking forward to seeing where this goes
I'm baaaaaaaaaaaaaaack!!
The administrator has disabled public write access.

Re: Hack: Fill missing issues in Series with ComicVineScraper 4 years 2 months ago #36003

  • actioncomics
  • actioncomics's Avatar
  • Offline
  • Senior Boarder
  • Posts: 43
  • Thank you received: 6
  • Karma: 2
This is exactly what i have been looking for. Works perfect....Thanks for creating this.
The administrator has disabled public write access.

Re: Hack: Fill missing issues in Series with ComicVineScraper 4 years 2 months ago #36012

  • sykoone
  • sykoone's Avatar
  • Offline
  • Expert Boarder
  • Posts: 153
  • Thank you received: 16
  • Karma: 5
I've been looking for something like this ever since the CV site update broke the old scripts that were available. Thank you. Now to rescrape my files and give it a spin.
The administrator has disabled public write access.

Re: Hack: Fill missing issues in Series with ComicVineScraper 4 years 2 months ago #36016

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Caliph wrote:
Also don't want to step on any of the CVS team's toes - that's a seriously nice code base to work with - so I couldn't pass up the opportunity to just implement it as a plug in to ComicVineScraper.

No worries, as long as you use your own API key (which I see you've done) then there's no problems. :P

If you want to make it so your plugin does not interfere with or use the Comic Vine Scraper settings, just find the code where the Configuration object is loaded and saved (it happens early in the script, before any scraping occurs.) You can either change the location of the loaded/saved settings (if you want to read/write your own settings) or you just skip loading the settings altogether and use the defaults from the Configuration object.
Last Edit: 4 years 2 months ago by cbanack.
The administrator has disabled public write access.

Re: Hack: Fill missing issues in Series with ComicVineScraper 4 years 2 months ago #36029

  • Caliph
  • Caliph's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 10
  • Thank you received: 13
  • Karma: 8
kenjio wrote:
I think (speaking as someone who has less than zero knowledge about developing scripts) that the priority would be to create a .crplugin that would not interfere with (overwrite) the Scraper's files (obviously this is presented as a hack at the moment)
Not sure that would be possible since I need to place a file in a pre-existing directory.

What may be possible is to do something like 'import ../Comic Vine Scraper/scrapeengine.py', but I am not sure how possible that is with Python. I would imagine it is but this way would also require me not replacing any of Comic Vine Scraper's files. Which would require cbanack to accept a patch from me that A sets the currently constant comicvine API key to be a get/set variable and B add a publicly accessible scrape function to scrapeengine.py. If he is amenable to accepting one I can look into whipping one up.

Actually B might not be necessary, I need to take another look at setting up my own config settings with CVS, which might let me use the already public "start scrape on books" method. This would, I think, also buy us a progress bar for free?
cbanack wrote:
No worries, as long as you use your own API key (which I see you've done) then there's no problems. :P
Yeah, I was just imagining my script doing something to get the original API key blocked which would have taken out every user of CVS. :whistle:
Last Edit: 4 years 2 months ago by Caliph.
The administrator has disabled public write access.

Re: Hack: Fill missing issues in Series with ComicVineScraper 4 years 2 months ago #36030

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 869
  • Thank you received: 108
  • Karma: 37
Caliph wrote:
Yeah, I was just imagining my script doing something to get the original API key blocked which would have taken out every user of CVS. :whistle:

And we would then hunt you down and burn you at the stake, but since you're not doing it, all is fine :)

Nice script btw.

fK
The administrator has disabled public write access.

Re: Hack: Fill missing issues in Series with ComicVineScraper 4 years 2 months ago #36031

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Why not just package up your files (including copies of whatever parts of the Comic Vine Scraper code you are using) into a separate .crplugin file? Then when users install it, it will be in its own folder, with no external dependencies and no side effects on other scripts. Just be sure to find the configuration code like I described earlier so you can make sure that your forked plugin doesn't interfere with the users' settings for Comic Vine Scraper.
The administrator has disabled public write access.

Re: Hack: Fill missing issues in Series with ComicVineScraper 4 years 1 month ago #36790

  • sykoone
  • sykoone's Avatar
  • Offline
  • Expert Boarder
  • Posts: 153
  • Thank you received: 16
  • Karma: 5
Any word on this becoming it's own plugin? I updated CVS and didn't think about the overwrites until I went to use this one.
The administrator has disabled public write access.

Hack: Fill missing issues in Series with ComicVineScraper 3 years 6 months ago #38771

I am having trouble getting this to work on ComicRack Portable? it works fine on my laptop and desktop but on the hardrive it is not working? any help?
The administrator has disabled public write access.
  • Page:
  • 1
  • 2
Time to create page: 0.312 seconds

Who's Online

We have 179 guests and 6 members online