Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper 1.0.44-47

Re: Comic Vine Scraper 1.0.44-46 6 years 1 month ago #16293

  • ph34r
  • ph34r's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
cbanack wrote:
What problem are you trying to solve? I don't know of any script that lets you manipulate covers for your comics, but ComicRack itself has some tools for doing that sort of thing.

my only problem is, i don't have any cover for my manga scan
since mostly they're from magazine, not comic book

i'll try make fileless, maybe i can do it this way!
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 1 month ago #16397

  • romsnesrom
  • romsnesrom's Avatar
  • Offline
  • Expert Boarder
  • Posts: 120
  • Thank you received: 12
  • Karma: 3
Hi,

I recently decided that my collection was already too big to manually manage and started trying software for helping me in that task. I even tried ComicRack and passed it favor of an online application. Turns out that it has some scaling problems and I started researching again. This time I did end up with Comic Rack, not by the program itself (not to knock anything away from it, it simply does not cover my needs alone), but for plug-ins like CVS and Library Organizer.

In my small scale tests (correct me if I am wrong) CVS behavior seems to be: scan comic -> fetch possible candidates -> prompt user for match -> scrape data -> scan next comic...

While that sequence is fine for a collection update it presents a problem when you have a fairly large collection to import at once. The steps that take more time are usually the ones where the scraper has to interact with Comic Vine. CVS working the way it does leave the user having to stand before it for a very long time (mostly, doing nothing but wait) because you cannot do anything else or the match phase will come again and it will not go further.

I do not know if this is made by design in order to lighten the load for Comic Vine but I ask: Is it possible to change the order that CVS execute the scraping steps? Exemplifying: Is it possible that the sequence can be altered to: scan comic [1 to n] (defined by number of comics or cache size) -> fetch possible candidates [1 to n] -> prompt user for match [1 to n] -> scrape data [1 to n] -> scan next batch...

Or just tell me it is not a real problem and I am just being a silly newbie :laugh:
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 1 month ago #16412

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
romsnesrom wrote:
I do not know if this is made by design in order to lighten the load for Comic Vine but I ask: Is it possible to change the order that CVS execute the scraping steps? Exemplifying: Is it possible that the sequence can be altered to: scan comic [1 to n] (defined by number of comics or cache size) -> fetch possible candidates [1 to n] -> prompt user for match [1 to n] -> scrape data [1 to n] -> scan next batch...

Hmmm...that's an interesting idea. It would be hard to entirely avoid interacting with the CV database (i.e. the slowest part) during the "prompt user for match" phase. Still, the improvement you describe is theoretically possible, I think, but it would also require rewriting most of the scraper, which I'm probably not going to do anytime soon!

If you've just scraped your collection for the first time, it might make you feel better to know that the whole process does get somewhat faster from now on. For one thing, rescraping your collection will not require any interaction, you can just start it and walk away. Also, the scraper caches many of the search results that were used in previous scrapes, and after that it only checks to make sure its cache is up to date whenever you scrape new comics in a series you've scraped before.
Last Edit: 6 years 1 month ago by cbanack.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 1 month ago #16420

  • romsnesrom
  • romsnesrom's Avatar
  • Offline
  • Expert Boarder
  • Posts: 120
  • Thank you received: 12
  • Karma: 3
I was asking if there was already a way (in current CVS) to do that. I agree that rewriting most of the code for that would probably not be very productive. Since there is no way to avoid that, I shall CVS my collection for the first time this weekend. FYI I still think CVS is awesome and Futurama rocks!
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 1 month ago #16421

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Yay Futurama!

If you're scraping a large collection for the first time, be sure to look in the preferences for the feature that determines whether your comic book metadata (i.e. the comic info that you scrape) gets stored in your actual comic book files, or only in the comicrack database (I can't remember exactly what the preference is called, and I'm at work right now so I can't look.)

I believe by default the scraped metadata only gets stored in the database, which I think is not ideal. It makes it tricky to retain all that data if you buy a new computer and move your collection. So you might want to change that setting before you get started.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 1 month ago #16422

  • romsnesrom
  • romsnesrom's Avatar
  • Offline
  • Expert Boarder
  • Posts: 120
  • Thank you received: 12
  • Karma: 3
I think you are referring to "allow writing of Book info into files". How difficult would it be to move my collection to another computer? Would not be sufficient to backup the XML, recreate the same directory tree and restore the XML?
Last Edit: 6 years 1 month ago by romsnesrom.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 1 month ago #16423

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
romsnesrom wrote:
I think you are referring to "allow writing of Book info into files".
Yes.
How difficult would it be to move my collection to another computer? Would not be sufficient to backup the XML, recreate the same directory tree and restore the XML?
Yeah, that should work (some people do not find those steps very easy!) Other people in this forum have done it, and would know better than I do if you're missing a step.

On the other hand, if you have all the metadata in your comic book files, you can just install a fresh copy of ComicRack onto a new computer, add all your comics to it the usual way, and you're done. (Unless of course you want to move your smartlists too.) I've done that with my collection twice.
Last Edit: 6 years 1 month ago by cbanack.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 3 weeks ago #16525

  • Tardy
  • Tardy's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 4
  • Karma: 0
Suddenly Comic Vine Scraper won't can't seem to find any of my titles. I have tried looking up "Batman" as well as several others and it responds:

Couldn't find any comic books that match the search terms:

'Batman'

Be sure that all search terms are spelled correctly! Your search should include all (or part) of a comic book's title, but NOT its issue number, publisher, publication date, story arc, etc.

Search the comic book database for:

Batman

It was working great last week. Any idea what could be happening? I am using Comic Vine Scraper 1.0.44-46, ComicRack 0.9.143 64 bit. I HAVE tried turning off my personal firewall just in case . . . I get the same issue.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 3 weeks ago #16526

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
I don't have any immediate solution for you, but I definitely know what's happening:

comicrack.cyolito.com/forum/13-scripts/1...-problem-anyone-else
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 3 weeks ago #16528

  • Tardy
  • Tardy's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 4
  • Karma: 0
Ahhh . . . sorry and thanks for the info. I went onto the ComicVine site, created an account and added to the list of people asking for a fix. Hope it helps.
The topic has been locked.
Time to create page: 0.212 seconds

Who's Online

We have 223 guests and 10 members online