Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper 1.0.28 - 1.0.32

Re: Comic Vine Scraper 1.0.28 - 1.0.32 7 years 3 weeks ago #10768

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 871
  • Thank you received: 109
  • Karma: 37
*subtle hint*
say, like a "real" book search by title/isbn, comparable to current series name/cvdb tag
*subtle hint*

fK
The topic has been locked.

Re:Comic Vine Scraper 1.0.28 - 1.0.32 7 years 3 weeks ago #10772

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
perezmu wrote:
Not that I miss ANY thing at all in the script, but, it is great to know you still look at ways to improve it!

Oh yeah, nothing to worry about there--I haven't lost interest in the scraper project at all. The amount of time I can dedicate fluctuates sometimes, but it's ok because the scraper is now pretty polished and useful even if sometimes I don't have a chance to add to it.
Now, here is an idea for you! I have't really looked into the code for a while, but it seems to me it is really modularized... I was playing with the idea of making plugins for a plugin... so that the base structure is reused, changing just the part that queries the database so it could be adapted to other databases... just food for thought, since the interface and options you offer are just superb, it would be great not to need to reinvent the wheel!

I have been thinking along the same lines since early this summer. I've been trying to change the code so that it would be (relatively) easy to substitute other databases in place of Comic Vine. I'm about 90% done. The key to adding support for an extra database is all in the files "dbmodels.py" and "db.py". (Basically, db.py would have to be reimplemented to use a different database to produce the same Issue and Series objects that are defined in dbmodels.py.)

This is the kind thing I'd even consider building directly into the Scraper myself, if I could find a reasonably popular alternate database that has a web API with the same key capabilities as Comic Vine (searching by title, cover images, indexes for individual issues and series, etc.) At this point, the Anime Vice website seems like a good candidate--if they ever open up their API.

@forkicks: "real" books? You mean like novels? I didn't think people used ComicRack to read that sort of material...
The topic has been locked.
The following user(s) said Thank You: OniTenshu

Re:Comic Vine Scraper 1.0.28 - 1.0.32 7 years 1 week ago #10927

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
Kurt_Wagner wrote:
And for unknown reason it adds the Marvel publisher as Marvel Comics, although on ComicVine it's listed simply as Marvel. later i'll check DC's issues for this problem and report.

This will be addressed in the next release of Comic Vine Scraper:

code.google.com/p/comic-vine-scraper/issues/detail?id=143
The topic has been locked.

Re: Comic Vine Scraper 1.0.28 - 1.0.32 7 years 1 week ago #10929

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 871
  • Thank you received: 109
  • Karma: 37
Hey cbanack. I just updated issue 149 with the info you requested, sorry for taking so long, but i didnt get google notification regarding updates on it, so i didn't see your updates.

fK
The topic has been locked.

Re: Comic Vine Scraper 1.0.28 - 1.0.32 7 years 2 days ago #11008

  • kenjio
  • kenjio's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 597
  • Thank you received: 127
  • Karma: 32
I just noticed while scraping "The Savage Dragon" that CVS is not recognizing issues numbered as such

"The Savage Dragon - 013a"

The issue is present on ComicVine (There is an issue 13 and a 13a - but only the 13 comes up in the scrape results)

===============

Correction: both issues appear in the scraper, but both identified as 13 - need to use teh cover image to distinguish them
I'm baaaaaaaaaaaaaaack!!
Last Edit: 7 years 2 days ago by kenjio.
The topic has been locked.

Re: Comic Vine Scraper 1.0.28 - 1.0.32 7 years 2 days ago #11009

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
kenjio wrote:
I just noticed while scraping "The Savage Dragon" ...
Thanks for pointing that out, I'll look into it for the next release. B)
The topic has been locked.

Re: Comic Vine Scraper 1.0.28 - 1.0.32 7 years 2 days ago #11010

  • kenjio
  • kenjio's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 597
  • Thank you received: 127
  • Karma: 32
Awesome :)

PS - next time, I'll not be lazy and file the bug report myself :P
I'm baaaaaaaaaaaaaaack!!
The topic has been locked.

Re: Comic Vine Scraper 1.0.28 - 1.0.32 7 years 2 days ago #11011

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
kenjio wrote:
Awesome :)
PS - next time, I'll not be lazy and file the bug report myself :P

No worries! :laugh:

I'm just happy when people let me know about problems. Lazy would be if you hadn't said anything at all. ;)
The topic has been locked.

Re: Comic Vine Scraper 1.0.28 - 1.0.32 7 years 2 hours ago #11035

  • rmagere
  • rmagere's Avatar
  • Offline
  • Gold Boarder
  • Posts: 221
  • Thank you received: 24
  • Karma: 7
I have encountered an odd problem: the scraper does (appears to) not display all of the issues that are actually within the comicvine database.

Examples are:

Cobb: Off the Leash (www.comicvine.com/cobb-off-the-leash/49-23493/) the scraper thinks that there is only issues 1 and 3 but not 2

and

Angel: Long Night's Journey (www.comicvine.com/angel-long-nights-journey/49-36748/) the scraper thinks that there ins only issues 1 to 4 but not 4.

Any idea of why that would be the case?

Thanks,

P.S. Screen-shot of what I get when I try to scrape Cobb Issue 2: img214.imageshack.us/img214/1925/cobb.png
Last Edit: 7 years 2 hours ago by rmagere.
The topic has been locked.

Re: Comic Vine Scraper 1.0.28 - 1.0.32 6 years 11 months ago #11039

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1327
  • Thank you received: 508
  • Karma: 182
@rmajere:

Is there any chance that you are using an older version of the scraper (version 1.0.31 or earlier)? The bug you are reporting was a known problem that was fixed for 1.0.32.

If you ARE using 1.0.32, then there is also another bug that I've fixed for 1.0.33 (which is not available yet) that might be causing the problem. In that case, you can either wait for 1.0.33, or if you are impatient, you might fix the problem by deleting your "localCache" directory:

C:\Users\<YOUR USER NAME>\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\localCache\
The topic has been locked.
Time to create page: 0.218 seconds

Who's Online

We have 296 guests and 5 members online