Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper 1.0.44-47

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16035

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Hey guys, I've released version 1.0.46 of Comic Vine Scraper, which adds one very minor bugfix. Since I haven't had any other complaints about 1.0.45, I consider 1.0.46 to be a stable release.

So if you've been waiting for the newest stable release of Comic Vine Scraper, you need wait no longer. Go download it now! :)
The topic has been locked.
The following user(s) said Thank You: 600WPMPO, kenjio

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16037

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1112
  • Thank you received: 64
  • Karma: 51
Thanks, Cory!
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16054

  • sticorp
  • sticorp's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 4
  • Karma: 0
Hi! Thanks for the amazing piece of software! I have a suggestion!

When we do a search for a serie, let's say 'x-men', it will return something like 300 series. If you click in a serie, it will show you THE FIRST issue of that serie on the right side, right? Then, you have to click "Show Issues" to see if that particular issue belongs to that serie/volume. My suggestion is this:

Let's say we are scraping 'X-men #46'. When we do a search for 'x-men', hundreds of possible series appear. When we click in a serie, I think CVS should try to load the picture of issue #46, so we don't have to keep clicking in 'Show Issues' to confirm if it is the right series. If there is no issue #46, just load the first issue with a message "couldn't find issue #46' so we know this is not the right serie/volume. I also think that the aditional covers browsing could be outside the 'Show Issues' if that functionality is implemented!

Well thanks!
Last Edit: 6 years 2 months ago by sticorp.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16060

  • RevQuixo
  • RevQuixo's Avatar
  • Offline
  • Gold Boarder
  • Posts: 279
  • Thank you received: 25
  • Karma: 12
Already suggested ages ago..unfortunately ComicVine's API makes it almost impossible to implement:

code.google.com/p/comic-vine-scraper/issues/detail?id=56
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16061

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Yup, it's one of the biggest flaws in the ComicVine API--I can't access a single comic using its name and issue # unless I download the entire series.

If and when they fix that, a feature like this will be one of the first things I look at.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16065

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
I have a query that has been on my mind for quite some time:

Is ComicVine's "Date Published" field accurate?

Example: Flashpoint #4 was published on August 3, 2011. When I scrape the book, ComicVine puts the date published as 10/2011. WTF?
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16067

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
Well, the date that is currently entered in ComicVine for that comic is 10/2011, so of course that's why the scraper scrapes that date...but I'm not sure if that error is just an occasional kind of thing, or if there are lots of incorrect published dates in the database.

You'd think the people who take the time to enter this data would try to get it right...?
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16068

  • Samael69
  • Samael69's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 381
  • Thank you received: 47
  • Karma: 21
600WPMPO wrote:
I have a query that has been on my mind for quite some time:

Is ComicVine's "Date Published" field accurate?

Example: Flashpoint #4 was published on August 3, 2011. When I scrape the book, ComicVine puts the date published as 10/2011. WTF?
Unfortunately, the data is only as good as those entering it.

I did have a minor issue with the scrapper as of late. I had a book, Feeding Ground #5. Some nimrod entered it on ComicVine as 3.95. After several weeks of waiting for someone to fix this, I went and fixed it myself. Then, when I tried to rescrape it, the 3.95 was still showing and not 5. Presumably, it was pulling from the cache rather than directly from ComicVine. Is there a time-out on the cache so that it will refresh after it get's so old? Can we have an option to ignore the local cache if need be? After several days, I simply deleted the entire scraper cache, rescraped the details, and all was well.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16069

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
@Samael69

Hmmm...clearing the scraper's cache shouldn't have any effect on which data gets scraped. Perhaps it just took some extra time for your changes at the ComicVine website to propogate into their public API?

Longer explanation

The scraper's cache does not contain copies of any of the data that gets scraped--all it does is remember the CVDB numbers for groups of comics. So, for example, if you scrape Amazing Spiderman, it will download and cache the CVDB numbers for all issues of that series, that way it can more quickly find any newly added CVDB numbers when you scrape that series next time. (It's terrible that the scraper even needs a cache, but until ComicVine fixes their API, it's the only way to make the scraper run quickly.)

Also, when you rescrape a comic, it does not use the cache at all, since your comic is already tagged with the right CVDB number.
The topic has been locked.

Re: Comic Vine Scraper 1.0.44-46 6 years 2 months ago #16070

  • Samael69
  • Samael69's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 381
  • Thank you received: 47
  • Karma: 21
I never scraped it to begin with since it had the wrong number. Perhaps what I should have done was scrape it anyway since the issue ID would have been correct. It was the "Show All Issues" pane where I was seeing the incorrect number (3.95). Does that one pull from the cache? I just know that after deleting the cache "Show All Issues" displayed properly. Perhaps it was just an aberration. It came to my attention because issue 5 started showing up on my missing issue list after being approved by the CV mods.
The topic has been locked.
Time to create page: 0.209 seconds

Who's Online

We have 163 guests and 6 members online