Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 8 months 1 week ago #49142

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 596
  • Thank you received: 150
  • Karma: 30
Till I have time to see how the new API works, I recommend using this:

File Attachment:

File Name: cvdb2.2-2.zip
File Size:9 KB


It will help with the "-" problem in most cases
The administrator has disabled public write access.

Comic Vine Scraper 8 months 1 week ago #49143

  • Kiljoy McCoy
  • Kiljoy McCoy's Avatar
  • Offline
  • Expert Boarder
  • Posts: 80
  • Thank you received: 1
  • Karma: 0
I switched back and tried to scrape 4 TPB and switched back. Yours works better still..
The administrator has disabled public write access.

Comic Vine Scraper 8 months 1 week ago #49147

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
beardyandy wrote:
@cbanack

Out of curiousity, I've only just notices that settings for the scraper (e.g. API code) are set outside the normal path
Scripts\Comic Vine Scraper\resources.py:
profile_dir = Environment.GetFolderPath(
   97           Environment.SpecialFolder.ApplicationData) + \
   98:          r"\Comic Vine Scraper"
   99        
  100        # set the standard locations for settings files

Sure you had a good reason for it - can you remember why? The fun of windows permissions?

Reason I'm asking is I use CR in portable mode and it's the one thing that isn't under the comicrack/data folder for me. Not a bit problem but I might move it.

Honestly, I don't remember what I was thinking, though I do remember moving it to a different folder at least twice. I think the idea was to prevent people from losing their settings when they uninstalled/reinstalled ComicRack...? At any rate, the settings folder can really be anywhere, so if you decide to move it, I'm sure it won't cause you major problems.
Last Edit: 8 months 1 week ago by cbanack.
The administrator has disabled public write access.

Comic Vine Scraper 8 months 1 week ago #49148

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
Xelloss wrote:
I suggest everybody using my patch to go back to the original script, as it works much similar to before now and my patch just doesn't make sense anymore

I have been (slowly) working on a new version of the Scraper that stops loading search results once they don't match all your search terms (similar to what Xelloss has done, I believe.) My fix seems to be working much more like it did all this search API change happened, but I still have to do some testing before I package it up and release it to everyone.
The administrator has disabled public write access.
The following user(s) said Thank You: jericko, LoLyfe

Comic Vine Scraper 8 months 1 week ago #49149

  • pweasel
  • pweasel's Avatar
  • Offline
  • Expert Boarder
  • Posts: 127
  • Thank you received: 18
  • Karma: 8
how was it to force fed to the scrapper the URL of an issue on comicvine?
CRW 0.9.178 x64 on Win10
CRA 1.80 on Nexus 10
The administrator has disabled public write access.

Comic Vine Scraper 8 months 1 week ago #49150

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 596
  • Thank you received: 150
  • Karma: 30
cbanack wrote:
Xelloss wrote:
I suggest everybody using my patch to go back to the original script, as it works much similar to before now and my patch just doesn't make sense anymore

I have been (slowly) working on a new version of the Scraper that stops loading search results once they don't match all your search terms (similar to what Xelloss has done, I believe.) My fix seems to be working much more like it did all this search API change happened, but I still have to do some testing before I package it up and release it to everyone.

With the new API update from Yesterday, I don't think it will be needed... the only update I think you will need to do is just erasing the " AND " join in the query and use just spaces as "ands" (and just for cases such as titles with "-")

The API is now working much similar as it used to do before now...
The administrator has disabled public write access.

Comic Vine Scraper 8 months 1 week ago #49151

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
Xelloss wrote:
With the new API update from Yesterday, I don't think it will be needed... the only update I think you will need to do is just erasing the " AND " join in the query and use just spaces as "ands" (and just for cases such as titles with "-")

The API is now working much similar as it used to do before now...

Well, that will certainly make things easier. Two questions:

1) does the " AND " join actually break anything, or is it just unnecessary now?

2) what is wrong with series titles with "-"? is there a bug that I can fix?
The administrator has disabled public write access.

Comic Vine Scraper 8 months 1 week ago #49152

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 596
  • Thank you received: 150
  • Karma: 30
I will answer both questions with the same answer XD

The "and" doesn't work as far as I can see as an AND, it rather is a word that the API ignore... (as the, is, etc)

The problem is that the API has some kind of algorithm so that both A-B and A B returns both A-B and A B

So if you search Spider-Man or Spiderman or even Spider Man, it will return Spider-man Spiderman and Spider Man

HOWEVER if you do a search as "Spider AND Man" (that it is what the script do if you write Spider-Man), it will only return "Spider Man" NOT "Spider-Man"

In other words it brokes anything that has a "-" in the middle of the name

Also "Spider AND Man" will return ONLY "Spider Man" and NOT "Man Spider", in other words, it doesn't work as an AND but rather a word that the API just ignore
Last Edit: 8 months 1 week ago by Xelloss.
The administrator has disabled public write access.

Comic Vine Scraper 8 months 1 week ago #49153

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1367
  • Thank you received: 550
  • Karma: 186
Perfect explanation...that's very helpful!
The administrator has disabled public write access.

Comic Vine Scraper 8 months 1 week ago #49154

  • Gemris
  • Gemris's Avatar
  • Offline
  • Senior Boarder
  • Posts: 42
  • Thank you received: 13
  • Karma: 6
This morning I also tried to do a search for an old issue of All-American Comics, a hyphenated title with the Golden-Age Green Lantern, but got only one result “All American Comics” by Marvel (non-hyphenated), which was the wrong series.

I was only able to find the correct listing by searching the terms “All Comics” and removing the hyphen and second word “American”. That was the only combination that worked, and then “All-American Comics” was at the top of the list.
Last Edit: 8 months 1 week ago by Gemris.
The administrator has disabled public write access.
Time to create page: 0.310 seconds

Who's Online

We have 112 guests and one member online