Welcome, Guest
Python Scripts for ComicRack

TOPIC: Comic Vine Scraper Patch [Not Official]

Comic Vine Scraper Patch [Not Official] 8 months 1 week ago #49080

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 596
  • Thank you received: 150
  • Karma: 30
Now try this one:

File Attachment:

File Name: cvdb-2-3-4-5-6.zip
File Size:9 KB


I added improvements for unicode characters

(for example now é is read in series and in search as e, so presente is the same as présente, according as how the API search works)
The administrator has disabled public write access.

Comic Vine Scraper Patch [Not Official] 8 months 1 week ago #49081

  • beardyandy
  • beardyandy's Avatar
  • Offline
  • Senior Boarder
  • Posts: 48
  • Thank you received: 5
  • Karma: 0
Sorry bud

'Scopestorage' object has no attribute 'cvs_scape'
It's a windows dialogue box and I can get you a screenshot later but a bit hard at the moment.

Just before that was a warning about a missing unicode module

EDIT:
cvdb1 is first error. It then exits the scraper code

on running again, without exiting comicrack - cvdb2
Attachments:
Last Edit: 8 months 1 week ago by beardyandy. Reason: getting screenshots
The administrator has disabled public write access.

Comic Vine Scraper Patch [Not Official] 8 months 1 week ago #49082

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 596
  • Thank you received: 150
  • Karma: 30
#$%#$% I thought the module was included in ComicRack XD
The administrator has disabled public write access.

Comic Vine Scraper Patch [Not Official] 8 months 1 week ago #49083

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 596
  • Thank you received: 150
  • Karma: 30
Try this one:

File Attachment:

File Name: cvdb-2-3-4-5-6-7.zip
File Size:9 KB


It is FAR from a perfect solution (it only recognises and correct 9 symbols, and ignore the rest), but at least it will work without python libraries...

The corrections are:

á - > a
é - > e
í - > i
ó - > o
ú ->u
ñ -> n
ö -> o
ä -> a
è -> e

(instead of the hundreads the library did Other non alfanumeric symbols are just deleted from the words)

I HATE problems with CR and Python libraries... U_U
Last Edit: 8 months 1 week ago by Xelloss.
The administrator has disabled public write access.

Comic Vine Scraper Patch [Not Official] 8 months 1 week ago #49088

  • beardyandy
  • beardyandy's Avatar
  • Offline
  • Senior Boarder
  • Posts: 48
  • Thank you received: 5
  • Karma: 0
Well done boss.

I've just been looking through the code

media.giphy.com/media/wSCAy1zJbcUG4/giphy.gif
The administrator has disabled public write access.
The following user(s) said Thank You: Xelloss

Comic Vine Scraper Patch [Not Official] 8 months 1 week ago #49089

  • beardyandy
  • beardyandy's Avatar
  • Offline
  • Senior Boarder
  • Posts: 48
  • Thank you received: 5
  • Karma: 0
Probably a silly question but as I'm trying to understand this a little - there aren't any help are they?

comicrack.cyolito.com/forum/10-bugs/5271...ies-missing-from-115

ironpython.net/documentation/dotnet/

msdn.microsoft.com/en-us/library/kdcak6ye(v=vs.110).aspx
The administrator has disabled public write access.

Comic Vine Scraper Patch [Not Official] 8 months 1 week ago #49090

  • EricS1980
  • EricS1980's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 10
  • Thank you received: 4
  • Karma: 0
I've come across a few that are returning no results with the scraper.

The series I'm unable to scrape are:
Coyotes
Reactor
VS

All three are giving me 'Search Failed. Couldn't find any comic books that match...' with the scraper using your latest version (cvdb-2-3-4-5-6-7-8). I reverted back to my original cvdb.py file, and Coyotes and Reactor were both found, but VS still couldn't be found.

I tried running all 3 searches through the api in my browser and they all came back with results (just 1 each for Coyotes and Reactor, but quite a few for VS):
comicvine.gamespot.com/api/search/?api_k...tes&resources=volume
comicvine.gamespot.com/api/search/?api_k...tor&resources=volume
comicvine.gamespot.com/api/search/?api_k...=VS&resources=volume

I was able to locate all 3 searching ComicVine directly, and was able to scrape all of them by manually entering the comcivine_volume ID into ComicRack and rescraping.

Hope this info is helpful. Let me know if you need anything else.

Thanks.
The administrator has disabled public write access.
The following user(s) said Thank You: Xelloss

Comic Vine Scraper Patch [Not Official] 8 months 1 week ago #49092

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 596
  • Thank you received: 150
  • Karma: 30
Will try to see what is happenning with those cases!

Thanks!!
The administrator has disabled public write access.

Comic Vine Scraper Patch [Not Official] 8 months 6 days ago #49093

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 596
  • Thank you received: 150
  • Karma: 30
Ok, many changes:

- I reorder all the code of the added lines to make it clearer to read and improve performance (nobody will notice this, but the second it took to filter things now is in the order of miliseconds becuase the previous code was a disaster in performance). This I think will make no difference for most users, but the previous code ashamed me XD

- The patch didn't work for queries that returned one result only, not it does (thanks EricS1980 for the bug report)

- Because of how the scraper filtered words, searches as "vs" or "is" didn't work before to improve matches, as the new search can ignore conflictive words if needed, I change how this words are filtered for these searches to work (thanks EricS1980 for the bug report)

- A lot of minor changes nobody will notice unless they read the code :P

If new bugs were added for all this changes, please tell me and I will see them :)
Last Edit: 8 months 6 days ago by Xelloss.
The administrator has disabled public write access.
The following user(s) said Thank You: EricS1980

Comic Vine Scraper Patch [Not Official] 8 months 6 days ago #49094

  • boshuda
  • boshuda's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 335
  • Thank you received: 86
  • Karma: 10
Ignore this. Xellos addressed it and more.
Last Edit: 8 months 6 days ago by boshuda.
The administrator has disabled public write access.
Time to create page: 1.194 seconds

Who's Online

We have 140 guests and 2 members online