Welcome, Guest
Python Scripts for ComicRack

TOPIC: ComicVineScraper - an ImportComicVineInfo clone

ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5238

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
Hi guys,

A while back, I grabbed perezmu's fine ImportComicVineInfo script, and began reading through it as a way to teach myself python. In the process, (because I write code for a living, so I can't help myself) I began reorganizing things and fixing a few problems. Ultimately I decided that it would be fun to make my own version of this script.

At this point, I haven't done anything amazing with it. I've fixed a few bugs, made it so that it has full unicode support, gave it a little bit better image handling, and fixed it so that it runs properly on ComicRack 0.9.115 (i.e. using IronPython 2.6).

It also uses the nifty new script package installer that cYo added in ComicRack 0.9.115.

I have some other plans for this script, but for the near term I won't be adding any big features--just focusing on bug fixes and improving the stability, performance and appearance of the existing script.

If you're interested in big new features, a quick browse through the forums here suggests the perezmu might be working on something more dramatic. If my version of this script tides people over until he's done, I'll be happy. :)
**************
Since this is really a fork of perezmu's code, and because I'm interested in encouraging users to use either one or both scripts, I've taken some pains to make it easy to distinguish between my script and perezmu's. To that end, I've:

1) renamed the script (me: "Comic Vine Scraper", him: "Import Comic Vine Info")
2) changed the wording of all UI text slightly (me: "scrape", him "import")
3) changed the script's icon slightly (me: green icon, him: red icon)

There should be no problem if you want to have both scripts installed side-by-side, so you can run one, and if it doesn't work, you can try the other.
****************

To get the latest version of ComicVineScraper, or to report bugs or problems with it, please visit the google code project page.

***************
Feel free to use this thread to post any comments, questions, or problems that you're having with ComicVineScraper. I'm planning to put in a couple hours a week doing bug fixes and improvements, so fire away. B)
Attachments:
Last Edit: 7 years 9 months ago by cbanack.
The administrator has disabled public write access.

Re: ComicVineScraper-a clone of ImportComicVineInfo 7 years 9 months ago #5240

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
Ok.. doesn't solve any problems that I had with the ImportComicVineInfo script.

It simply says in the end "scrape complete" and doesn't update the comic.

Thanks anyway for trying out a solution. Please don't give up.. thanks
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The administrator has disabled public write access.

Re:ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5243

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1114
  • Thank you received: 64
  • Karma: 51
cbanack wrote:
Feel free to use this thread to post any comments, questions, or problems that you're having with Comic Vine Scraper. I'm planning to put in a couple hours a week doing bug fixes and improvements, so fire away. B)

Hi man!

First of all, thanks a lot for this! I appreciate your go on this script... I keep wanting to start with the newest version with all the features suggested in the corresponding thread, but I still seem to be unable to find the time, so please, update yours with whatever you think is useful from that thread :P

Now, a *bug* :blush:

Trying to import a comic that has an issue number not listed in comicvine crashes. Debug log:
C:\Documents and Settings\apm\Datos de programa/cYo/ComicRack/Scripts/ComicVineScraper/localCache/28062.dat

__Debug__:__book__:__fillIssuesList__: getting issues info from local cache

__Debug__:__book__:__fillIssuesList__: data retrieved : '192531', '5'], ['186913', '4'], ['181391', '3'], ['175141', '2'], ['172310', '1'

__Debug__:__book__:__fillIssuesList__: data ids :

__Debug__:__book__:__fillIssuesList__: getting issues info from internet

__Debug__:__book__:__fillIssuesList__: Issues in this series URL: api.comicvine.com/volume/28062/?api_key=...ml&field_list=issues

__Debug__:__book__:__fillIssuesList__: Exiting

__Debug__:__book__:__fillIssuesList__: Return Value '192531', '5'], ['186913', '4'], ['181391', '3'], ['175141', '2'], ['172310', '1'

__Debug__:__book__: Calling 'searchForIssue'

__Debug__:__book__:__searchForIssue__

__Debug__:__book__:__searchForIssue__: Issue is not an Integer: 1.5

__Debug__:__book__:__searchForIssue__: Searching for a numerical match for: 5

__Debug__:__book__:__searchForIssue__: Searching for a numerical match for: 4

__Debug__:__book__:__searchForIssue__: Searching for a numerical match for: 3

__Debug__:__book__:__searchForIssue__: Searching for a numerical match for: 2

__Debug__:__book__:__searchForIssue__: Searching for a numerical match for: 1

__Debug__:__book__:__searchForIssue__: Launching selectForm for user selection

__Debug__:__book__:__searchForIssue__:__SelectIssueForm__: Launching Select Issue Form

[Errno 2] C:\Documents and Settings\apm\Datos de programa\cYo\ComicRack\Scripts\Comic Vine Scraper\ComicVincomicvinelogo.png

Well this is an easy one, I just renamed comicvinelogo.png and it works just fine!

Also, a question... have you modified the local cache files structure (I probably would :P) or are they still compatible with mine so I can just copy them to your directory!

Thanks again!
The administrator has disabled public write access.

Re:ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5245

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
ok.. almost everything's working fine again..!!

I uninstalled comicrack to the core & wiped out every registry/database setting related to it, and rebooted.

Then installed cr v0.9.115. As sugessted by cYo, I entered the last value in comicrack.ini as "false". Installed both ComicVineScraper & ImportComicVineInfo. Now, both are working fine..

However, many-a-times, they stil tend to hang midway, crashing the program.

Thanks to both perezmu & cbanack for their efforts..!!
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The administrator has disabled public write access.

Re:ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5253

  • cYo
  • cYo's Avatar
  • Offline
  • Moderator
  • Posts: 3476
  • Thank you received: 676
  • Karma: 181
Tried it with some issues and it worked clean and fast :)

About frame settings: I have both on and it works. My earlier comment was just meant as stuff you can try if there are problems. Actually I do not know about a reason why it should be influenced by the frame settings.
The administrator has disabled public write access.

Re:ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5255

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
Another bug ?

When the "display series covers thumbs" option is unchecked, the dialog box closes without updating the comic. When I enable this option, everything works fine..
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
Last Edit: 7 years 9 months ago by 600WPMPO.
The administrator has disabled public write access.

Re:ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5257

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
perezmu wrote:
Well this is an easy one, I just renamed comicvinelogo.png and it works just fine!

Oops! I'll fix that up for everyone tonight. Thanks for finding it.
Also, a question... have you modified the local cache files structure (I probably would :P) or are they still compatible with mine so I can just copy them to your directory!

No modifications yet, so you should be able to copy them over without any problems.

At some point I might look at a different mechanism for caching, but what's there now works fine. I've got limited time to work on this until summer, so I have to pick and choose what I do. Mostly I'll try to track down bugs for now, I think.
The administrator has disabled public write access.

Re:ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5258

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
600WPMPO wrote:
Then installed cr v0.9.115. As sugessted by cYo, I entered the last value in comicrack.ini as "false". Installed both ComicVineScraper & ImportComicVineInfo. Now, both are working fine..

This probably isn't necessary, as the script runs fine for a number of people (myself included) when frames are turned on. Moreover, I am developing the script with the assumption that frames are turned on, because that gives me access to more python features. So if you turn frames off (i.e. last two values in comcrack.ini set to false), that will eventually cause bugs in the script.
However, many-a-times, they stil tend to hang midway, crashing the program.

Yeah, this is something I've run into too. I think it has more to do with the comicvine website (or our internet connections) getting bogged down. The script needs to handle that situation more gracefully...it's something I plan to work on.

However, if you find an example of a scrape that hangs like this every time (even when you restart comic rack, and even if you wait a few hours or a day between tries), then there's a good chance that you've found different bug altogether!
The administrator has disabled public write access.

Re:ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5261

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
Added an update to fix the (actually rather serious) bug that perezmu found.

The link to it is in the first comment of this post (it's the same link that you got the first version from, but now it gives you version 1.0.1 instead of 1.0.0).

Recommend that everyone upgrade to 1.0.1 before reporting any additional bugs, as the problem that I just fixed could have all kinds of irritating side effects! :)
Last Edit: 7 years 9 months ago by cbanack.
The administrator has disabled public write access.

Re:ComicVineScraper - an ImportComicVineInfo clone 7 years 9 months ago #5266

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1328
  • Thank you received: 508
  • Karma: 182
600WPMPO wrote:
Another bug ?

When the "display series covers thumbs" option is unchecked, the dialog box closes without updating the comic. When I enable this option, everything works fine..

Does this problem still occur for version 1.0.1? I just tried it out, and it seems to be working for me. Also, knowing the nature of the bug I fixed for 1.0.1, I could easily see it being the cause of this issue...
The administrator has disabled public write access.
Time to create page: 0.609 seconds

Who's Online

We have 271 guests and 3 members online