Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper

Comic Vine Scraper 1 month 2 weeks ago #48127

  • Xelloss
  • Xelloss's Avatar
  • Online
  • Platinum Boarder
  • Posts: 372
  • Thank you received: 98
  • Karma: 20
What my script do is search for "List of covers and their creator" and everything that comes after that and delete it, I am working in one that add this info but in a correct way. When it is ready I will release it in this forum :)
The administrator has disabled public write access.

Comic Vine Scraper 1 month 2 weeks ago #48128

  • Xelloss
  • Xelloss's Avatar
  • Online
  • Platinum Boarder
  • Posts: 372
  • Thank you received: 98
  • Karma: 20
If what you want to do is just delete the extra text, use this:

File Attachment:

File Name: DeleteCoverData.zip
File Size:1 KB


It os what I am using now... Remember it will delete all the text from the alternate covers... (you will have to rescrap the books for recovering this in the future)

It also counts the covers and save the result in a custom value called "Covers", before deleting the text.
The administrator has disabled public write access.
The following user(s) said Thank You: romsnesrom, n8thagr8

Comic Vine Scraper 1 month 2 weeks ago #48130

  • n8thagr8
  • n8thagr8's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 11
  • Karma: 0
You're amazing, thank you!
The administrator has disabled public write access.

Comic Vine Scraper 1 month 2 weeks ago #48131

  • Xelloss
  • Xelloss's Avatar
  • Online
  • Platinum Boarder
  • Posts: 372
  • Thank you received: 98
  • Karma: 20
No, thank you and I must apoligise for posting a script not well tested...

Thanks to this topic I revised this script (it was never made to be published in the forum, and I shouldn't have published it without testing it accordingly) and I improve it a lot... Here is the final version with all the fixes... please replace the other one with this.

Modifications:

* It now has a decent performance according with the simple task it makes... it should work with a lot of comics in less than a second...
* It recognises the data to read even if the person who wrote it used Capitals or not... and even if it uses a few different words than the most commons (based in all the books in my library)
* It will now ignore books who were already edited by the script before, and will not overwrite the "Covers" custom value with 1 the second time it runs
* Better number of covers recognition

Again, sorry for publishing a script with such a bad quility...

New version:

File Attachment:

File Name: DeleteCoverData-2.zip
File Size:1 KB
Last Edit: 1 month 2 weeks ago by Xelloss.
The administrator has disabled public write access.
The following user(s) said Thank You: romsnesrom, n8thagr8

Comic Vine Scraper 1 month 4 days ago #48178

  • boshuda
  • boshuda's Avatar
  • Online
  • Gold Boarder
  • Posts: 285
  • Thank you received: 63
  • Karma: 7
cbanack wrote:
Drybonz wrote:
I was wondering if there was a way to have the scraper run in the background, or minimized, so that we can open books, etc while it is working. If not, is this a feature that could be added? Thank you.

Unfortunately, I suspect something like that would be too much work. I don't really think ComicRack is set up to run plugins in the background, and the Scraper certainly doesn't work in the background anyway. Though it is a good idea; if I were writing the whole thing over from scratch, I'd try to make it more thread aware (i.e. more able to run in the background.)

But these days I am only supporting the scraping the the form that it currently exists, not adding new features.

I believe ComicRack explicitly prevents the spawning of new threads from scripts anyway. I haven't confirmed it in any way, but I ran into something long ago that lead me to that conclusion.
The administrator has disabled public write access.

Comic Vine Scraper 1 month 4 days ago #48179

  • boshuda
  • boshuda's Avatar
  • Online
  • Gold Boarder
  • Posts: 285
  • Thank you received: 63
  • Karma: 7
n8thagr8 wrote:
Hey

I noticed lately that the scraper keeps adding this (or some variation) to the plot summary of each book:

"List of covers and their creators:CoverNameCreator(s)Sidebar LocationACover AJamie McKelvie & Matthew Wilson1BCover BMeredith McClaren2CJonathan Hickman Month Variant Cover CJonathan Hickman3"

That's at the end of the plot summary i scraped for Wicked + Divine 30, for another example, this is the full plot summary metadata for Secret Empire #10:

"Can there be any redemption for Captain America as the SECRET EMPIRE starts to crumble?List of covers and their creators:CoverNameCreator(s)Sidebar LocationRegRegular CoverMark Brooks1VarVariant CoverJ. Scott Campbell2VarVillain Variant CoverDan Mora & Edgar Delgado3VarCivil Warrior Variant CoverGabriele Dell'Otto4VarAction Figure Variant CoverJohn Tyler Christopher5VarShield Variant CoverSteve McNiven6VarHydra Heroes Variant CoverAndrea Sorrentino7VarJack Kirby's 100th Birthday Variant CoverJack Kirby, Dick Ayers, Paul Mounts & Joe Frontirre8VarGenerations Variant CoverAlex Ross9"

Anyone know what causes this or how to fix it? It does not do this for every comic I scrape (probably more than half though) and I have no idea what is different about the ones it does and doesn't do this for.

Thanks

I take care of it with the Data Manager, which I always run as part of my normal workflow. To add the rule I use, create a new rule and copy the following into the Text line of your new rule. Then click on the little Update icon to the right of the text box.

<<Summary.ContainsAnyOf:List of covers||CoverNameCreator>> => <<Summary.RegexReplace:List of covers.*||>> <<Summary.RegexReplace:CoverNameCreator.*||>>
The administrator has disabled public write access.
Time to create page: 0.249 seconds

Who's Online

We have 213 guests and 7 members online