Welcome, Guest
Python Scripts for ComicRack

TOPIC: Duplicates Manager (v Alpha - 0.6)

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49196

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 580
  • Thank you received: 144
  • Karma: 29
I have never used this script, so I don't know exactly what it does... But from what I have read here, it is a duplicate identifier and manager (copy data, delte files, etc)

I manage those things in my collection with two script I made, perhaps they can be of use to you...

1- The SameXDifferentY script, that let me do smartlist that identified duplicates (for example, same comicvine_comic field)

2- The Autocomplete script, that if it finds comics with the same comicvine_comic in your library, it just copy the data missing from one into the other (and viceversa)

The deleting of duplicates I do manually, as I want to look at them very carefully to decide which one to delete and which one to keep (Although I use smartlist for this also, for example to find duplicate fileless comics)

About the exceptions, as boshuda said, enable the console and send the python error message here, that is the easiest way to debug an exception...

I would do it myself, but it is imposible for me to duplicate the exact conditions you are working with...
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49197

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1128
  • Thank you received: 64
  • Karma: 51
Xelloss,

In this case, I would really advise you to have a look at the dupes manager.

What it does is to remove dupes based on a set of rules you decide: for each group of duplicate comics found, it will remove/keep comics based on the rules as you define: noads/c2c, scanner, pages... have a look at the wiki and examples...
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49198

  • boshuda
  • boshuda's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 334
  • Thank you received: 85
  • Karma: 10
perezmu wrote:
Xelloss,

In this case, I would really advise you to have a look at the dupes manager.

What it does is to remove dupes based on a set of rules you decide: for each group of duplicate comics found, it will remove/keep comics based on the rules as you define: noads/c2c, scanner, pages... have a look at the wiki and examples...

He's right. It's nice. Once you get it tweaked, it's pretty good. And while tweaking it, you can set it up to just tell you what it's going to do, but not actually do it. And after that you configure it to put the files into a dupes directory. I use it first on duplicates after doing a quick visual scan to verify the covers match up (to minimize mis-matches and to merge files to get all the covers). Then I use it on a list created from your SameXDifferentY script. I'm hoping the Krandor modificaton to check against the cvdb code makes this even smoother. If the Format is different, for example, Dupes Manager won't see them as dupes (neither will ComicRack, though, and that's where you SameXDiffY comes in handy).
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49199

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 580
  • Thank you received: 144
  • Karma: 29
I was doing just that (and reading the code), good piece of software!

I don't find much use of it for me because I don't deal with many duplicates (because how I deal with comics in CR) and the few I have I like to examine them myself (to see resolution for example), and things I haven't thought about till seeing them... (another would be the case if I had to manage with hundreds of duplicates but that is not the case in my case XD)
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49200

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 580
  • Thank you received: 144
  • Karma: 29
boshuda wrote:
perezmu wrote:
Xelloss,

In this case, I would really advise you to have a look at the dupes manager.

What it does is to remove dupes based on a set of rules you decide: for each group of duplicate comics found, it will remove/keep comics based on the rules as you define: noads/c2c, scanner, pages... have a look at the wiki and examples...

He's right. It's nice. Once you get it tweaked, it's pretty good. And while tweaking it, you can set it up to just tell you what it's going to do, but not actually do it. And after that you configure it to put the files into a dupes directory. I use it first on duplicates after doing a quick visual scan to verify the covers match up (to minimize mis-matches and to merge files to get all the covers). Then I use it on a list created from your SameXDifferentY script. I'm hoping the Krandor modificaton to check against the cvdb code makes this even smoother. If the Format is different, for example, Dupes Manager won't see them as dupes (neither will ComicRack, though, and that's where you SameXDiffY comes in handy).

It does cover match up?! O_O

THAT I am interested in... What do you mean by match up? Does it do the merge covers automatically? I can't find much information about that in what I have read about the scirpt so far

I am interested in the cover match up as I wanted to add that to many of my scripts and for future ideas
Last Edit: 4 months 1 week ago by Xelloss.
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49201

  • boshuda
  • boshuda's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 334
  • Thank you received: 85
  • Karma: 10
Xelloss wrote:
I was doing just that (and reading the code), good piece of software!

I don't find much use of it for me because I don't deal with many duplicates (because how I deal with comics in CR) and the few I have I like to examine them myself (to see resolution for example), and things I haven't thought about till seeing them... (another would be the case if I had to manage with hundreds of duplicates but that is not the case in my case XD)
Fair point. I often have hundreds of dupes, where I try to keep just the highest quality version. I'm beginning to somewhat regret that as the older paper scans are being replaced by modern digital versions w/o the little data box inside (I forget the name now, the one with the subscription info that ComicVine wants to use as the holy grail of information). But, I still have the original DC & Marvel chronological backups which are almost all taken from real paper scans.Xelloss wrote:
boshuda wrote:
perezmu wrote:
Xelloss,

In this case, I would really advise you to have a look at the dupes manager.

What it does is to remove dupes based on a set of rules you decide: for each group of duplicate comics found, it will remove/keep comics based on the rules as you define: noads/c2c, scanner, pages... have a look at the wiki and examples...

He's right. It's nice. Once you get it tweaked, it's pretty good. And while tweaking it, you can set it up to just tell you what it's going to do, but not actually do it. And after that you configure it to put the files into a dupes directory. I use it first on duplicates after doing a quick visual scan to verify the covers match up (to minimize mis-matches and to merge files to get all the covers). Then I use it on a list created from your SameXDifferentY script. I'm hoping the Krandor modificaton to check against the cvdb code makes this even smoother. If the Format is different, for example, Dupes Manager won't see them as dupes (neither will ComicRack, though, and that's where you SameXDiffY comes in handy).

It does cover match up?! O_O

THAT I am interested in... What do you mean by match up? Does it do the merge covers automatically? I can't find much information about that in what I have read about the scirpt so far

I am interested in the cover match up as I wanted to add that to many of my scripts and for future ideas
No, it doesn't do cover match ups. I mean I put the thumbnails side by side, then do a visual scan as I scroll down. It's one last check against an incorrect scrape. It also sometimes shows additional covers. That is still a manual process to merge the covers. It takes almost zero time to do and I find it kind of relaxing.

To merge the covers into one book I pick the lowest quality copy of the book and set all of the pages except the cover(s) I want to keep to "delete". Then I select both books and do a Right-Click Export->Combine Books. It will create a single copy of that book without the pages marked as delete.
Warning: Spoiler! [ Click to expand ]
I then go in and manually reorder the pages if necessary and do a fresh export to CBZ. I think Export->Combine Books is provided by CR, but it might be one of the custom exports that I made. I can't remember anymore.

It's a bit of a pain but it happens infrequently enough that I don't really care. And it's kind of like flipping through a random book that I might otherwise not get a good look at. Like going to your local comic book shop when they existed and flipping through that Wednesday's shipment. Each book would need to be manually looked at anyway to decide which copy of a particular cover I want to keep, and to see if there are additional items I want (sketch pages, back covers, pin-ups, etc).

I applaud your thought of creating a script to pull the multiple covers together, and you've done some crazy stuff so I'm curious to see what you come up with. I've considered doing it too, but I get easily distracted and I specifically tend to get lost in the weeds of edge cases :).

For example, you would need to either mark the covers manually, or have some pretty good recognition software to determine a) Which are the covers (and ignore those splash pages that look like covers, are 'internal' covers, etc) and b) Determine which cover is different/similar enough to actually be the cover(s) you want to keep, and which covers need to be discared. That gets even more complicated on the barely different covers where they might just change an article of clothing, or a color, or even just the UPC box. Assuming you just require the user to set which covers they want to keep, I couldn't seem to find a way to hook into the File Export routines. So you would need to figure out how to access that hook (assuming it exists) or create the cbz files from the various images and the metadata. Including putting the comic info into the new file, tell ComicRack about your new file, and tell it to no longer use the existing file. Which would require quite a bit of filesystem work, and maybe some weird hacking into the supplied 7zip library. It's certainly doable, but it has quite a few roadblocks. I would almost recommend a completely separate project that takes two comics as input to create the new single comic. Then write a separate CR plugin to pass the comics to that external program.

I suppose at a minimum the CRC hashing (or whatever) that cbanack did in Comic Vine Scraper to match covers could be pulled in to help remove some of the manual scanning of covers, but I really just now thought of that. I can be dense sometimes, lol.
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49202

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 580
  • Thank you received: 144
  • Karma: 29
hahahaha!

I don't have many dupes as I don't usually add many comics without knowing what I am adding... (this only happen in 0 week cases, which are not dupes, or with specific collections, that I specifically add to choose the best version of each comic manually). Also, as much as I have played with page resolution (my main criteria for choosing one comic over other, or updating comic versions) it is a bit buggy how the resolution fields work in CR (you have to preview the page first for it to load and I have had some problems with ads and so that have different resolution. I still have to do some work with this though). So manual is my only option, and I really enjoy deleting worse version of comics and seeing which ones are XD (as I mentioned... not very common in my collection unless I am specifically updated the comics with better versions)

I DID have issues FINDING dupes before... and I would have used this script for that. But I have managed to solve that before knowing about this script U_U

About merging comics, I DO EXACTLY THE SAME (using the CR export option). And have exactly the same issues... Your answer is like scraped from my mind XD

About doing it automatically... I have thought about all the issues you are addressing and they are the reason I have never made a script for that.... In my case the recognition of covers is not a problem, as part of my system includes recognising covers and ads (manually) and then deleting the ads... However recognising "same" covers of different scanners would be a hashing nightmare I am not messing with :P (at least for now). Also I would have to work with ways to remake comic files from scripts (export option in CR) which I don't really know how to do.. (working with rar and zip libraries in CR is always a nightmare btw, been there with the scanner script coding)
Last Edit: 4 months 1 week ago by Xelloss.
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49203

  • WraithTDK
  • WraithTDK's Avatar
  • Offline
  • Senior Boarder
  • Posts: 41
  • Thank you received: 1
  • Karma: 0
Wow. Didn't expect to see notifications from this thread again. Never could get this plugin to run.
I am currently reading every Marvel Superhero comic book every printed, in chronological order, and blogging about the milestones, footnotes, and other interesting moments I read at http://www.wraithscomicjourney. I'll be adding DC when I hit 1985, and other companies when they launch.
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49209

  • boshuda
  • boshuda's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 334
  • Thank you received: 85
  • Karma: 10
WraithTDK wrote:
Wow. Didn't expect to see notifications from this thread again. Never could get this plugin to run.
No? If you pull in the Hitlist's or similar it's a life-saver.
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 4 months 1 week ago #49215

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 580
  • Thank you received: 144
  • Karma: 29
Ok, I have finished understanfding how this script works and it is quite interesting... I will play with it a bit and see if I can give it any use :)

Btw, it has given me some new ideas for smartlist scripts :3

In some way it works with the same idea as my FirstXinEachY smartlist but in a much more specific and advance way
Last Edit: 4 months 1 week ago by Xelloss.
The administrator has disabled public write access.
Time to create page: 0.392 seconds

Who's Online

We have 213 guests and 2 members online