Welcome, Guest
Python Scripts for ComicRack

TOPIC: Duplicates Manager (v Alpha - 0.6)

Duplicates Manager (v Alpha - 0.6) 1 month 1 week ago #48819

  • pueblo
  • pueblo's Avatar
  • Offline
  • Junior Boarder
  • Posts: 26
  • Thank you received: 3
  • Karma: 0
Hi Guys,
Could anyone modify this script so it would first look at CVDB values (comicvine_issue etc.)?
Few posts above krandor mentioned, that he made some rough hack to make it work in similar way.
Did anyone managed to fully integrate this feature?
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 1 month 1 week ago #48821

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 282
  • Thank you received: 31
  • Karma: 5
pueblo wrote:
Hi Guys,
Could anyone modify this script so it would first look at CVDB values (comicvine_issue etc.)?
Few posts above krandor mentioned, that he made some rough hack to make it work in similar way.
Did anyone managed to fully integrate this feature?

Not that I’m aware of. I’m still just using my hacked version which is working good for me. It’s not elegant but it works. I would one Day like to learn more programming to integrate it more nicely but I’m not there right now.
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 1 month 1 week ago #48822

  • pueblo
  • pueblo's Avatar
  • Offline
  • Junior Boarder
  • Posts: 26
  • Thank you received: 3
  • Karma: 0
krandor wrote:
pueblo wrote:
Hi Guys,
Could anyone modify this script so it would first look at CVDB values (comicvine_issue etc.)?
Few posts above krandor mentioned, that he made some rough hack to make it work in similar way.
Did anyone managed to fully integrate this feature?

Not that I’m aware of. I’m still just using my hacked version which is working good for me. It’s not elegant but it works. I would one Day like to learn more programming to integrate it more nicely but I’m not there right now.

Cold you please send me your hacked version?
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 1 month 1 week ago #48823

  • perezmu
  • perezmu's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1126
  • Thank you received: 64
  • Karma: 51
Actually, it would be great if you could share it here!
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 1 month 1 week ago #48824

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 282
  • Thank you received: 31
  • Karma: 5
perezmu wrote:
Actually, it would be great if you could share it here!

I'll do that tonight.

The one caveat on it is that you only want to run it against files that have CVDB values in it because it has no fallback mechanic in it so I'd expect bad results if run against non-CVDB files.. For myself, I have a smartlist of all files with CVDB values and run it against that. Like I said, it's a rough hack but it works.

IIRC (been a while since I did it), what i did was take the CVDB value and threw it into the series field in the script so each different CVDB became in effect a separate "series" but that did what I wanted and to make sure different items that may have the same name and volume year are treated differently. I had run into issues with some GNs for a while but Secret Wars was really what made me do this - too many series getting relaunched with same name in the same year that looked the same to DM.

Anyway, I'll post it tonight and hopefully some of you may find it useful. .
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 1 month 1 week ago #48832

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 282
  • Thank you received: 31
  • Karma: 5
Here is the file I modified. Just overwrite the one in the folder and you should be good.

NOTE : DO NOT RUN ON FILES THAT DO NOT HAVE CVDB VALUES! Results could be unpredictable and probably bad.

At some point I'd like to try to make a cleaner version of this but for now it does the job I need and if it helps others then more the better.

NOTE 2 : It won't let me attach a .py file so here it is saved as a .txt. Rename to .py and overwrite the .py in the duplicate manager script folder.
Attachments:
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 1 month 3 days ago #48866

  • pueblo
  • pueblo's Avatar
  • Offline
  • Junior Boarder
  • Posts: 26
  • Thank you received: 3
  • Karma: 0
I've managed to tweak this script a little bit further. I've used krandor's hack as inspiration ;)
Now it works for scrapped and not scrapped comics. I also added support for Volume End.
I know it's ugly, but it works ;p

How it works:
Scrapped files are always compared by CVDB_ID (value from in Notes/Tag).
If 'CVDBSKIP' is present in Notes, file is not compared by CVDB_ID, but as usual.

Not scrapped files (CVDB_ID is missing), are compared as usual.

I've added support for Volume End. You need to have this value in custom field named: VolumeEnd
So, if file is not scrapped, comparision is done by: Series+Number+Volume+VolumeEnd+Format.

To install, you need to change extension from 'txt' to 'py' and replace original file (make a backup first!).
Attachments:
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 1 month 3 days ago #48868

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 282
  • Thank you received: 31
  • Karma: 5
pueblo wrote:
I've managed to tweak this script a little bit further. I've used krandor's hack as inspiration ;)
Now it works for scrapped and not scrapped comics. I also added support for Volume End.
I know it's ugly, but it works ;p

How it works:
Scrapped files are always compared by CVDB_ID (value from in Notes/Tag).
If 'CVDBSKIP' is present in Notes, file is not compared by CVDB_ID, but as usual.

Not scrapped files (CVDB_ID is missing), are compared as usual.

I've added support for Volume End. You need to have this value in custom field named: VolumeEnd
So, if file is not scrapped, comparision is done by: Series+Number+Volume+VolumeEnd+Format.

To install, you need to change extension from 'txt' to 'py' and replace original file (make a backup first!).

Awesome. Great work.
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 1 week 2 days ago #49157

  • krandor
  • krandor's Avatar
  • Offline
  • Gold Boarder
  • Posts: 282
  • Thank you received: 31
  • Karma: 5
Having issue with this script throwing exceptions. Below are screenshots of before and after on one that threw exceptions and the log file as well.

https://imgur.com/a/EHGH1
https://imgur.com/a/bi1ui

And the logs.....


COMICRACK DUPLICATES MANAGER V 0.9

Parsing 5 ecomics


============= Beginning options reading ==================================

Successfully read the following options:

Line 6:
Line 7:
Line 8:



============= Beginning rules reading ==================================

Successfully read the following rules:

Line 10:
Line 11:
Line 14:
Line 15:
Line 16:
Line 17:
Line 18:



============= Beginning options parsing ==================================

Using the following options:

MOVEFILES = TRUE
UPDATEINFO = FALSE
VERBOSE = FALSE
DEBUG = FALSE
COVERPAGES = 4
SIZEMARGIN = 0
C2C_NOADS_GAP = 5
REMOVEFROMLIB = TRUE


============= Beginning rules parsing ==================================


============= End of rules parsing ======================================



============= Begining dupes identification ==================================

Parsing 5 ecomics
Found 1 different series
Found 1 different series with more than one issue
Found 1 different series with dupes
Found 1 groups of dupes, with a total of 5 ecomics.

============= End of dupes identification=====================================



============= Beginning dupes processing =====================================


= PROCESSING GROUP_____
= 652604 #22

_________________ pagecount remove fileless _________________
The administrator has disabled public write access.

Duplicates Manager (v Alpha - 0.6) 6 days 12 hours ago #49178

  • boshuda
  • boshuda's Avatar
  • Offline
  • Gold Boarder
  • Posts: 318
  • Thank you received: 76
  • Karma: 9
krandor wrote:
Having issue with this script throwing exceptions. Below are screenshots of before and after on one that threw exceptions and the log file as well.

https://imgur.com/a/EHGH1
https://imgur.com/a/bi1ui

And the logs.....


COMICRACK DUPLICATES MANAGER V 0.9

Parsing 5 ecomics


============= Beginning options reading ==================================

Successfully read the following options:

Line 6:
Line 7:
Line 8:



============= Beginning rules reading ==================================

Successfully read the following rules:

Line 10:
Line 11:
Line 14:
Line 15:
Line 16:
Line 17:
Line 18:



============= Beginning options parsing ==================================

Using the following options:

MOVEFILES = TRUE
UPDATEINFO = FALSE
VERBOSE = FALSE
DEBUG = FALSE
COVERPAGES = 4
SIZEMARGIN = 0
C2C_NOADS_GAP = 5
REMOVEFROMLIB = TRUE


============= Beginning rules parsing ==================================


============= End of rules parsing ======================================



============= Begining dupes identification ==================================

Parsing 5 ecomics
Found 1 different series
Found 1 different series with more than one issue
Found 1 different series with dupes
Found 1 groups of dupes, with a total of 5 ecomics.

============= End of dupes identification=====================================



============= Beginning dupes processing =====================================


= PROCESSING GROUP_____
= 652604 #22

_________________ pagecount remove fileless _________________

Start ComicRack with -ssc
-ssc is Show Script Console
With this you get a look at all your script output (good for debugging messages).

Right-click on your shortcut and select Properties. Add the -ssc to the very end of the Target box:
"C:\Program Files\ComicRack\ComicRack.exe" -ssc

look for something like:
file.write() argument must be string or read-only character buffer, not list
in the Script Output window.

If you're seeing that it's POSSIBLY in processfunctions.py line 225
for comic in fileless_nothumb:
        logfile.write('removing... '+ comic[SERIES]+' #' + comic[NUMBER] + ' (fileless + no cover)\n')  
    for comic in fileless_thumb:
        logfile.write('removing... '+ comic[SERIES]+' #' + comic[NUMBER] + ' (fileless + custom cover)\n')    
    for comic in to_keep:
        logfile.write('keeping... '+ comic[FILENAME]+' (pages '+str(comic[PAGECOUNT])+')\n')

Stop reading here, try out the script output method, and copy-paste the output from the script output window. I wouldn't be surprised if you've just never hit one of those lines that has comic[NUMBER] in it.

My version is pretty heavily modified, so I can only really give you pointers. IIRC, If you modify the code a bit to liberally add logfile.flush() lines after all of the logfile.write() calls you get a clearer picture of what's going on. If the program encounters an exception before the next flush(), you'll get an inaccurate idea of where the script actually was. Similarly make sure you have debug enabled in the dmrules.dat file (
@ MOVEFILES True
@ REMOVEFROMLIB True
@ C2C_NOADS_GAP 120
@ debug True
@ UPDATEINFO True)

Also remember you can attach to it using Visual Studio and step through the script, evaluating values and other cool things using the steps outlined at http://comicrack.cyolito.com/dokuwiki/doku.php?id=developer:debugging. You could probably do it a little more lightweight with the also freely available windbg, but I haven't tried so I'm not positive. It works with the latest free Visual Studio, though.

I can't ask you to send me those files and that version of duplicate manager to examine myself, because the files can't be shared. Or I'd take a look. It's probably getting lost in here somewhere:
t_group = globals()[t_rule[0]](*t_rule[1:])     ### this is the trick to call a function using a string with its name
in duplicatesmanager.py

Which will probably call
def remove_pagecount_fileless(options, cr, dgroup, logfile):
in processfunctions.py. However, there's virtually zero logging in there.
The administrator has disabled public write access.
Time to create page: 0.364 seconds

Who's Online

We have 249 guests and 4 members online