Welcome, Guest
General discussion about ComicRack

TOPIC: Scanners technical question... and posible forum communitie project

Scanners technical question... and posible forum communitie project 8 months 3 weeks ago #48647

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 582
  • Thank you received: 146
  • Karma: 29
ok, I have some free time today, I will seeit now :)
Last Edit: 8 months 3 weeks ago by Xelloss.
The administrator has disabled public write access.

Scanners technical question... and posible forum communitie project 8 months 3 weeks ago #48648

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 582
  • Thank you received: 146
  • Karma: 29
1) A suggestion - is there any way to add any feedback about progress. (Or where do I look for logging?). I probably need to do a clean install of comicrack but it keeps crashing (when doing other things as well). Or at least when it thinks it's finished. (see question 4)

I hadn't thought about it... I could make something for that I guess...

If the progress was stopped in the middle of it, you can know what % of comics you have with hashs loaded by doing a smartlist that look for comics with customvalue "page_hashes" not equal to "". (remember once a comics has this value completed it will not load it again unless you manually delete it)

Remember only the first time this process is done, as once it has finished once, it will only do it for new comics...

2) Would it be possible to store the hashvalue in the xml of the cbz?

Unfortunetely custom values are not saved in comic files, so the answer would be no... All the same, the comics already have the hashtags inside their compress file header (which is where I look for them)

3) In another thread you kindly suggested this could be used for files with Scanner unknown or no info - is it simple for me to edit that code? I've had a quick look but don't really know python. I'll keep looking though

I will do some more work on this in the future for the suer to be able to edit it easily, but if you want to do it manually... tell me what you exactly want to do and I will tell you what to edit

4) Interestingly I'm seeing a few files that have valid scanner info in them that aren't being hashed, but this could relate to question 1, I'm not sure it's finishing but I'll keep playing - trying to do a few things at once here.

All cbz and cbr files in your library should have hash information value filled (with or without scanner info) once the first time the script finish running (it is the first condition it must finish to do anything else)

5) I thought, reading previous posts, that it would try hashing only selected files. Pressing the button up top seems to be hashing everything (?) Maybe I didn't select 1 file but I think I did.

No. One thing is hashing the comics... This is done with every comic and inside the custom values. Another completely different thing is the file that save the relation between these hashes and the scanner... This one is store in a separate file in your documents (for now) and it will be deleted with the button to refresh (so that it can reload new scanner information). The second one takes less than a minute to be made, the first one can take more than an hour if you have MANY comics (and will only be done once)

6) Out of interest - how many and which pages is it hashing? The values in page_hash don't seem to be 1 per page so I presume you're doing a subset. Which surely must be the same every time? Or are the values totals?

Every page is hashed if the script works ok... the hash is a combination of the crc and the size of the file... if you have any example where the number of hashes is not the number of pages, it is a bug, please send it to me and I will try to fix it

The hashing comparition is done page by page, so EVERY page has a hash made

I hope this answer all your questions :)
Last Edit: 8 months 3 weeks ago by Xelloss.
The administrator has disabled public write access.

Scanners technical question... and posible forum communitie project 8 months 2 weeks ago #48653

  • beardyandy
  • beardyandy's Avatar
  • Offline
  • Senior Boarder
  • Posts: 47
  • Thank you received: 5
  • Karma: 0
It does, thank you very much.

Mostly ;) One last one...

I'm still a little confused about when the hash is created. Is it done by your script, or when creating the cbr/cbz, or are you talking about two different hashes. One being the page_hashes, and the other being "All the same, the comics already have the hashtags inside their compress file header (which is where I look for them)"

Just to confirm, would you expect the hashes to change on conversion from cbr to cbz using the export function in comicrack (I've never noticed is the export to cbz changes the file size before).

I will have a large hashfile for you soon, some more sorting to do then are you ok if I PM you a dropbox for it?
The administrator has disabled public write access.

Scanners technical question... and posible forum communitie project 8 months 2 weeks ago #48658

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 582
  • Thank you received: 146
  • Karma: 29
The hash I make for every page is based in two things: The crc of the image file in the compressed file and the original size of the uncompressed image...

The two things are in the cbr and cbz headers (split in two values of course) and don't change if you export from cbz to cbr or cbz to cbr (as they are properties of the uncompressed files themselves). I use crc hash to make my hash because zip and rar files alrady store them in the file header and you don't have to read all the file to know them.

About when hashes are created, they are created first of all in all the comics... then they are read one by one from the custom value, and it is here when the cache file with all the links between hashes and scanners is made... (one thing is the hashes, another is the cache file with the relations)

About sizes, the compressed size change of course, but I use uncompressed file size, so the scipt don't care if the page is in a cbr or cbz file :)

About the cache file, I would be more than grateful to receive it, THANKS!
Last Edit: 8 months 2 weeks ago by Xelloss.
The administrator has disabled public write access.

Scanners technical question... and posible forum communitie project 6 months 1 week ago #49008

  • beardyandy
  • beardyandy's Avatar
  • Offline
  • Senior Boarder
  • Posts: 47
  • Thank you received: 5
  • Karma: 0
The administrator has disabled public write access.

Scanners technical question... and posible forum communitie project 6 months 1 week ago #49022

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 582
  • Thank you received: 146
  • Karma: 29
I will reply your messages here, as the message box is horrible for large replies :P
By the way - this is a work in progress but I'm creating a scanners list so I can see ones that need sorting.

github.com/beardedfool/Comicrack/blob/master/scanners

Not sure it has value yet but maybe

Thanks! I will look at it and compare it with mine... Really I am not much consistent or detalied with scanner names and format, but I tried using a standard that I apply with a script that works in 80%/70% of the cases (the rest I change manually)

My standard is:

"Scanner:" + Group1 (if present) + "-" + (group2 if any) + "-" + Scanner1 + "-" + (scanner2 if any)

Again, I am not telling that I am 100% changing everything perctly to this starndard, as most clearing I do is manually looking at the stacks

I will later post a list of scanners fields I have if you want
That's great, glad it helped!
Could you send me the rules Alan Scott used as well please. All when you get a chance.
Do I just add it to my cache file? Is it a problem if there are duplicates?

I'm just cleaning up all mine through DM rules to I'll keep to the same standard.

(also see here comicrack.cyolito.com/forum/7-general/40...tion-smartlist#49007 - but I'll change to that standard)

How to you put mulitple scanners e.g. G85-Fawkes-Empire
I was going to use + sign

As I mentioned, I used a script I made myself for this, with a lot of random rules... I didn't post it because I am constantly changing it and it is FAR of perfect yet... But it helps to do the manual clearing a bit...

If you want, I can send you my script, but I warn you that I am not sure you will find it useful

I don't have the Alan Scott one anymore U_U (I replace it with yours when I finish using it), but I will try to see if I have a copy somewhere. On the other hand you can ask him for the file :)

I used him and yours as a base to know the scanners, and then standirized it to my likings, I am not expecting my script to just complete the data perfectly, but help to complete them as people prefer them...

About DM, I don't use it much, only for VERY simple rules. I find making my scripts in python much easier than playing with regex complex expressions :P
Sorry! Last message!!

If you use DMrules for the cleanup - would you mind sharing that as well please.

As I said, I use a script, not DM

Btw, I will upload my cache today. It is MUCH smaller than yours. Always use other caches as GUIDES, never expect it to complete the field in a perfect way :P
Last Edit: 6 months 1 week ago by Xelloss.
The administrator has disabled public write access.
Time to create page: 0.455 seconds

Who's Online

We have 165 guests and 5 members online