Welcome, Guest
General discussion about ComicRack

TOPIC: Best Practices for Backing-up Large Libraries?

Best Practices for Backing-up Large Libraries? 1 year 5 months ago #45564

  • Gemris
  • Gemris's Avatar
  • Offline
  • Junior Boarder
  • Posts: 38
  • Thank you received: 12
  • Karma: 6
I have a VERY large library, and I'm hoping someone has a good suggestion on how to back up changes and additions as I make them.

Currently, I process and organize all new comics and/or changes into the library on my hard drive, but in order to back that up, I then have to re-transfer my entire library to my back-up external drive (or remember issue-by-issue any updates or new comics, which usually go into existing series folders).

I'm assuming there is no way for comicrack to automatically apply all changes to both libraries at once, so how do you all apply updates to your backups? Copying my entire collection every time can't be good for the drives and takes 3 hours minimum.

Does anyone have better practices that they can share? Thanks!
The administrator has disabled public write access.

Best Practices for Backing-up Large Libraries? 1 year 5 months ago #45667

  • nfp1971
  • nfp1971's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 8
  • Thank you received: 3
  • Karma: 0
Having recently bought myself a m-disc blu-ray recorder, i would also like to hear from anyone with a good backup plan.

I look forward to start backing up my comics and bd.

Thanks!
The administrator has disabled public write access.

Best Practices for Backing-up Large Libraries? 1 year 5 months ago #45668

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 455
  • Thank you received: 117
  • Karma: 24
I used to have a back up admin job, and what you have to take in account is... Do you usually change your files or file ubication? (Even micro changes as add a value or change massively a field). If not, there are tons of software to sync folders only modifying what is new... That is how most backup jobs professionally work... (or incremental/diferential backups if you want a not directly usable backup in case of failure)

Whole backups in cases of large size data is only recommended once in a while (a year for example), as incremental/sync backups are much better in many ways...

The problem, however, with comics files in cb# formats, is that data is stored in small xmls inside "large" comic files... So when you add only a character in a comic, the whole comic has to be copied again... This problem is even worse when you use scripts such as organizers that move and raname files using internal data, as they "create new comic files" regularly...

If you don't move/rename and don't change comic files often, you can use a sync program (any back up softaware, even tiny freeware ones as robocopy) has an option to do this...

To know when was the last time you modified your files or folders to know if this is a good idea for your collection, just search for modify dates in your folders and comics in your comic folder... That will you give you an idea of how much data each sync would have to copy :)
Last Edit: 1 year 5 months ago by Xelloss.
The administrator has disabled public write access.
The following user(s) said Thank You: nfp1971

Best Practices for Backing-up Large Libraries? 1 year 5 months ago #45669

  • Gemris
  • Gemris's Avatar
  • Offline
  • Junior Boarder
  • Posts: 38
  • Thank you received: 12
  • Karma: 6
Thank you. I'll have to see if I can find something user friendly to do periodic incremental back-ups of changes, then reserve full back-ups to once or twice a year. I guess it's a matter of finding the right software. There is no comicrack plugin though, correct? Any suggestions on a program?
Thanks again.
The administrator has disabled public write access.

Best Practices for Backing-up Large Libraries? 1 year 5 months ago #45679

  • Xelloss
  • Xelloss's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 455
  • Thank you received: 117
  • Karma: 24
All the programs I used to work with were paid or for Linux, but there is a lot of good free ones out there...
The administrator has disabled public write access.

Best Practices for Backing-up Large Libraries? 1 year 4 months ago #45723

  • Nomadtla
  • Nomadtla's Avatar
  • Offline
  • Junior Boarder
  • Posts: 22
  • Thank you received: 10
  • Karma: 5
The problem I have found with "sync" style programs, particularly free ones, is that you lose some Metadata. Anything that moves the files outside of ComicRack will lose some metadata. Seemed to lose metadata on about 50% of files when I was using FreeFileSync. Now if you just want to keep the comic files and would not mind rescraping and some reorganization this is fine. If any loss of data bothers you, like it does me, the best I have found is what you are doing in your original post.

That is what I do about once every 3 months. Just export my library from inside ComicRack. Usually do it in groups (publisher) overnight for about 3-4 days.
The administrator has disabled public write access.

Best Practices for Backing-up Large Libraries? 1 year 4 months ago #45725

  • Gemris
  • Gemris's Avatar
  • Offline
  • Junior Boarder
  • Posts: 38
  • Thank you received: 12
  • Karma: 6
Nomadtla wrote:
The problem I have found with "sync" style programs, particularly free ones, is that you lose some Metadata. Anything that moves the files outside of ComicRack will lose some metadata. Seemed to lose metadata on about 50% of files when I was using FreeFileSync. Now if you just want to keep the comic files and would not mind rescraping and some reorganization this is fine. If any loss of data bothers you, like it does me, the best I have found is what you are doing in your original post.

That is what I do about once every 3 months. Just export my library from inside ComicRack. Usually do it in groups (publisher) overnight for about 3-4 days.

I appreciate the responses and tips but I'm confused by this. Assuming I've converted all files to cbz and saved the data into the file itself, I don't see how they would lose metadata. Once I've done that, the data is part of the file, not just ComicRack. I share files with my cousin regularly, and anything cbz loses no data, so is it because the file is being synced prior to the metadata being saved?

Sometimes I add a lot of stuff within a few days/weeks and the idea of losing that much in-between 3-4 month back-ups makes me cringe.
The administrator has disabled public write access.

Best Practices for Backing-up Large Libraries? 1 year 4 months ago #45854

  • boshuda
  • boshuda's Avatar
  • Offline
  • Gold Boarder
  • Posts: 295
  • Thank you received: 64
  • Karma: 8
I think they lose metadata because some of it is saved in an alternate file stream by ComicRack. Don't quote me on that, I'm not positive, but I believe I remember something like that. The backup tools often fail to retrieve that data.

I've found the best method for backing them up is to make sure, as part of my work flow, the books are exported to CBZ format and then I use an auto-sync tool. I use d-synchronize ([url=http://http://dimio.altervista.org/eng/][/url]). There's also a script to backup your ComicRack application data called Backup Manager.
The administrator has disabled public write access.

Best Practices for Backing-up Large Libraries? 1 year 3 months ago #45943

Proper planning helps a long way. These are my practices when it comes to backing-up comic files:

1. Back-up only the complete series (comic), per arcs (manga), volumes (manga) or crossovers (major comic events). Anything less than that should not be priority for back-up.

2. Once decided, scrape any metadata you can or make inputs of your own (optional). (Opinion: I don't bother with this except genre, date, number and volume)

3. After scraping, rename your file in comicrack library and make sure it has the same name in your windows file browser as you see fit [automation→file rename]. For example, I renamed 'Scarlet Witch' (2016) to 'ANAD Scarlet Witch' and 'Extraordinary X-Men' to 'ANAD X-Men Extraordinary'. 'X-Factor' to 'X-Men X-Factor' and 'Uncanny X-Men' to 'X-Men Uncanny'.

4. Delete any advertisement on the comic so as to help reduce file size.

5. After finalizing everything, back-up the files to your desired drive or file directory by exporting them. Check the 'embed book info' box, remove deleted pages and file format should be 'cbz' for uniformity, page format should be 'preserve original', template should be 'filename'.


Here is a look of my organization in comicrack and should look the same as with file explorer after automation.
The administrator has disabled public write access.
Time to create page: 0.203 seconds

Who's Online

We have 169 guests and 3 members online