using option lateLinks

You can use storeBackup as one program (storeBackup.pl) which does everything alone or you can split the different tasks into several pieces. There is mostly one advantage to run different programs for the different tasks: the time for backup itself from the perspective of the saved computer (or) data is lower.

It makes sense to use option lateLinks if you store your Backup on an nfs server and if you think it's a good idea to speed up (see performance). Configuring lateLinks is a little bit more complicated than using storeBackup.pl as a standalone programe because you have to manage multiple programs.

StoreBackup.pl as a standalone program does the following tasks:

1. The link consistency of all backups (from all backup series) is checked. We will see late what this means.
2. Loading information from one or more old backups. This task is like an initialisation, where it gets file names, md5 sums, dates, times and some other information from the old backups.
3. Checking for all files to backup if another file with that specific content is already in those old backups from where the file names, md5 sums etc. were loaded.
4. The changed data is transferred to the backup: By copying, by compression or by hard linking. Naturally, also the directory structure is generated. For every file, the owner and permissions are set in the same way as in the source directory.
5. The permissions and owners of the directories are set to the same values as in the source directory.
6. Depending on the rules defined with the keep$*$ options of storeBackup.pl, old backups are deleted.

If you start storeBackup.pl with option lateLinks, then the transfer of data (see step 4) and the actions on the remote file system are reduced to the absolutely necessary minimum:

• The changed or new files (including special files) are copied. Changed files which should be compressed are only copied if option lateCompress is set. It depends on your situation if usage of lateCompress makes sense or not.
• Hard links are not generated in the new backup.
• Directories are only created if they are needed for copying / compressing.
• An additional file is created in the new backup: .storeBackupLinks/linkFile.bz2. It contains all the information what should have been done to complete a ``full'' backup with all hard links, directory entries and compressions. The correct permissions (which are also not set) are stored in the file .md5CheckSums in the top level of the backup. This file is also generated in a ``full'' backup run of storeBackup.pl. It is used for restoring data (storeBackupRecover.pl).

Independent of option lateLinks you can always configure storeBackup.pl to not perform step 6, the deletion of old backups. Especially if you are writing your backup on an nfs mount, this will take some time and lengthen you backup. Use storeBackupDel.pl (which can read the configuration file of storeBackup.pl) to split the deletion of old backups from the direct backup process.

It is important to understand that using option lateLinks creates an unfinished backup. Such backups do contain all the data which was intended to be backed up, so the core backup is complete. But storeBackup's job is not completed in the following ways:

• Directory entries are missing.
• Files are not compressed (if you use option lateCompress).
• Hard links are not set at all.
• Permissions are not set correctly, neither for files nor for directories.
• Not that all the information that's necessary to complete to a full backup is available! In the case of hard links this means that there is a reference in the file linkFile.bz2 which points to a file in an older backup. In that older backup, there also can be only a reference in its file linkFile.bz2, and so on. Naturally, at one point there must (and will) be a real file. But you should be aware: if you delete one of these referenced backups, you will destroy all backups which are referencing (directly in indirectly) to that backup. Only delete backups with storeBackup.pl or storeBackupDel.pl -- never use ``rm'' or something similar! These two programs take care of the dependency just described. If you really want to delete files with ``rm'', then make sure, that storeBackupUpdateBackup.pl completed all backups successfully. When all links are set, there are absolutely no dependencies (beside hard links) between the different backups.

The following picture shows two cross linked backup series (from different computers).¹¹ You see, that the references resulting from the lateLinks option can be complex. The hard links are never a problem, because there is no original or ``real'' reference -- every hard link is an original pointer to the file or more precise to the inode. The file will only be deleted if the last hard link has gone. But the references created by lateLinks are just some file names in a file which has nothing to do with the file system.

To complete an unfinished backup (make all those nasty linking and compressing and so on), use storeBackupUpdateBackup.pl. It also will analyse your backups (below ``backupDir'') and find the right order to complete them. After running storeBackupUpdateBackup.pl successfully your backups will be in the same state as if you ran your backup without option lateLinks. Among some others, file .storeBackupLinks/linkFile.bz2 is deleted and everything is hard linked (and compressed), also the permissions are set like in the source directory (except if set option ignorePerms in storeBackup.pl).

If you use option lateLinks, your should run storeBackupUpdateBackup.pl regularly, eg. every night and check if there were some ERROR messages.

Summary:

• An unfinished backup is a backup that was made with lateLinks and that has not yet been finished via storeBackupUpdateBackup.pl.
• You are cannot make a new full backup (that is, a backup without using lateLinks) when this new backup refers to a prior unfinished backup. This is simply because you cannot hard link to files which are not there.
• When using storeBackup.pl or storeBackupDel.pl, you cannot delete earlier backups to which an unfinished backup refers. (You could circumvent this using rm to delete, but that is inappropriate and not recommended.)