Rules can be defined in storeBackup.pl (options exceptRule, includeRule and comprRule) and in storeBackupSearch.pl (option searchRule). Both support the definition in configuration files and on the command line.
This part of the description shows how to use rules in storeBackup. If you are not familiar with pattern matching and perl you should try to change the examples very carefully a little bit. But you can run easily into error messages you will not understand.
First, all the examples are explained for being written in a configuration file. Mostly I will use the key word from storeBackup.pl (exceptRule19), but the syntax of rule definition is identical to the ones you can use for includeRule and searchRule. Later, we will see how to use rules on the command line.
All the values we are talking about now are the ones from the files backed up at the point in time when the backup was performed, not from the files in the backup!
In general, rules are a piece of perl with some specialities. We start with some easy and typical examples:
exceptRule = '$size > 1610612736'
(Take care of the quotes. Generate a configuration file with storeBackup.pl or storeBackupSearch.pl and read the comments in the beginning how quoting and environment variables are interpreted.)
This rule will match for all files with more than 1.5GB () bytes. $size represents the size of each individual file. In this example, all files bigger than 1.5GB will not be saved. This is not very easy to read, and you can write instead:
exceptRule = '$size > &::SIZE("1.5G")'
(Take care of all quotes.) This will have the same effect as the rule before.
&::SIZE is a function which calculates the real value from the string "1.5G".
You can use identifiers from 'k' to 'P' with the following meaning:
|1k||1 kilobyte = 1024 Byte|
|1M||1 Megabyte = Byte|
|1G||1 Gigabyte = Byte|
|1T||1 Terabyte = Byte|
|1P||1 Petabyte = Byte|
Eg: &::SIZE("0.4T") is valid, while &::SIZE("1G1M") is not.
exceptRule = '$file =~ m#\.bak$#'
(Take care of the quotes.) This rule will match for all files ending with '.bak' which means they will not be saved. $file represents the individual file name with the relative path below the parameter of option sourceDir from storeBackup.pl. If you do not understand the strange thing right to $file, it is called pattern matching or regular expression. See man perlretut (perl regular expressions tutorial) for detailed explanation. But you should be able to expand this to simple needs:
exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#'
(Take care of the quotes and all blanks.) This rule will match and therefore not save files ending with '.bak' or '.mpg'.
exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#' or '$file =~ m#\.avi$#'
It should not be a surprise, that you will not backup files ending with '.bak', '.mpg' or '.avi'.
Now we want to create a rule which will prevent the backup of all files which end with '.bak', '.mpg' or '.avi' and also all files bigger than 500 Megabyte:
exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#' or '$file =~ m#\.avi$#' or '$size > &::SIZE("0.5G")'
If you set 'debug = 2', you can see if and how the rule matches for individual files. If you set 'debug = 1', you can see if the rule matches for each file. With 'debug = 0' (default), you will not get a message.
You can use the following 'preset variables':
|$file||file name with relative path from original sourceDir|
|$size||size of the file in bytes|
|$mode||mode of the file (integer, use 0... to compare with octal value, e.g. $mode = 0644|
|$ctime||creation time in seconds since epoch (Jan 1 1970), see below|
|$mtime||modify time in seconds since epoch, see below|
|$uid||user id (string if defined in operating systems), e.g. $uid eq "bob"|
|$uidn||user id (numerical value), e.g. $uidn = 1001|
|$gid||group id (string if defined in operating system), see $uid|
|$gidn||group id (numerical value), see $uidn|
|$type||type of the file, can be one of SbcFpl, see option exceptTypes in storeBackup.pl|
||use contents of environment variable XXX|
If you use $ctime or $mtime, it is not pure fun to calculate the
number of seconds since epoch (Jan 1st 1970) every time. For this
reason, storeBackup supports a special function &::DATE to make your
searchRule = '$mtime > &::DATE("14d")' and '$mtime < &::DATE("3d12h")'
With this search rule (in storeBackupSearch.pl) you will find all files which are younger than exactly 14 days and older than 3 days and 12 hours. The syntax understood by &::DATE is:
|YYYY.MM.DD_hh.mm.ss||same format as backup dirs|
|2008.04.30||specifies April 30 2008, 0:00,|
|2008.04.30_14.03.05||specifies April 30 2008, at 2 o'clock, 3 min. and 5 sec. in the afternoon.|
You already saw some possibilities to group the checking of the ``variables'' by using and and or. You can use:
and, or, not, (, )
Everything is like in perl (to be honest, it is evaluated by the perl
interpreter). But you should surround each of these with one (or more)
blanks (white spaces) if you want debug = 2 to work correctly!
searchRule = ( '$mtime < &::DATE("14d")' and '$mtime > &::DATE("3d12h")' ) and not '$file =~ m#\.bak$'
Finds all files younger than 14 days and older than 3 days, 12 hours,
only if they do not end with .bak.
See how and, not, ( and ) have at least one white space surrounding it.
StoreBackup allows you to control its behavior via rules by storing special flag files in the source directory. This can be done for a specific directory or additionally for its subdirectories.
You define in the configuration file of storeBackup.pl
then all files in directories with the default ``marker'' file .storeBackupMark will not be saved because the function MARK_DIR returns 1 in that case.
You may also use another marker file, e.g. .dontBackup:
exceptRule= '&::MARK_DIR($file, ".dontBackup")'
Then - naturally - you have to create a file .dontBackup in the directories you do not want to backup:
$ touch .dontBackup
If you do not want to backup all sudirectories below the one with file .dontBackup in it, define the rule acting recursively:
exceptRule= '&::MARK_DIR_REC($file, ".dontBackup")'
Because this is a rule, you can also reverse the result by configuring:
exceptRule= not '&::MARK_DIR($file, ".backupThisDir")'
As a name for the marker file, you now use e.g. .backupThisDir. All backup without this marker will not be backuped now.
You can do any combination with other rule elements. If e.g. you only want to save files from root below directories marked with .saveRootOnly, define:
exceptRule= not ( '&::MARK_DIR_REC($file, ".saveRootOnly")' and '$uid eq "root"' )
Now think you do not want to save recursively all files in directories with .dontBackup and you only want to save files from user root recursively in directories with .saveRootOnly. In this case you can define the following:
exceptRule = '&::MARK_DIR_REC($file, ".dontBackup")' or not ( '&::MARK_DIR_REC($file, ".saveRootOnly")' and '$uid eq "root"' )
You can use combinations with different marker files in different or the same rules.
Naturally, you can also use MARK_DIR and MARK_DIR_REC in other rules, especially for option comprRule.
Using rules on the command line
Let's take a look at:
exceptRule = '$size > &::SIZE("1.5G")'
If we try to use the command line like this:
--exceptRule '$size > &::SIZE("1.5G")' ### WRONG ###
we will get some nasty error messages because the shell strips the single quotes and storeBackup tries to interpret the result the same way as in the configuration file (see description in each configuration file at the top). Here, storeBackup will complain about not knowing the environment variable $size. (The $-sign is not masked any more because the shell removed the single quote.) So we have to mask the $-sign. We also have to mask the double quotes, because normally, storeBackup will interpret them as grouping quotes and will not bypass them directly to perl. The right way specifying this option is:
--exceptRule '\$size > &::SIZE(\"1.5G\")' ### CORRECT ###
We have to write example 4 in the following way:
--searchRule '( \$mtime < &::DATE(\"14d\") and \ \$mtime > &::DATE(\"3d12h\") ) and not \$file =~ m#\.bak\$#'
In case of problems, you should read the perl error massage which shows what perl really gets. Beside this, option --print will show each parameter after being parsed through shell and storeBackup. You can use --print in combination with configuration files, too.
The following predefined functions are available for rules:
"3d2h". See example 3 in this section for details.