Defining rules

Rules can be defined in storeBackup.pl (options exceptRule, includeRule and comprRule) and in storeBackupSearch.pl (option searchRule). Both support the definition in configuration files and on the command line.

This part of the description shows how to use rules in storeBackup. If you are not familiar with pattern matching and perl you should try to change the examples very carefully a little bit. But you can run easily into error messages you will not understand.

First, all the examples are explained for being written in a configuration file. Mostly I will use the key word from storeBackup.pl (exceptRule¹⁹), but the syntax of rule definition is identical to the ones you can use for includeRule and searchRule. Later, we will see how to use rules on the command line.

All the values we are talking about now are the ones from the files backed up at the point in time when the backup was performed, not from the files in the backup!

In general, rules are a piece of perl with some specialities. We start with some easy and typical examples:

EXAMPLE 1:

exceptRule = '$size > 1610612736'

(Take care of the quotes. Generate a configuration file with storeBackup.pl or storeBackupSearch.pl and read the comments in the beginning how quoting and environment variables are interpreted.)

This rule will match for all files with more than 1.5GB ($1.5 * 1024^3$) bytes. $size represents the size of each individual file. In this example, all files bigger than 1.5GB will not be saved. This is not very easy to read, and you can write instead:

exceptRule = '$size > &::SIZE("1.5G")'

(Take care of all quotes.) This will have the same effect as the rule before. &::SIZE is a function which calculates the real value from the string "1.5G". You can use identifiers from 'k' to 'P' with the following meaning:

1k	1 kilobyte = 1024 Byte
1M	1 Megabyte = $1024^2$ Byte
1G	1 Gigabyte = $1024^3$ Byte
1T	1 Terabyte = $1024^4$ Byte
1P	1 Petabyte = $1024^5$ Byte

Eg: &::SIZE("0.4T") is valid, while &::SIZE("1G1M") is not.

EXAMPLE 2:

exceptRule = '$file =~ m#\.bak$#'

(Take care of the quotes.) This rule will match for all files ending with '.bak' which means they will not be saved. $file represents the individual file name with the relative path below the parameter of option sourceDir from storeBackup.pl. If you do not understand the strange thing right to $file, it is called pattern matching or regular expression. See man perlretut (perl regular expressions tutorial) for detailed explanation. But you should be able to expand this to simple needs:

exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#'

(Take care of the quotes and all blanks.) This rule will match and therefore not save files ending with '.bak' or '.mpg'.

exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#'
	   or '$file =~ m#\.avi$#'

It should not be a surprise, that you will not backup files ending with '.bak', '.mpg' or '.avi'.

Now we want to create a rule which will prevent the backup of all files which end with '.bak', '.mpg' or '.avi' and also all files bigger than 500 Megabyte:

exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#'
	   or '$file =~ m#\.avi$#' or '$size > &::SIZE("0.5G")'

If you set 'debug = 2', you can see if and how the rule matches for individual files. If you set 'debug = 1', you can see if the rule matches for each file. With 'debug = 0' (default), you will not get a message.

You can use the following 'preset variables':

$file	file name with relative path from original sourceDir
$size	size of the file in bytes
$mode	mode of the file (integer, use 0... to compare with octal value, e.g. $mode = 0644
$ctime	creation time in seconds since epoch (Jan 1 1970), see below
$mtime	modify time in seconds since epoch, see below
$uid	user id (string if defined in operating systems), e.g. $uid eq "bob"
$uidn	user id (numerical value), e.g. $uidn = 1001
$gid	group id (string if defined in operating system), see $uid
$gidn	group id (numerical value), see $uidn
$type	type of the file, can be one of SbcFpl, see option exceptTypes in storeBackup.pl
$ENV{XXX}	use contents of environment variable XXX

If you use $ctime or $mtime, it is not pure fun to calculate the number of seconds since epoch (Jan 1st 1970) every time. For this reason, storeBackup supports a special function &::DATE to make your live cosy:

EXAMPLE 3:

searchRule = '$mtime > &::DATE("14d")' and '$mtime < &::DATE("3d12h")'

With this search rule (in storeBackupSearch.pl) you will find all files which are younger than exactly 14 days and older than 3 days and 12 hours. The syntax understood by &::DATE is:

1. d	day
   h	hour
   m	minute
   s	second

   
   
   So "3d2h50s" means 3 days, 2 hours and 50 seconds. With the function above, you specify ``now'' minus that period.
   
   

2. YYYY.MM.DD	year.month.day
   YYYY.MM.DD_hh.mm.ss	same format as backup dirs
   2008.04.30	specifies April 30 2008, 0:00,
   2008.04.30_14.03.05	specifies April 30 2008, at 2 o'clock, 3 min. and 5 sec. in the afternoon.

   
   
   With the function &::DATE, you also specify a fixed point in time.

You already saw some possibilities to group the checking of the ``variables'' by using and and or. You can use:

and, or, not, (, )

Everything is like in perl (to be honest, it is evaluated by the perl interpreter). But you should surround each of these with one (or more) blanks (white spaces) if you want debug = 2 to work correctly!

EXAMPLE 4:

searchRule = ( '$mtime < &::DATE("14d")' and '$mtime > &::DATE("3d12h")' )
	     and not '$file =~ m#\.bak$'

Finds all files younger than 14 days and older than 3 days, 12 hours, but only if they do not end with .bak.
See how and, not, ( and ) have at least one white space surrounding it.

EXAMPLE 5:

StoreBackup allows you to control its behavior via rules by storing special flag files in the source directory. This can be done for a specific directory or additionally for its subdirectories.

You define in the configuration file of storeBackup.pl

exceptRule= '&::MARK_DIR($file)'

then all files in directories with the default ``marker'' file .storeBackupMark will not be saved because the function MARK_DIR returns 1 in that case.

You may also use another marker file, e.g. .dontBackup:

exceptRule= '&::MARK_DIR($file, ".dontBackup")'

Then - naturally - you have to create a file .dontBackup in the directories you do not want to backup:

$ touch .dontBackup

If you do not want to backup all sudirectories below the one with file .dontBackup in it, define the rule acting recursively:

exceptRule= '&::MARK_DIR_REC($file, ".dontBackup")'

Because this is a rule, you can also reverse the result by configuring:

exceptRule= not '&::MARK_DIR($file, ".backupThisDir")'

As a name for the marker file, you now use e.g. .backupThisDir. All backup without this marker will not be backuped now.

You can do any combination with other rule elements. If e.g. you only want to save files from root below directories marked with .saveRootOnly, define:

exceptRule= not ( '&::MARK_DIR_REC($file, ".saveRootOnly")'
                  and '$uid eq "root"' )

Now think you do not want to save recursively all files in directories with .dontBackup and you only want to save files from user root recursively in directories with .saveRootOnly. In this case you can define the following:

exceptRule = '&::MARK_DIR_REC($file, ".dontBackup")'
	     or
 	     not ( '&::MARK_DIR_REC($file, ".saveRootOnly")'
                  and '$uid eq "root"' )

You can use combinations with different marker files in different or the same rules.

Naturally, you can also use MARK_DIR and MARK_DIR_REC in other rules, especially for option comprRule.

Using rules on the command line

Let's take a look at:

exceptRule = '$size > &::SIZE("1.5G")'

If we try to use the command line like this:

--exceptRule '$size > &::SIZE("1.5G")'               ### WRONG ###

we will get some nasty error messages because the shell strips the single quotes and storeBackup tries to interpret the result the same way as in the configuration file (see description in each configuration file at the top). Here, storeBackup will complain about not knowing the environment variable $size. (The $-sign is not masked any more because the shell removed the single quote.) So we have to mask the $-sign. We also have to mask the double quotes, because normally, storeBackup will interpret them as grouping quotes and will not bypass them directly to perl. The right way specifying this option is:

--exceptRule '\$size > &::SIZE(\"1.5G\")'            ### CORRECT ###

We have to write example 4 in the following way:

--searchRule '( \$mtime < &::DATE(\"14d\") and \
   \$mtime > &::DATE(\"3d12h\") ) and not \$file =~ m#\.bak\$#'

In case of problems, you should read the perl error massage which shows what perl really gets. Beside this, option --print will show each parameter after being parsed through shell and storeBackup. You can use --print in combination with configuration files, too.

The following predefined functions are available for rules:

&::DATE(timePeriod)

Calculates the relative time from ``now'' and lets you compare it with $ctime or $mtime. Set timePeriod to e.g. "3d2h". See example 3 in this section for details.

&::SIZE(size)

Returns the size given in human readable form (e.g. 1.5G) as a plain number (in bytes). See example 1 in this section for details.

&::MARK_DIR($file [, markerFile])

Checks, if markerFile exists in the directory of $file. Default for markerFile is .storeBackupMark. See example 5 in this section for details.

&::MARK_DIR_REC($file [, markerFile])

Checks, if markerFile exists in the directory or any higher level directory of $file. Default for markerFile is .storeBackupMarkRec. See example 5 in this section for details.