Defining rules

Rules can be defined in storeBackup.pl (options excludeRule, includeRule and comprRule) and in storeBackupSearch.pl (option searchRule). Both support the definition in configuration files and on the command line.

This part of the description shows how to use rules in storeBackup. If you are not familiar with pattern matching and perl you should try to change the examples very carefully a little bit. But you can run easily into error messages you will not understand.

First, all the examples are explained for being written in a configuration file. Mostly I will use the key word from storeBackup.pl (exceptRule), but the rules are identical to the ones you can use for includeRule and searchRule. Later, we will see how to use rules on the command line.

All the values we a talking about now, are the ones from the files backed up at the point in time when the backup was performed, not from the files in the backup!

In general, rules are a piece of perl with some specialities. We start with some easy and typical examples:

EXAMPLE 1:

exceptRule = '$size > 1610612736'

(Take care of the quotes. Generate a configuration file with storeBackup.pl or storeBackupSearch.pl and read the comments in the beginning how quoting and environment variables are interpreted.)

This rule will match for all files with more than 1.5GB ($1.5 * 1024^3$) bytes. $size represents the size of each individual file. In this example, all files bigger than 1.5GB will not be saved. This is not very easy to read, and you can write instead:

exceptRule = '$size > &::SIZE("1.5G")'

(Take care of all quotes.) This will have the same effect as the rule before. &::SIZE is a function which calculates the real value from the string "1.5G". You can use identifiers from 'k' to 'P' with the following meaning:

1k	1 kilobyte = 1024 Byte
1M	1 Megabyte = $1024^2$ Byte
1G	1 Gigabyte = $1024^3$ Byte
1T	1 Terabyte = $1024^4$ Byte
1P	1 Petabyte = $1024^5$ Byte

Eg: &::SIZE("0.4T") is valid, while &::SIZE("1G1M") is not.

EXAMPLE 2:

exceptRule = '$file =~ m#\.bak$#'

(Take care of the quotes.) This rule will match for all files ending with '.bak' which means they will not be saved. $file represents the individual file name with the relative path below the parameter of option sourceDir from storeBackup.pl. If you do not understand the strange thing right to $file, it's called pattern matching or regular expression. See man perlretut (perl regular expressions tutorial) for detailed explanation. But you should be able to expand this to simple needs:

exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#'

(Take care of the quotes and all blanks.) This rule will match and therefore not save files ending with '.bak' or '.mpg'.

exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#'
	   or '$file =~ m#\.avi$#'

It should not be a surprise, that you will not backup files ending with '.bak', '.mpg' or '.avi'.

Now we want to create a rule which will prevent the backup of all files which end with '.bak', '.mpg' or '.avi' and also all files bigger than 500 Megabyte:

exceptRule = '$file =~ m#\.bak$#' or '$file =~ m#\.mpg$#'
	   or '$file =~ m#\.avi$#' or '$size > &::SIZE("0.5G")'

If you set 'debug = 2', you can see if and how the rule matches for individual files. If you set 'debug = 1', you can see if the rule matches for each file. With 'debug = 0' (default), you will not get a message.

You can use the following 'preset variables':

$file	file name with relative path from original sourceDir
$size	size of the file in bytes
$mode	mode of the file (integer, use 0... to compare with octal value, eg. $mode = 0644
$ctime	creation time in seconds since epoch (Jan 1 1970), see below
$mtime	modify time in seconds since epoch, see below
$uid	user id (string if defined in operating systems), eg. $uid eq "bob"
$uidn	user id (numerical value), eg. $uidn = 1001
$gid	group id (string if defined in operating system), see $uid
$gidn	group id (numerical value), see $uidn
$type	type of the file, can be one of SbcFpl, see option exceptTypes in storeBackup.pl

If you use $ctime or $mtime, it's not pure fun to calculate the number of seconds since epoch every time. For this reason, storeBackup supports a special function &::DATE to make your live cosy:

EXAMPLE 3:

searchRule = '$mtime > &::DATE("14d")' and '$mtime < &::DATE("3d12h")'

With this search rule (in storeBackupSearch.pl) you will find all files which are younger than exactly 14 days and older than 3 days and 12 hours. The syntax understood by &::DATE is:

1. d	day
   h	hour
   m	minute
   s	second

   
   
   So "3d2h50s" means 3 days, 2 hours and 50 seconds. With the function above, you specify ``now'' minus that period.
   
   

2. YYYY.MM.DD	year.month.day
   YYYY.MM.DD_hh.mm.ss	same format as backup dirs
   2008.04.30	specifies April 30 2008, 0:00,
   2008.04.30_14.03.05	specifies April 30 2008, at 2 o'clock, 3 min. and 5 sec. in the afternoon.

   
   
   With the function &::DATE, you specify a fixed point in time also.

You already saw some possibilities to group the checking of the ``variables'' by using: and and or. You can use:

and, or, not, (, )

Everything is like in perl. (To be honest, it is evaluated by the perl interpreter.). But you should surround each of these with one (or more) blanks (white spaces) if you want debug = 2 to work correctly!

EXAMPLE 4:

searchRule = ( '$mtime < &::DATE("14d")' and '$mtime > &::DATE("3d12h")' )
	     and not '$file =~ m#\.bak$'

Finds all files younger than 14 days and older than 3 days, 12 hours, but only if they do not end with .bak.
See how and, not, ( and ) have at least one white space surrounding it.

using rules on the command line

Let's take a look at:

exceptRule = '$size > &::SIZE("1.5G")'

If we try to use the command line like this:

--exceptRule '$size > &::SIZE("1.5G")'               ### WRONG ###

we will get some nasty error messages because the shell strips the single quotes and storeBackup tries to interpret the result the same way as in the configuration file (see description in each configuration file at the top). Here, storeBackup will complain about not knowing the environment variable $size. (The $-sign is not masked any more because the shell removed the single quote.) So we have to mask the $-sign. We also have to mask the double quotes, because normally, storeBackup will interpret them as grouping quotes and will not bypass them directly to perl. The right way specifying this option is:

--exceptRule '\$size > &::SIZE(\"1.5G\")'            ### CORRECT ###

We have to write example 4 in the following way:

--searchRule '( \$mtime < &::DATE(\"14d\") and \
   \$mtime > &::DATE(\"3d12h\") ) and not \$file =~ m#\.bak\$#'

In case of problems, you should read the perl error massage which shows what perl really gets. Beside this, option --print will show each parameter after being parsed through shell and storeBackup. You can use --print in combination with configuration files also.