Memory Problem

Ben Escoto
Wed, 12 Dec 2001 12:49:24 -0800

Content-Type: text/plain; charset=us-ascii

Looking at the Python web site, it appears there have been a number of
memory leaks reported with python 2.2b2.  For instance, try:

def bar(a):
    a + (1/0)

while 1:
    try: bar([])
    except: pass

This should run in constant memory but apparently takes up unlimited
amounts of memory on some systems.  So, python itself has some memory
leaks, and that might be your problem.  However, this doesn't explain
why on my system rdiff-backup takes 7MB of memory to process 7GB but
but on yours it takes 200+MB to process 80MB (!).

    Perhaps there is something different about the files you are
backing up, or the options you are using.  Or maybe python has more
memory leaks on your system for some reason.  rdiff-backup is based
heavily around generators, so you could also try something like:

def foo():
    i = 0L
    while 1:
        yield i
        i += 1

for i in foo(): j = 3-i

This should run in constant space, and does on my system.  But if it
doesn't on yours, that would explain everything.

Ben Escoto

Content-Type: application/pgp-signature

Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Exmh version 2.5 01/15/2001