[rproxy-devel] Re: bottlenecks
Shirish H. Phatak
Wed, 19 Jun 2002 11:01:31 -0400
I have run the librsync compiled binaries through a bunch of
different memory checkers already. There is a buffer leak in rs_delta_*,
which my last patch should take care of. Other than that I haver not
Donovan Baarda wrote:
>I've Cc'd this to the rproxy-devel list for reference.
>On Tue, Jun 18, 2002 at 03:07:13AM -0700, Ben Escoto wrote:
>>>>>>>"DG" == dean gaudet <firstname.lastname@example.org>
>>>>>>>wrote the following on Mon, 17 Jun 2002 12:43:24 -0700 (PDT)
>>The internal librsync procedure used up only about 1/20th of the CPU
>>time as the external rdiff procedure, meaning I suppose that 95% of
>>the CPU time was spent forking, setting up the external rdiff process,
>>etc. Of course this is for small files; as the file sizes became
>>large, the forking time would go to 0 as a percentage of the total.
>> Oh, and the reason that test3.py had such a large real time is
>>because it ended up using hundreds of MB of memory and my system
>>started swapping like crazy. I guess all this efficiency won't be
>>worth much if there is a huge memory leak in pysync or librsync...
>I'm glad to see pysync's librsync wrapper being useful. 1/20th is a
>The memory problem...
>It is quite possible that there is a memory leak either in librsync or the
>Python wrapper. The Python wrapper is a first cut using swig, so it is
>possible swig introduced the memory leak, or I did with a bad swig wrapper.
>Librsync is not exactly clean code either, and I've already seen some dodgey
>memory alloc/free stuff in there that can be cleaned up, though I'm not sure
>it exactly leaks.
>There are two things that need doing; improving librsync, and improving the
>I've decided that I'm going to improve librsync a bit first. After that I'm
>going to fix up the Python wrapper using the development versions of swig.
>None of this is going to happen quickly unless someone decides to pay me for
>it (my wife is hassling me to get a real job so that we can buy food).
>However, because I'm a programming addict, there is a chance I'll slowly
>starve to death after my wife leaves me as I code it up anyway :-)