--bwlimit how to

Ben Escoto bescoto@stanford.edu
Tue, 03 Sep 2002 22:06:38 -0700

Content-Type: text/plain; charset=us-ascii

>>>>> "T" == trevor  <trevor@tecnopolis.ca>
>>>>> wrote the following on Tue, 3 Sep 2002 13:33:09 -0500 (CDT)

  T> You might also want to put a URL to a source for cstream in there
  T> as only people who've read your list archives will have a clue
  T> what we're talking about.

Ok, updated.

  T> The only other option would be to put cstream in the ssh command
  T> like:

  T> rdiff-backup --remote-schema 'ssh %s '\''cstream -v 1 -t 10000 |
  T> rdiff-backup --server'\' localsrcdir
  T> 'user@foo.bar.com::/remotebakdir'

  T> I'm wracking my brain trying to determine what the difference
  T> between that and the last option would be, if any.

If ssh compression is enabled, this could result in rdiff-backup being
limited by a much lower real network rate than intended...  If you are
making comparisons, remember that by default rdiff-backup enables ssh
compression (this is overridden by the --remote-schema option).

  T> So if you want to update your FAQ, use the 2 examples I've marked
  T> with *'s as they, in theory, should be the best methods.

Hmm, because of the compression issue your first way still seems more
flexible to me.

  T> I'm only guessing at this, as I'm hard pressed to find an easy
  T> method of testing this.  However, this would make it extremely
  T> bursty and somewhat contrary to what I want to achieve with
  T> --bwlimit.

Perhaps you could mail the author?  Doesn't qualify as a "test", but
hopefully will result in valid information.. :)

  T> This leads me to a suggestion.  rdiff-backup should either
  T> multithread or fork into 2 processes so that while a file's data
  T> is transferring, it can be calculating the next file and getting
  T> it ready for potential transfer.  That way you keep the data pipe
  T> busy at all times, instead of the think - transfer - think -
  T> transfer burstiness it seems to do now.

  T> I'm not saying that this is imperative or you should triple the
  T> complexity of the code to do this... (I am a programmer and know
  T> how this stuff works)... but if you ever do a rewrite it would be
  T> something to think about.

Yep, perhaps for version 2.0.  I considered threads when initially
writing rdiff-backup but decided to keep it simple.  Note that the
burstiness can at most result in a factor of two speed reduction
compared to the optimal (always thinking+always writing).

    If I were adding threads I'd add more than two I think, so that
multiprocessor systems could work on multiple files simultaneously.

Ben Escoto

Content-Type: application/pgp-signature

Version: GnuPG v1.0.7 (GNU/Linux)
Comment: Exmh version 2.5 01/15/2001