ftp recursively

[Jeff Schwab]

I need to move a directory tree (~9GB) from one machine to another on
the same LAN. What's the best (briefest and most portable) way to do
this in Python?

I see that urllib has some support for getting files by FTP, but that it
has some trouble distinguishing files from directories.

http://docs.python.org/lib/module-urllib.html

"The code handling the FTP protocol cannot differentiate
between a file and a directory."

I tried it anyway, but got an authentication problem. I don't see a
"how to log into the FTP server" section on docs.python.org . Is there a
tutorial I should read?

I am particularly looking for a quick, "good enough" solution that I can
use this afternoon. Thanks in advance to any kind-hearted soul who
chooses to help me out.

[Paul Rubin]

Re: ftp recursively

Jeff Schwab <jeff@schwabcen ter.comwrites:

I need to move a directory tree (~9GB) from one machine to another on
the same LAN. What's the best (briefest and most portable) way to do
this in Python?

os.popen("rsync ...")

[Gabriel Genellina]

Re: ftp recursively

En Tue, 18 Mar 2008 18:25:28 -0300, Jeff Schwab <jeff@schwabcen ter.com>
escribió:

I need to move a directory tree (~9GB) from one machine to another on
the same LAN. What's the best (briefest and most portable) way to do
this in Python?

See Tools/scripts/ftpmirror.py in your Python installation.

--
Gabriel Genellina

[Jeff Schwab]

Re: ftp recursively

Gabriel Genellina wrote:

En Tue, 18 Mar 2008 18:25:28 -0300, Jeff Schwab <jeff@schwabcen ter.com>
escribió:
>

>I need to move a directory tree (~9GB) from one machine to another on
>the same LAN. What's the best (briefest and most portable) way to do
>this in Python?

>
See Tools/scripts/ftpmirror.py in your Python installation.

Thank you, that's perfect. Thanks to Arnaud as well, for the pointer to
ftplib, which might useful for other purposes as well.

Per the earlier advice of other posters (including one whose message
seems mysteriously to have disappeared from c.l.python), I just stuck
with the Unix tools I already knew: I ended up tarring the whole 9GB,
ftping it as a flat file, and untarring it on the other side. Of
course, the motivation wasn't just to get the files from point A to
point B using Unix (which I already know how to do), but to take
advantage of an opportunity to learn some Python; next time, I'll try
the ftpmirror.py script if it's generic enough, or ftplib if there are
more specific requirements.

[Paul Rubin]

Re: ftp recursively

Jeff Schwab <jeff@schwabcen ter.comwrites:

ftping it as a flat file, and untarring it on the other side. Of
course, the motivation wasn't just to get the files from point A to
point B using Unix (which I already know how to do), but to take
advantage of an opportunity to learn some Python; next time, I'll try
the ftpmirror.py script if it's generic enough, or ftplib if there are
more specific requirements.

I see, that wasn't clear in your original post. You should look at
the os.walk function if you want to know how to traverse a directory
tree (maybe you are already doing this). Also, for security reasons,
it's getting somewhat uncommon, and is generally not a good idea to
run an ftpd these days, even on a LAN. It's more usual these days
to transfer all files by rcp or rsync tunnelled through ssh.

[Steven D'Aprano]

Re: ftp recursively

On Wed, 19 Mar 2008 23:53:22 -0700, Jeff Schwab wrote:

The point of rsync is to keep a local directory tree in sync with a
remote one, by transferring only change-sets that are conceptually
similar to patches. If you're only transferring files once, there's no
particular benefit (AFAIK) to using rsync rather than some kind of
recursive ftp.

Avoiding re-inventing the wheel is a big benefit.


--
Steven