file system iteration

[rick]

In Unix, the file system hierarchy is like a tree that has a base or
'root' that exposes objects (files and folders) that can easily be
iterated over.


\ \ | / /
\ \ | / /
\ \|/ /
\ | /
\|/
|
|
Root

So, when I do os.chdir('/') I am at the base of the tree and can now use
something like os.walk() to work with all of the file system objects.

In Windows, the file system is disjointed and there is now real 'root'
At least none that I can see. It looks more like this:

| | | | | | |
|_|_|_|_|_|_|
A B C D E F G

How do you guys handle this when working with scripts that need to touch
all files and folders on a Windows machine? I've been looping through
A-Z like this:

import os.path

paths = []

if os.path.isdir(' A:/'):
paths.append('A :/')

if os.path.isdir(' B:/'):
paths.append('B :/')

....

That's a kludge, but it works OK. I'm sure WMI may have a function that
returns mounted volumes, but under the circumstances currently, I can
only use the standard Python library. Any ideas on how to do this better?

Thanks

[Gerrit Holl]

Re: file system iteration

On 2006-10-09 14:45:35 +0200, rick wrote:

import os.path
>
paths = []
>
if os.path.isdir(' A:/'):
paths.append('A :/')
>
if os.path.isdir(' B:/'):
paths.append('B :/')
>
...
>
That's a kludge, but it works OK. I'm sure WMI may have a function that
returns mounted volumes, but under the circumstances currently, I can
only use the standard Python library. Any ideas on how to do this better?

The very least you can try:

import string
string.ascii_up percase

for c in string.ascii_up percase:
if os.path.isdir(' %s:/' % c):
...

etc.
But I suppose there should be a better way.

Gerrit.

[rick]

Re: file system iteration

Gerrit Holl wrote:

The very least you can try:
>
import string
string.ascii_up percase
>
for c in string.ascii_up percase:
if os.path.isdir(' %s:/' % c):
...
>
etc.
But I suppose there should be a better way.

Oh yes, I do that. I spelled out the example very explicitly for
clarity. I don't actually type in A-Z :)

[Georg Brandl]

Re: file system iteration

rick wrote:

In Unix, the file system hierarchy is like a tree that has a base or
'root' that exposes objects (files and folders) that can easily be
iterated over.
>
>
\ \ | / /
\ \ | / /
\ \|/ /
\ | /
\|/
|
|
Root
>
So, when I do os.chdir('/') I am at the base of the tree and can now use
something like os.walk() to work with all of the file system objects.
>
In Windows, the file system is disjointed and there is now real 'root'
At least none that I can see. It looks more like this:
>
| | | | | | |
|_|_|_|_|_|_|
A B C D E F G
>
How do you guys handle this when working with scripts that need to touch
all files and folders on a Windows machine? I've been looping through
A-Z like this:

Which application needs to walk over ALL files? Normally, you just have a
starting path and walk over everything under it.

In Unix, things aren't so clear either. For example, there are symbolic links
that make the tree more complicated. Or different file system mounted on
different mount points, perhaps not even representing real files like the
/proc filesystem. All that needs caution when iterating over "all files".

Georg

[rick]

Re: file system iteration

Georg Brandl wrote:

Which application needs to walk over ALL files? Normally, you just have a
starting path and walk over everything under it.

Searching for a file by name. Scanning for viruses. Etc. There are lots
of legitimate reason to walk all paths from a central starting point, no???

[Fredrik Lundh]

Re: file system iteration

"rick" <ath-admin@vt.eduwro te:

>Which application needs to walk over ALL files? Normally, you just have a
>starting path and walk over everything under it.

>
Searching for a file by name. Scanning for viruses. Etc. There are lots
of legitimate reason to walk all paths from a central starting point, no???

what's the difference between a "starting path" and a "starting point" ?

</F>

[rick]

Re: file system iteration

Fredrik Lundh wrote:

what's the difference between a "starting path" and a "starting point" ?

None. What starting path or point would you suggest under Windows? Is
there something obvious that I'm missing? I see no starting point under
windows as my initial question clearly stated.

[Jonathan Hartley]

Re: file system iteration

Georg Brandl wrote:

>Which application needs to walk over ALL files?

How about 'updatedb' for starters, the index-maintainer for the common
*nix command-line utility 'locate'.

I'm pretty sure that os.walk( ) deals with symbolic links (by not
visiting them) and ' /proc' type complexities by not doing anything to
walked directories that '/proc' type entries cannot deal with. I think
(no sarcasm intended) the point of offering a directory-like interface
to '/proc' was so one can perform directory-like operations on it.

--

Jonathan Hartley
tartley@tartley .com
+44 7737 062 225

[Georg Brandl]

Re: file system iteration

rick wrote:

Georg Brandl wrote:
>

>Which application needs to walk over ALL files? Normally, you just have a
>starting path and walk over everything under it.

>
Searching for a file by name. Scanning for viruses. Etc. There are lots
of legitimate reason to walk all paths from a central starting point, no???

Yes. Still, the user may not want to scan all files, or exclude non-locally
mounted filesystem etc.

So you'll always have to give the user control over where to start, and
therefore there's no problem in letting him choose which drives he wants
to search on.

Georg

[Georg Brandl]

Re: file system iteration

Jonathan Hartley wrote:

Georg Brandl wrote:

>Which application needs to walk over ALL files?

>
How about 'updatedb' for starters, the index-maintainer for the common
*nix command-line utility 'locate'.
>
I'm pretty sure that os.walk( ) deals with symbolic links (by not
visiting them) and ' /proc' type complexities by not doing anything to
walked directories that '/proc' type entries cannot deal with. I think
(no sarcasm intended) the point of offering a directory-like interface
to '/proc' was so one can perform directory-like operations on it.

Sure, and I don't say that this is not useful.

But for all applications mentioned in the thread (virus scanning, searching
for a file by name, updating the locate db), including /proc is not very
useful, to say the least.

Georg

[Duncan Booth]

Re: file system iteration

rick <ath-admin@vt.eduwro te:

Georg Brandl wrote:
>

>Which application needs to walk over ALL files? Normally, you just
>have a starting path and walk over everything under it.

>
Searching for a file by name. Scanning for viruses. Etc. There are
lots of legitimate reason to walk all paths from a central starting
point, no???

Personally I'd get pretty annoyed if my virus scanner started gratuitously
scanning network drives and CD's.