how to get files in a directory

[Anand K Rayudu]

Hi all,

I am trying to find a way to get the files recursively in a given
directory,

The following code is failing, can some one please suggest what could be
problem here


from os import walk,join

for root,dir,files in os.walk("E:\myD ir1\MyDir2"):
for i in dir:
for j in files:
fille = root+i+j
print file

Surprisingly if i give os.walk("E:\myD ir1") the above code works, but
not if i have 2 levels of directories.

Thanks & Best Regards,
Anand

[Dan Perl]

Re: how to get files in a directory

First of all, Jeremy already pointed out that your code has several errors.
I'll assume though that your real code is more complicated and you just
tried to show us a simplification of your code.

On the other hand, without the real code we cannot tell why it works for 1
level and it does not work for 2 levels. As someone pointed out already, it
migh be because you are using "\" instead of "\\" for path separators.
Python will accept "\c" as if it was with double backslashes but it will
reject "\x" because that is a special character.

Based on the example that you gave, you are also misunderstandin g os.walk.
os.walk returns 3 values (I'll use the names in your code):
- root - the parent directory at this point in the traversal of the tree
- dir - the list of children directories in root (right below root)
- files - the list of files in root
So the files in the 'files' list are not under dir but under root. So it
does not make sense to loop over 'files' inside the loop over 'dir' or to
join root+i+j. Both your 'i' and your 'j' are just under 'root'.

I'm not sure either what you are trying to do with "root+i+j". You probably
need os.path.join(ro ot,i) and os.path.join(ro ot,j)

Dan

"Anand K Rayudu" <ary@esi-group.com> wrote in message
news:mailman.40 78.1096469771.5 135.python-list@python.org ...[color=blue]
>
> Hi all,
>
> I am trying to find a way to get the files recursively in a given
> directory,
>
> The following code is failing, can some one please suggest what could be
> problem here
>
>
> from os import walk,join
>
> for root,dir,files in os.walk("E:\myD ir1\MyDir2"):
> for i in dir:
> for j in files:
> fille = root+i+j
> print file
>
> Surprisingly if i give os.walk("E:\myD ir1") the above code works, but not
> if i have 2 levels of directories.
>
> Thanks & Best Regards,
> Anand
>
>[/color]

[wes weston]

Re: how to get files in a directory

Anand K Rayudu wrote:[color=blue]
>
> Hi all,
>
> I am trying to find a way to get the files recursively in a given
> directory,
>
> The following code is failing, can some one please suggest what could be
> problem here
>
>
> from os import walk,join
>
> for root,dir,files in os.walk("E:\myD ir1\MyDir2"):
> for i in dir:
> for j in files:
> fille = root+i+j
> print file
>
> Surprisingly if i give os.walk("E:\myD ir1") the above code works, but
> not if i have 2 levels of directories.
>
> Thanks & Best Regards,
> Anand
>
>[/color]

Anand,
Does this help?
wes

def GetFilesWithExt Recursively(pat h,ext_list): #ext_list is like [".h",".cpp" ,...]
dir_list = []
file_list = []
if not os.path.isdir(p ath):
print "\a"
return [],[]

filenames = os.listdir( path )

for fn in filenames:
if path[-1] == "/":
xfn = path + fn
else:
xfn = path + "/" + fn
if( os.path.isdir( xfn ) ):
file_list = file_list + GetFilesWithExt Recursively( xfn ,ext_list)
elif( os.path.isfile( xfn ) ):
front,back = os.path.splitex t(xfn)
if back in ext_list:
file_list.appen d( xfn )
return file_list

[Elbert Lev]

Re: how to get files in a directory

Anand K Rayudu <ary@esi-group.com> wrote in message news:<mailman.4 078.1096469771. 5135.python-list@python.org >...[color=blue]
> Hi all,
>
> I am trying to find a way to get the files recursively in a given
> directory,
>
> The following code is failing, can some one please suggest what could be
> problem here
>
>
> from os import walk,join
>
> for root,dir,files in os.walk("E:\myD ir1\MyDir2"):
> for i in dir:
> for j in files:
> fille = root+i+j
> print file
>
> Surprisingly if i give os.walk("E:\myD ir1") the above code works, but
> not if i have 2 levels of directories.
>
> Thanks & Best Regards,
> Anand[/color]


All is wrong!
0. Wrong import
1. not "E:\myDir1\MyDi r2", but "E:\\myDir1\\My Dir2"
2. dir and files are in the SAME directory. In other words if the
structure is:
myDir1
MyDir2
somefile.py
the first itteration will give an EMPTY dir= [] and files =
['somefile.py']


Here is an example from the manual:

This example displays the number of bytes taken by non-directory files
in each directory under the starting directory, except that it doesn't
look under any CVS subdirectory

import os
from os.path import join, getsize
for root, dirs, files in os.walk('python/Lib/email'):
print root, "consumes",
print sum([getsize(join(ro ot, name)) for name in files]),
print "bytes in", len(files), "non-directory files"
if 'CVS' in dirs:
dirs.remove('CV S') # don't visit CVS directories

so if you want just print full path of the files in E:\myDir1\MyDir 2

for root,dir,files in os.walk("E:\myD ir1\MyDir2"):
for j in files:
fille = os.path.join(ro ot,j)
print fille

Do not touch dir. It is supposed to be used to restrict the search.
If you do not modify dir it walk will progress into root's
subdirectories.

[Michael Kent]

Re: how to get files in a directory

Anand K Rayudu <ary@esi-group.com> wrote in message news:<mailman.4 078.1096469771. 5135.python-list@python.org >...[color=blue]
> I am trying to find a way to get the files recursively in a given
> directory,
>
> The following code is failing, can some one please suggest what could be
> problem here[/color]

You don't really give enough information, but I'll make some
reasonable guesses. My guess is that you have a directory tree
"E:\myDir1\MyDi r2", and that "MyDir2" contains files but no
subdirectories.

[color=blue]
> for root,dir,files in os.walk("E:\myD ir1\MyDir2"):[/color]

Here you are iterating over a directory tree. For each directory in
the tree, os.walk returns the path to the directory, a list of the
subdirectories in that directory, and a list of non-directory files in
that directory.
[color=blue]
> for i in dir:[/color]

Here you are examining each of the subdirectories of the current
directory. You are ignoring any files in that directory. So, for the
top-level directory you gave to os.walk, "E:\myDir1\MyDi r2", you are
ignoring the actual files in that directory, and examining only its
subdirectories. My guess is that there are no subdirectories, so you
get no results.
[color=blue]
> Surprisingly if i give os.walk("E:\myD ir1") the above code works, but
> not if i have 2 levels of directories.[/color]

Unsurprisingly. If there are any files in "myDir1", your code would
ignore them, and only find files in its subdirectory, "MyDir2".

[Lonnie Princehouse]

Re: how to get files in a directory

Backslash begins an escape sequence in a string literal.
You have a few options:

0. Use a forward slash (which will work in Python for DOS paths):
"E:/myDir1/myDir2""

1. Use the escape sequence for backslash: "E:\\myDir1\\my Dir2"

2. Use a raw string: r"E:\myDir1\myD ir2"

Further reference:

http://www.python.org/doc/current/ref/strings.html

[color=blue]
> Surprisingly if i give os.walk("E:\myD ir1") the above code works, but
> not if i have 2 levels of directories.[/color]

Yes, that is kind of surprising. Also, your nested loop...

for i in dir:
for j in files:
fille = root+i+j
print file

.... doesn't make any sense. "dir" is a list of sub-directories in
root, while "files" is a list of non-directory files in root. The
files in "files" are not in the subdirectories.