Creating a List of Empty Lists

**Duncan Booth** · Jul 18 '05, 06:20 AM

Re: Creating a List of Empty Lists

michael@foord.n et (Fuzzyman) wrote in
news:8089854e.0 312040649.4a7f1 715@posting.goo gle.com:
[color=blue]
> I eventually discovered that (as a silly example) :
> a = [[]]*10
> b=-1
> while b < 10:
> b += 1
> a[b] = b
> print a
>
> Produced :
> [ [9], [9], [9]......
>
> Which isn't at all what I intended....... ........
> What is the correct, quick way of doing this (without using a loop and
> appending...) ?[/color]

The recommended way these days is usually:

a = [ [] for i in range(10) ]

That still has a loop and works by appending empty lists, but at least its
just a single expression. Also you can incorporate the next stage of your
initialisation quite easily:

a = [ [b] for b in range(10) ]

--
Duncan Booth duncan@rcp.co.u k
int month(char *p){return(1248 64/((p[0]+p[1]-p[2]&0x1f)+1)%12 )["\5\x8\3"
"\6\7\xb\1\x9\x a\2\0\4"];} // Who said my code was obscure?

**Daniel Dittmar** · Jul 18 '05, 06:20 AM

Re: Creating a List of Empty Lists

Fuzzyman wrote:[color=blue]
> Pythons internal 'pointers' system is certainly causing me a few
> headaches..... When I want to copy the contents of a variable I find
> it impossible to know whether I've copied the contents *or* just
> created a new pointer to the original value....
>
> For example I wanted to initialize a list of empty lists....
>
> a=[ [], [], [], [], [] ][/color]
[...][color=blue]
> What is the correct, quick way of doing this (without using a loop and
> appending...) ?[/color]

[color=blue][color=green][color=darkred]
>>> l = [ [] for i in xrange (3)]
>>> l[/color][/color][/color]
[[], [], []][color=blue][color=green][color=darkred]
>>> l [0].append ('a')
>>> l[/color][/color][/color]
[['a'], [], []]

Daniel

**Anton Vredegoor** · Jul 18 '05, 06:20 AM

Re: Creating a List of Empty Lists

michael@foord.n et (Fuzzyman) wrote:
[color=blue]
>Pythons internal 'pointers' system is certainly causing me a few
>headaches... .. When I want to copy the contents of a variable I find
>it impossible to know whether I've copied the contents *or* just
>created a new pointer to the original value....
>
>For example I wanted to initialize a list of empty lists....
>
>a=[ [], [], [], [], [] ]
>
>I thought there has to be a *really* easy way of doing it - after a
>bit of hacking I discovered that :
>a = [[]]*10 appeared to work
>
>however - using it in my program called bizarre crashes....
>I eventually discovered that (as a silly example) :
>a = [[]]*10
>b=-1
>while b < 10:
> b += 1
> a[b] = b
>print a
>
>Produced :
>[ [9], [9], [9]......
>
>Which isn't at all what I intended....... ........
>What is the correct, quick way of doing this (without using a loop and
>appending... ) ?[/color]

Here it produced an IndexError. After changing "b < 10" into "b < 9"
the code produced:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

I see some other posters already gave you the answer. I'll do
something different and give you the question :-)

n = 4
agents = [[]]*n
print agents
agents[0].append('Smith' )
print agents
neos = map(list,[[]]*n)
print neos
neos[0].append('Neo')
print neos

output is:

[[], [], [], []]
[['Smith'], ['Smith'], ['Smith'], ['Smith']]
[[], [], [], []]
[['Neo'], [], [], []]

The question is:

Why is "Smith" copied to all elements in the matrix?

(or is that another movie :-)

Anton

**Samuel Tardieu** · Jul 18 '05, 06:21 AM

Re: Creating a List of Empty Lists

>>>>> "Anton" == Anton Vredegoor <anton@vredegoo r.doge.nl> writes:

Anton> Why is "Smith" copied to all elements in the matrix?

:)

The construct

[[]] * n

gives you a list with n references to the same list. When you modify
one of the elements, all the references will see the changes.

You can use one of the three (there are more)

([[]]*n)[:]
[[] for _ in range(n)]
map (lambda _: [], range(n))

to get n different copies of [].

Sam
--
Samuel Tardieu -- sam@rfc1149.net -- http://www.rfc1149.net/sam

**Erik Max Francis** · Jul 18 '05, 06:21 AM

Re: Creating a List of Empty Lists

Samuel Tardieu wrote:
[color=blue]
> You can use one of the three (there are more)
>
> ([[]]*n)[:][/color]

This won't work. [:] makes a shallow copy. This will make a different
containing list, but the contained lists will still be identical:
[color=blue][color=green][color=darkred]
>>> s = [[]]*10
>>> s = [[]]*5
>>> t = s[:]
>>> id(s)[/color][/color][/color]
1076779980[color=blue][color=green][color=darkred]
>>> id(t)[/color][/color][/color]
1076780300[color=blue][color=green][color=darkred]
>>> map(lambda (x, y): x is y, zip(s, t))[/color][/color][/color]
[True, True, True, True, True][color=blue][color=green][color=darkred]
>>> s[0].append(2)
>>> s[/color][/color][/color]
[[2], [2], [2], [2], [2]][color=blue][color=green][color=darkred]
>>> t[/color][/color][/color]
[[2], [2], [2], [2], [2]]

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \
\__/ I'm sharing the joy / I'm glowing like sunshine
-- Chante Moore

**Francis Avila** · Jul 18 '05, 06:21 AM

Re: Creating a List of Empty Lists

Fuzzyman wrote in message
<8089854e.03120 40649.4a7f1715@ posting.google. com>...[color=blue]
>Pythons internal 'pointers' system is certainly causing me a few
>headaches... .. When I want to copy the contents of a variable I find
>it impossible to know whether I've copied the contents *or* just
>created a new pointer to the original value....[/color]

You don't need to divine the rules for copy vs. reference: Python NEVER
copies values, ONLY dereferences names, unless you EXPLICITLY ask for a copy
(which Python-the-language doesn't have any special machinery for; you have
to use the copy module or figure out how to copy the object yourself.)

It would help if you stopped thinking in terms of variables and pointers (as
in C) and thought instead in terms of names and objects. Python doesn't
have variables in the C sense, where the variable name stands for its value
ONLY up to the compilation stage, at which time the variable names cease to
exist. Rather, in Python, a "variable" is a name which points to an object.
This behavior is implemented as a key:value pair in a dictionary (i.e.,
__dict__), where the key is a string holding the name and the value is the
object itself. A Python "pointer" is a sort of indexed name, like C arrays;
hence the square-bracket syntax. However, even though the name is strictly
speaking unnamed (i.e., there is no corresponding string object in the
namespace dictionary), yet it is still purely a referent to a real object,
and not a real object itself.

Another way to interpret "pointer" in a Pythonic framework is to say its a
weak-reference: i.e., a reference to an object which does not increase that
object's reference count. However, there is no clear correspondence between
C's "variable/pointer" concepts and what Python does, only analogy.

When no names point to a given object, that object is a candidate for
garbage collection.

All these concepts are illustrated in the following two lines:[color=blue][color=green][color=darkred]
>>>[[]]*2[/color][/color][/color]
[[], []]

This means:

- Create an empty list.
- Create a list which references that empty list.
- Create an integer '2'. (This step may be optimized away by pre-created
integer objects; pre-creating an object is called "interning" it--these
objects are immortal. You can make one with the intern() builtin.)
- Call the __mul__ method of the one-element list, with an argument of 2.
- The __mul__ method creates a new list, and inserts references to its own
elements into this new list--twice. References/names can *only* be copied
(by definition), not pointed to.
- __mul__ then returns the new list.
- This new list is not bound to any name, and so the object is impossible to
access. It is now a candidate for garbage collection.

So, in this one line, four objects were created and destroyed, not five.

The following behavior should now make sense:
[color=blue][color=green][color=darkred]
>>> L = [[]]*2
>>> L[/color][/color][/color]
[[], []][color=blue][color=green][color=darkred]
>>> L[0].append(1)
>>> L[/color][/color][/color]
[[1], [1]]

L[0] and L[1] are two different names, but the object they both point to has
been modified. However:
[color=blue][color=green][color=darkred]
>>> L[0] = 1
>>> L[/color][/color][/color]
[1, [1]]

Here, the name L[0] was made to point to a new object, the integer 1.

The only mutable objects you usually have to worry about are dicts and
lists.

--
Francis Avila

**Fuzzyman** · Jul 18 '05, 06:21 AM

Re: Creating a List of Empty Lists

"Francis Avila" <francisgavila@ yahoo.com> wrote in message news:<vsvd2uq3m f7q5b@corp.supe rnews.com>...[color=blue]
> Fuzzyman wrote in message
> <8089854e.03120 40649.4a7f1715@ posting.google. com>...[color=green]
> >Pythons internal 'pointers' system is certainly causing me a few
> >headaches... .. When I want to copy the contents of a variable I find
> >it impossible to know whether I've copied the contents *or* just
> >created a new pointer to the original value....[/color]
>
>
> You don't need to divine the rules for copy vs. reference: Python NEVER
> copies values, ONLY dereferences names, unless you EXPLICITLY ask for a copy
> (which Python-the-language doesn't have any special machinery for; you have
> to use the copy module or figure out how to copy the object yourself.)
>[/color]
[snip...] Interesting discussion reluctantly snipped.......[color=blue]
>
> The only mutable objects you usually have to worry about are dicts and
> lists.[/color]

Right - because if I do something like :

a = 4
b = a
a = 5
print b

It prints 4... not 5.
In other words - the line b = a creates a name pointing to the object
4, rather than a name pointing to the contents of a.....

I think I see what you mean - since the object 4 is immutable......
the line a = 5 destroys the old name a and creates a new one pointing
to object 5, rather than changing what the name a is pointing to.

Since lists and dictionaries are mutable..... changing the contents
modifies the object rather than destroying the refereence tothe old
one and creating a new one.....

Hmmmmmm... thanks.........

Fuzzy

**Andrew Koenig** · Jul 18 '05, 06:21 AM

Re: Creating a List of Empty Lists

[color=blue]
> Right - because if I do something like :
>
> a = 4
> b = a
> a = 5
> print b
>
> It prints 4... not 5.
> In other words - the line b = a creates a name pointing to the object
> 4, rather than a name pointing to the contents of a.....[/color]

There's no difference between "the object 4" and "the contents of a", so the
"rather than" makes no sense in this context.

After executing

b = a

the names "a" and "b" refer to the same object.

**Robin Munn** · Jul 18 '05, 06:25 AM

Re: Creating a List of Empty Lists

r.e.s. <r.s@XXmindspri ng.com> wrote:[color=blue]
> "David M. Cooke" wrote ...[color=green]
>> "r.e.s." wrote:
>>[color=darkred]
>> > But something's wrong with that explanation, because
>> > of the following:
>> >
>> > >>> a = 'gobble'
>> > >>> a is 'gobble'
>> > True
>> >
>> > Surely `'gobble'` is not the name of an already-existing
>> > object, so I expected exactly the same result as for
>> > `100` and `[]`. What's going on there?[/color]
>>
>> Actually, your second use of 'gobble' *is* an already-existing object.
>> Python 'interns' string constants, i.e., reuses them.[/color]
><snip>
>
> Thanks. That explains it.[/color]

But take note that this behavior is not guaranteed anywhere in the language
reference. As someone else said in this thread, don't count on interning.
Sometimes it will happen, and sometimes it won't:
[color=blue][color=green][color=darkred]
>>> a = 'gobble'
>>> a is gobble[/color][/color][/color]
True[color=blue][color=green][color=darkred]
>>> b = 'foo bar'
>>> b is 'foo bar'[/color][/color][/color]
False

The rule of thumb is that Python interns a string if it's likely to be used
as a name (i.e., only alphanumerics and underscores). The string 'foo bar'
has a space and would be an invalid name, so it wasn't interned.

--
Robin Munn
rmunn@pobox.com

**Robin Becker** · Jul 18 '05, 06:25 AM

Re: Creating a List of Empty Lists

In article <Xns9447973F01B Fduncanrcpcouk@ 127.0.0.1>, Duncan Booth
<duncan@NOSPAMr cp.co.uk> writes
.......[color=blue]
>
>The recommended way these days is usually:
>
> a = [ [] for i in range(10) ]
>
>That still has a loop and works by appending empty lists, but at least its
>just a single expression. Also you can incorporate the next stage of your
>initialisati on quite easily:
>
> a = [ [b] for b in range(10) ]
>[/color]
I seem to remember the fastest way to do this was map(list,n*[[]]) from
a couple of earlier threads, but is that true in 2.3?
--
Robin Becker

**Emile van Sebille** · Jul 18 '05, 06:25 AM

Re: Creating a List of Empty Lists

Robin Munn:[color=blue]
> But take note that this behavior is not guaranteed anywhere in the[/color]
language[color=blue]
> reference. As someone else said in this thread, don't count on[/color]
interning.[color=blue]
> Sometimes it will happen, and sometimes it won't:
>[color=green][color=darkred]
> >>> a = 'gobble'
> >>> a is gobble[/color][/color]
> True[color=green][color=darkred]
> >>> b = 'foo bar'
> >>> b is 'foo bar'[/color][/color]
> False
>
> The rule of thumb is that Python interns a string if it's likely to[/color]
be used[color=blue]
> as a name (i.e., only alphanumerics and underscores). The string[/color]
'foo bar'[color=blue]
> has a space and would be an invalid name, so it wasn't interned.
>[/color]

And don't think you can get around it using intern():

Python 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] on
win32[color=blue][color=green][color=darkred]
>>> b = intern('foo bar')
>>> a = 'foo bar'
>>> a is 'foo bar'[/color][/color][/color]
False[color=blue][color=green][color=darkred]
>>> b is 'foo bar'[/color][/color][/color]
False

That apparent space requirement should really be better documented.
[color=blue][color=green][color=darkred]
>>> b = intern('foo_bar ')
>>> b is 'foo_bar'[/color][/color][/color]
True[color=blue][color=green][color=darkred]
>>> a = 'foo_bar'
>>> a is 'foo_bar'[/color][/color][/color]
True

From the docs on intern():
"Interning strings is useful to gain a little performance on
dictionary lookup "

and while it continues:
"...names used in Python programs are automatically interned, and
the dictionaries used to hold module, class or instance attributes
have interned keys"

It probably should specifically state that only valid identifiers will
be intern()'d and optionally return an error or warning otherwise.

Emile van Sebille
emile@fenx.com

**Skip Montanaro** · Jul 18 '05, 06:25 AM

Re: Creating a List of Empty Lists

Python 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] on win32[color=blue][color=green][color=darkred]
>>> b = intern('foo bar')
>>> a = 'foo bar'
>>> a is 'foo bar'[/color][/color][/color]
False[color=blue][color=green][color=darkred]
>>> b is 'foo bar'[/color][/color][/color]
False

Emile> That apparent space requirement should really be better documented.

The fact that the current implementation of CPython automatically interns
strings which look like identifiers is simply an efficiency consideration.
It's not part of the language definition, so doesn't bear documenting. The
correct way to compare two strings is using '==' (which is independent of
CPython's implementation details), not 'is'.

Skip

**Emile van Sebille** · Jul 18 '05, 06:25 AM

Re: Creating a List of Empty Lists

Skip Montanaro:[color=blue]
> [ quoting me ]
> Python 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit[/color]
(Intel)] on win32[color=blue][color=green][color=darkred]
> >>> b = intern('foo bar')
> >>> a = 'foo bar'
> >>> a is 'foo bar'[/color][/color]
> False[color=green][color=darkred]
> >>> b is 'foo bar'[/color][/color]
> False
>
> Emile> That apparent space requirement should really be better[/color]
documented.[color=blue]
>
> The fact that the current implementation of CPython automatically[/color]
interns[color=blue]
> strings which look like identifiers is simply an efficiency[/color]
consideration.[color=blue]
> It's not part of the language definition, so doesn't bear[/color]
documenting. The[color=blue]
> correct way to compare two strings is using '==' (which is[/color]
independent of[color=blue]
> CPython's implementation details), not 'is'.
>
> Skip
>[/color]

OK. But does intern() intern? I (thought I) only used is to show
that it wasn't intern()'d, and as the documentation holds intern'ing
up as an optimization technique, does it _only_ apply to strings that
look like identifiers? How else might you know?

Emile van Sebille
emile@fenx.com

**Francis Avila** · Jul 18 '05, 06:25 AM

Re: Creating a List of Empty Lists

Skip Montanaro wrote in message ...[color=blue]
> The
>correct way to compare two strings is using '==' (which is independent of
>CPython's implementation details), not 'is'.[/color]

I think it's better to say that the only time you *use* 'is' is when you
*know* the object you're comparing to is a singleton (None, True, or False).

Every other time, use '=='. And don't be deceived by 'is' working
sometimes.
--
Francis Avila

Creating a List of Empty Lists

Creating a List of Empty Lists

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment