memory allocation for Python list

[dmitrey]

hi all,
I have a python list of unknown length, that sequentially grows up via
adding single elements.
Each element has same size in memory (numpy.array of shape 1 x N, N is
known from the very beginning).
As I have mentioned, I don't know final length of the list, but
usually I know a good approximation, for example 400.

So, how can I optimize a code for the sake of calculations speedup?
Currently I just use

myList = []

for i in some_range:
...
myList.append(e lement)
...

Thank you in advance,
Dmitrey

[bearophileHUGS@lycos.com]

Re: memory allocation for Python list

dmitrey:

As I have mentioned, I don't know final length of the list, but
usually I know a good approximation, for example 400.

There is no reserve()-like method, but this is a fast enough operation
you can do at the beginning:

l = [None] * 400

It may speed up your code, but the final resizing may kill your
performance anyway. You can try it. Just using Psyco is probably
better.

Bye,
bearophile

[bearophileHUGS@lycos.com]

Re: memory allocation for Python list

dmitrey:

As I have mentioned, I don't know final length of the list, but
usually I know a good approximation, for example 400.

You may want to use collections.deq ue too, it doesn't produce a Python
list, but it's quite fast in appending (it's a linked list of small
arrays).

Bye,
bearophile