yet another noob question

[Jason Nordwick]

Re: yet another noob question

better (sorry, still learning Python):

def cross(x,y): return [a+b for a in x for y in b]

Jason Nordwick wrote:

Or without filter:
>
from operator import add
def pr(x): print x
def cross(x,y): return reduce(add, [[a+b for b in y] for a in x])
x=map(pr, reduce(cross, [map(str,range(1 ,6))]*5))
>
>
>
mike_wilson1333 wrote:

>I would like to generate every unique combination of numbers 1-5 in a 5
>digit number and follow each combo with a newline. So i'm looking at
>generating combinations such as: (12345) , (12235), (55554) and so on.
>What would be the best way to do this? So, basically i'm looking for a
>list of all combinations of 1-5 in a 5 digit unique number. Also, when
>I send the list to print there can't be any duplicates of the combos.
>>
>>
> Thanks,
>>
> Mike
>>

>

[Stargaming]

Re: yet another noob question

Jason Nordwick schrieb:

Or without filter:
>
from operator import add
def pr(x): print x
def cross(x,y): return reduce(add, [[a+b for b in y] for a in x])
x=map(pr, reduce(cross, [map(str,range(1 ,6))]*5))
>

[...]

reduce(add, list) is the same as sum(list) and is only half as fast as sum:

>>from timeit import Timer
>>sum(Timer("su m(range(500))") .repeat(200, 1000))/200

0.0556937865007 98058

>>sum(Timer("re duce(add, range(500))", "from operator import

add").repeat(20 0, 1000))/200
0.1082086103122 0445

>>>

Also note that reduce will be removed in Python 3000.

[bearophileHUGS@lycos.com]

Re: yet another noob question

Stargaming:

Also note that reduce will be removed in Python 3000.

Then let's use it until it lasts! :-)

bearophile

[Gerard Flanagan]

Re: yet another noob question


mike_wilson1333 wrote:

I would like to generate every unique combination of numbers 1-5 in a 5
digit number and follow each combo with a newline. So i'm looking at
generating combinations such as: (12345) , (12235), (55554) and so on.
What would be the best way to do this? So, basically i'm looking for a
list of all combinations of 1-5 in a 5 digit unique number. Also, when
I send the list to print there can't be any duplicates of the combos.
>
>
Thanks,
>
Mike

Hi Mike

mod5 = ['1','2','3','4' ,'5']

X = [ ''.join([a,b,c,d,e])
for a in mod5
for b in mod5
for c in mod5
for d in mod5
for e in mod5 ]

assert len(X) == 5**5
assert len(set(X)) == 5**5 #no duplicates


Gerard

[bearophileHUGS@lycos.com]

Re: yet another noob question

Gerard Flanagan:

mod5 = ['1','2','3','4' ,'5']
X = [ ''.join([a,b,c,d,e])
for a in mod5
for b in mod5
for c in mod5
for d in mod5
for e in mod5 ]

A modified version of your one is the faster so far:

v = "12345"
r = [a+b+c+d+e for a in v for b in v for c in v for d in v for e in v]

Bye,
bearophile

[Jason Nordwick]

Re: yet another noob question

Somehow my other response to the list got lost. I'm still learning Python, but this seems much better than my first attempt:

def pr(x): print x
def cross(x,y): return [a+b for a in x for b in y]
x=map(pr, reduce(cross, [map(str,range(1 ,6))]*5))

-j

Stargaming wrote:

Jason Nordwick schrieb:

>Or without filter:
>>
>from operator import add
>def pr(x): print x
>def cross(x,y): return reduce(add, [[a+b for b in y] for a in x])
>x=map(pr, reduce(cross, [map(str,range(1 ,6))]*5))
>>

[...]
>
reduce(add, list) is the same as sum(list) and is only half as fast as sum:

>>from timeit import Timer
>>sum(Timer("su m(range(500))") .repeat(200, 1000))/200

0.0556937865007 98058

>>sum(Timer("re duce(add, range(500))", "from operator import

add").repeat(20 0, 1000))/200
0.1082086103122 0445

>>>

>
Also note that reduce will be removed in Python 3000.

[Jason Nordwick]

Re: yet another noob question



Stargaming wrote:

>
Also note that reduce will be removed in Python 3000.

What will replace it?

-j

[bearophileHUGS@lycos.com]

Re: yet another noob question

Jason Nordwick:

Stargaming wrote:

Also note that reduce will be removed in Python 3000.

What will replace it?

Nothing, I presume. You will have to write a function to find another
way to solve problems.

Bye,
bearophile

[Jason Nordwick]

Re: yet another noob question

*mouth agape*

Wow. That really sucks. I make extensive use of reduce. It seems to run more than twice as fast as a for loop.

>>t = Timer('bino.red uceadd(bino.bit s)', 'import bino')
>>s = Timer('bino.loo padd(bino.bits) ', 'import bino')
>>t.timeit(10 )

1.2373670396656 564

>>s.timeit(10 )

2.6450051612705 039

>>t.timeit(10 0)

11.312374896809 501

>>s.timeit(10 0)

25.817132345032 689

where

bits = map(lambda x:randint(0,1), xrange(1000000) )

def reduceadd(v):
return reduce(add, v)

def loopadd(v):
sum = 0
for x in v:
sum = add(sum, x)
return sum



(Yes, I know there are better ways to sum up a list, but I just wanted to test the performance of the loop against reduce. In most of my reduce usages, the function passed to reduce is much more complex.)
-j


bearophileHUGS@ lycos.com wrote:

Jason Nordwick:

>Stargaming wrote:

>>Also note that reduce will be removed in Python 3000.

>What will replace it?

>
Nothing, I presume. You will have to write a function to find another
way to solve problems.
>
Bye,
bearophile
>

[AlbaClause]

Re: yet another noob question

mike_wilson1333 wrote:

I would like to generate every unique combination of numbers 1-5 in a 5
digit number and follow each combo with a newline. So i'm looking at
generating combinations such as: (12345) , (12235), (55554) and so on.
What would be the best way to do this? So, basically i'm looking for a
list of all combinations of 1-5 in a 5 digit unique number. Also, when
I send the list to print there can't be any duplicates of the combos.
>
>
Thanks,
>
Mike

This is one of those questions that reminded me that I don't know Python.
I launched my Python interpreter, and forgot how to even print a string.
Duh!

Oh well...

--
--
There are several things that I will never be:
* I will never be attracted to females.
* I will never enjoy the company of others.
Exactly how these realities bode for my enemy, is not of my concern.

[Stargaming]

Re: yet another noob question

Jason Nordwick wrote:
<benchmark here>

>
bearophileHUGS@ lycos.com wrote:
>

>Jason Nordwick:
>>

>>Stargaming wrote:
>>>
>>>Also note that reduce will be removed in Python 3000.
>>>
>>What will replace it?

>>
>>
>Nothing, I presume. You will have to write a function to find another
>way to solve problems.
>>
>Bye,
>bearophile
>>

>

Well, *perhaps* there will be a new function product() -- and there we
go having replaced every arithmetical use of reduce() because Guido
cannot read them. Nice.

Read on: "The fate of reduce() in Python 3000"
http://www.artima.com/weblogs/viewpost.jsp?thread=98196 -- found on "PEP
3100 -- Miscellaneous Python 3.0 Plans"

http://www.python.org/dev/peps/pep-3100/#id63

Regards,
Stargaming
--

[starGaming@gmail.com]

Re: yet another noob question


Jason Nordwick wrote:

*mouth agape*
>
Wow. That really sucks. I make extensive use of reduce. It seems to run more than twice as fast as a for loop.
>

>t = Timer('bino.red uceadd(bino.bit s)', 'import bino')
>s = Timer('bino.loo padd(bino.bits) ', 'import bino')
>t.timeit(10)

1.2373670396656 564

>s.timeit(10)

2.6450051612705 039

>t.timeit(100 )

11.312374896809 501

>s.timeit(100 )

25.817132345032 689
>
where
>
bits = map(lambda x:randint(0,1), xrange(1000000) )
>
def reduceadd(v):
return reduce(add, v)
>
def loopadd(v):
sum = 0
for x in v:
sum = add(sum, x)
return sum
>
>
>
(Yes, I know there are better ways to sum up a list, but I just wanted to test the performance of the loop against reduce. In most of my reduce usages, the function passed to reduce is much more complex.)

[...]

Assuming Guido van Rossum is right and reduce() is only useful for
arithmetic expressions (could you show an example where you use it
else?), you could take advantage of more "built-ins".
To spoiler the result of this, the loop-variant is slightly (in this
test: 2ms) faster.
The results on my amd64 3k+ were:
Reduce: 0.5686828074520 .56633643382
For-loop: 0.56633643382

#!/usr/bin/env python
#
#

from random import randint
from timeit import Timer
from operator import add

def bits():
return [randint(0,1) for x in xrange(1000)]
# use def & list comprehension since lambda/map will be removed in
Python 3.0, too


print bits()[0:5]
print bits()[0:5] # it's working.

def reducefunc(x):
return reduce(add,x)
def loopfunc(x):
y = 0
for i in x: y += i
return y

x = bits()
print reducefunc(x)
print loopfunc(x) # both give proper output

print sum(Timer("redu cefunc(bits())" , "from __main__ import bits,
reducefunc").re peat(20,100))/20
print sum(Timer("loop func(bits())", "from __main__ import bits,
loopfunc").repe at(20,100))/20

[Jason Nordwick]

Re: yet another noob question

I use reduce to also do indexing, hashing with upsert semantics of lists of key-value pairs, transitioning through a state table, etc...

Somebody else pointed out to me how odd it is of Python to be ditching reduce when Guido van Rossum was hired by Google, and Google is literally built on map and reduce (http://en.wikipedia.org/wiki/Mapreduce).

Here is a code fragment I pulled from what I am currently working on (with a little modification to make things simple):

First build a tree:

>>def byn(a,n): return [[a[i] for i in x] for x in zip(*[range(y,len(a), n) for y in range(n)])]

....

>>def whilep(pred,f,x ):

.... while pred(x): x = f(x)
.... return x
....

>>t = whilep(lambda x:1<len(x), lambda x:byn(x,2), map(chr,range(9 7,97+16)))
>>>
>>t

[[[[['a', 'b'], ['c', 'd']], [['e', 'f'], ['g', 'h']]], [[['i', 'j'], ['k', 'l']], [['m', 'n'], ['o', 'p']]]]]

Now if I had the list [0,1,0,0,1] and needed to find what letter it encodes:

>>t[0][1][0][0][1]

'j'

But sometimes, I want to be able to index branches or the leaves are at different depths, so I essentially have a list of indexes to follow:

>>def indexDepth(a,i) : return reduce(lambda x,y: x[y], i, a)

....

>>indexDepth( t,[0,1,0,0,1])

'j'


Also, I completely understand your timings, but you are not timing reduce against a hand coded loop, but instead the operator '+' against the function add, as the function symbol lookup and call seem to have a heavy price. Reduce was one of the nice ways to eliminate some of those lookups.

-j


starGaming@gmai l.com wrote:

Jason Nordwick wrote:

>*mouth agape*
>>
>Wow. That really sucks. I make extensive use of reduce. It seems to run more than twice as fast as a for loop.
>>

>>>>t = Timer('bino.red uceadd(bino.bit s)', 'import bino')
>>>>s = Timer('bino.loo padd(bino.bits) ', 'import bino')
>>>>t.timeit(10 )

>1.237367039665 6564

>>>>s.timeit(10 )

>2.645005161270 5039

>>>>t.timeit(10 0)

>11.31237489680 9501

>>>>s.timeit(10 0)

>25.81713234503 2689
>>
>where
>>
>bits = map(lambda x:randint(0,1), xrange(1000000) )
>>
>def reduceadd(v):
> return reduce(add, v)
>>
>def loopadd(v):
> sum = 0
> for x in v:
> sum = add(sum, x)
> return sum
>>
>>
>>
>(Yes, I know there are better ways to sum up a list, but I just wanted to test the performance of the loop against reduce. In most of my reduce usages, the function passed to reduce is much more complex.)

[...]
>
Assuming Guido van Rossum is right and reduce() is only useful for
arithmetic expressions (could you show an example where you use it
else?), you could take advantage of more "built-ins".
To spoiler the result of this, the loop-variant is slightly (in this
test: 2ms) faster.
The results on my amd64 3k+ were:
Reduce: 0.5686828074520 .56633643382
For-loop: 0.56633643382
>
#!/usr/bin/env python
#
#
>
from random import randint
from timeit import Timer
from operator import add
>
def bits():
return [randint(0,1) for x in xrange(1000)]
# use def & list comprehension since lambda/map will be removed in
Python 3.0, too
>
>
print bits()[0:5]
print bits()[0:5] # it's working.
>
def reducefunc(x):
return reduce(add,x)
def loopfunc(x):
y = 0
for i in x: y += i
return y
>
x = bits()
print reducefunc(x)
print loopfunc(x) # both give proper output
>
print sum(Timer("redu cefunc(bits())" , "from __main__ import bits,
reducefunc").re peat(20,100))/20
print sum(Timer("loop func(bits())", "from __main__ import bits,
loopfunc").repe at(20,100))/20
>

[Steve Holden]

Re: yet another noob question

Jason Nordwick wrote:

I use reduce to also do indexing, hashing with upsert semantics of lists of key-value pairs, transitioning through a state table, etc...
>
Somebody else pointed out to me how odd it is of Python to be ditching reduce when Guido van Rossum was hired by Google, and Google is literally built on map and reduce (http://en.wikipedia.org/wiki/Mapreduce).
>

That seems a bit literal. Just because they use a tool called MapReduce
that doesn't imply that thay chose to implement it with the Python map()
and reduce() functions. It's a distributed application, in case you
hadn't noticed ...

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[Jason Nordwick]

Re: yet another noob question

That isn't what I meant. If there was a a point (and I'm not really sure that I'm even trying to make one), the point was that Google makes heavy use of reduce-like functionality, essentially implementing a distributed reduce across a cluster. From what I hear, they use a lot of Python and hired van Rossum for a reason. It just seems odd (don't read anything into this than mere cocked eyebrows) that the language designer that they hired and obviously have a lot of trust in would decide that reduce was essentially pointless. Google's distributed reduce seems to point in opposite way.

However, if reduce could be rolled into the list comprehension syntax, that would be even better. Or take it that extra step and roll a grouping functionality in there too, then you would have map, reduce, group, and filter all in one construct. You could call it select (joins are merely indexes into other structures).

-j


Steve Holden wrote:

Jason Nordwick wrote:

>I use reduce to also do indexing, hashing with upsert semantics of lists of key-value pairs, transitioning through a state table, etc...
>>
>Somebody else pointed out to me how odd it is of Python to be ditching reduce when Guido van Rossum was hired by Google, and Google is literally built on map and reduce (http://en.wikipedia.org/wiki/Mapreduce).
>>

>
That seems a bit literal. Just because they use a tool called MapReduce
that doesn't imply that thay chose to implement it with the Python map()
and reduce() functions. It's a distributed application, in case you
hadn't noticed ...
>
regards
Steve