random number generator thread safety

[Mike Brown]

I have questions about thread safety in the 'random' module.

When using the random.Random class (be it Mersenne Twister or Wichmann-Hill
based), is it sufficiently thread-safe (preserving entropy and guarding
against attack) to just have each thread work with its own random.Random
instance? Or do I need to wrap my calls to each instance's methods to use
locks? Wichmann-Hill in particular has the warning in its .random()
vulnerability; do I need to make an exception for that case?

What about when using the random.SystemRa ndom class (be it based on
/dev/urandom or CryptGenRandom based)? Should I be doing anything in my code
to ensure that /dev/urandom, for example, is polled serially? (I thought I
read somewhere that this was advisable). SystemRandom claims it has no state,
which may be true at the Python class level, but is that guaranteed to be true
for the underlying RNG?

Also, side note: The doc string for random.SystemRa ndom says "Not available on
all systems (see os.urandom() for details)" ...but os.urandom()'s doc string
provides no details. Actually, os.urandom() and random.SystemRa ndom are
"available" on all systems (they exist and can be called/instantiated), but
os.urandom() can raise a NotImplementedE rror (any time it is called, in
theory), therefore random.SystemRa ndom's methods can fail with a bubbled-up
NotImplementedE rror (on non-win32 OSes, at least). At the very least, this
should be documented. I'm not in a position to submit a patch, but will submit
a documentation bug report if so advised.

Thanks,
Mike

[Raymond Hettinger]

Re: random number generator thread safety

Mike Brown wrote:[color=blue]
> I have questions about thread safety in the 'random' module.
>
> When using the random.Random class (be it Mersenne Twister or Wichmann-Hill
> based), is it sufficiently thread-safe (preserving entropy and guarding
> against attack) to just have each thread work with its own random.Random
> instance? Or do I need to wrap my calls to each instance's methods to use
> locks? Wichmann-Hill in particular has the warning in its .random()
> vulnerability; do I need to make an exception for that case?[/color]

Thread-safety has nothing to do with preserving entropy or guarding
against attack. All of the entropy in an MT sequence is contained in
the seed (upto 624 bytes) and that entropy is preserved through all
subsequent calls.

Create unique instances of MT generators when you want to be able to
reproduce the sequence later.

Nothing in the random module provides cryptographic guarantees. If you
want crypto-strength, then use real encryption. Search SourceForge for
patches that show how to use MD5 and other cryptographic hash functions
as a basis for random number generation.

As far as I'm concerned, there is no reason other than backwards
compatability to use the Wichmann-Hill generator -- the Mersenne
Twister is a much better generator.



Raymond

[Paul Rubin]

Re: random number generator thread safety

"Raymond Hettinger" <python@rcn.com > writes:[color=blue]
> Thread-safety has nothing to do with preserving entropy or guarding
> against attack. All of the entropy in an MT sequence is contained in
> the seed (upto 624 bytes) and that entropy is preserved through all
> subsequent calls.[/color]

I think the concern is that there can be a thread switch after some
intermediate result is computed but before the state is updated. That
would mean two threads can get random numbers that are identical or
anyway correlated. Whether that can happen with Python's MT, I don't
know.
[color=blue]
> Nothing in the random module provides cryptographic guarantees. If you
> want crypto-strength, then use real encryption. Search SourceForge for
> patches that show how to use MD5 and other cryptographic hash functions
> as a basis for random number generation.[/color]

Or use os.urandom.

[Raymond Hettinger]

Re: random number generator thread safety

> > Thread-safety has nothing to do with preserving entropy or guarding[color=blue][color=green]
> > against attack. All of the entropy in an MT sequence is contained in
> > the seed (upto 624 bytes) and that entropy is preserved through all
> > subsequent calls.[/color]
>
> I think the concern is that there can be a thread switch after some
> intermediate result is computed but before the state is updated.[/color]

The concern expressed by the OP was "preserving entropy or guarding
against attack".
[color=blue]
> That
> would mean two threads can get random numbers that are identical or
> anyway correlated. Whether that can happen with Python's MT, I don't
> know.[/color]

It can't. That is what the docs mean when they say that MT is
thread-safe.