time.clock() going backwards??

**Michiel Sikma** · Aug 25 '06, 04:15 PM

Re: time.clock() going backwards??

Op 25-aug-2006, om 16:13 heeft Giovanni Bajo het volgende geschreven:

Hello,
>
Is it possible this to be a bug in Python itself
(maybe, shooting at the moon, in the conversion between the 64bit
performance
counter and the floating point representation returned by time.clock
()), or
could it be a bug in Windows itself or the mother board drivers
(buf if so,
wouldn't other application start going mad)?
--
Giovanni Bajo

Very interesting! While I couldn't possibly know the answer to this,
I do wonder which version of Python you were using. Is it the
bleeding edge 2.5 release candidate?

Michiel

**K.S.Sreeram** · Aug 25 '06, 04:35 PM

Re: time.clock() going backwards??

Giovanni Bajo wrote:

Hello,

I experimented something very strange, a few days ago. I was debugging an
application at a customer's site, and the problem turned out to be that
time.clock() was going "backwards" , that is it was sometimes (randomically)
returning a floating point value which was "less than" the value returned by
the previous invokation. The computer was a pretty fast one (P4 3Ghz I think,
running Windows XP), and this happened only between very close invokations of
time.clock().

I have triple-verified this, including printing the repr() of the floating
point number and verifying it was really minor than the previous value by a few
microseconds. In other words, I'm absolutely positive that it's not a mistake
on my side, but that time.clock() was really apparently "jumping backward".
This was confusing the hell out of my application, of course, and I just hacked
it so to ignore these bogus reads, and just reading off again. Since the error
was just of a few microseconds, reading time.clock() again produces a number
which was higher than what I had before, and thus OK for my application.

I was wondering if someone had experimented this behaviour before. I tried
googling but to no effect. Is it possible this to be a bug in Python itself
(maybe, shooting at the moon, in the conversion between the 64bit performance
counter and the floating point representation returned by time.clock()), or
could it be a bug in Windows itself or the mother board drivers (buf ifso,
wouldn't other application start going mad)?

From the MSDN docs for QueryPerformanc eCounter:

Remarks:
On a multiprocessor computer, it should not matter which processor is
called. However, you can get different results on different processors
due to bugs in the basic input/output system (BIOS) or the hardware
abstraction layer (HAL). To specify processor affinity for a thread, use
the SetThreadAffini tyMask function.

Are you running on a multi-core or a multi-processor machine?

The best way to handle this is probably to make your application
'tolerant' to such cases.

[sreeram;]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFE7yhsrgn 0plK5qqURAp1sAJ 9mOO0UbZUhjZQsj ew3roLGQHUabACg vwCJ
I8wDMKiuL2xF/dY7d2b1ccA=
=hRd5
-----END PGP SIGNATURE-----

**Terry Reedy** · Aug 25 '06, 07:55 PM

Re: time.clock() going backwards??

"Giovanni Bajo" <noway@sorry.co mwrote in message
news:vkFHg.8252 9$_J1.759243@tw ister2.libero.i t...

Hello,
>
I experimented something very strange, a few days ago. I was debugging an
application at a customer's site, and the problem turned out to be that
time.clock() was going "backwards" , that is it was sometimes
(randomically)
returning a floating point value which was "less than" the value returned
by
the previous invokation. The computer was a pretty fast one (P4 3Ghz I
think,
running Windows XP), and this happened only between very close
invokations of
time.clock().

I seem to remember this being mentioned before on the list, perhaps by Tim
Peters. Perhaps he will chime in.

tjr

**Claudio Grondi** · Aug 25 '06, 08:55 PM

Re: time.clock() going backwards??

Terry Reedy wrote:

"Giovanni Bajo" <noway@sorry.co mwrote in message
news:vkFHg.8252 9$_J1.759243@tw ister2.libero.i t...
>

>>Hello,
>>
>>I experimented something very strange, a few days ago. I was debugging an
>>application at a customer's site, and the problem turned out to be that
>>time.clock( ) was going "backwards" , that is it was sometimes
>>(randomically )
>>returning a floating point value which was "less than" the value returned
>>by
>>the previous invokation. The computer was a pretty fast one (P4 3Ghz I
>>think,
>>running Windows XP), and this happened only between very close
>>invokations of
>>time.clock( ).

>
>
I seem to remember this being mentioned before on the list, perhaps by Tim
Peters. Perhaps he will chime in.
>
tjr

If I remember it right, the cause of such a problem is updating the
clock by accessing a time server over a network. Just any such access
results in adjusting the time a bit and leads eventually to such problems.

Claudio Grondi

**Tim Peters** · Aug 25 '06, 09:15 PM

Re: time.clock() going backwards??

[Giovanni Bajo[

>I experimented something very strange, a few days ago. I was debugging an
>application at a customer's site, and the problem turned out to be that
>time.clock() was going "backwards" , that is it was sometimes
>(randomicall y) returning a floating point value which was "less than" the
>value returned by the previous invokation. The computer was a pretty fast
>one (P4 3Ghz I think, running Windows XP), and this happened only between
>very close invokations of time.clock().

[Terry Reed]

I seem to remember this being mentioned before on the list, perhaps by Tim
Peters. Perhaps he will chime in.

No real need ;-) BIOS or HAL bug on a multiprocessor (or maybe even
hyperthreaded) box is by far the most likely cause (and the only cause
ever identified for sure by people who followed up). Python's C code
slinging QueryPerformanc eCounter is exceedingly simple, and isn't a
suspect because of that. It's on the edge of vague possibility that
Microsoft's compiler generates non-monotonic code for converting
64-bit integer to double: i.e., Python's C code obviously relies on
that if i and j are _int64 satisfying i >= j, then (double)i >=
(double)j. If that type-conversion code /is/ monotonic (which is
almost certainly so), then the only ways the C code can fail are if
the HW counter overflows (in which case time.clock() would "jump" a
huge amount), or if the sequence of values returned by
QueryPerformanc eCounter is itself non-monotonic at times (which is
consistent with known BIOS/HAL bugs).

**Sander Steffann** · Aug 25 '06, 09:35 PM

Re: time.clock() going backwards??

Hi,

>>>I experimented something very strange, a few days ago. I was debugging an
>>>applicatio n at a customer's site, and the problem turned out to be that
>>>time.clock () was going "backwards" , that is it was sometimes
>>>(randomicall y)
>>>returning a floating point value which was "less than" the value returned
>>>by the previous invokation.

>
If I remember it right, the cause of such a problem is updating the
clock by accessing a time server over a network. Just any such access
results in adjusting the time a bit and leads eventually to such problems.

Usualy time is synchronized by making the clock run a little faster or a
little slower until it is in sync with the timeserver. It shouldn't go
backwards... A bad time-sync-program might ofcourse do this, but with a
decent NTP client this shouldn't happen.

- Sander

**skip@pobox.com** · Aug 26 '06, 12:25 AM

Re: time.clock() going backwards??

>I seem to remember this being mentioned before on the list, perhaps by Tim

>Peters. Perhaps he will chime in.

ClaudioIf I remember it right, the cause of such a problem is updating
Claudiothe clock by accessing a time server over a network. Just any
Claudiosuch access results in adjusting the time a bit and leads
Claudioeventual ly to such problems.

Wouldn't that affect time.time (time since the start of the Epoch), not
time.clock (cpu time used by the current process)?

Skip

**Tim Roberts** · Aug 28 '06, 04:45 AM

Re: time.clock() going backwards??

"Tim Peters" <tim.peters@gma il.comwrote:

>[Giovanni Bajo[

>>I experimented something very strange, a few days ago. I was debugging an
>>application at a customer's site, and the problem turned out to be that
>>time.clock( ) was going "backwards" , that is it was sometimes
>>(randomically ) returning a floating point value which was "less than" the
>>value returned by the previous invokation. The computer was a pretty fast
>>one (P4 3Ghz I think, running Windows XP), and this happened only between
>>very close invokations of time.clock().

>
>[Terry Reed]

>I seem to remember this being mentioned before on the list, perhaps by Tim
>Peters. Perhaps he will chime in.

>
>No real need ;-) BIOS or HAL bug on a multiprocessor (or maybe even
>hyperthreade d) box is by far the most likely cause (and the only cause
>ever identified for sure by people who followed up). Python's C code
>slinging QueryPerformanc eCounter is exceedingly simple, and isn't a
>suspect because of that. It's on the edge of vague possibility that
>Microsoft's compiler generates non-monotonic code for converting
>64-bit integer to double:

It is much simpler than that. With a multiprocessor HAL, including on a
dual-core or hyperthreaded system, QueryPerformanc eCounter returns the raw
cycle counter (RDTSC). However, on Windows XP, the operating system does
not synchronize the cycle counters on multiple processors, and they can be
actually be millions of cycles apart.

This was a change from previous systems. On NT4 and Win2000, the operating
actually rewrote the cycle counters on the second (and beyond) processors
to align them to the first processor, so the delta was usually only a dozen
or two cycles. XP does not appear to do that. I think that is a huge
mistake, since it renders QueryPerformanc eCounter non-monotonic.
--
- Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

**Claudio Grondi** · Aug 28 '06, 07:05 AM

Re: time.clock() going backwards??

Tim Roberts wrote:

"Tim Peters" <tim.peters@gma il.comwrote:
>
>

>>[Giovanni Bajo[
>>

>>>>I experimented something very strange, a few days ago. I was debugging an
>>>>applicati on at a customer's site, and the problem turned out to be that
>>>>time.clock( ) was going "backwards" , that is it was sometimes
>>>>(randomical ly) returning a floating point value which was "less than" the
>>>>value returned by the previous invokation. The computer was a pretty fast
>>>>one (P4 3Ghz I think, running Windows XP), and this happened only between
>>>>very close invokations of time.clock().

>>
>>[Terry Reed]
>>

>>>I seem to remember this being mentioned before on the list, perhaps by Tim
>>>Peters. Perhaps he will chime in.

>>
>>No real need ;-) BIOS or HAL bug on a multiprocessor (or maybe even
>>hyperthreaded ) box is by far the most likely cause (and the only cause
>>ever identified for sure by people who followed up). Python's C code
>>slinging QueryPerformanc eCounter is exceedingly simple, and isn't a
>>suspect because of that. It's on the edge of vague possibility that
>>Microsoft's compiler generates non-monotonic code for converting
>>64-bit integer to double:

>
>
It is much simpler than that. With a multiprocessor HAL, including on a
dual-core or hyperthreaded system, QueryPerformanc eCounter returns the raw
cycle counter (RDTSC). However, on Windows XP, the operating system does
not synchronize the cycle counters on multiple processors, and they can be
actually be millions of cycles apart.
>
This was a change from previous systems. On NT4 and Win2000, the operating
actually rewrote the cycle counters on the second (and beyond) processors
to align them to the first processor, so the delta was usually only a dozen
or two cycles. XP does not appear to do that. I think that is a huge
mistake, since it renders QueryPerformanc eCounter non-monotonic.

How does it come, that processors on same mainboard run at different
speeds? Do they have separate clock-pulse generators?

I can remember, that (at least on very old motherboards) the clock-pulse
generator was a separate element and the processor just used it, so I
would expect, that even in case of multiple processors, if there were
only one clock-pulse generator for all of them, they were not be able to
run at different speeds.

Claudio Grondi

**Rob Williscroft** · Aug 28 '06, 02:45 PM

Re: time.clock() going backwards??

Claudio Grondi wrote in news:ecu546$sn9 $1@newsreader2. netcologne.de in
gmane.comp.pyth on.general:

Tim Roberts wrote:

>"Tim Peters" <tim.peters@gma il.comwrote:

>>
>It is much simpler than that. With a multiprocessor HAL, including
>on a dual-core or hyperthreaded system, QueryPerformanc eCounter
>returns the raw cycle counter (RDTSC). However, on Windows XP, the
>operating system does not synchronize the cycle counters on multiple
>processors, and they can be actually be millions of cycles apart.
>>
>This was a change from previous systems. On NT4 and Win2000, the
>operating actually rewrote the cycle counters on the second (and
>beyond) processors to align them to the first processor, so the delta
>was usually only a dozen or two cycles. XP does not appear to do
>that. I think that is a huge mistake, since it renders
>QueryPerforman ceCounter non-monotonic.

>
How does it come, that processors on same mainboard run at different
speeds? Do they have separate clock-pulse generators?

I don't see any claim above that they run at different speeds, only
that the counters are several million cycles apart, IOW running at the
same speed but with different initial values, or more likely starting
at different times.

For processors that run at (say) 2GHz, several million (say 10 million)
represents a difference of 10e6/2e9 = 0.005 seconds between when the
processors were sufficiently powered up to start counting cycles.

Rob.
--

http://www.victim-prime.dsl.pipex.com/

**Claudio Grondi** · Aug 28 '06, 03:25 PM

Re: time.clock() going backwards??

Rob Williscroft wrote:

Claudio Grondi wrote in news:ecu546$sn9 $1@newsreader2. netcologne.de in
gmane.comp.pyth on.general:
>
>

>>Tim Roberts wrote:
>>

>>>"Tim Peters" <tim.peters@gma il.comwrote:

>
>

>>>It is much simpler than that. With a multiprocessor HAL, including
>>>on a dual-core or hyperthreaded system, QueryPerformanc eCounter
>>>returns the raw cycle counter (RDTSC). However, on Windows XP, the
>>>operating system does not synchronize the cycle counters on multiple
>>>processors , and they can be actually be millions of cycles apart.
>>>
>>>This was a change from previous systems. On NT4 and Win2000, the
>>>operating actually rewrote the cycle counters on the second (and
>>>beyond) processors to align them to the first processor, so the delta
>>>was usually only a dozen or two cycles. XP does not appear to do
>>>that. I think that is a huge mistake, since it renders
>>>QueryPerform anceCounter non-monotonic.

>>
>>How does it come, that processors on same mainboard run at different
>>speeds? Do they have separate clock-pulse generators?

>
>
I don't see any claim above that they run at different speeds, only
that the counters are several million cycles apart, IOW running at the
same speed but with different initial values, or more likely starting
at different times.
>
For processors that run at (say) 2GHz, several million (say 10 million)
represents a difference of 10e6/2e9 = 0.005 seconds between when the
processors were sufficiently powered up to start counting cycles.
>
Rob.

If it were so, than why can't the delta of time between the processors
be set to exact zero?
I assume, that it is known how many cycles adjusting the value will
take, so it could be done exactly ... hmmm ...

Claudio Grondi

**Grant Edwards** · Aug 28 '06, 03:45 PM

Re: time.clock() going backwards??

On 2006-08-28, Claudio Grondi <claudio.grondi @freenet.dewrot e:

>I don't see any claim above that they run at different speeds, only
>that the counters are several million cycles apart, IOW running at the
>same speed but with different initial values, or more likely starting
>at different times.
>>
>For processors that run at (say) 2GHz, several million (say 10 million)
>represents a difference of 10e6/2e9 = 0.005 seconds between when the
>processors were sufficiently powered up to start counting cycles.

If it were so, than why can't the delta of time between the processors
be set to exact zero?

This is

I assume, that it is known how many cycles adjusting the value will
take, so it could be done exactly ... hmmm ...
>
Claudio Grondi

--
Grant Edwards grante Yow! LIFE is a
at never-ending INFORMERCIAL!
visi.com

**Fredrik Lundh** · Aug 28 '06, 03:45 PM

Re: time.clock() going backwards??

Claudio Grondi wrote:

If it were so, than why can't the delta of time between the processors
be set to exact zero?

someone just wrote:

>>>This was a change from previous systems. On NT4 and Win2000, the
>>>operating actually rewrote the cycle counters on the second (and
>>>beyond) processors to align them to the first processor, so the delta
>>>was usually only a dozen or two cycles. XP does not appear to do
>>>that. I think that is a huge mistake, since it renders
>>>QueryPerform anceCounter non-monotonic.

so if you really want to know, you may have to ask microsoft.

</F>

**Fredrik Lundh** · Aug 28 '06, 03:55 PM

Re: time.clock() going backwards??

so if you really want to know, you may have to ask microsoft.

footnote: judging from

Microsoft Support

http://support.microsoft.com/?id=896256

Microsoft Support is here to help you with Microsoft products. Find how-to articles, videos, and training for Microsoft Copilot, Microsoft 365, Windows 11, Surface, and more.

this might be related to advanced power management features in XP. that
note also mentions a hotfix for this.

</F>