Python 2.5.1 broken os.stat module

[=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=]

Re: Python 2.5.1 broken os.stat module

It appears that you may have missed part of my tests. Sorry it was such a

long reply but I was trying to provide a lot of detail so that others had a
clear understanding of what was going on.

Please understand that it is *extremely* tedious to follow your
messages. It would have been much better if they had been short and
to the point.

Changing the timezone will defintely change the "textual representation" of
all timestamps just as you say (even though the actual UTC value has *not*
changed), however, when DST starts/ends the "textual representation" of the
timestamps on some files WILL ALSO CHANGE when the automatically adjust for
DST setting is turned on.

Right, and that's because switching to DST means that YOU MOVE TO A
DIFFERENT TIME ZONE.

EST != EDT.

While it is true that I did change the timezone in the first part of the
test (to establish the baseline), in the second part of the tests where the
difference you are referring to occured, the timezone was *not* changed, the
only thng that occured was that DST started.

And that meant that the time zone *did* change.

When the date progressed to a point after DST had started Windows now
reports the timestamp on that *unmodified* file now is 01/01/2007 08:00 PM.
>
I did not change the timezone, the only thing that occurred was DST started.

So you did change the timezone.

Over the years this issue has caused all sorts of problems for backup
programs and CVS (greatly detailed in the article I provided the link for).

That is very hard to believe. CVS and backup programs use the UTC time
stamp, and completely ignore the timezone. So when you agree that the
UTC time stamp did not change, CVS and the backup programs will work
just fine.

There was a long-standing problem with changing time-stamps ON FAT.
On a FAT file system, the actual UTC time stamps change when the
timezone changes (including the change to and from DST). *That*
was a big problem for backup programs and CVS, and is now fixed
with NTFS.

You mixed up my tests, in that case as shown above the timezone did *not*
change, the only thing that changed was that DST started and the file was
created during a time when DST was not in effect.

Right, so the timezone did change.

Regards,
Martin

[Joe Salmeri]

Re: Python 2.5.1 broken os.stat module

| You are misinterpreting what you are seeing. Windows is not claiming
| that the modification time changes when DST starts. Instead, it is
| claiming that the *same* time now has a *different* textual
| representation, because the computer now has moved to a different
| time zone.
>
Joe, read the last sentence again. Most of your reply shows that you have
missed this basic point.

My reply specifically indicated that I was referring to the textual
presentation that Windows is showing.

I understand that a DST change and a timezone change produce the same
results on the offset from GMT but I don't consider them the same thing.

As I attempted to describe the issue with the timestamps changing when DST
starts/ends is not my *opinion* it is behavior that Microsoft (and others)
have documented (and I agree with).

Consider this situation from the end user perspective.

I create a file on 01/02/2007 at 07:00 PM

Windows shows the last write time as 01/02/2007 07:00 PM

I never make any changes to the file.

Fast forward in time and it is now 06/03/2007.

When I do a directory on the file Windows now shows the last write time as
01/02/2007 08:00 PM.

I understand WHY this is happening, but like many others do not agree with
the behavior.

The reason Windows does this is because although my timeszone did not change
(as far as I am concerned) DST has started (a timezone change as far as you
are concerned) so it now shows the time as 1 hour more than it used to
because the GMT offset has changed.

From the end user perspective it appears this file has now been modified an
hour later than it showed it was modifed before DST started.

Now let's look at another common user activity.

I watch this TV show on 01/02/2007 at 07:00 PM. It comes on every day at
07:00 PM

Fast forward in time it is now 06/03/2007.

When I watch that same show it comes on at 07:00 PM (even though DST has
started) and if I look the TV listings it still shows that the older show on
01/02/2007 came on at 07:00 PM. It did not show the older show now on as
08:00 PM which is what Windows is doing with the timestamp.

When you compare those two situations it does not make sense that Windows
shows a different timestamp.

I would recommend reading the links in the articles where Microsoft
documents the situation and developers for the CVS project discuss having to
deal with the problems.

Here's a clip from the article:

This behaviour is responsible for a flood of questions to the
various support lists for CVS, following the first Sunday in April and the
last Sunday in October, with
scores of people complaining that CVS now reports erroneously that
their files have been modified. This is commonly known as the "red file bug"
because the
WinCVS shell uses red icons to indicate modified files.

[Joe Salmeri]

Re: Python 2.5.1 broken os.stat module

Please understand that it is *extremely* tedious to follow your

messages. It would have been much better if they had been short and
to the point.

Sometimes it seems that it is impossible to please people.

When messages are short then people complain that they did not get
sufficient details, or examples.

I provide details and tests so that anyone could easily attempt to duplicate
exactly what I said was happening and you complain that they are too long.

Baring any typo mistakes on my part, performing the tests should produce
the exact results that I stated were happening.

That is very hard to believe. CVS and backup programs use the UTC time
stamp, and completely ignore the timezone. So when you agree that the
UTC time stamp did not change, CVS and the backup programs will work
just fine.

The CVS problems were mentioned in the article I linked to and were
discussed by people involved in CVS development. Whether you choose to
believe all those people or not is up to you.

See my reply to Terry, from the end user perspective, the Windows behavior
does not make sense.

Short enough?

[Joe Salmeri]

Re: Python 2.5.1 broken os.stat module


""Martin v. Löwis"" <martin@v.loewi s.dewrote in message
news:4662F98C.8 070709@v.loewis .de...

>It appears that you may have missed part of my tests. Sorry it was such
>a
>long reply but I was trying to provide a lot of detail so that others had
>a
>clear understanding of what was going on.

>
Please understand that it is *extremely* tedious to follow your
messages. It would have been much better if they had been short and
to the point.
>

>Changing the timezone will defintely change the "textual representation"
>of
>all timestamps just as you say (even though the actual UTC value has
>*not*
>changed), however, when DST starts/ends the "textual representation" of
>the
>timestamps on some files WILL ALSO CHANGE when the automatically adjust
>for
>DST setting is turned on.

>
Right, and that's because switching to DST means that YOU MOVE TO A
DIFFERENT TIME ZONE.
>
EST != EDT.
>

>While it is true that I did change the timezone in the first part of the
>test (to establish the baseline), in the second part of the tests where
>the
>difference you are referring to occured, the timezone was *not* changed,
>the
>only thng that occured was that DST started.

>
And that meant that the time zone *did* change.
>

>When the date progressed to a point after DST had started Windows now
>reports the timestamp on that *unmodified* file now is 01/01/2007 08:00
>PM.
>>
>I did not change the timezone, the only thing that occurred was DST
>started.

>
So you did change the timezone.
>

>Over the years this issue has caused all sorts of problems for backup
>programs and CVS (greatly detailed in the article I provided the link
>for).

>
That is very hard to believe. CVS and backup programs use the UTC time
stamp, and completely ignore the timezone. So when you agree that the
UTC time stamp did not change, CVS and the backup programs will work
just fine.
>
There was a long-standing problem with changing time-stamps ON FAT.
On a FAT file system, the actual UTC time stamps change when the
timezone changes (including the change to and from DST). *That*
was a big problem for backup programs and CVS, and is now fixed
with NTFS.
>

>You mixed up my tests, in that case as shown above the timezone did *not*
>change, the only thing that changed was that DST started and the file was
>created during a time when DST was not in effect.

>
Right, so the timezone did change.
>
Regards,
Martin

[Joe Salmeri]

Re: Python 2.5.1 broken os.stat module

There is a conflict with the answers that you and Terry have given.

The original issue I raised was that I Python 2.5.1 and Windows did not have
the same textual represenation for a localize date.

You have stood true to the statements that Python 2.5.1 is correct and that
previous versions were wrong.

I agree with you that Python 2.5.1 is correct which means that Windows is
showing the incorrect dates.

Since that point in time you and Terry have disagreed with my statements
regarding timezones and DST.

All dates in my below comments are the textual representation of those dates
from Windows or from Python 2.5.1

For the test file created at 01/02/2007 07:00 PM.

From that date up until just before DST starts both Windows AND Python 2.5.1
will report that date as 01/02/2007 07:00 PM.

Once DST starts Python 2.5.1 will still report that date as 01/02/2007 07:00
PM, however Windows will now report that
date as 01/02/2007 at 08:00 PM.

I agree and believe that Python 2.5.1 is reporting the correct textual
representation of that timestamp, however, using
*your* logic that a DST change *is a timezone change* that would mean that
Python 2.5.1 is wrong and should be reporting the
date 1 hour later just as Windows is reporting.

You cannot have it both ways.

If a DST change is the same as a timezone change then Python 2.5.1 should be
reporting the dates that Windows reports.
If a DST change is NOT the same as a timezone change then Python 2.5.1
should be reporting the dates that it currently is.

[Joe Salmeri]

Re: Python 2.5.1 broken os.stat module

Perspective is often the source of problems with communication.

You view timezones and DST as offsets from GMT. I understand and respect
that perspective.

When I think of timezones and DST I think of the timezone setting and the
DST setting in Windows.

These settings are two separate settings in Windows that can be configured
individually and independent of each other.

When I think of a time I think of

somedate, sometime, Eastern Standard Time (EST)

OR

somedate, sometime, Eastern Standard Time Daylight Saving Time (EST DST)

If you view things from my perspective then I would hope my comments become
more clear.

Last night I was talking to my wife about this discussion (I need to get a
life :-)) and she came up with an analogy
that I like even better than my TV analogy.

Suppose she has a baby 01/02/2007 07:00 PM.

Today, a time after DST has started for this region someone asks her when
her baby was born.

Here answer would be 01/02/2007 07:00 PM.

If we ask Windows that same question it would be 01/02/2007 08:00 PM.

Please see my other response to Martin because using your logic of "a DST
change is a timezone change" that would mean that the textual representation
of dates by Python 2.5.1 is WRONG. I don't believe that to be true but you
cannot have it both ways, if a DST change is a timezone change then Python
should show the same dates as Windows does, if it is not the same then
Python should show the dates that it is.

[=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=]

Re: Python 2.5.1 broken os.stat module

Joe Salmeri schrieb:

There is a conflict with the answers that you and Terry have given.

No, there isn't. See some of my earlier replies: both windows and python
are correct, despite the fact they give different results.

When Windows renders a time stamp, it always uses the current UTC
offset. That is the only way to guarantee that a time stamp uniquely
identifies points in time: i.e. two files with the same time stamp
where also modified at the same time.

OTOH, Python's localtime function tries to apply the UTC offset that
was likely in force at the time indicated by the time stamp. I say
likely, because it probably doesn't know exactly when DST started in
what year (I might be wrong - Python defers to some Microsoft API
to perform the conversion). While this apparently better matches what
you expect, it also is ambiguous: two files authored at 2:15 a.m.
might still be an hour apart, if that is on the day when DST switches
to standard time.

The original issue I raised was that I Python 2.5.1 and Windows did not have
the same textual represenation for a localize date.
>
You have stood true to the statements that Python 2.5.1 is correct and that
previous versions were wrong.

Yes - but not because localtime reported the wrong value, but because
stat() didn't report UTC times.

I agree with you that Python 2.5.1 is correct which means that Windows is
showing the incorrect dates.

And yet, I claim they are both correct - just different.

Since that point in time you and Terry have disagreed with my statements
regarding timezones and DST.

I don't see how. Terry claims the Windows result is correct, and so do I.

From that date up until just before DST starts both Windows AND Python 2.5.1
will report that date as 01/02/2007 07:00 PM.

EST, that is.

Once DST starts Python 2.5.1 will still report that date as 01/02/2007 07:00
PM, however Windows will now report that
date as 01/02/2007 at 08:00 PM.

EDT.

01/02/2007 7:00 PM EST == 01/02/2007 8:00 PM EDT

So they print indeed the same output, even though the text is different.

I agree and believe that Python 2.5.1 is reporting the correct textual
representation of that timestamp, however, using
*your* logic that a DST change *is a timezone change* that would mean that
Python 2.5.1 is wrong and should be reporting the
date 1 hour later just as Windows is reporting.

No. Python prints the stamp according to the time zone that was in force
when the file was created. Of course, it can't be sure (since the
computer may have lived in an entirely different time zone at the
time), but it assumes that the computer has not moved, and it assumes
to know when DST started in 2007 (which it probably does know correctly;
I wouldn't be so sure about 1981).

You cannot have it both ways.

Sure can.

If a DST change is the same as a timezone change then Python 2.5.1 should be
reporting the dates that Windows reports.

You are misunderstandin g what localtime() does.

If a DST change is NOT the same as a timezone change then Python 2.5.1
should be reporting the dates that it currently is.

You are misunderstandin g that Windows does *not* use localtime(),
but windows_localti me(). Both consider the current timezone, but in
a different way.

Regards,
Martin

[Joe Salmeri]

Re: Python 2.5.1 broken os.stat module

I have tried (unsuccessfully ) to get you to view things from the end user
perspective.

I wish that you would consider looking at what the end user sees because
that is what really matters.

Without end users we would not need to develop software would we?

This entire conversation was VERY nicely summarized (by you) with the
following statement:

01/02/2007 7:00 PM EST == 01/02/2007 8:00 PM EDT

I agree with that statement 100%.

Python displays the date as 01/02/2007 7:00 PM EST
Windows displays the date as 01/02/2007 8:00 PM EDT.

Python and Windows are BOTH correct!

However, there is a problem. Windows does *not* display the date like that.

Windows displays the date as 01/02/2007 08:00 PM.

Windows does *not* give *any* indication when it made the switch from EST to
EDT!!!

All the end user sees is that the file said it was last changed at 07:00 PM
one day and then the next
day after DST started it now says the file was last changed at 08:00 PM.

The user is left scratching their head.

I prefer not to leave my end users in that situation.

From Python I could resolve this using strftime with %Z to show the timezone
/ EST / EDT but
from Windows you cannot get it to show you the timezone / DST value.

The only place in Windows that I have found that indicates EST / EDT is in
Control Panel
Date and Time Properties.

Since the dir command and Explorer do not display the timezone / DST
information the end
experience is confusion when the EST / EDT switch occurs.

You understand what happend, I understand what happen, but the typical user
does not.

[Terry Reedy]

Re: Python 2.5.1 broken os.stat module


"Joe Salmeri" <JoeSalmeri@hot mail.comwrote in message
news:D_WdnYzFdu e9O_nbnZ2dnUVZ_ uKknZ2d@comcast .com...
|I have tried (unsuccessfully ) to get you to view things from the end user
| perspective.

But that perspective is not directly relevant to *your* topic line. When
you make a claim that os.stat is 'broken' and bugged, you are making a
claim about the *programmer* experience -- in particular, experiencing a
discrepancy between performance and reasonable expectation based on the
manuals. Martin and I are both concerned about that particular claim
(which now appears to be resolved in the negative).

>From the developer point of view, you, the programmer, are the user. What

you present to your users (who may or may not be the 'end' users) is your
responsibility.

| However, there is a problem. Windows does *not* display the date like
that.

Python developers cannot fix that. Besides which, Python is a
cross-platform language, originally developed on *nix.

| The user is left scratching their head.
|
| I prefer not to leave my end users in that situation.

Then do not. I am sure Martin also agrees. But if you want to discuss
writing applications for end users, perhaps you should start a new thread
with a different subject line. It would attract different respondents.

Terry Jan Reedy

[Joe Salmeri]

Re: Python 2.5.1 broken os.stat module

But that perspective is not directly relevant to *your* topic line. When

you make a claim that os.stat is 'broken' and bugged, you are making a
claim about the *programmer* experience -- in particular, experiencing a
discrepancy between performance and reasonable expectation based on the
manuals. Martin and I are both concerned about that particular claim
(which now appears to be resolved in the negative).

The subject line reflects what *originally* appeared to be the source of the
problem since a comparison of the results did not match previous Python
versions.

For other developers that run into this difference (which did not occur
before) I believe it is benefitial to them to understand why the difference
is now occurring and why BOTH Python and Windows are displaying CORRECT
results even though they do NOT match what they display.

For those same developers, I also posted a localtime_windo ws much which
allows them to massage the values returned by Python localtime so that they
match the values that Windows displays if they have the need and/or
requirement to do so.

Often times in development what "appears" to be the source of the problem
turns out that it is not the actual problem that is occurring.

It is still beneficial to have that link from what "appears" to be the
problem to the details of what "actually" is the source of the problem.

For another developer that runs into this difference their most likely
initial conclusion would be that something in Python 2.5.1 is broke
especially when they research the changes in Python 2.5.1 and find that
os.stat was changed. As a general rule, when something is broke, suspect
the last thing that you did even if it does not seem like it should be the
source of the problem.

Python developers cannot fix that. Besides which, Python is a
cross-platform language, originally developed on *nix.

Really, I thought they could fix anything :-)