State of Beta 2

[Nigel J. Andrews]

Re: State of Beta 2

On Fri, 12 Sep 2003, Ron Johnson wrote:
[color=blue]
> On Fri, 2003-09-12 at 10:50, Andrew Rawnsley wrote:[color=green]
> > Small soapbox moment here...
> >
> > ANYTHING that can be done to eliminate having to do an initdb on
> > version changes would make a lot of people do cartwheels. 'Do a
> > dump/reload' sometimes comes across a bit casually on the lists (my
> > apologies if it isn't meant to be), but it can be be incredibly onerous
> > to do on a large production system. That's probably why you run across
> > people running stupid-old versions.[/color]
>
> And this will become even more of an issue as it's PG's popularity
> grows with large and 24x7 databases.[/color]

And dump/reload isn't always such a casual operation to do. I initialise a
database from dump but I have to fiddle the sql on the reload to make it work.
The odd thing is I never thought it a bug, just something to work around, until
someone else has been persuing it on the list as one (it's the create schema
thing).


--
Nigel J. Andrews


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postg resql.org

[Dennis Gearon]

Re: State of Beta 2

Ron Johnson wrote:
[color=blue]
>On Fri, 2003-09-12 at 10:50, Andrew Rawnsley wrote:
>
>[color=green]
>>Small soapbox moment here...
>>
>>ANYTHING that can be done to eliminate having to do an initdb on
>>version changes would make a lot of people do cartwheels. 'Do a
>>dump/reload' sometimes comes across a bit casually on the lists (my
>>apologies if it isn't meant to be), but it can be be incredibly onerous
>>to do on a large production system. That's probably why you run across
>>people running stupid-old versions.
>>
>>[/color]
>
>And this will become even more of an issue as it's PG's popularity
>grows with large and 24x7 databases.
>
>[/color]
He is right, it might be a good idea to head this problem 'off at the
pass'. I am usually pretty good at predicting technilogical trends. I've
made some money at it. And I predict that Postgres will eclipse MySQL
and be in the top 5 of all databases deployed. But it does have some
achilles tendon's.


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

[Dennis Gearon]

Re: State of Beta 2

Ron Johnson wrote:
[color=blue]
>On Fri, 2003-09-12 at 10:50, Andrew Rawnsley wrote:
>
>[color=green]
>>Small soapbox moment here...
>>
>>ANYTHING that can be done to eliminate having to do an initdb on
>>version changes would make a lot of people do cartwheels. 'Do a
>>dump/reload' sometimes comes across a bit casually on the lists (my
>>apologies if it isn't meant to be), but it can be be incredibly onerous
>>to do on a large production system. That's probably why you run across
>>people running stupid-old versions.
>>
>>[/color]
>
>And this will become even more of an issue as it's PG's popularity
>grows with large and 24x7 databases.
>
>[/color]
He is right, it might be a good idea to head this problem 'off at the
pass'. I am usually pretty good at predicting technilogical trends. I've
made some money at it. And I predict that Postgres will eclipse MySQL
and be in the top 5 of all databases deployed. But it does have some
achilles tendon's.


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

[Kaare Rasmussen]

Re: State of Beta 2

> He is right, it might be a good idea to head this problem 'off at the[color=blue]
> pass'. I am usually pretty good at predicting technilogical trends. I've[/color]

Well, the only solution I can see is to make an inline conversion tool that
knows about every step from earlier versions.

I believe this has been discussed before, but it does not seem to be a small
or an easy task to implement.

--
Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582
Kaki Data tshirts, merchandize Fax: 3816 2501
Howitzvej 75 Åben 12.00-18.00 Email: kar@kakidata.dk
2000 Frederiksberg Lørdag 12.00-16.00 Web: www.suse.dk

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

[Kaare Rasmussen]

Re: State of Beta 2

> He is right, it might be a good idea to head this problem 'off at the[color=blue]
> pass'. I am usually pretty good at predicting technilogical trends. I've[/color]

Well, the only solution I can see is to make an inline conversion tool that
knows about every step from earlier versions.

I believe this has been discussed before, but it does not seem to be a small
or an easy task to implement.

--
Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582
Kaki Data tshirts, merchandize Fax: 3816 2501
Howitzvej 75 Åben 12.00-18.00 Email: kar@kakidata.dk
2000 Frederiksberg Lørdag 12.00-16.00 Web: www.suse.dk

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

[Ron Johnson]

Upgrading (was Re: State of Beta 2)

On Fri, 2003-09-12 at 17:01, Kaare Rasmussen wrote:[color=blue][color=green]
> > He is right, it might be a good idea to head this problem 'off at the
> > pass'. I am usually pretty good at predicting technilogical trends. I've[/color]
>
> Well, the only solution I can see is to make an inline conversion tool that
> knows about every step from earlier versions.
>
> I believe this has been discussed before, but it does not seem to be a small
> or an easy task to implement.[/color]

Does the "on-disk structure" really change that much between major
versions?

--
-----------------------------------------------------------------
Ron Johnson, Jr. ron.l.johnson@c ox.net
Jefferson, LA USA

"Vanity, my favorite sin."
Larry/John/Satan, "The Devil's Advocate"


---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Ron Johnson]

Upgrading (was Re: State of Beta 2)

On Fri, 2003-09-12 at 17:01, Kaare Rasmussen wrote:[color=blue][color=green]
> > He is right, it might be a good idea to head this problem 'off at the
> > pass'. I am usually pretty good at predicting technilogical trends. I've[/color]
>
> Well, the only solution I can see is to make an inline conversion tool that
> knows about every step from earlier versions.
>
> I believe this has been discussed before, but it does not seem to be a small
> or an easy task to implement.[/color]

Does the "on-disk structure" really change that much between major
versions?

--
-----------------------------------------------------------------
Ron Johnson, Jr. ron.l.johnson@c ox.net
Jefferson, LA USA

"Vanity, my favorite sin."
Larry/John/Satan, "The Devil's Advocate"


---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Joshua D. Drake]

Re: State of Beta 2

Hello,

The initdb is not always a bad thing. In reality the idea of just
being able to "upgrade" is not a good thing. Just think about the
differences between 7.2.3 and 7.3.x... The most annoying (although
appropriate) one being that integers can no longer be ''.

If we provide the ability to do a wholesale upgrade many things would
just break. Heck even the connection protocol is different for 7.4.


J

Dennis Gearon wrote:
[color=blue]
> Ron Johnson wrote:
>[color=green]
>> On Fri, 2003-09-12 at 10:50, Andrew Rawnsley wrote:
>>
>>[color=darkred]
>>> Small soapbox moment here...
>>>
>>> ANYTHING that can be done to eliminate having to do an initdb on
>>> version changes would make a lot of people do cartwheels. 'Do a
>>> dump/reload' sometimes comes across a bit casually on the lists (my
>>> apologies if it isn't meant to be), but it can be be incredibly
>>> onerous to do on a large production system. That's probably why you
>>> run across people running stupid-old versions.
>>>[/color]
>>
>>
>> And this will become even more of an issue as it's PG's popularity
>> grows with large and 24x7 databases.
>>
>>[/color]
> He is right, it might be a good idea to head this problem 'off at the
> pass'. I am usually pretty good at predicting technilogical trends.
> I've made some money at it. And I predict that Postgres will eclipse
> MySQL and be in the top 5 of all databases deployed. But it does have
> some achilles tendon's.
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postg resql.org so that your
> message can get through to the mailing list cleanly[/color]


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-222-2783 - jd@commandpromp t.com - http://www.commandprompt.com
The most reliable support for the most reliable Open Source database.



---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Joshua D. Drake]

Re: State of Beta 2

Hello,

The initdb is not always a bad thing. In reality the idea of just
being able to "upgrade" is not a good thing. Just think about the
differences between 7.2.3 and 7.3.x... The most annoying (although
appropriate) one being that integers can no longer be ''.

If we provide the ability to do a wholesale upgrade many things would
just break. Heck even the connection protocol is different for 7.4.


J

Dennis Gearon wrote:
[color=blue]
> Ron Johnson wrote:
>[color=green]
>> On Fri, 2003-09-12 at 10:50, Andrew Rawnsley wrote:
>>
>>[color=darkred]
>>> Small soapbox moment here...
>>>
>>> ANYTHING that can be done to eliminate having to do an initdb on
>>> version changes would make a lot of people do cartwheels. 'Do a
>>> dump/reload' sometimes comes across a bit casually on the lists (my
>>> apologies if it isn't meant to be), but it can be be incredibly
>>> onerous to do on a large production system. That's probably why you
>>> run across people running stupid-old versions.
>>>[/color]
>>
>>
>> And this will become even more of an issue as it's PG's popularity
>> grows with large and 24x7 databases.
>>
>>[/color]
> He is right, it might be a good idea to head this problem 'off at the
> pass'. I am usually pretty good at predicting technilogical trends.
> I've made some money at it. And I predict that Postgres will eclipse
> MySQL and be in the top 5 of all databases deployed. But it does have
> some achilles tendon's.
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postg resql.org so that your
> message can get through to the mailing list cleanly[/color]


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-222-2783 - jd@commandpromp t.com - http://www.commandprompt.com
The most reliable support for the most reliable Open Source database.



---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Ron Johnson]

need for in-place upgrades (was Re: State of Beta 2)

On Fri, 2003-09-12 at 17:48, Joshua D. Drake wrote:[color=blue]
> Hello,
>
> The initdb is not always a bad thing. In reality the idea of just
> being able to "upgrade" is not a good thing. Just think about the
> differences between 7.2.3 and 7.3.x... The most annoying (although
> appropriate) one being that integers can no longer be ''.[/color]

But that's just not going to cut it if PostgreSQL wants to be
a serious "player" in the enterprise space, where 24x7 systems
are common, and you just don't *get* 12/18/24/whatever hours to
dump/restore a 200GB database.

For example, there are some rather large companies whose fac-
tories are run 24x365 on rather old versions of VAX/VMS and
Rdb/VMS, because the DBAs can't even get the 3 hours to do
in-place upgrades to Rdb, much less the time the SysAdmin needs
to upgrade VAX/VMS to VAX/OpenVMS.

In our case, we have systems that have multiple 300+GB databases
(working in concert as one big system), and dumping all of them,
then restoring (which includes creating indexes on tables with
row-counts in the low 9 digits, and one which has gone as high
as 2+ billion records) is just totally out of the question.
[color=blue]
> If we provide the ability to do a wholesale upgrade many things would
> just break. Heck even the connection protocol is different for 7.4.[/color]

But what does a *closed* database care about changed communications
protocols? When you reopen the database after an upgrade the
postmaster and client libs start yakking away in a slightly diff-
erent language, but so what?
[color=blue]
> Dennis Gearon wrote:
>[color=green]
> > Ron Johnson wrote:
> >[color=darkred]
> >> On Fri, 2003-09-12 at 10:50, Andrew Rawnsley wrote:
> >>
> >>
> >>> Small soapbox moment here...
> >>>
> >>> ANYTHING that can be done to eliminate having to do an initdb on
> >>> version changes would make a lot of people do cartwheels. 'Do a
> >>> dump/reload' sometimes comes across a bit casually on the lists (my
> >>> apologies if it isn't meant to be), but it can be be incredibly
> >>> onerous to do on a large production system. That's probably why you
> >>> run across people running stupid-old versions.
> >>>
> >>
> >>
> >> And this will become even more of an issue as it's PG's popularity
> >> grows with large and 24x7 databases.
> >>
> >>[/color]
> > He is right, it might be a good idea to head this problem 'off at the
> > pass'. I am usually pretty good at predicting technilogical trends.
> > I've made some money at it. And I predict that Postgres will eclipse
> > MySQL and be in the top 5 of all databases deployed. But it does have
> > some achilles tendon's.[/color][/color]


--
-----------------------------------------------------------------
Ron Johnson, Jr. ron.l.johnson@c ox.net
Jefferson, LA USA

"The UN couldn't break up a cookie fight in a Brownie meeting."
Larry Miller


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

[Ron Johnson]

need for in-place upgrades (was Re: State of Beta 2)

On Fri, 2003-09-12 at 17:48, Joshua D. Drake wrote:[color=blue]
> Hello,
>
> The initdb is not always a bad thing. In reality the idea of just
> being able to "upgrade" is not a good thing. Just think about the
> differences between 7.2.3 and 7.3.x... The most annoying (although
> appropriate) one being that integers can no longer be ''.[/color]

But that's just not going to cut it if PostgreSQL wants to be
a serious "player" in the enterprise space, where 24x7 systems
are common, and you just don't *get* 12/18/24/whatever hours to
dump/restore a 200GB database.

For example, there are some rather large companies whose fac-
tories are run 24x365 on rather old versions of VAX/VMS and
Rdb/VMS, because the DBAs can't even get the 3 hours to do
in-place upgrades to Rdb, much less the time the SysAdmin needs
to upgrade VAX/VMS to VAX/OpenVMS.

In our case, we have systems that have multiple 300+GB databases
(working in concert as one big system), and dumping all of them,
then restoring (which includes creating indexes on tables with
row-counts in the low 9 digits, and one which has gone as high
as 2+ billion records) is just totally out of the question.
[color=blue]
> If we provide the ability to do a wholesale upgrade many things would
> just break. Heck even the connection protocol is different for 7.4.[/color]

But what does a *closed* database care about changed communications
protocols? When you reopen the database after an upgrade the
postmaster and client libs start yakking away in a slightly diff-
erent language, but so what?
[color=blue]
> Dennis Gearon wrote:
>[color=green]
> > Ron Johnson wrote:
> >[color=darkred]
> >> On Fri, 2003-09-12 at 10:50, Andrew Rawnsley wrote:
> >>
> >>
> >>> Small soapbox moment here...
> >>>
> >>> ANYTHING that can be done to eliminate having to do an initdb on
> >>> version changes would make a lot of people do cartwheels. 'Do a
> >>> dump/reload' sometimes comes across a bit casually on the lists (my
> >>> apologies if it isn't meant to be), but it can be be incredibly
> >>> onerous to do on a large production system. That's probably why you
> >>> run across people running stupid-old versions.
> >>>
> >>
> >>
> >> And this will become even more of an issue as it's PG's popularity
> >> grows with large and 24x7 databases.
> >>
> >>[/color]
> > He is right, it might be a good idea to head this problem 'off at the
> > pass'. I am usually pretty good at predicting technilogical trends.
> > I've made some money at it. And I predict that Postgres will eclipse
> > MySQL and be in the top 5 of all databases deployed. But it does have
> > some achilles tendon's.[/color][/color]


--
-----------------------------------------------------------------
Ron Johnson, Jr. ron.l.johnson@c ox.net
Jefferson, LA USA

"The UN couldn't break up a cookie fight in a Brownie meeting."
Larry Miller


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

[Tom Lane]

Re: State of Beta 2

Kaare Rasmussen <kar@kakidata.d k> writes:[color=blue]
> I believe this has been discussed before, but it does not seem to be a small
> or an easy task to implement.[/color]

Yes, it's been discussed to death, and it isn't easy. See the archives
for Lamar Owen's eloquent rants on the subject, and various hackers'
followups as to the implementation issues.

What it comes down to IMHO is that (a) there are still a lot of bad,
incomplete, or shortsighted decisions embedded in Postgres, which cannot
really be fixed in 100% backwards compatible ways; (b) there are not all
that many people competent to work on improving Postgres, and even fewer
who are actually being paid to do so; and (c) those who are volunteers
are likely to work on things they find interesting to fix. Finding ways
to maintain backwards compatibility without dump/reload is not in the
"interestin g" category. It is in the category of things that will only
happen if people pony up money to pay someone to do uninteresting work.
And for all the ranting, I've not seen any ponying.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postg resql.org

[Tom Lane]

Re: State of Beta 2

Kaare Rasmussen <kar@kakidata.d k> writes:[color=blue]
> I believe this has been discussed before, but it does not seem to be a small
> or an easy task to implement.[/color]

Yes, it's been discussed to death, and it isn't easy. See the archives
for Lamar Owen's eloquent rants on the subject, and various hackers'
followups as to the implementation issues.

What it comes down to IMHO is that (a) there are still a lot of bad,
incomplete, or shortsighted decisions embedded in Postgres, which cannot
really be fixed in 100% backwards compatible ways; (b) there are not all
that many people competent to work on improving Postgres, and even fewer
who are actually being paid to do so; and (c) those who are volunteers
are likely to work on things they find interesting to fix. Finding ways
to maintain backwards compatibility without dump/reload is not in the
"interestin g" category. It is in the category of things that will only
happen if people pony up money to pay someone to do uninteresting work.
And for all the ranting, I've not seen any ponying.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postg resql.org

[Alvaro Herrera]

Re: State of Beta 2

On Fri, Sep 12, 2003 at 03:48:48PM -0700, Joshua D. Drake wrote:
[color=blue]
> The initdb is not always a bad thing. In reality the idea of just
> being able to "upgrade" is not a good thing. Just think about the
> differences between 7.2.3 and 7.3.x... The most annoying (although
> appropriate) one being that integers can no longer be ''.[/color]

But it would be much easier if one wasn't forced to create a dump and
then restore it. One would still need to change the applications, but
that doesn't force downtime.

[color=blue]
> If we provide the ability to do a wholesale upgrade many things would
> just break. Heck even the connection protocol is different for 7.4.[/color]

But the new client libpq _can_ talk to older servers.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
FOO MANE PADME HUM

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to majordomo@postg resql.org)

[Alvaro Herrera]

Re: State of Beta 2

On Fri, Sep 12, 2003 at 03:48:48PM -0700, Joshua D. Drake wrote:
[color=blue]
> The initdb is not always a bad thing. In reality the idea of just
> being able to "upgrade" is not a good thing. Just think about the
> differences between 7.2.3 and 7.3.x... The most annoying (although
> appropriate) one being that integers can no longer be ''.[/color]

But it would be much easier if one wasn't forced to create a dump and
then restore it. One would still need to change the applications, but
that doesn't force downtime.

[color=blue]
> If we provide the ability to do a wholesale upgrade many things would
> just break. Heck even the connection protocol is different for 7.4.[/color]

But the new client libpq _can_ talk to older servers.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
FOO MANE PADME HUM

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to majordomo@postg resql.org)