Re: embedded/"serverles s" (Re: serverless postgresql)
Personally, I would use something like berkelydb for an embedded database
engine. As a general rule, if it needs to be embedded, you don't need sql.
I'm sure you could think of areas where you could use a full blown sql
server, but in practice that generally isn't the case. And like others
have already mentioned, if you are embedding a database in software that a
user has control of, you want it as SIMPLE as possible.
Chris
----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa .us>
To: "Rick Gigger" <rick@alpinenet working.com>
Cc: "Chris Travers" <chris@travelam ericas.com>; "David Garamond"
<lists@zara.6.i sreserved.com>; "Jeff Bowden" <jlb@houseofdis traction.com>;
"pgsql-general" <pgsql-general@postgre sql.org>
Sent: Friday, January 16, 2004 1:14 PM
Subject: Re: embedded/"serverless " (Re: [GENERAL] serverless postgresql)
[color=blue]
> "Rick Gigger" <rick@alpinenet working.com> writes:[color=green][color=darkred]
> >> ... just adds potential for mucking things up. I can see the bug
> >> reports now: "I decided I'd make the shutdown routine 'kill -9' the
> >> postmaster because I didn't like the multi-second delay for a normal
> >> shutdown. Now my database is corrupt."[/color][/color]
>[color=green]
> > I recall a discussion a while back where people were complaining that[/color][/color]
some[color=blue][color=green]
> > os (I think it was mac os x) would just kill all proccesses after[/color][/color]
something[color=blue][color=green]
> > like 30 seconds on system shutdown if they didn't quit fast enough on[/color][/color]
their[color=blue][color=green]
> > own. The response was that because of how postgres uses wal files to
> > prevent db corruption this was not an issue. Why does this not apply in
> > this situation?[/color]
>
> Because a system shutdown doesn't leave anything else running. kill -9
> on the postmaster leaves orphaned backends still running, and orphaned
> shared memory segments still in existence. The latter could prevent you
> from starting a fresh postmaster (because two shmem segments could
> exceed the kernel's SHMMAX) until you do manual cleanup, which most
> people don't know how to do. If you do succeed in starting another
> postmaster with a fresh shared memory segment, you will have two
> independent sets of backends modifying the database files with no
> interlocking, which is a sure recipe for data corruption.
>
> There is an interlock that is supposed to prevent this catastrophic
> scenario, but I do not trust it unreservedly (and not at all on
> non-Unix-derived platforms, because it depends on some rather arcane
> features of the SysV shared memory API, which might not be implemented
> fully on other platforms).
>
> This is why the "don't kill -9 the postmaster" TIP is still around, BTW.
> It has nothing to do with crash safety.
>[color=green]
> > By saying "the client can scribble on the database anyway" do you do you
> > mean the client app actually writing over the db files on disk? It[/color][/color]
seems[color=blue][color=green]
> > like this would only be a problem with an exceptionally stupid[/color][/color]
programmer.[color=blue][color=green]
> > How could that happen on accident?[/color]
>
> Scribbling on memory that doesn't belong to you isn't something one does
> intentionally, either --- but it happens. However, my real concern here
> is not so much with program bugs as clueless users. We see enough cases
> already of people removing lock files or "unnecessar y" log files; I fear
> it'd be a lot worse if those files were sitting in the user's home
> directory.
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postg resql.org
>[/color]
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
Personally, I would use something like berkelydb for an embedded database
engine. As a general rule, if it needs to be embedded, you don't need sql.
I'm sure you could think of areas where you could use a full blown sql
server, but in practice that generally isn't the case. And like others
have already mentioned, if you are embedding a database in software that a
user has control of, you want it as SIMPLE as possible.
Chris
----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa .us>
To: "Rick Gigger" <rick@alpinenet working.com>
Cc: "Chris Travers" <chris@travelam ericas.com>; "David Garamond"
<lists@zara.6.i sreserved.com>; "Jeff Bowden" <jlb@houseofdis traction.com>;
"pgsql-general" <pgsql-general@postgre sql.org>
Sent: Friday, January 16, 2004 1:14 PM
Subject: Re: embedded/"serverless " (Re: [GENERAL] serverless postgresql)
[color=blue]
> "Rick Gigger" <rick@alpinenet working.com> writes:[color=green][color=darkred]
> >> ... just adds potential for mucking things up. I can see the bug
> >> reports now: "I decided I'd make the shutdown routine 'kill -9' the
> >> postmaster because I didn't like the multi-second delay for a normal
> >> shutdown. Now my database is corrupt."[/color][/color]
>[color=green]
> > I recall a discussion a while back where people were complaining that[/color][/color]
some[color=blue][color=green]
> > os (I think it was mac os x) would just kill all proccesses after[/color][/color]
something[color=blue][color=green]
> > like 30 seconds on system shutdown if they didn't quit fast enough on[/color][/color]
their[color=blue][color=green]
> > own. The response was that because of how postgres uses wal files to
> > prevent db corruption this was not an issue. Why does this not apply in
> > this situation?[/color]
>
> Because a system shutdown doesn't leave anything else running. kill -9
> on the postmaster leaves orphaned backends still running, and orphaned
> shared memory segments still in existence. The latter could prevent you
> from starting a fresh postmaster (because two shmem segments could
> exceed the kernel's SHMMAX) until you do manual cleanup, which most
> people don't know how to do. If you do succeed in starting another
> postmaster with a fresh shared memory segment, you will have two
> independent sets of backends modifying the database files with no
> interlocking, which is a sure recipe for data corruption.
>
> There is an interlock that is supposed to prevent this catastrophic
> scenario, but I do not trust it unreservedly (and not at all on
> non-Unix-derived platforms, because it depends on some rather arcane
> features of the SysV shared memory API, which might not be implemented
> fully on other platforms).
>
> This is why the "don't kill -9 the postmaster" TIP is still around, BTW.
> It has nothing to do with crash safety.
>[color=green]
> > By saying "the client can scribble on the database anyway" do you do you
> > mean the client app actually writing over the db files on disk? It[/color][/color]
seems[color=blue][color=green]
> > like this would only be a problem with an exceptionally stupid[/color][/color]
programmer.[color=blue][color=green]
> > How could that happen on accident?[/color]
>
> Scribbling on memory that doesn't belong to you isn't something one does
> intentionally, either --- but it happens. However, my real concern here
> is not so much with program bugs as clueless users. We see enough cases
> already of people removing lock files or "unnecessar y" log files; I fear
> it'd be a lot worse if those files were sitting in the user's home
> directory.
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postg resql.org
>[/color]
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
Comment