Postgre and Web Request

[Hervé Piedvache]

Re: Postgre and Web Request

Le mercredi 28 Avril 2004 20:56, Joshua D. Drake a écrit :[color=blue][color=green]
> > 1. With the OS? Will it crash?[/color]
>
> See above. And actually if you really did have a situation where 10,000
> users accessed the site at the exact same time you would need a pretty
> large cluster to handle it... regardless of PostgreSQL.[/color]

Excuse for my stupid question, Joshua, but how do you make a PostgreSQL
Cluster ??? I'm really interesting about this solution ...

regards,
--
Bill Footcow


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

[Hervé Piedvache]

Re: Postgre and Web Request

Le mercredi 28 Avril 2004 20:56, Joshua D. Drake a écrit :[color=blue][color=green]
> > 1. With the OS? Will it crash?[/color]
>
> See above. And actually if you really did have a situation where 10,000
> users accessed the site at the exact same time you would need a pretty
> large cluster to handle it... regardless of PostgreSQL.[/color]

Excuse for my stupid question, Joshua, but how do you make a PostgreSQL
Cluster ??? I'm really interesting about this solution ...

regards,
--
Bill Footcow


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

[Shridhar Daithankar]

Re: Postgre and Web Request

Hervé Piedvache wrote:
[color=blue]
> Le mercredi 28 Avril 2004 20:56, Joshua D. Drake a écrit :
>[color=green][color=darkred]
>>> 1. With the OS? Will it crash?[/color]
>>
>>See above. And actually if you really did have a situation where 10,000
>>users accessed the site at the exact same time you would need a pretty
>>large cluster to handle it... regardless of PostgreSQL.[/color]
> Excuse for my stupid question, Joshua, but how do you make a PostgreSQL
> Cluster ??? I'm really interesting about this solution ...[/color]

You seem to misunderstand. In postgresql, a database installation is called as
cluster. When you initdb, it initializes a database area where more than one
databases can be created. Since there are more than one databases, such an
installation is called as database cluster.

It is not like a beowulf cluster..:-)

HTH

Shridhar

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to majordomo@postg resql.org)

[Shridhar Daithankar]

Re: Postgre and Web Request

Hervé Piedvache wrote:
[color=blue]
> Le mercredi 28 Avril 2004 20:56, Joshua D. Drake a écrit :
>[color=green][color=darkred]
>>> 1. With the OS? Will it crash?[/color]
>>
>>See above. And actually if you really did have a situation where 10,000
>>users accessed the site at the exact same time you would need a pretty
>>large cluster to handle it... regardless of PostgreSQL.[/color]
> Excuse for my stupid question, Joshua, but how do you make a PostgreSQL
> Cluster ??? I'm really interesting about this solution ...[/color]

You seem to misunderstand. In postgresql, a database installation is called as
cluster. When you initdb, it initializes a database area where more than one
databases can be created. Since there are more than one databases, such an
installation is called as database cluster.

It is not like a beowulf cluster..:-)

HTH

Shridhar

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to majordomo@postg resql.org)

[Hervé Piedvache]

Re: Postgre and Web Request

Le jeudi 29 Avril 2004 08:56, Shridhar Daithankar a écrit :[color=blue]
> Hervé Piedvache wrote:[color=green]
> > Le mercredi 28 Avril 2004 20:56, Joshua D. Drake a écrit :[color=darkred]
> >>> 1. With the OS? Will it crash?
> >>
> >>See above. And actually if you really did have a situation where 10,000
> >>users accessed the site at the exact same time you would need a pretty
> >>large cluster to handle it... regardless of PostgreSQL.[/color]
> >
> > Excuse for my stupid question, Joshua, but how do you make a PostgreSQL
> > Cluster ??? I'm really interesting about this solution ...[/color]
>
> You seem to misunderstand. In postgresql, a database installation is called
> as cluster. When you initdb, it initializes a database area where more than
> one databases can be created. Since there are more than one databases, such
> an installation is called as database cluster.
>
> It is not like a beowulf cluster..:-)[/color]

OK ... but sorry I may misunderstand something more ... but why Joshua is
talking about a large cluster ... ? Why the need of several database for one
application ?
And how can you maintain one application with several database ??
(replication ?, I'm not sure there is a realtime solution of replication with
PostgreSQL able to accep 10000 insert by second ... for the moment !?)

And second point ... to talk about beowulf cluster .... is there any solution
for PostgreSQL ? We here talking many time about MySQL solution for that ...
why PostgreSQL do not have this kind of solution, yet ?

Regards,
--
Hervé

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

[Hervé Piedvache]

Re: Postgre and Web Request

Le jeudi 29 Avril 2004 08:56, Shridhar Daithankar a écrit :[color=blue]
> Hervé Piedvache wrote:[color=green]
> > Le mercredi 28 Avril 2004 20:56, Joshua D. Drake a écrit :[color=darkred]
> >>> 1. With the OS? Will it crash?
> >>
> >>See above. And actually if you really did have a situation where 10,000
> >>users accessed the site at the exact same time you would need a pretty
> >>large cluster to handle it... regardless of PostgreSQL.[/color]
> >
> > Excuse for my stupid question, Joshua, but how do you make a PostgreSQL
> > Cluster ??? I'm really interesting about this solution ...[/color]
>
> You seem to misunderstand. In postgresql, a database installation is called
> as cluster. When you initdb, it initializes a database area where more than
> one databases can be created. Since there are more than one databases, such
> an installation is called as database cluster.
>
> It is not like a beowulf cluster..:-)[/color]

OK ... but sorry I may misunderstand something more ... but why Joshua is
talking about a large cluster ... ? Why the need of several database for one
application ?
And how can you maintain one application with several database ??
(replication ?, I'm not sure there is a realtime solution of replication with
PostgreSQL able to accep 10000 insert by second ... for the moment !?)

And second point ... to talk about beowulf cluster .... is there any solution
for PostgreSQL ? We here talking many time about MySQL solution for that ...
why PostgreSQL do not have this kind of solution, yet ?

Regards,
--
Hervé

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

[Shridhar Daithankar]

Re: Postgre and Web Request

Hervé Piedvache wrote:
[color=blue]
> Le jeudi 29 Avril 2004 08:56, Shridhar Daithankar a écrit :
>[color=green]
>>Hervé Piedvache wrote:
>>[color=darkred]
>>>Le mercredi 28 Avril 2004 20:56, Joshua D. Drake a écrit :
>>>
>>>>> 1. With the OS? Will it crash?
>>>>
>>>>See above. And actually if you really did have a situation where 10,000
>>>>users accessed the site at the exact same time you would need a pretty
>>>>large cluster to handle it... regardless of PostgreSQL.
>>>
>>>Excuse for my stupid question, Joshua, but how do you make a PostgreSQL
>>>Cluster ??? I'm really interesting about this solution ...[/color]
>>
>>You seem to misunderstand. In postgresql, a database installation is called
>>as cluster. When you initdb, it initializes a database area where more than
>>one databases can be created. Since there are more than one databases, such
>>an installation is called as database cluster.
>>
>>It is not like a beowulf cluster..:-)[/color]
>
>
> OK ... but sorry I may misunderstand something more ... but why Joshua is
> talking about a large cluster ... ? Why the need of several database for one
> application ?[/color]

Yeah right. After I sent the last reply, I think Joshua might actually meant a
cluster as in several machines working together. But anyways, what I explained
is also correct but not sure now, if this is the same context.
[color=blue]
> And how can you maintain one application with several database ??[/color]

Well, you can use dblink to link databases from different machines in a central
"hub" but that should be in the database design from the start. Furthermore
there are some limitations to it. Check dblink module in contrib. It could be
made to do some interesting work..:-)
[color=blue]
> (replication ?, I'm not sure there is a realtime solution of replication with
> PostgreSQL able to accep 10000 insert by second ... for the moment !?)[/color]

It can accept that many inserts per second given enough disk bandwidth and few
concurrent connection. I wouldn't be surprised if it reaches that rate with a
fiber channel disk array with 100+ connections concurrently inserting data on a
middle to high end machine.

Disk bandwidth is very crucial for databases.
[color=blue]
> And second point ... to talk about beowulf cluster .... is there any solution
> for PostgreSQL ? We here talking many time about MySQL solution for that ...
> why PostgreSQL do not have this kind of solution, yet ?[/color]

Ohh. good question.. there are so many facets of this problems that it would
take a book...:-)

There are many replication solutions for postgresql. Most of them do async
replication quite nicely. Synchronous multimaster replication isn't there AFAIK.
But I could be out of touch..

Check gborg(http://gborg.postgresql.org) on replication. I seem not to reach it now.

Shridhar



---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

[Shridhar Daithankar]

Re: Postgre and Web Request

Hervé Piedvache wrote:
[color=blue]
> Le jeudi 29 Avril 2004 08:56, Shridhar Daithankar a écrit :
>[color=green]
>>Hervé Piedvache wrote:
>>[color=darkred]
>>>Le mercredi 28 Avril 2004 20:56, Joshua D. Drake a écrit :
>>>
>>>>> 1. With the OS? Will it crash?
>>>>
>>>>See above. And actually if you really did have a situation where 10,000
>>>>users accessed the site at the exact same time you would need a pretty
>>>>large cluster to handle it... regardless of PostgreSQL.
>>>
>>>Excuse for my stupid question, Joshua, but how do you make a PostgreSQL
>>>Cluster ??? I'm really interesting about this solution ...[/color]
>>
>>You seem to misunderstand. In postgresql, a database installation is called
>>as cluster. When you initdb, it initializes a database area where more than
>>one databases can be created. Since there are more than one databases, such
>>an installation is called as database cluster.
>>
>>It is not like a beowulf cluster..:-)[/color]
>
>
> OK ... but sorry I may misunderstand something more ... but why Joshua is
> talking about a large cluster ... ? Why the need of several database for one
> application ?[/color]

Yeah right. After I sent the last reply, I think Joshua might actually meant a
cluster as in several machines working together. But anyways, what I explained
is also correct but not sure now, if this is the same context.
[color=blue]
> And how can you maintain one application with several database ??[/color]

Well, you can use dblink to link databases from different machines in a central
"hub" but that should be in the database design from the start. Furthermore
there are some limitations to it. Check dblink module in contrib. It could be
made to do some interesting work..:-)
[color=blue]
> (replication ?, I'm not sure there is a realtime solution of replication with
> PostgreSQL able to accep 10000 insert by second ... for the moment !?)[/color]

It can accept that many inserts per second given enough disk bandwidth and few
concurrent connection. I wouldn't be surprised if it reaches that rate with a
fiber channel disk array with 100+ connections concurrently inserting data on a
middle to high end machine.

Disk bandwidth is very crucial for databases.
[color=blue]
> And second point ... to talk about beowulf cluster .... is there any solution
> for PostgreSQL ? We here talking many time about MySQL solution for that ...
> why PostgreSQL do not have this kind of solution, yet ?[/color]

Ohh. good question.. there are so many facets of this problems that it would
take a book...:-)

There are many replication solutions for postgresql. Most of them do async
replication quite nicely. Synchronous multimaster replication isn't there AFAIK.
But I could be out of touch..

Check gborg(http://gborg.postgresql.org) on replication. I seem not to reach it now.

Shridhar



---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

[Tatsuo Ishii]

Re: Postgre and Web Request

> > (replication ?, I'm not sure there is a realtime solution of replication with[color=blue][color=green]
> > PostgreSQL able to accep 10000 insert by second ... for the moment !?)[/color][/color]

pgpool(annouced at general/announce list) is a synchronous single
master replication solution and it runs UPDATE/INSERT about 2/3 speed
of a normal PostgreSQL. So if you are lucky enough to aquire good
hardware which can run normal PostgreSQL with 15000 INSERTs, you could
obtain that speed with replication.
[color=blue][color=green]
> > And second point ... to talk about beowulf cluster .... is there any solution
> > for PostgreSQL ? We here talking many time about MySQL solution for that ...
> > why PostgreSQL do not have this kind of solution, yet ?[/color]
>
> Ohh. good question.. there are so many facets of this problems that it would
> take a book...:-)
>
> There are many replication solutions for postgresql. Most of them do async
> replication quite nicely. Synchronous multimaster replication isn't there AFAIK.
> But I could be out of touch..[/color]

PGCluster (http://www.csra.co.jp/~mitani/jpug/p.../en/index.html) is a
synchronous multimaster replication solution.
--
Tatsuo Ishii

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

[Tatsuo Ishii]

Re: Postgre and Web Request

> > (replication ?, I'm not sure there is a realtime solution of replication with[color=blue][color=green]
> > PostgreSQL able to accep 10000 insert by second ... for the moment !?)[/color][/color]

pgpool(annouced at general/announce list) is a synchronous single
master replication solution and it runs UPDATE/INSERT about 2/3 speed
of a normal PostgreSQL. So if you are lucky enough to aquire good
hardware which can run normal PostgreSQL with 15000 INSERTs, you could
obtain that speed with replication.
[color=blue][color=green]
> > And second point ... to talk about beowulf cluster .... is there any solution
> > for PostgreSQL ? We here talking many time about MySQL solution for that ...
> > why PostgreSQL do not have this kind of solution, yet ?[/color]
>
> Ohh. good question.. there are so many facets of this problems that it would
> take a book...:-)
>
> There are many replication solutions for postgresql. Most of them do async
> replication quite nicely. Synchronous multimaster replication isn't there AFAIK.
> But I could be out of touch..[/color]

PGCluster (http://www.csra.co.jp/~mitani/jpug/p.../en/index.html) is a
synchronous multimaster replication solution.
--
Tatsuo Ishii

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

[Stefan Sturm]

Re: Postgre and Web Request

Hello,
[color=blue]
> Depending on your web development environment (java, php, .NET) etc,
> you should be able to use some mechanism that will provide a pool of
> connections to the database. Each request does not open a new
> connection (and then release it), but insteads gets a connection from
> the pool to use, and returns it back to the pool when done.[/color]

Where can I find some examples for connection pooling with php? Or must I
just use persistence connections?

Greetings,
Stefan Sturm



---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Stefan Sturm]

Re: Postgre and Web Request

Hello,
[color=blue]
> Depending on your web development environment (java, php, .NET) etc,
> you should be able to use some mechanism that will provide a pool of
> connections to the database. Each request does not open a new
> connection (and then release it), but insteads gets a connection from
> the pool to use, and returns it back to the pool when done.[/color]

Where can I find some examples for connection pooling with php? Or must I
just use persistence connections?

Greetings,
Stefan Sturm



---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Robert Treat]

Re: Postgre and Web Request

On Wed, 2004-04-28 at 17:58, Chris Browne wrote:[color=blue]
> listas@miti.com .br ("Kilmer C. de Souza") writes:[color=green]
> > Oww ... sorry man ...
> > I make a mistake ... there are 10.000 users and 1.000 from 10.000 try to
> > access at the same time the database.
> > Can you help me again with this condition?[/color]
>
> The issues don't really change. Opening 1000 concurrent connections
> means spawning 1K PostgreSQL processes, which will reserve a pile of
> memory, and cause a pretty severe performance problem.
>[/color]

I think you need some qualifiers to that statement, since opening the
processes themselves should cause little to no problems at all if given
the right hardware. The main database I work on is currently set to
handle up to 825 simultaneous connections during peak times and that is
with perl dbi style connection pooling. If it weren't for i/o issues,
I'm pretty sure PostgreSQL would have no problems at all running that
load, which really only means we need to get a faster disk system set
up. (Currently the data and wal live on a single 10,000 rpm SCSI drive).
While I agree with everyone else in this thread that the OP is not
likely to ever need such a high connection count, there's no reason that
PostgreSQL can't support it given you have enough RAM, fast enough
disks, and you don't shoot yourself in the foot with FK/Locking issues
in the app.


Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL


---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

[Robert Treat]

Re: Postgre and Web Request

On Wed, 2004-04-28 at 17:58, Chris Browne wrote:[color=blue]
> listas@miti.com .br ("Kilmer C. de Souza") writes:[color=green]
> > Oww ... sorry man ...
> > I make a mistake ... there are 10.000 users and 1.000 from 10.000 try to
> > access at the same time the database.
> > Can you help me again with this condition?[/color]
>
> The issues don't really change. Opening 1000 concurrent connections
> means spawning 1K PostgreSQL processes, which will reserve a pile of
> memory, and cause a pretty severe performance problem.
>[/color]

I think you need some qualifiers to that statement, since opening the
processes themselves should cause little to no problems at all if given
the right hardware. The main database I work on is currently set to
handle up to 825 simultaneous connections during peak times and that is
with perl dbi style connection pooling. If it weren't for i/o issues,
I'm pretty sure PostgreSQL would have no problems at all running that
load, which really only means we need to get a faster disk system set
up. (Currently the data and wal live on a single 10,000 rpm SCSI drive).
While I agree with everyone else in this thread that the OP is not
likely to ever need such a high connection count, there's no reason that
PostgreSQL can't support it given you have enough RAM, fast enough
disks, and you don't shoot yourself in the foot with FK/Locking issues
in the app.


Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL


---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

[Bill Moran]

Re: Postgre and Web Request

Stefan Sturm wrote:[color=blue]
> Hello,
>[color=green]
>>Depending on your web development environment (java, php, .NET) etc,
>>you should be able to use some mechanism that will provide a pool of
>>connections to the database. Each request does not open a new
>>connection (and then release it), but insteads gets a connection from
>>the pool to use, and returns it back to the pool when done.[/color]
>
> Where can I find some examples for connection pooling with php? Or must I
> just use persistence connections?[/color]

php handles connection pooling more or less automatically ... as long as
you use pg_pconnect() instead of pg_connect().

You have to be careful using connection pooling. For example, if you create
a temporary table and forget to delete it when the the script completes, the
next time the script runs, it's likely that the connection will be reused
and the script will cause an error because the temp table already exists.

Here's some docs at the php web site that may help:

http://us4.php.net/manual/en/features.persistent-connections.php

--
Bill Moran
Potential Technologies

http://www.potentialtech.com

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to majordomo@postg resql.org)