shared memory on OS X - 7.4beta4

**Adam Witney** · Nov 12 '05, 12:23 AM

Re: shared memory on OS X - 7.4beta4

I don't know what this error is particularly, but just to say that 7.4beta5
compiles and starts up fine on my OSX 10.2.8 machine

Cheers

Adam

[color=blue]
> I get this when I try to start up a freshly compiled beta4 on OS X
> 10.2.6:
>
> FATAL: could not create shared memory segment: Invalid argument
>
> I saw a previous thread on this for beta2. It sounded like this was a
> bug that was to be fixed in beta3, but I'm still having this problem
> with beta4.
>
> Thanks,
>
> John DeSoi, Ph.D.
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postg resql.org so that your
> message can get through to the mailing list cleanly[/color]

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

**Vivek Khera** · Nov 12 '05, 12:23 AM

Re: shared memory on OS X - 7.4beta4

>>>>> "JD" == John DeSoi <jd@icx.net> writes:

JD> I get this when I try to start up a freshly compiled beta4 on OS X
JD> 10.2.6:

JD> FATAL: could not create shared memory segment: Invalid argument

How much SHM are you requesting (shared_buffers ), and does the system
allow for that much? There's some bootup script you can modify in OSX
to up that limit if you need. Check google and/or Postgres' techdocs
site.

--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: khera@kciLink.c om Rockville, MD +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

**John DeSoi** · Nov 12 '05, 12:23 AM

Re: shared memory on OS X - 7.4beta4

On Friday, October 24, 2003, at 11:26 AM, Vivek Khera wrote:
[color=blue]
> How much SHM are you requesting (shared_buffers ), and does the system
> allow for that much? There's some bootup script you can modify in OSX
> to up that limit if you need. Check google and/or Postgres' techdocs
> site.[/color]

I did a default configure and did not change anything in the .conf
file. It requests 10444800 bytes. But even if I drop max_connections
down to 10, it still won't start. Yes, I can modify SHMMAX for the
operating system to make it work but I'm trying to understand why I
need to do this when it was not necessary under 7.3 or an earlier 7.4
development build (pre beta 1). I was hoping it is a bug, because I
think it hurts the adoption of Postgresql if I have to worry about a
lot of low level OS settings just to setup a simple database.

Best,

John DeSoi, Ph.D.

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

**John DeSoi** · Nov 12 '05, 12:24 AM

Re: shared memory on OS X - 7.4beta4

On Friday, October 24, 2003, at 10:19 AM, Adam Witney wrote:
[color=blue]
> I don't know what this error is particularly, but just to say that
> 7.4beta5
> compiles and starts up fine on my OSX 10.2.8 machine[/color]

OK, I compiled beta 5 and put my rest SHMMAX back to the default
settings as provided by OS X. Still the same error:

FATAL: could not create shared memory segment: Invalid argument
DETAIL: Failed system call was shmget(key=5432 001, size=10444800,
03600).
HINT: This error usually means that PostgreSQL's request for a shared
memory segment exceeded your kernel's SHMMAX parameter. You can either
reduce the request size or reconfigure the kernel with larger SHMMAX.
To reduce the request size (currently 10444800 bytes), reduce
PostgreSQL's shared_buffers parameter (currently 1000) and/or its
max_connections parameter (currently 100).
If the request size is already small, it's possible that it is less
than your kernel's SHMMIN parameter, in which case raising the request
size or reconfiguring SHMMIN is called for.
The PostgreSQL documentation contains more information about shared
memory configuration.

Here is how I configured it:

../configure --prefix=/Unix/postgresql --enable-locale
--without-readline --with-includes=/sw/include --with-libraries=/sw/lib
--with-openssl=/sw/lib

Next I upgraded to 10.2.8. Same issue.

So you have not modified your memory settings in
/System/Library/StartupItems/SystemTuning/SystemTuning and you don't
get this error?

I'm using a PowerBook G4 with 512M of memory.

Thanks,

John DeSoi, Ph.D.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

**Adam Witney** · Nov 12 '05, 12:24 AM

Re: shared memory on OS X - 7.4beta4

On 26/10/03 1:01 am, "John DeSoi" <jd@icx.net> wrote:
[color=blue]
>
> On Friday, October 24, 2003, at 10:19 AM, Adam Witney wrote:
>[color=green]
>> I don't know what this error is particularly, but just to say that
>> 7.4beta5
>> compiles and starts up fine on my OSX 10.2.8 machine[/color]
>
> OK, I compiled beta 5 and put my rest SHMMAX back to the default
> settings as provided by OS X. Still the same error:
>
> FATAL: could not create shared memory segment: Invalid argument
> DETAIL: Failed system call was shmget(key=5432 001, size=10444800,
> 03600).
> HINT: This error usually means that PostgreSQL's request for a shared
> memory segment exceeded your kernel's SHMMAX parameter. You can either
> reduce the request size or reconfigure the kernel with larger SHMMAX.
> To reduce the request size (currently 10444800 bytes), reduce
> PostgreSQL's shared_buffers parameter (currently 1000) and/or its
> max_connections parameter (currently 100).
> If the request size is already small, it's possible that it is less
> than your kernel's SHMMIN parameter, in which case raising the request
> size or reconfiguring SHMMIN is called for.
> The PostgreSQL documentation contains more information about shared
> memory configuration.
>
> Here is how I configured it:
>
> ./configure --prefix=/Unix/postgresql --enable-locale
> --without-readline --with-includes=/sw/include --with-libraries=/sw/lib
> --with-openssl=/sw/lib
>
>
> Next I upgraded to 10.2.8. Same issue.
>
> So you have not modified your memory settings in
> /System/Library/StartupItems/SystemTuning/SystemTuning and you don't
> get this error?[/color]

Ah, yes I did change these... Mine are set to

sysctl -w kern.sysv.shmma x=4194304
sysctl -w kern.sysv.shmmi n=1
sysctl -w kern.sysv.shmmn i=32
sysctl -w kern.sysv.shmse g=8
sysctl -w kern.sysv.shmal l=1024
[color=blue]
> I'm using a PowerBook G4 with 512M of memory.[/color]

Same as me

Do you have many other applications running? Or an old version of
PostgreSQL... I found when running the regression tests I had to shut down
my 7.3.2 installation

Cheers

adam

adam

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

**John DeSoi** · Nov 12 '05, 12:24 AM

Re: shared memory on OS X - 7.4beta4

Adam,

On Sunday, October 26, 2003, at 10:16 AM, Adam Witney wrote:
[color=blue]
> Ah, yes I did change these... Mine are set to
>
> sysctl -w kern.sysv.shmma x=4194304
> sysctl -w kern.sysv.shmmi n=1
> sysctl -w kern.sysv.shmmn i=32
> sysctl -w kern.sysv.shmse g=8
> sysctl -w kern.sysv.shmal l=1024[/color]

OK, this is what I was after. I did not have to change any of this for
7.3, but now 7.4 does not work unless on a standard OS X installation
unless the user modifies these system settings. I can live with it, but
I was hoping it might be a bug that could be resolved before the final
7.4. Every person that installs on OS X is going to run into this.
Tracking it down and understanding what you have to change is not
simple for us OS X users that are not familiar with Unix :).

Best,

John DeSoi, Ph.D.

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postg resql.org

**Alvaro Herrera** · Nov 12 '05, 12:24 AM

Re: shared memory on OS X - 7.4beta4

On Sun, Oct 26, 2003 at 04:41:16PM -0500, John DeSoi wrote:
[color=blue]
> On Sunday, October 26, 2003, at 10:16 AM, Adam Witney wrote:
>[color=green]
> >Ah, yes I did change these... Mine are set to
> >
> > sysctl -w kern.sysv.shmma x=4194304
> > sysctl -w kern.sysv.shmmi n=1
> > sysctl -w kern.sysv.shmmn i=32
> > sysctl -w kern.sysv.shmse g=8
> > sysctl -w kern.sysv.shmal l=1024[/color]
>
>
> OK, this is what I was after. I did not have to change any of this for
> 7.3, but now 7.4 does not work unless on a standard OS X installation
> unless the user modifies these system settings.[/color]

Huh, this is strange. IIRC during the installation process, initdb is
supposed to determine which values actually work; those are affected by
the shmmax operating system mostly, and the idea is to enable the
installation process to pick more reasonable values out-of-the-box for
memory usage that will work on your system.

How did you install Postgres, and what values did initdb pick for
shared_buffers and max_connections ?

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Puedes vivir solo una vez, pero si lo haces bien, una vez es suficiente"

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to majordomo@postg resql.org)

**Adam Witney** · Nov 12 '05, 12:24 AM

Re: shared memory on OS X - 7.4beta4

On 26/10/03 9:41 pm, "John DeSoi" <jd@icx.net> wrote:
[color=blue]
> Adam,
>
> On Sunday, October 26, 2003, at 10:16 AM, Adam Witney wrote:
>[color=green]
>> Ah, yes I did change these... Mine are set to
>>
>> sysctl -w kern.sysv.shmma x=4194304
>> sysctl -w kern.sysv.shmmi n=1
>> sysctl -w kern.sysv.shmmn i=32
>> sysctl -w kern.sysv.shmse g=8
>> sysctl -w kern.sysv.shmal l=1024[/color]
>
>
> OK, this is what I was after. I did not have to change any of this for
> 7.3, but now 7.4 does not work unless on a standard OS X installation
> unless the user modifies these system settings. I can live with it, but
> I was hoping it might be a bug that could be resolved before the final
> 7.4. Every person that installs on OS X is going to run into this.
> Tracking it down and understanding what you have to change is not
> simple for us OS X users that are not familiar with Unix :).[/color]

Well I had to change this for 7.3 to run and I never set it back to try 7.4
with the defaults

But I think the 7.4 installation process checks for the best values of
max_connections and shared_buffers that for the current settings shmmax
settings

But someone else on the list is probably better to answer the technical
details of this

adam

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postg resql.org so that your
message can get through to the mailing list cleanly

**John DeSoi** · Nov 12 '05, 12:24 AM

Re: shared memory on OS X - 7.4beta4

On Sunday, October 26, 2003, at 05:51 PM, Alvaro Herrera wrote:
[color=blue]
> Huh, this is strange. IIRC during the installation process, initdb is
> supposed to determine which values actually work; those are affected by
> the shmmax operating system mostly, and the idea is to enable the
> installation process to pick more reasonable values out-of-the-box for
> memory usage that will work on your system.
>
> How did you install Postgres, and what values did initdb pick for
> shared_buffers and max_connections ?[/color]

Here were my steps to install and attempt to run it:

Install (clean install to a new directory)

../configure --prefix=/Unix/postgresql --enable-locale
--without-readline --with-includes=/sw/include --with-libraries=/sw/lib
--with-openssl=/sw/lib
make
make install

Initdb:

initdb -D /data/pgtest --no-locale

Startup:

pg_ctl start -D /data/pgtest -l /data/pgtest/logfile -o '-i -p 5432'

the postgresql.conf has
max_connections = 100
shared_buffers = 1000

And here are are the default settings for OS X 10.2.6:

sysctl -w kern.sysv.shmma x=4194304
sysctl -w kern.sysv.shmmi n=1
sysctl -w kern.sysv.shmmn i=32
sysctl -w kern.sysv.shmse g=8
sysctl -w kern.sysv.shmal l=1024

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

**Alvaro Herrera Munoz** · Nov 12 '05, 12:25 AM

Re: shared memory on OS X - 7.4beta4

On Sun, Oct 26, 2003 at 10:17:50PM -0500, John DeSoi wrote:
[color=blue]
> On Sunday, October 26, 2003, at 05:51 PM, Alvaro Herrera wrote:
>[color=green]
> >How did you install Postgres, and what values did initdb pick for
> >shared_buffe rs and max_connections ?[/color]
>
> the postgresql.conf has
> max_connections = 100
> shared_buffers = 1000
>
> And here are are the default settings for OS X 10.2.6:
>
> sysctl -w kern.sysv.shmma x=4194304[/color]

Well, I can only say that if I lower shmmax to that value, initdb picks
max_connections = 50
shared_buffers = 300

I remember there was a subtle bug in earlier versions corrected after
beta4. Can you try beta5 and see if it works for you? But with beta4 I
get 400/20 instead of 300/50, so this is probably not the same problem.

1000/100 are actually the largest probed values, so maybe initdb is
failing to probe or the postgresql.conf is not written correctly. Does
the initdb output actually say that it is going to use 1000/100? And is
the generated postgresql.conf by that initdb run the file that gets
actually used?

Maybe this is useless anyway, because I'm on Linux. Someone with an
actual OS X system could probably be a lot more helpful.

--
Alvaro Herrera (<alvherre[@]dcc.uchile.cl>)
"Cuando no hay humildad las personas se degradan" (A. Christie)

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

**John DeSoi** · Nov 12 '05, 12:25 AM

Re: shared memory on OS X - 7.4beta4

On Sunday, October 26, 2003, at 11:13 PM, Alvaro Herrera Munoz wrote:
[color=blue]
>
> I remember there was a subtle bug in earlier versions corrected after
> beta4. Can you try beta5 and see if it works for you? But with beta4
> I
> get 400/20 instead of 300/50, so this is probably not the same problem.[/color]

I mentioned earlier in the thread -- I did try beta5 -- same result.
[color=blue]
>
> 1000/100 are actually the largest probed values, so maybe initdb is
> failing to probe or the postgresql.conf is not written correctly. Does
> the initdb output actually say that it is going to use 1000/100? And
> is
> the generated postgresql.conf by that initdb run the file that gets
> actually used?
>[/color]

I don't recall seeing the initdb output, but I'm sure the generated
postgresql.conf is the one being used. I lowered max_connections and
saw lower numbers in the failed request when I tried to start the
server.

Best,

John DeSoi, Ph.D.

#-----------------------------------------------------------------------
----
# CONNECTIONS AND AUTHENTICATION
#-----------------------------------------------------------------------
----

# - Connection Settings -

#tcpip_socket = false
max_connections = 100
# note: increasing max_connections costs about 500 bytes of shared
# memory per connection slot, in addition to costs from shared_buffers
# and max_locks_per_t ransaction.
#superuser_rese rved_connection s = 2
#port = 5432
#unix_socket_di rectory = ''
#unix_socket_gr oup = ''
#unix_socket_pe rmissions = 0777 # octal
#virtual_host = '' # what interface to listen on; defaults to any
#rendezvous_nam e = '' # defaults to the computer name

# - Security & Authentication -

#authentication _timeout = 60 # 1-600, in seconds
#ssl = false
#password_encry ption = true
#krb_server_key file = ''
#db_user_namesp ace = false

#-----------------------------------------------------------------------
----
# RESOURCE USAGE (except WAL)
#-----------------------------------------------------------------------
----

# - Memory -

shared_buffers = 1000 # min 16, at least max_connections *2, 8KB each
#sort_mem = 1024 # min 64, size in KB
#vacuum_mem = 8192

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

**Adam Witney** · Nov 12 '05, 12:25 AM

Re: shared memory on OS X - 7.4beta4

On 27/10/03 5:46 am, "John DeSoi" <jd@icx.net> wrote:
[color=blue]
>
> On Sunday, October 26, 2003, at 11:13 PM, Alvaro Herrera Munoz wrote:
>[color=green]
>>
>> I remember there was a subtle bug in earlier versions corrected after
>> beta4. Can you try beta5 and see if it works for you? But with beta4
>> I
>> get 400/20 instead of 300/50, so this is probably not the same problem.[/color]
>
> I mentioned earlier in the thread -- I did try beta5 -- same result.
>[color=green]
>>
>> 1000/100 are actually the largest probed values, so maybe initdb is
>> failing to probe or the postgresql.conf is not written correctly. Does
>> the initdb output actually say that it is going to use 1000/100? And
>> is
>> the generated postgresql.conf by that initdb run the file that gets
>> actually used?
>>[/color]
>
> I don't recall seeing the initdb output, but I'm sure the generated
> postgresql.conf is the one being used. I lowered max_connections and
> saw lower numbers in the failed request when I tried to start the
> server.[/color]

My current settings are

max_connections = 32
shared_buffers = 100

But the settings picked automatically by the installation were

max_connections = 50
shared_buffers = 300

But I found this would not always start as I often have lots of applications
open (this is my laptop) and I had trouble running both 7.3 and 7.4beta5 at
the same time

Cheers

Adam

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postg resql.org

**Jim Crate** · Nov 12 '05, 12:26 AM

Re: shared memory on OS X - 7.4beta4

>> And here are are the default settings for OS X 10.2.6:[color=blue][color=green]
>>
>> sysctl -w kern.sysv.shmma x=4194304[/color]
>
>Well, I can only say that if I lower shmmax to that value, initdb picks
>max_connection s = 50
>shared_buffe rs = 300[/color]

I just did a fresh install of 7.4b5 on a fresh install of MacOS 10.3, and initdb
picked those same values for postgresql.conf . PostgreSQL started up the first
time successfully, but then refused to start again even after restarting the
computer until I reduced shared_buffers.

Now to figure out why sysctl -w kern.sysv.shmma x=<some_reasona ble_value> doesn't
seem to work on 10.3...

Jim Crate

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

**Eric Soroos** · Nov 12 '05, 12:26 AM

Re: shared memory on OS X - 7.4beta4

Slightly off topic confirmations.. .
[color=blue][color=green][color=darkred]
>>> And here are are the default settings for OS X 10.2.6:
>>>
>>> sysctl -w kern.sysv.shmma x=4194304[/color][/color][/color]

Seems to be the default on 10.2.8 as well.

10.1.5 doesn't seem to have a kern.sysv.* section:

[broccoli:~] erics% sysctl kern.sysv.shmma x
second level name sysv in kern.sysv.shmma x is invalid
[color=blue][color=green]
>>
>> Well, I can only say that if I lower shmmax to that value, initdb
>> picks
>> max_connections = 50
>> shared_buffers = 300[/color]
>
> I just did a fresh install of 7.4b5 on a fresh install of MacOS 10.3,
> and initdb
> picked those same values for postgresql.conf . PostgreSQL started up
> the first
> time successfully, but then refused to start again even after
> restarting the
> computer until I reduced shared_buffers.[/color]

I've had cases in 10.1.x where after hammering on the database for a
while and having to restart it, there was no longer enough shared
memory to start postgres, requiring a restart of the computer. On 10.1,
I couldn't even get it to change the shmmax value, necessitating some
very low buffer numbers (nearly out of the box for pg 7.2, yes, that
bad).

eric

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

PostgreSQL: Not Found

http://www.postgresql.org/docs/faqs/FAQ.html

shared memory on OS X - 7.4beta4

shared memory on OS X - 7.4beta4

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment