Python/C and PYTHONPATH

**Jeff Epler** · Jul 18 '05, 07:18 AM

Re: Python/C and PYTHONPATH

You'd need to tell us how you're invoking Python. When the Python
executable is invoked without arguments, the current directory is on the
sys.path:

$ python
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.[color=blue][color=green][color=darkred]
>>> import sys
>>> sys.path[0][/color][/color][/color]
''

When you run a script, the directory of that script is in sys.path[0]:
$ echo "import sys; print sys.path[0]" > /tmp/script.py
$ python /tmp/script.py
/tmp

Some Python editors may put the directory where they reside on sys.path,
but not the current directory. They might do other startup that is
different from normal. The newest Python I have on this machine
is 2.3b1, and its idle puts these elements at the start of Pythonpath:

Python 2.3b1 (#16, May 19 2003, 10:22:28)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "copyright" , "credits" or "license" for more information.
IDLE 0.8 -- press F1 for help[color=blue][color=green][color=darkred]
>>> import sys
>>> sys.path[:2][/color][/color][/color]
['/home/jepler', '/usr/local/bin']

.... so instead of "the current directory" ('' or '.'), sys.path[0]
is "the directory idle was invoked from", and that's followed by "the
directory where the idle script lives".

Jeff

**Tero Pihlajakoski** · Jul 18 '05, 07:18 AM

Re: Python/C and PYTHONPATH

Jeff Epler <jepler@unpytho nic.net> wrote:[color=blue]
> You'd need to tell us how you're invoking Python. When the Python[/color]

Take a look at:
http://www.python.org/doc/current/ex...embedding.html. Basically,
it's about same (and I tested with it too, same results: failing)
[color=blue]
> executable is invoked without arguments, the current directory is on the
> sys.path:[/color]
[color=blue]
> $ python
> Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
> [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
> Type "help", "copyright" , "credits" or "license" for more information.[color=green][color=darkred]
>>>> import sys
>>>> sys.path[0][/color][/color]
> ''[/color]
[color=blue]
> When you run a script, the directory of that script is in sys.path[0]:
> $ echo "import sys; print sys.path[0]" > /tmp/script.py
> $ python /tmp/script.py
> /tmp[/color]

If I run "python" as a shell (python <scriptname>) , everything works ok.
Says path is ok: [ '', '<thelib>', ... ]. However, from C, something works
differently: PyRun_SimpleStr ing("import sys\nprint sys.path"); Shows no ''
(the rest is ok). The question is: Why is there no '', and what is the
"official/safe" way to set it? Just insert it to the list? (Setting
"PYTHONPATH = " helps, but I'd like it to run "off the box")

Tero

**Samuel Walters** · Jul 18 '05, 07:18 AM

Re: Python/C and PYTHONPATH

Fair warning: At this point, I couldn't even get the example you quoted
to link properly. Of course, I didn't dig too deep, as I'm in a hurry.

throw this c code into a .h file (my name for it is envdump.h)

---code---
#include <stdio.h>

extern char **environ;

void fdump_env(FILE *fd)
{
char **ptr;
int i = 0;
ptr = environ;
while(ptr[i] != NULL)
{
fprintf(fd, "%i : %s\n", ptr[i]);
i++;
}
}

void dump_env()
{
char **ptr;
int i = 0;
ptr = environ;
while(ptr[i] != NULL)
{
printf("%i : %s\n", i, ptr[i]);
i++;
}
}
---code---

Now, just inside the main function (or, perhaps, wherever you please, but
I'd try the line right before the first if statement) call dump_env() and
it will spit out all the environment variables that your C program
received. You might find something there that's of interest.

Or, call setenv ("man 3 setenv") to manually set a python-path.
you can use that in combo with getenv ("man 3 getenv") to append to an
existing PYTHONPATH, if you so choose.

Give it a go, please let me know how it turns out. If this doesn't shed
some light on the matter I might have another idea once I'm not rushing
around.

HTH

Sam Walters

--
Never forget the halloween documents.

Page not found – Open Source Initiative

http://www.opensource.org/halloween/

""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

**Tero Pihlajakoski** · Jul 18 '05, 07:18 AM

Re: Python/C and PYTHONPATH

Samuel Walters <swalters_usene t@yahoo.com> wrote:
[color=blue]
> Fair warning: At this point, I couldn't even get the example you quoted
> to link properly. Of course, I didn't dig too deep, as I'm in a hurry.[/color]
[color=blue]
> ---code---[/color]

.... snip ...
[color=blue]
> Or, call setenv ("man 3 setenv") to manually set a python-path.
> you can use that in combo with getenv ("man 3 getenv") to append to an
> existing PYTHONPATH, if you so choose.[/color]
[color=blue]
> Give it a go, please let me know how it turns out. If this doesn't shed
> some light on the matter I might have another idea once I'm not rushing
> around.[/color]

(Sorry for the delay, I was offline)

I'll see if it's actually the C-part that's causing problems, but I
worked it around by adding:

PyRun_SimpleStr ing("import sys\nsys.path.i nsert(0,'')");

right after the Py_Initialize() .

Works ok. Guess I'm "allowed" to do that(?)

Thanks,
- Tero

--

**Samuel Walters** · Jul 18 '05, 07:19 AM

Re: Python/C and PYTHONPATH

|Thus Spake Tero Pihlajakoski On the now historical date of Mon, 05 Jan
2004 22:39:19 +0000|[color=blue]
> I'll see if it's actually the C-part that's causing problems, but I worked
> it around by adding:
>
> PyRun_SimpleStr ing("import sys\nsys.path.i nsert(0,'')");
>
> right after the Py_Initialize() .
>
> Works ok. Guess I'm "allowed" to do that(?)[/color]
IMLK (In My Limited Knowledge) that seems okay, but it also feels a bit
ham-handed. try this snippet:

---untested code---
#include <string.h>
#include <stdlib.h>
/* prototyping may not be neccessary... dunno...*/
extern char *getenv(const char *name);
extern char *setenv(const char *name, const char *value, int overwrite);
/* comment them out if gcc complains about redeclarations. */

/* append something to pythonpath */
/* returns 0 on failure, 1 on creation of the env variable and 2 on an append
oh, and -1 in cases of catastrophic miscompilation */
int myPyPathAppend( char *extrapath)
{
char *buffer = NULL;
/* size to taste... You should do this dynamically to avoid future buffer overrun attacks*/
char eventual_path[1024] = { 0 };
/* take note: after getenv, buffer points to an external constant character buffer.
do NOT try to modify it directly. use strcpy(char *dest, char *src)
(he says knowingly... is she into photography mate?)
*/
if( (buffer = getenv("PYTHONP ATH")) == NULL )
{
/* we're here because PYTHONPATH is not already part of the environment. */

setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONP ATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 1;
}
return -1; /* dead code... should never reach here */
}
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */

/* find the "=" in the buffer...
from string.h
extern char *strstr (__const char *__haystack, __const char *__needle)
there's a better way to do this, but I can't recall the function off the top of my head
*/
buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */

/* copy the old PYTHONPATH string */
strcpy(eventual _path, buffer);
strcat(eventual _path, ";");
strcat(eventual _path, extrapath);

setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONP ATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 2;
}
return -1; /* dead code... should never reach here */
}
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */

/* find the "=" in the buffer...
from string.h
extern char *strstr (__const char *__haystack, __const char *__needle)
there's a better way to do this, but I can't recall the function off the top of my head
*/
buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */

/* copy the old PYTHONPATH string */
strcpy(eventual _path, buffer);
strcat(eventual _path, ";");
strcat(eventual _path, extrapath);

setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONP ATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 2;
}
return -1; /* dead code... should never reach here */
}
return -1; /* deader code... should *really* never reach here */
}
---untested code---
I haven't tested, compiled or even read through this code.
I'm late for a party and still added comments
That means you get punctuation patrol :-P
Check the semicolons, check the braces
Hey, I hear that in some companies they call this teamwork methodology
"extreme-programming" We're buzzword compliant!

HTH
(Danm... I'm such a code monkey)

--
Never forget the halloween documents.

Page not found – Open Source Initiative

http://www.opensource.org/halloween/

""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

**Tero Pihlajakoski** · Jul 18 '05, 07:19 AM

Re: Python/C and PYTHONPATH

Samuel Walters <swalters_usene t@yahoo.com> wrote:[color=blue]
> |Thus Spake Tero Pihlajakoski On the now historical date of Mon, 05 Jan
> 2004 22:39:19 +0000|[color=green]
>> I'll see if it's actually the C-part that's causing problems, but I worked
>> it around by adding:
>>
>> PyRun_SimpleStr ing("import sys\nsys.path.i nsert(0,'')");
>>
>> right after the Py_Initialize() .
>>
>> Works ok. Guess I'm "allowed" to do that(?)[/color]
> IMLK (In My Limited Knowledge) that seems okay, but it also feels a bit
> ham-handed. try this snippet:[/color]

Ok, there are comments here, somewhere:
[color=blue]
> ---untested code---
> #include <string.h>
> #include <stdlib.h>
> /* prototyping may not be neccessary... dunno...*/
> extern char *getenv(const char *name);
> extern char *setenv(const char *name, const char *value, int overwrite);
> /* comment them out if gcc complains about redeclarations. */[/color]
[color=blue]
> /* append something to pythonpath */
> /* returns 0 on failure, 1 on creation of the env variable and 2 on an append
> oh, and -1 in cases of catastrophic miscompilation */
> int myPyPathAppend( char *extrapath)
> {
> char *buffer = NULL;
> /* size to taste... You should do this dynamically to avoid future buffer overrun attacks*/
> char eventual_path[1024] = { 0 };
> /* take note: after getenv, buffer points to an external constant character buffer.
> do NOT try to modify it directly. use strcpy(char *dest, char *src)
> (he says knowingly... is she into photography mate?)
> */
> if( (buffer = getenv("PYTHONP ATH")) == NULL )
> {
> /* we're here because PYTHONPATH is not already part of the environment. */
>
> setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
> /* did it go happen .. you should check this more rigorously*/
> if( (buffer = getenv("PYTHONP ATH")) == NULL)
> {
> /* we failed... abend. */
> return 0;
> }
> else
> {
> /* success! cheers! */
> return 1;
> }
> return -1; /* dead code... should never reach here */
> }
> else
> {
> /* PYTHONPATH already exists. append ';', then our new path and update it. */[/color]

Here. I might not want to add ';' or ':', depending on the OS (probably
not)? I can solve this with #ifdefs for WIN32 and Linux, but everytime I
want to run it on a new system, I'd have to find out the delimiter... It
also needs a buffer underrun check on that [1024]. I'll stick with the
PyRun_... for now, but I'll definitely save this code, so thanks. Again.

Also, found this piece from sys.path docs (now that my net is up and
running):
... "A program is free to modify this (sys.path) list for its own
purposes." ...

[color=blue]
>
> /* find the "=" in the buffer...
> from string.h
> extern char *strstr (__const char *__haystack, __const char *__needle)
> there's a better way to do this, but I can't recall the function off the top of my head
> */
> buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */
>
> /* copy the old PYTHONPATH string */
> strcpy(eventual _path, buffer);
> strcat(eventual _path, ";");
> strcat(eventual _path, extrapath);[/color]
[color=blue]
> setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
> /* did it go happen .. you should check this more rigorously*/
> if( (buffer = getenv("PYTHONP ATH")) == NULL)
> {
> /* we failed... abend. */
> return 0;
> }
> else
> {
> /* success! cheers! */
> return 2;
> }
> return -1; /* dead code... should never reach here */
> }[/color]

One if and two elses, have you started "getting ready" for the party
already? ;) Or maybe it's fuzzy logic ;)
[color=blue]
> else
> {
> /* PYTHONPATH already exists. append ';', then our new path and update it. */
> /* find the "=" in the buffer...[/color]

.... snip ...
[color=blue]
> }
> return -1; /* deader code... should *really* never reach here */
> }
> ---untested code---
> I haven't tested, compiled or even read through this code.
> I'm late for a party and still added comments
> That means you get punctuation patrol :-P
> Check the semicolons, check the braces
> Hey, I hear that in some companies they call this teamwork methodology
> "extreme-programming" We're buzzword compliant![/color]

- Tero

--

**Jeff Epler** · Jul 18 '05, 07:19 AM

Re: Python/C and PYTHONPATH

On Mon, Jan 05, 2004 at 04:17:52PM +0000, Tero Pihlajakoski wrote:[color=blue]
> Jeff Epler <jepler@unpytho nic.net> wrote:[color=green]
> > You'd need to tell us how you're invoking Python. When the Python[/color]
>
> Take a look at:
> http://www.python.org/doc/current/ex...embedding.html. Basically,
> it's about same (and I tested with it too, same results: failing)[/color]

Here's the program I came up with:

#include <Python.h>

int
main(int argc, char *argv[])
{
Py_SetProgramNa me(argv[0]);
Py_Initialize() ;
PyRun_SimpleStr ing("import sys; print sys.path, sys.executable" );
PySys_SetArgv(a rgc-1, argv+1);
PyRun_SimpleStr ing("import sys; print sys.path, sys.executable" );
Py_Finalize();
return 0;
}

This program demonstrates that the initial sys.path element
corresponding to the location of the script is created when
PySys_SetArgv() is called:

$ ./embed
['/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]
['', '/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]

That first element depends on the location of the script, as shown here:

$ ./embed /tmp/x
['/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]
['/tmp', '/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]

I don't know where this is documented---PySys_SetArgv is mentioned in
the "api" document, but this side-effect is not:

Welcome to Python.org

http://www.python.org/doc/current/api/embedding.html#l2h-35

The official home of the Python Programming Language

Welcome to Python.org

http://www.python.org/doc/current/api/initialization.html#l3h-713

The official home of the Python Programming Language

You might want to explore just what PySys_SetArgv and submit a
documentation patch on sourceforge, both for the api document and for
the ext document. Interestingly, calling PySys_SetArgv multiple times
inserts multiple items in sys.path.

Jeff

**Samuel Walters** · Jul 18 '05, 07:19 AM

Re: Python/C and PYTHONPATH

Ack... definitely not pep-7 compliant code

And to think... A co-worker once called me "obsessive" about indention.
(They say "obsessive, " I say "consistent and legible.")

Well, It seems PAN likes to perform a stealth word-wrap when you edit from
the non-attached editor. (I used vim because I'm comfy hammering out code
there) Nope, on second look, I must have accidentally mashed some keys
while in vim. That extraneous "else" block is just a copy of the "else"
block.

If you run it through indent, and clean up a couple of oddball line-wraps
it will at least be legible. Remove the unneeded prototypes, embarrassing
extra else block, and it will compile. Remove the whole block about
searching for the = and fix the typo in the second setenv call (extrapath
should be eventual_path) and it will run without segfault. Then replace
the ";" with ":" and it will run properly on a linux system. For myself,
I made the snippet a little more general and added the option of choosing
which pathlist env variable and changed the function name to
envPathAppend. Then it's not a bad little snippet.

|Thus Spake Tero Pihlajakoski On the now historical date of Tue, 06 Jan
2004 01:04:31 +0000|
[color=blue]
> Here. I might not want to add ';' or ':', depending on the OS (probably
> not)? I can solve this with #ifdefs for WIN32 and Linux, but everytime I
> want to run it on a new system, I'd have to find out the delimiter... It
> also needs a buffer underrun check on that [1024]. I'll stick with the
> PyRun_... for now, but I'll definitely save this code, so thanks. Again.[/color]

That buffer overrun is not the only bit of braindead logic in that
snippet. The "success/failure" condition testing leaves a couple of loose
ends.
[color=blue]
> Also, found this piece from sys.path docs (now that my net is up and
> running):
> ... "A program is free to modify this (sys.path) list for its own
> purposes." ...[/color]

Hmmm... I suppose it's a matter of aesthetics and scope of the project
you're working on. If you're writing a big program for which the python
interpretor is only one small or medium part of the functionality and the
majority is c/c++, then you're opening up a whole big realm of
cross-platform issues. However, if the bulk of your code is python, then
perhaps you should have the main program be in python with your extra c
code as a module. That would be cleaner and probably more portable.

I suspect that at this point you're just trying to tinker with the
interface, then use what's most comfortable. Now at least, you have an
option. Besides, it seems that someone already posted a cleaner fix than
I could have given you.

HTH

Sam Walters.

--
Never forget the halloween documents.

Page not found – Open Source Initiative

http://www.opensource.org/halloween/

""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Python/C and PYTHONPATH

Python/C and PYTHONPATH

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment