#!/usr/bin/python or #!/usr/bin/env python?

[Erik Max Francis]

Re: #!/usr/bin/python or #!/usr/bin/env python?

Stephan Kuhagen wrote:

Yes, the same happens with all Tcl-Scripts. I like to see this as a bug in
"file", not in the scripting...

How does that make sense? `file` cannot possibly understand the
semantics of files at that level, at least not without executing them.
And that's exactly what you _don't_ want to do when you're using `file` ...

The file _is_ a /bin/sh executable. You're just having that /bin/sh
executable run something else -- how could `file` figure that out
without a ridiculously complicated set of rules that rise to the level
of a sh interpreter -- thereby, defeating the purpose?

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Only those who dare to fail greatly can ever achieve greatly.
-- Robert F. Kennedy

[Tobias Brox]

Re: #!/usr/bin/python or #!/usr/bin/env python?

[Erik Max Francis

How does that make sense? `file` cannot possibly understand the
semantics of files at that level, at least not without executing them.
And that's exactly what you _don't_ want to do when you're using `file` ...

This is very off-topic, but if it's fairly common to begin tcl-scripts
as a /bin/sh-file with "exec tcl" at one of the first lines, I think
"file" ought to be able to recognize it.

"""exec" python is clearly an obscure hack not used by many, so I
don't see why "file" should ever recognize that :-)

--
Tobias Brox, 69°42'N, 18°57'E

[Erik Max Francis]

Re: #!/usr/bin/python or #!/usr/bin/env python?

Tobias Brox wrote:

This is very off-topic, but if it's fairly common to begin tcl-scripts
as a /bin/sh-file with "exec tcl" at one of the first lines, I think
"file" ought to be able to recognize it.
>
"""exec" python is clearly an obscure hack not used by many, so I
don't see why "file" should ever recognize that :-)

The point is, they're all part of the same tactic -- the particulars of
sh. Special casing each one is a task without an end. People will come
up with variants that will do the right thing but foil `file`,
intentionally or unintentionally -- just as we've seen in this thread.

The right way to approach this with `file` is to acknowledge that such
tricks are inherently sh-specific and leave it identified as a sh file.
Because that is, of course, exactly what it is.

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Only those who dare to fail greatly can ever achieve greatly.
-- Robert F. Kennedy

[Stephan Kuhagen]

Re: #!/usr/bin/python or #!/usr/bin/env python?

Tobias Brox wrote:

This is very off-topic,

Sorry for starting that...

but if it's fairly common to begin tcl-scripts
as a /bin/sh-file with "exec tcl" at one of the first lines, I think
"file" ought to be able to recognize it.
>
"""exec" python is clearly an obscure hack not used by many, so I
don't see why "file" should ever recognize that :-)

That's what I meant. It is okay for a Python-script not to be recognized by
"file" that way, it is enough that I can do it, if I need to. But file
should recognize this for Tcl, because it is common there. And if it needs
to work for Tcl only, one can construct a simple mechanism for "file" to
check this.

Stephan

[ZeD]

Re: #!/usr/bin/python or #!/usr/bin/env python?

Erik Max Francis wrote:

The file _is_ a /bin/sh executable. You're just having that /bin/sh
executable run something else -- how could `file` figure that out
without a ridiculously complicated set of rules that rise to the level
of a sh interpreter -- thereby, defeating the purpose?

but...

$ cat test.py
#!/usr/bin/env python

print "Hello, world"
$ file test.py
file.py: a python script text executable

following what you said, test.py is a /usr/bin/env script, not a python one.

--
Under construction

[Tobias Brox]

Re: #!/usr/bin/python or #!/usr/bin/env python?

[Erik Max Francis]

The point is, they're all part of the same tactic -- the particulars of
sh. Special casing each one is a task without an end. People will come
up with variants that will do the right thing but foil `file`,
intentionally or unintentionally -- just as we've seen in this thread.

I have no clue about Tcl, but it was said some postings ago that "all
Tcl-scripts are recognized as sh". If this is true, I think it ought
to be fixed. Searching for one of the very first lines to begin with
#, end with \, and followed by "exec tcl" doesn't sound like
rocket-science to me, though I've neither studied magic(4) or Tcl
scripting conventions.

I do understand that it would still be many ways to start a Tcl-script
without getting recognized by the above logic, as well as ways to
trick file into reporting a real shell script to be a Tcl-script.
IMHO a 90% hit rate is far better than 0% hit rate.

The right way to approach this with `file` is to acknowledge that such
tricks are inherently sh-specific and leave it identified as a sh file.
Because that is, of course, exactly what it is.

A shell script containing some inline tcl is a shell script, though
when the only shell-command is "start up tcl" and the rest of the file
is tcl-code, I really don't think it can be defined as a "shell
script" anymore. Particularly not if almost all tcl-scripts are
started that way.

--
Tobias Brox, 69°42'N, 18°57'E

[msoulier]

Re: #!/usr/bin/python or #!/usr/bin/env python?

John Salerno wrote:

I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other?

I personally prefer being explicit over implicit, but then, that is in
the Zen of Python.

I work on machines with multiple interpreters installed. I find picking
the first one in the path to be very annoying. That means that it's
going to work differently for different people based on their PATH.

Mike

[Christophe]

Re: #!/usr/bin/python or #!/usr/bin/env python?

Stephan Kuhagen a écrit :

>Always prefer to use env over a hardcoded path, because that hardcoded
>path will invariably be wrong. (Yes, for those about to nitpick, it's
>conceivable that env might be somewhere other than /usr/bin. However,
>that is very rare and results in a no-win situations regardless of the
>issue of where Python is installed.)

>
Don't yell at me for bringing in another language, but I really like the
trick, Tcl does:
>

> #!/bin/sh
> # The next line starts Tcl \
> exec tclsh "$0" "$@"

>
This works by the somewhat weird feature of Tcl, that allows comments to be
continued in the next line with "\" at the end of the comment-line. It
looks unfamiliar, but has several advantages, I think. First it's really
VERY unlikely, that there is no /bin/sh (while I found systems with
different places for env), and you can add some other features at or before
the actual call of the interpreter, i.e. finding the right or preferred
version... - This way I coded a complete software-installer, that runs
either as a Tcl/Tk-Script with GUI, or as bash-script, when no Tcl is
available. - I really would like to have something like that for python,
but I did not find a way to achieve that, yet.
>
Regards
Stephan
>

What about that one ( windows .bat file ) :

@echo off
rem = """-*-Python-*- script
rem -------------------- DOS section --------------------
rem You could set PYTHONPATH or TK environment variables here
python -x %~f0 %*
goto exit

"""
# -------------------- Python section --------------------
import sys
print sys.argv


DosExitLabel = """
:exit
rem """


I'm sure it can be updated to work on Unix too :)

[Aahz]

Re: #!/usr/bin/python or #!/usr/bin/env python?

In article <fj5Cg.2658$No6 .51984@news.tuf ts.edu>,
John Salerno <johnjsal@NOSPA Mgmail.comwrote :

>
>I understand the difference, but I'm just curious if anyone has any
>strong feelings toward using one over the other? I was reading that a
>disadvantage to the more general usage (i.e. env) is that it finds the
>first python on the path, and that might not be the proper one to use. I
>don't know if that's a real issue most of the time, but it's at least
>something to consider.

The main argument against the env method is that I've seen a fair number
of sysadmins claim that it's less secure. I'm not competent to judget
that claim myself, but I prefer to play safe and stay away from env.
--
Aahz (aahz@pythoncra ft.com) <* http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it." --Brian W. Kernighan

[Erik Max Francis]

Re: #!/usr/bin/python or #!/usr/bin/env python?

ZeD wrote:

but...
>
$ cat test.py
#!/usr/bin/env python
>
print "Hello, world"
$ file test.py
file.py: a python script text executable
>
following what you said, test.py is a /usr/bin/env script, not a python one.

That's rather a silly interpretation of what I said, I think. After all::

% cat test.gargle
#!/usr/bin/env gargle

asdfasflsafhkls fhklsdfhsdf
max@oxygen:~/tmp% file test.gargle
test.gargle: a gargle script text executable

`file` knows how /usr/bin/env is used because it's not only a standard
Unix idiom, it's the primary purpose for which it exists. That's not
the case with endless variants of /bin/sh scripts whose first statements
involve running another interpreter in some clever way that's not
detected by _that_ interpreter's processing, which is what the
previously discussed tricks amount to.

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Punctuality is the virtue of the bored.
-- Evelyn Waugh

[Erik Max Francis]

Re: #!/usr/bin/python or #!/usr/bin/env python?

Tobias Brox wrote:

A shell script containing some inline tcl is a shell script, though
when the only shell-command is "start up tcl" and the rest of the file
is tcl-code, I really don't think it can be defined as a "shell
script" anymore. Particularly not if almost all tcl-scripts are
started that way.

The problem is that there are endless ways to do that, and figuring out
all the cases makes `file` an sh interpreter, not the magic number
detector it's supposed to be.

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Punctuality is the virtue of the bored.
-- Evelyn Waugh

[olsongt@verizon.net]

Re: #!/usr/bin/python or #!/usr/bin/env python?


Aahz wrote:

In article <fj5Cg.2658$No6 .51984@news.tuf ts.edu>,
John Salerno <johnjsal@NOSPA Mgmail.comwrote :

I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other? I was reading that a
disadvantage to the more general usage (i.e. env) is that it finds the
first python on the path, and that might not be the proper one to use. I
don't know if that's a real issue most of the time, but it's at least
something to consider.

>
The main argument against the env method is that I've seen a fair number
of sysadmins claim that it's less secure. I'm not competent to judget
that claim myself, but I prefer to play safe and stay away from env.
--
Aahz (aahz@pythoncra ft.com) <* http://www.pythoncraft.com/
>

Basically, someone could inject an arbirtrary script called 'python'
into your path that does whatever (rm -fr /) under your user context
when you run the script. But the same thing would happen if you run
'python test.py' instead of '/usr/local/bin/python test.py' to run a
script that doesn't have a she-bang or hasn't been flagged as
executable. Some admins will use a fully-qualified path for every
command to guard against this; I think that can be overkill.

[Erik Max Francis]

Re: #!/usr/bin/python or #!/usr/bin/env python?

olsongt@verizon .net wrote:

Basically, someone could inject an arbirtrary script called 'python'
into your path that does whatever (rm -fr /) under your user context
when you run the script. But the same thing would happen if you run
'python test.py' instead of '/usr/local/bin/python test.py' to run a
script that doesn't have a she-bang or hasn't been flagged as
executable. Some admins will use a fully-qualified path for every
command to guard against this; I think that can be overkill.

The primary guard for this is not having publicly-writable things in
your PATH. In other words, this is the argument for not putting things
like /tmp or . (because you might cd to somewhere publicly writable like
/tmp) in your PATH, not really for avoiding /usr/bin/env in hash bangs.

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
We must all hang together, or, most assuredly, we will all hang
separately. -- John Hancock

[olsongt@verizon.net]

Re: #!/usr/bin/python or #!/usr/bin/env python?


Erik Max Francis wrote:

olsongt@verizon .net wrote:
>

Basically, someone could inject an arbirtrary script called 'python'
into your path that does whatever (rm -fr /) under your user context
when you run the script. But the same thing would happen if you run
'python test.py' instead of '/usr/local/bin/python test.py' to run a
script that doesn't have a she-bang or hasn't been flagged as
executable. Some admins will use a fully-qualified path for every
command to guard against this; I think that can be overkill.

>
The primary guard for this is not having publicly-writable things in
your PATH. In other words, this is the argument for not putting things
like /tmp or . (because you might cd to somewhere publicly writable like
/tmp) in your PATH, not really for avoiding /usr/bin/env in hash bangs.
>

Yeah, I agree 100%. I guess the best way to put it is that if random
people are creating scripts called 'python' in your path, you've got
bigger problems than /usr/bin/env calling the wrong program or a plain
old 'python test.py' doing the same.

[Stephan Kuhagen]

Re: #!/usr/bin/python or #!/usr/bin/env python?

Erik Max Francis wrote:

The problem is that there are endless ways to do that, and figuring out
all the cases makes `file` an sh interpreter, not the magic number
detector it's supposed to be.

It makes it into a pattern-matcher, not an interpreter. But that it is
already.

But right, there are endless ways to do that, but only one or a very small
subset is common. The way I cited is the way mentioned in the
Tclsh-manpage, so it can be (not must, but can) as the standard-header of a
Tcl-Script on Unix-like systems.

Even if the construct sometimes differs a little bit, "file" should be able
to identify, since the manpage of "file" says, that it is not a pure
magic-number-detector. The first part explains how the magic-number-thing
works and then what is done, when that fails:

"If a file does not match any of the entries in the magic file, it is
examined to see if it seems to be a text file.
[...]
Once file has determined the character set used in a text-type file, it will
attempt to determine in what language the file is written. The language
tests look for particular strings (cf names.h) that can appear anywhere in
the first few blocks of a file. For example, the keyword .br indicates
that the file is most likely a troff(1) input file, just as the keyword
struct indicates a C program. These tests are less reliable than the
previous two groups, so they are performed last."

This is not the most reliable way, as the man-page says, but it should work:
if in the first some blocks you can find a statement with a continued
comment-line and the exec-trick, "file" can at least guess, that it is a
Tcl-file. So this would be a valid method to detect that files type,
because a troff-file is a troff file, even when there is no .br in the
first few blocks, but "file" tries to identify it anyway. "file" is not
meant to be to ignore troff files, just because they are sometimes hard to
detect. The same applies to Tcl-files, I think. Not perfectly reliable, but
worth a try, since it is, according to the Tclsh-manpage, the common
header-pattern for a Tcl-script.

So "file" can not be perfect and reliable in every case, but it should try
to take a good guess. If you do not care about the Tcl-headers (and why
should you, this is comp.lang.pytho n... ;-), you are right with your
reasoning. But if you accept, that file can not be perfect anyway and want
it to be as good as possible, then it is some kind of bug or missing
feature in "file" that it recognizes (or tries to) some morphing file
formats but not another (which is fairly wide spread, even if Tcl is not a
modern buzz-word-language these days).

Stephan