Choosing Perl/Python for my particular niche

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Tassilo v. Parseval

    #16
    Re: Choosing Perl/Python for my particular niche

    Also sprach Cameron Laird:
    [color=blue]
    > I generally favor Python over Perl; in the absence of
    > more details, I think there's mild evidence that those
    > who don't specialize in programming are happier over the
    > long term with the readability of the former. They're
    > very close, though. The one most certain advantage
    > Python boasts is a dimension that I'd think important
    > to you, although you haven't mentioned it: Python's
    > interfaces to C++ are MUCH easier to manage than Perl's.
    > Perl6 will change this. For now, though, it's FAR
    > easier to practice "dual-level programming" with Python
    > and C++. If you have a large existing library of C++
    > work, I think that tips the balance toward Python.
    > Understand, SWIG and other alternatives make Perl-to-C++
    > links possible; with Python's facilities, though,
    > including Pyrex, they're *fun*.[/color]

    When it comes to embedding C/C++ into Perl, SWIG is the last thing that
    I'd recommend. It may be capable of transforming such a library into a
    Perl module on its own with no user intervention. However, C/C++ doesn't
    map particularly well onto Perl from an interface point-of-view: A 1:1
    translation of a C++ API doesn't look particularly good.

    Those things should be done manually. How much work this is depends on
    the library. Some only require a few lines of XS. XS' learning curve is
    admittedly a bit steep. Not because it is difficult in itself but rather
    because the documentation leaves a few things to be desired. There are
    some actions on completing perlxstut.pod on its way right now, though.

    There is also Inline::CPP, although I don't think it is as mature as
    Inline::C. And since you have mentioned Tcl, even that can be inlined
    with the help of Inline::Tcl.

    Tassilo
    --
    $_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_ $;//::niam/s~=)]3[))_$-3(rellac(=_$({
    pam{rekcahbus}) (rekcah{lrePbus })(lreP{rehtona bus})!JAPH!qq(r ehtona{tsuJbus# ;
    $_=reverse,s+(? <=sub).+q#q!'"q q.\t$&."'!#+sex isexiixesixeseg ;y~\n~~dddd;eva l

    Comment

    • Cameron Laird

      #17
      Combinatorics of dual-language use (was: Choosing Perl/Python for my particular niche)

      In article <c44ine$ad6$1@n ets3.rz.RWTH-Aachen.DE>,
      Tassilo v. Parseval <tassilo.parsev al@post.rwth-aachen.de> wrote:

      Comment

      • Fred Ma

        #18
        Re: Choosing Perl/Python for my particular niche


        "Tassilo v. Parseval" wrote:[color=blue]
        >
        > Also sprach Fred Ma:
        >[color=green]
        > > No XML processing or database interaction in what I do.[/color]
        >
        > Who knows, though. The desire to do certain things grows with the skills one
        > acquired.[/color]

        You might be right. Some digital CAD data is stored in XML.
        [color=blue]
        > This is a strong reason to use Perl. While it may not always be desirable to
        > toe the line, here it is. The availability of tools that can easily be
        > integrated into one's own work will save you a lot of work on the long run. I
        > didn't know that Perl was particularly strong in the field of IC design. If
        > so, good then.
        >
        > Apart from this particular case, it's generally a good thing when pre-written
        > code exists that can be used. The amount of such code for Perl is immense
        > (several thousand libraries) and gathered all in one place (the CPAN) with a
        > unified interface to access and install them and a vast infrastructure built
        > around it.
        >
        > I don't know how much or whether at all the situation for Python has
        > significantly improved over the recent past. A while ago at least there was no
        > such thing.[/color]

        SNIP
        [color=blue]
        > As with text processing, both Python and Perl offer all the essential things
        > needed for that. On a cursory glance, the score of them is tied.
        >
        > Perl wins when it comes to one-liners and maybe replacing shell scripts. As I
        > tend to forget the syntax of bash's scripting language easily, I've replaced
        > it entirely with Perl.
        >
        > Perl's bad reputation for big projects is probably due to the variance of
        > those people using Perl. One large group is sys-admins who use Perl in an
        > entirely different way as compared to someone building larger systems.
        > However, there are enough very large Perl projects that show that it can be
        > used for that just as easily (at least as long as one keeps the sys-admins
        > away from the source;-).
        >[color=green]
        > > I've seen mention of parrot and perl6, which is quite a ways off. I'm not
        > > sure how much that should weigh into my decision, since it isn't real yet.[/color]
        >
        > It shouldn't. Perl6 is mostly yet another leap towards fitness for large
        > projects. I don't think it will gain much for those more hackish tasks. Also,
        > no one quite knows when Perl6 will be due.[/color]


        Thanks for the commentary on Perl, for clarifying the source of unweldiness in
        big Perl projects, as well as its nonuniversality .

        Fred

        Comment

        • Petri

          #19
          Re: Choosing Perl/Python for my particular niche

          In article <40652B0D.7C313 F77@doe.carleto n.ca>, Fred Ma says...[color=blue]
          > One thing I expect to have to do is to modify design
          > files. For example, there is a tool which takes ASCII
          > hardware desscription language (HDL) and converts it
          > to a C++ (augmented by hardware simulation library).
          > The translator is freeware, so has limitations which I
          > have to make up for by tweaking the HDL code.[/color]

          The translator, is it doing the Verilog stuff you mention later in the thread?
          Why not check if you can't replace the translator altogether with an existing
          Perl module:


          Well, I know nothing about your problem area, but it's a tip in case you hadn't
          thought about checking CPAN.


          Petri

          Comment

          • Paddy McCarthy

            #20
            Re: Choosing Perl/Python for my particular niche

            Hi Fred,
            If there are particular C/C++ packages of algorithms that you use then
            you might try searching for the package name +python on the web. You
            might find someone who has already turned the library into aPython
            module.

            There are several Scientific libraries for Python that have good
            graphing capabilities as well as speed advantages for certain
            operations. - Try installing the Enthought Python distribution for
            windows at: http://www.enthought.com/python/

            Another good page is ScriptEDA:
            http://www-cad.eecs.berkeley.edu/~pinhong/scriptEDA/ - I was using the
            gate-level verilog parser a month ago.

            On Scripting language choice, I find TCL to be in many EDA tools, but
            its datastructures and syntax make it a language I use when I have
            to, rather than a language of choice.

            Perl IS used a lot in EDA but I like to think of it as being dangerous
            in a lot of hands as great Engineers start scripting larger and larger
            applications in Perl and usually end up with something that is hard to
            maintain and hard to write. Perl does have its sweet spot - I do like
            the `perl -p -i -e` mewthod of making sed like alterations to files
            in-place, butthe languages support for subroutine arguments - compared
            to Pythons; and the need to compute and pass around references to
            lists and hashes for simple nested datastructures - things like that
            shout small programs only to me.

            Python is my language of choice for a large amount of programming in
            the EDA field (when given the choince :-).

            Perl has a large library, Python has a large library - what you need
            to do is do some web searches for libraries in your field to see which
            is more appropriate -

            When I have to write an algorithm from scratch then I do it in Python.
            It does get out of the way and allow me to concentrate on the
            algorithm most of the time. (Although I wish it had constrained random
            generation of integers like Vera or Specman :-).

            I hope I've helped, Paddy.

            Comment

            • Cameron Laird

              #21
              Appreciation for the importance of readability (was: Choosing Perl/Python for my particular niche)

              In article <40657D14.3FBE7 E82@doe.carleto n.ca>,
              Fred Ma <fma@doe.carlet on.ca> wrote:

              Comment

              • Fred Ma

                #22
                Re: Choosing Perl/Python for my particular niche

                Hi, Paddy,

                Wow, there's more information coming in than I expected. Having
                a bit of a nonengineering day (at least not at the ground level)
                keeping up. I did ask for it.

                Paddy McCarthy wrote:[color=blue]
                >
                > Hi Fred,
                > If there are particular C/C++ packages of algorithms that you use then
                > you might try searching for the package name +python on the web. You
                > might find someone who has already turned the library into aPython
                > module.[/color]

                Actually, verilator (does verilog to SystemC, which is a C++ library
                that provides HDL simulation capability) is a Perl-based module.

                As for the glue functionality I need to modify the my original verilog
                source, there's no specific functionality. I'm still trying to think
                of ways in which to modify to code to work around the translation
                limitations of verilator. So the filter will be very much ad-hoc.
                [color=blue]
                > There are several Scientific libraries for Python that have good
                > graphing capabilities as well as speed advantages for certain
                > operations. - Try installing the Enthought Python distribution for
                > windows at: http://www.enthought.com/python/
                >
                > Another good page is ScriptEDA:
                > http://www-cad.eecs.berkeley.edu/~pinhong/scriptEDA/ - I was using the
                > gate-level verilog parser a month ago.[/color]

                Wow. The screen shots on for enthought are impressive. The software at
                scriptEDA reminds me that I'm an a newbie in this area. The level to
                which interfacing environments and languages have been brought boggles
                the mind. I think that I need to take a bite of Perl and/or Python
                before I even consider looking at the offerings there with any degree
                of appreciation. For the computation engine and post-process/graphing,
                I'll stick with C++/matlab for now, due to lack of thesis time, and
                because I've got a familiar system (or "idiom"?) setup to use them
                together. As I mentioned in another post, I'll start with Perl as
                the starting point, and keep tabs on Python, something that is feasible
                because of its readability.
                [color=blue]
                > On Scripting language choice, I find TCL to be in many EDA tools, but
                > its datastructures and syntax make it a language I use when I have
                > to, rather than a language of choice.
                >
                > Perl IS used a lot in EDA but I like to think of it as being dangerous
                > in a lot of hands as great Engineers start scripting larger and larger
                > applications in Perl and usually end up with something that is hard to
                > maintain and hard to write. Perl does have its sweet spot - I do like
                > the `perl -p -i -e` mewthod of making sed like alterations to files
                > in-place, butthe languages support for subroutine arguments - compared
                > to Pythons; and the need to compute and pass around references to
                > lists and hashes for simple nested datastructures - things like that
                > shout small programs only to me.[/color]

                I got the same impression, but there have been other explanations given
                in this thread for the unweldiness of big projects, as well as the fact
                that this is not universal. Myself, I will be using "P..." for glue
                functionality, so I won't run the risk of creating dangerously
                unmaintainable code (finger crossed here). Besides, it's for the thesis,
                which has to be finished soon, so I won't have the problem of looking
                back a ways to figure out code written long ago. By the time I get to
                an industrial situation (fingers crossed, since jobs are leaving North
                America), I hope to have a bit for breathing time to ramp up on Python.
                [color=blue]
                > Python is my language of choice for a large amount of programming in
                > the EDA field (when given the choince :-).
                >
                > Perl has a large library, Python has a large library - what you need
                > to do is do some web searches for libraries in your field to see which
                > is more appropriate -[/color]

                Well...I did it...verilog to SystemC boils down to verilator unless one
                has money to shell out. Despite its professed limitations, I am still
                astounded to read even the help pages. That degree of sophistication,
                for free. Kind of makes the thesis look like small time, though of
                course, the challenges are of a different nature.
                [color=blue]
                > When I have to write an algorithm from scratch then I do it in Python.
                > It does get out of the way and allow me to concentrate on the
                > algorithm most of the time. (Although I wish it had constrained random
                > generation of integers like Vera or Specman :-).
                >
                > I hope I've helped, Paddy.[/color]

                A great deal. Thanks.

                Fred

                Comment

                • Fred Ma

                  #23
                  Re: Tangent on embeddability (was: Choosing Perl/Python for myparticular niche)

                  Cameron Laird wrote:[color=blue]
                  >
                  > In article <roy-3EDA29.08465327 032004@reader1. panix.com>,
                  > Roy Smith <roy@panix.co m> wrote:
                  > .
                  > .
                  > .[color=green]
                  > >2) It's dead simple to embed in a C/C++ application. This is one place
                  > >where it really outshines Python. It's literally two lines of C to get
                  > >a Tcl interpreter up and running. It's equally trivial to extend Tcl by
                  > >providing functions in C. Compare this to the native Python/C
                  > >interface, boost.python, or swig, and you really appreciate the
                  > >difference!
                  > >
                  > >For most general purpose tasks, I reach for Python first. But if I
                  > >needed to provide a quick scripting interface to an existing C library,
                  > >or wanted to embed some easy scriptability into a large C/C++ app, Tcl
                  > >is what I would be looking at.[/color]
                  >
                  > Me, too, to a large extent. I think you overstate Tcl's
                  > advantage, though, Roy, in a couple of ways. First, I
                  > agree that Tcl is objectively easier and more reliable
                  > in embedding; Python still has blemishes in ease-of-use
                  > and portability in this role. It can be hard getting
                  > all of Python embedded correctly, especially, in my ex-
                  > perience, when working on multiple platforms.
                  >
                  > However, Tcl has historically been so easy that it hasn't
                  > bothered documenting the process. Python has a definitive
                  > (if aging) embedding manual that I regard as more inviting
                  > than what Tcl offers.
                  >
                  > Also, ctypes, Boost and Pyrex *are* successful, and appear
                  > to me progressing faster than any of their Tcl cousins.
                  >
                  > Yet another way to look at the comparison is that a sig-
                  > nificant number of people rely on SWIG, and, to them,
                  > Tcl and Python are indistinguishab le.
                  >
                  > Follow-ups narrowed. Most of Perl's response to all of
                  > this is, "Wait 'til 6!"[/color]


                  Thanks, Roy & Cameron, for the glimpse into TCL. I had just
                  a passing bout with it in trying to use Synopsys's TCL based
                  shell. It was far too brief to be a fair assessment, since
                  the documentation wasn't quite up to it, and I fell back to
                  their conventional shell (which itself was suppose to be
                  a brief detour to get some design information mulched out
                  from some verilog files). Certainly didn't get to the level
                  of using it to interface to C libraries. For the most part,
                  however, I'll be using "P..." for data conversion rather
                  than coordination of different library routines. I've got
                  a pretty good template worked out with matlab on top and
                  C++ on the bottom. I'll probably stick with this model until
                  I graduate. I'm looking at "P..." to help with converting
                  design information from verilog to SystemC (C++ library for
                  digital hardware simulation) so that I can use it in the
                  matlab/C++ model, but it won't be part of the main line of
                  computation.

                  Fred

                  Comment

                  • Fred Ma

                    #24
                    Re: Choosing Perl/Python for my particular niche

                    Paul McGuire wrote:[color=blue]
                    >
                    > "Fred Ma" <fma@doe.carlet on.ca> wrote in message
                    > news:40652B0D.7 C313F77@doe.car leton.ca...
                    > <snip>[color=green]
                    > > One thing I expect to have to do is to modify design
                    > > files. For example, there is a tool which takes ASCII
                    > > hardware desscription language (HDL) and converts it
                    > > to a C++ (augmented by hardware simulation library).
                    > > The translator is freeware, so has limitations which I
                    > > have to make up for by tweaking the HDL code.[/color]
                    >
                    > If you eventually find yourself in the Python realm, please look into the
                    > pyparsing text parsing module, more information at
                    > http://pyparsing.sourceforge.net/. I have implemented an easily-extended
                    > 99% Verilog parser using this module, and it may provide some shortcuts for
                    > you in dealing with your HDL files.
                    >
                    > -- Paul McGuire[/color]


                    Hi, Paul,

                    I took a look at your website. I've decided to go Perl for now,
                    and ramp up on Python on the side. I think a parser has a higher
                    level of intelligence than regex'ing, but I hesitate to jump into
                    it at the moment because thesis time is running out. I may do some
                    adhoc regex'ing, either with sed/Perl/gvim (it's quick and dirty,
                    but suitable for the time crunch of my current circumstance, which
                    is a hard deadline on the thesis). Also, I'm doing the quick-and-
                    dirty because of limitations of an *existing* verilog parser (and
                    translator) which I don't want to delve into right now, for the
                    same reason. But thanks for the heads up. If things work out in
                    the long run, and I get to know Python, I know there is a verilog
                    parser to keep an eye out for.

                    Fred

                    Comment

                    • Fred Ma

                      #25
                      Re: Choosing Perl/Python for my particular niche

                      Petri wrote:[color=blue]
                      >
                      > In article <40652B0D.7C313 F77@doe.carleto n.ca>, Fred Ma says...[color=green]
                      > > One thing I expect to have to do is to modify design
                      > > files. For example, there is a tool which takes ASCII
                      > > hardware desscription language (HDL) and converts it
                      > > to a C++ (augmented by hardware simulation library).
                      > > The translator is freeware, so has limitations which I
                      > > have to make up for by tweaking the HDL code.[/color]
                      >
                      > The translator, is it doing the Verilog stuff you mention later in the thread?
                      > Why not check if you can't replace the translator altogether with an existing
                      > Perl module:
                      > http://search.cpan.org/search?query=verilog&mode=all
                      >
                      > Well, I know nothing about your problem area, but it's a tip in case you hadn't
                      > thought about checking CPAN.
                      >
                      > Petri[/color]

                      Actually, I was spending much of last week installing many
                      of those tools! The translater (verilator) is Perl-based,
                      written by Wilson Snyder (author of many of those tools)
                      and relies on many of those tools. But thanks for the tip...

                      Fred

                      Comment

                      • Fred Ma

                        #26
                        Re: Choosing Perl/Python for my particular niche

                        Jim Keenan wrote:[color=blue]
                        >
                        > You are at Carleton University, which was the site of Yet
                        > Another Perl Conference::Can ada in May 2003. So you have an excellent
                        > pool of local Perl experts to draw upon -- and there are many more not
                        > far away in Montreal, Toronto and Kitchener/Waterloo. Look them up
                        > via http://www.pm.org/groups/north_america.html .[/color]


                        Thanks for the pointer, Jim.

                        Fred

                        Comment

                        • Fred Ma

                          #27
                          Re: Combinatorics of dual-language use

                          Paul Prescod wrote:[color=blue]
                          >
                          > Neither Pyrex nor Inline::C are going to be great for C++. They were
                          > both created by C programmers for C programmers (I'd like to see this
                          > change about Pyrex some day). BOOST::Python, on the other hand, is
                          > probably a good fit.[/color]


                          I just checked out Boost (visited there before based on a pointer
                          by Scott Meyers, author of a book on the STL). I wasn't quite
                          prepared to put the time into getting to know libraries at
                          that time, and I really needed to get to know basic standard
                          C++ library before delving into expansions of them, but the
                          example for the python interface looks very cool. My current
                          use of a "P..." type scripting language, though, wouldn't be to
                          replace my matlab wrapper around C++ (the main engine until I
                          graduate). It would be to mulch data to get it in a palatable
                          form for the engine code. It's good to know about the python
                          interface, though, for future investigation. Thanks.

                          Fred

                          Comment

                          • Michele Simionato

                            #28
                            Re: Appreciation for the importance of readability (was: Choosing Perl/Python for my particular niche)

                            claird@lairds.c om (Cameron Laird) wrote in message news:<106ce9eae d1sk61@corp.sup ernews.com>...[color=blue]
                            > Some scientists are very successful with "slash-and-burn"
                            > techniques. That manifests itself in this area as "con-
                            > science-free" coding: they write whatever programs get
                            > them the results they're after, and don't care that no
                            > one else can reproduce their results, nor can they them-
                            > selves, six months later. A LOT of that goes on.
                            >
                            > It makes me uncomfortable. A lesson I've learned over
                            > and over is that computer programs live far longer than
                            > you expect. It pays to do 'em better at the beginning,
                            > 'cause you're likely to live with them quite a while.
                            >
                            > I'm not a successful scientist, though; odds are long
                            > against me ever being one.
                            >[/color]

                            The relation between programming and science is rather complex.
                            There are still lots of sciences where the programming part is
                            very minor and the "slash-and-burn" technique is the right thing
                            to do. For instance, I have worked with very successful scientists
                            using "slash-and-burn". Their requirements were the following:

                            1. we want to spend at most 5% of our time doing coding and 95% doing
                            science;
                            2. our group is composed by two persons and our code will never be
                            released;
                            3. if a Ph. D. student reach the group and has to study the code, we
                            will explain it to him in person;
                            4. the largest program we will ever write is a thousand lines long;
                            for the rest we will use standard routines and write as little code
                            as possible;
                            5. we don't care at all about fancy graphics: the output will be a list
                            of numbers write on a file and displayed by gnuplot;
                            6. all we care is numerical performance in floating point computations;
                            the logic of the program will be trivial.

                            They actually used old fashioned Fortran 77 and I think they made the perfect
                            choice given their requirements; they essentially had to do numerical
                            integrations and solving differential equations, tasks were Fortran is
                            incredibly strong. Other persons I know doing particle Physics and doing a
                            good deal of programming (good deal means something like 50% of their time
                            spent on programming; spending more than 50% means that you are actually
                            a programmer, and scientists don't want to do that) were using a mixture
                            of Fortran and C. Here I am referring to people doing numerically
                            intensive tasks; I have seen a trend to switch to C++ for large projects,
                            since there is a perception than C++ scales better than Fortran. In
                            any case, the core routines are still in Fortran and always will be.
                            For scripting tasks, scripting languages are used; I know a few
                            astronomers who were using Perl to control the telescope movement.
                            Many physicists/astronomers use Python too. Specialized languages
                            and tools (Mathematica, Maple, Matlab, Octave, Yorick, etc. etc.)
                            have their place.

                            Finally, it may be weird in XXI century, but there are still scientists
                            not doing programming. For instance, the only programming language I used
                            for my scientific research was Latex, to write the papers; my reseach tools
                            were paper and pencil and blackboard, which are still as effective as
                            ever.

                            Michele Simionato

                            Comment

                            • Ville Vainio

                              #29
                              Re: Choosing Perl/Python for my particular niche

                              >>>>> "Cameron" == Cameron Laird <claird@lairds. com> writes:
                              [color=blue][color=green]
                              >> Since you happily use Perl, Python or Tcl, I have a
                              >> question. How do you decided which one to use? :-)
                              >>
                              >> Robert[/color][/color]

                              Cameron> Most often, whatever is already at hand. It'll be
                              Cameron> interesting some day to come across an organization that
                              Cameron> doesn't already have a standard in place.

                              Many organizations don't, esp. if the actual product they produce is
                              not written in either of the languages. Some code may exist, but there
                              is no "official" standard. And if there is, there is a good
                              possibility of changing it if you are able to demonstrate clear
                              technical superiority in skunkworks projects. And w/ Python, you
                              are. Been there, done that :-).

                              --
                              Ville Vainio http://tinyurl.com/2prnb

                              Comment

                              • Max M

                                #30
                                Re: Choosing Perl/Python for my particular niche

                                Fred Ma wrote:
                                [color=blue]
                                > I took a look at your website. I've decided to go Perl for now,
                                > and ramp up on Python on the side.[/color]


                                That's fine. Good luck. Many of us have tried it.

                                Just remember, that when you have been swimming under the dark cold
                                waters of Perl for a few weeks, you can allways swim up to the surface
                                and gasp a few fresh breaths of python.

                                You will then find that those few breaths will take you as far as the
                                previuos weeks did.


                                ;-)

                                regards Max M

                                Comment

                                Working...