Leo + Python: the ultimate scripting tool

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Jegenye 2001 Bt

    #16
    Re: Leo + Python: the ultimate scripting tool: Conclusion


    Edward K. Ream <edreamleo@char ter.net> wrote in message
    news:vqq9rgp1gc l9cb@corp.super news.com...[color=blue][color=green]
    > > Sounds good. However, I won't try it until there's a shell-based mode;
    > > I do too much of my work remotely.[/color]
    >
    > Could you be more-specific about what you mean by "shell-based" mode?[/color]
    What[color=blue]
    > would have to be different in Leo?
    >
    > Are you talking about a version of Leo that doesn't put up a gui? If so,[/color]
    a[color=blue]
    > null-gui plugin might suffice. If Leo can't do what you want already, I
    > suspect that adding this feature would be easy.
    >
    > Edward
    >[/color]

    No, I think he means using Leo with a text terminal a la browsing the web
    with lynx.
    Btw, a web based interface would be absolutely cool.. Do you see chance to
    integrate Leo with, say, Zope?

    Miklós

    --
    PRISZNYÁK Miklós
    ---
    Jegenye 2001 Bt. ( jegenye2000@par khosting.moc ... to send mail increment
    the number and reverse the domainname.. )
    Egyedi szoftverkészíté s, tanácsadás
    Custom software development, consulting




    Comment

    • Edward K. Ream

      #17
      Re: Leo + Python: the ultimate scripting tool: Conclusion

      > Thanx so much for your thought-provoking, stimulating and clarifying
      ideas/comments.

      You are welcome. This is an exciting time for me.
      [color=blue]
      > Say, in a model building environment, first you dream about some fuzzy[/color]
      things...

      I'm glad you mentioned this. It's so important to be able to explore
      _easily_ while still being confused about what it is one is trying to do.
      On Leo's SF Forum I described how I ended up creating the script-find and
      script-change commands. I'll repeat it here, because it illustrates
      something crucial about executing Python scripts in Leo.

      [quote]
      As late as yesterday I wasn't sure how this would work, or if these new
      options were even necessary. Here is a summary of what I did yesterday.

      1. As usual, just being able to begin this project was very important. It
      turned out that the ability to execute test scripts rapidly without leaving
      Leo was crucial. I was able to run through many ideas quickly so that no
      intellectual momentum got wasted.

      2. I was quite confused about what was needed. After all, there are already
      a lot of find/change scripts in scripts/leoFindScript.p y. But these scripts
      are not interactive. I wanted to see if I could write an interactive find
      script without using Leo's Find panel. If I could, I might simplify the
      entire process: no changes required to Leo's find/change code.

      I soon discovered, however, exactly why Leo's Find panel is necessary. The
      problem is that interactive scripts need a _nonmodal_ dialog always present
      so that the user can choose "yes", "no" or "cancel". However, interacting
      with that non-modal dialog would be quite a problem for the script. Rather
      than reinventing the wheel, it is _so_ much easier just to use the Find
      panel that already exists. And of course, it makes sense to do this.

      3. So ok, we do, in fact, need Script Search and Script Change options in
      Leo's Find panel. The next big big question was: how to do this? Always
      present in my awareness was the fact the present find/change code is
      complex. A few experiments with hacking into this code made me leery of
      messing with this code. For one thing, the code is all Tk based.

      4. I messed around with some more test scripts. Don't remember how it
      happened, but finally a huge Aha happened: the Find/Change scripts in the
      Find panel don't need any help at all from Leo! They can essentially do
      everything on their own! Here is how I entered this Aha in my diary:

      - Leo doesn't clear app.searchDict. However, a _script_ can a) init itself
      and b) bring up the find window (Eureka!)

      - The find script is responsible for traversing the tree.

      - The find script is responsible for highlighting the found text or
      otherwise indicating to the change script what the found text was. For
      example, the re find script can set app.searchDict["m"] to the match object.

      - The find script must communicate with the change script. Leo does
      nothing.

      - The change script must change the text, usually by using
      v.setBodyString OrPane.

      BTW, the first Eureka seems almost irrelevant now, but somehow it was really
      important at the time. Anyway, you can see that these notes clearly
      indicate that Leo need not get intimately involved in "managing" the
      search/change scripts. This basic principle is the real Aha, and it made
      the rest of the work routine.

      5. With this clear vision in mind, it became routine to add support for
      find/change scripts. What I did:

      a) Added support for new c.pattern_match _flag and c.pattern_searc h_flag
      ivars. There was a digression until a fixed a config bug that was blocking
      initing of these ivars.

      b) Changed the top-level find/change code so that it simply calls code to
      execute the scripts in the find/change text area if the corresponding ivars
      were set.

      That's all!!
      [end quote]

      The point is this: I did _not_ have a clear picture of how script-find and
      script-change would work when I started, and moreover what picture I had was
      confused and wrong. Yet somehow it was easy to noodle around until the
      creative unconscious created an initial Aha. After that, it was smooth
      sailing.
      [color=blue]
      > Biology and bioinformatics (among many other domains) might gain a lot[/color]
      using the Leo/Python great combo.

      I've thought so too. One thing I didn't mention in my "seven reasons" is
      that Leo's outlines are isomorphic to dags (directed acyclic graphs). Dags
      are much _more_ powerful than completely general graphs. Indeed, the
      transitive closure of a general graph is just the entire graph again, so
      Leo's clones would not be possible (would not be well-defined) in a general
      graph.

      Edward
      --------------------------------------------------------------------
      Edward K. Ream email: edreamleo@chart er.net
      Leo: Literate Editor with Outlines
      Leo: http://webpages.charter.net/edreamleo/front.html
      --------------------------------------------------------------------


      Comment

      • Edward K. Ream

        #18
        Re: Leo + Python: the ultimate scripting tool: Conclusion

        > No, I think he means using Leo with a text terminal a la browsing the web
        with lynx.

        Whatever Aahz means, this idea has merit.
        [color=blue]
        > Btw, a web based interface would be absolutely cool..
        > Do you see chance to integrate Leo with, say, Zope?[/color]

        There is some great work being done on Leo. I've been very interested in a
        Leo + Zope combination, though I still don't understand what it means. Leo
        as a "web app" would indeed be cool. There have been discussion of using
        Flash plugins to host Leo as well...

        One thing I mustn't forget to mention: Rodrigo B. is working on "Leo on the
        Net" (LeoN for short). This will allow collaborative editing over the net
        of a single .leo file. This is based on recent computer science research
        and is leading-edge stuff. This is real. See the screen shots at:
        http://ryalias.freezope.org/souvenirs/leon/et2003 I would like Rodrigo to
        present his work at PyCon 2004. We shall see :-) For more details search on
        LeoN on Leo's SourceForge forums.

        Much of the reason for the 4.x code base is to support such things.

        Edward
        --------------------------------------------------------------------
        Edward K. Ream email: edreamleo@chart er.net
        Leo: Literate Editor with Outlines
        Leo: http://webpages.charter.net/edreamleo/front.html
        --------------------------------------------------------------------


        Comment

        • Douglas Alan

          #19
          Re: Leo + Python: the ultimate scripting tool: Conclusion

          aahz@pythoncraf t.com (Aahz) writes:
          [color=blue][color=green][color=darkred]
          >>> Sounds good. However, I won't try it until there's a shell-based mode;
          >>> I do too much of my work remotely.[/color][/color][/color]
          [color=blue][color=green]
          >>That's what TightVNC is for.[/color][/color]
          [color=blue]
          > Ugh. While I love what my new DSL can do, I don't think that running GUI
          > applications remotely is one of the things it *should* do, especially not
          > when text works so much better.[/color]

          I do so for hours every day with no problem.

          That's not to say that an ascii-only version of Leo might not be a
          good thing, but rather that in this day and age, not having one should
          hardly be a show-stopper.

          |>oug

          Comment

          • Mark Hahn

            #20
            Re: Leo + Python: the ultimate scripting tool: Conclusion

            This may be a really off-the-wall question, but how different would Leo be
            if the outline were just directories and text files? Would it be a set of
            file utilities and a kind of super-editor?

            It seems to me that a lot of Leo features can be found in standard
            directories and files.

            "Edward K. Ream" <edreamleo@char ter.net> wrote in message
            news:vqq69dh7rg 5ne7@corp.super news.com...[color=blue]
            > Conclusion
            >
            > All the power of Leo derives directly from the power, flexibility and
            > dynamism of Python. But making structure explicit as Leo outlines do adds
            > an important new dimension, literally and figuratively, to Python.
            > Moreover, Leo's Find/Change dialog creates a new environment from which to
            > run Python scripts. At long last the full power of the unification of code
            > and data is becoming apparent.
            >
            > Edward
            > --------------------------------------------------------------------
            > Edward K. Ream email: edreamleo@chart er.net
            > Leo: Literate Editor with Outlines
            > Leo: http://webpages.charter.net/edreamleo/front.html
            > --------------------------------------------------------------------
            >
            >[/color]


            Comment

            • Edward K. Ream

              #21
              Re: Leo + Python: the ultimate scripting tool: Conclusion

              > how different would Leo be if the outline were just directories and text
              files? Would it be a set of file utilities and a kind of super-editor?

              I'm glad you asked this question. There are several kinds of replies. I
              suspect point 3 answers your question most directly, but points 1 and 2 seem
              like necessary background:

              1. Yes, you could simulate the effects I talk about in various ways.
              Because directories are hierarchical, one can no doubt simulate almost any
              kind of hierarchical operation using files and directories. Whether that
              simulation is easy, safe, powerful and natural is another matter. It is
              vastly easier (and safer) to create, delete and reorganize nodes in Leo
              outlines than it would be to do the corresponding operations on files and
              directories.

              The main topic of my postings has been the happy interaction between Leo
              outlines and scripts. These interactions would not be nearly so strong if
              all hierarchies had to simulated via the file system. I used the term
              "momentum" in one of the postings. Intellectual momentum is really
              important, and we don't want to slow that momentum by simulating things
              indirectly that can much more easily be done directly.

              For example, I presume unit test would typically be organized in folders to
              make it possible to create unit tests dynamically. And yes, you could use
              something like grep to organize search and replace operations. However,
              grep is just another wimpy re tool, so the simulation is weak.

              2. Leo itself has many file-oriented features. In particular, Leo provides
              a way of associating parts of outlines with so-called "derived files". Leo
              can change derived files based on changes to Leo outlines. Very
              importantly, Leo can do the reverse: Leo can change outlines based on
              changes to derived files.

              3. I have devoted a _huge_ amount of wondering, over a period of at least 10
              years, about whether one could eliminate .leo files entirely, somehow
              replacing .leo files with the union of all derived files. It turns out that
              the answer must be "no". Without going into details, trying to organize
              derived files without a .leo file would be like trying to work in an IDE
              without a project file. The analogy isn't exact, but the conclusion is
              inescapable: Leo needs .leo files. Moreover, Leo needs information _not_
              contained in derived files in order to make derived files friendly to cvs.
              In the 4.0 code base this extra information is the tnodeList attribute of
              <v> elements of .leo files. Finally, it is just plain more convenient to
              squirrel away information in an outline than it would be to create separate
              files. In other words, we wouldn't want to give up .leo files even if it
              were possible.

              In short, there are extremely strong reasons why Leo works the way it does.
              If you want more details, look at the detailed design discussions that have
              appeared on Leo's SF forums over the past year or so. I'm always glad to
              talk about Leo, so feel free to ask more questions if you like.

              Edward

              P.S. You question got me thinking along a new and amusing line: .leo files
              simulate (in quite a few ways) a file system, with clones, for example,
              acting much like shortcuts or links. Actually, implementing Leo's outline
              operations is quite a bit more complex than implementing the core of a
              unix-like file system. I can say this because I've done both.

              EKR
              --------------------------------------------------------------------
              Edward K. Ream email: edreamleo@chart er.net
              Leo: Literate Editor with Outlines
              Leo: http://webpages.charter.net/edreamleo/front.html
              --------------------------------------------------------------------


              Comment

              • Aahz

                #22
                Re: Leo + Python: the ultimate scripting tool: Conclusion

                In article <lcd6c2pjwq.fsf @gaffa.mit.edu> ,
                Douglas Alan <nessus@mit.edu > wrote:[color=blue]
                >aahz@pythoncra ft.com (Aahz) writes:[color=green]
                >>Douglas Alan removed one attribution:[color=darkred]
                >>>Douglas Alan removed Aahz's attribution:
                >>>>
                >>>> Sounds good. However, I won't try it until there's a shell-based mode;
                >>>> I do too much of my work remotely.
                >>>
                >>>That's what TightVNC is for.[/color]
                >>
                >> Ugh. While I love what my new DSL can do, I don't think that running GUI
                >> applications remotely is one of the things it *should* do, especially not
                >> when text works so much better.[/color]
                >
                >I do so for hours every day with no problem.[/color]

                That's not what I said.
                [color=blue]
                >That's not to say that an ascii-only version of Leo might not be a
                >good thing, but rather that in this day and age, not having one should
                >hardly be a show-stopper.[/color]

                That's your opinion. I disagree, at least to the extent that I won't
                use any editing environment that doesn't have a text-only version.
                --
                Aahz (aahz@pythoncra ft.com) <*> http://www.pythoncraft.com/

                "It is easier to optimize correct code than to correct optimized code."
                --Bill Harlan

                Comment

                • Edward K. Ream

                  #23
                  Re: Leo + Python: the ultimate scripting tool: Conclusion

                  > I won't use any editing environment that doesn't have a text-only version.

                  Your call, of course.

                  I just found the following at:



                  [quote]
                  In a world of X displays, one might ask ``why bother''? It's true that
                  character-cell display terminals are an obsolete technology, but there are
                  niches in which being able to do fancy things with them are still valuable.
                  One is on small-footprint or embedded Unixes that don't carry an X server.
                  Another is for tools like OS installers and kernel configurators that may
                  have to run before X is available.
                  [end quote]

                  It seems to me that simulating Leo on a character-cell display is going to
                  be all work and no gain. I have no interest in designing a dumbed-down
                  model for Leo that would work in paleolithic environments. Besides, Emacs
                  probably has a curses mode. Why would I want to compete with that?

                  So I am not going to do a text-only mode for Leo unless somebody comes up
                  with a pretty good reason why Leo would be an appropriate tool in
                  non-windowing environments. Starting up Leo in script mode is still on the
                  to-do list.

                  Edward
                  --------------------------------------------------------------------
                  Edward K. Ream email: edreamleo@chart er.net
                  Leo: Literate Editor with Outlines
                  Leo: http://webpages.charter.net/edreamleo/front.html
                  --------------------------------------------------------------------


                  Comment

                  • Aahz

                    #24
                    Why text-only? (was Re: Leo + Python: the ultimate scripting tool: Conclusion)

                    In article <vqsmdi8tub9c74 @corp.supernews .com>,
                    Edward K. Ream <edreamleo@char ter.net> wrote:[color=blue]
                    >
                    >It seems to me that simulating Leo on a character-cell display is
                    >going to be all work and no gain. I have no interest in designing a
                    >dumbed-down model for Leo that would work in paleolithic environments.
                    >Besides, Emacs probably has a curses mode. Why would I want to compete
                    >with that?[/color]

                    That's fine, although for me you're competing with vi[m]. There are two
                    reasons why I stick with my requirement for text-only systems:

                    * As I said earlier, I do much of my work from text consoles, and much of
                    that is also remote. I think trying to run GUIs over Net connections is
                    poor use of bandwidth. Not to mention the fact that until a month ago,
                    my only Net access was still direct-dial shell (yes, vt100 emulator, no
                    PPP). Even now that I do have DSL, I still do much of my GUI browsing
                    (when forced to use JavaScript) with images turned off.

                    * Requiring a text-only system is a reasonable proxy for requiring a
                    keyboard-centric system, because text-only systems by definition have to
                    work with a keyboard. Too many GUI-based systems have at least one
                    oddball corner that just doesn't work well with the keyboard, and it's
                    invariably one of those corners that I need to be productive.

                    This is an old argument, of course. I've been using the .sig I'm
                    including below for more than five years.
                    --
                    Aahz (aahz@pythoncra ft.com) <*> http://www.pythoncraft.com/

                    I surf faster than you do, monkey boy. (My take on Netscape vs. Lynx)

                    Comment

                    • Jegenye 2001 Bt

                      #25
                      Re: Leo + Python: the ultimate scripting tool: Conclusion


                      Edward K. Ream <edreamleo@char ter.net> wrote in message
                      news:vqqq2mk11j e1e6@corp.super news.com...
                      [color=blue]
                      > One thing I mustn't forget to mention: Rodrigo B. is working on "Leo on[/color]
                      the[color=blue]
                      > Net" (LeoN for short). This will allow collaborative editing over the net
                      > of a single .leo file. This is based on recent computer science research
                      > and is leading-edge stuff. This is real. See the screen shots at:
                      > http://ryalias.freezope.org/souvenirs/leon/et2003 I would like Rodrigo to
                      > present his work at PyCon 2004. We shall see :-) For more details search[/color]
                      on[color=blue]
                      > LeoN on Leo's SourceForge forums.
                      >
                      > Much of the reason for the 4.x code base is to support such things.
                      >
                      > Edward[/color]

                      Thanks, Edward, I'll check out this. But before that a little ranting..
                      Speaking of a multi-user version, security comes to mind.
                      Like you described, the scripts are omnipotent within the .leo files and
                      itself Leo.. and this is no problem if I use my own outlines.
                      However when it comes to collaborating and sharing outlines... well, you
                      know. (You just cannot check each and every line in that way cool outline
                      someone wrote.)
                      Then we have the Microsoft (macro virus) symptom.
                      So I wonder how this is going to be addressed with LeoN... This is not
                      easy, say, the Bastion/rexec thing was removed from Python...
                      Zope has its own access right management and that could be perhaps used
                      somehow and most importantly in a transparent way..

                      Best,
                      Miklós


                      Comment

                      • Edward K. Ream

                        #26
                        Re: Leo + Python: the ultimate scripting tool: Conclusion

                        > Speaking of a multi-user version, security comes to mind.

                        Thanks for mentioning this. Security has been discussed at length on Leo's
                        SF forums. It is a _really_ serious problem for an environment as powerful
                        as Leo and Python. The general principle is: you (or your scripts) must
                        NEVER execute a script blindly. As you say, that is not so easy to do.
                        Certainly rexec would not help at all: a Leo script could create a time-bomb
                        and still be legal as far as rexec is concerned. The only solution is a)
                        know what scripts you are executing and b) know who you are working with.
                        Using Leo + Python is like using a chain saw: you had best keep it under
                        control, and you had best know what you are doing.

                        Edward
                        --------------------------------------------------------------------
                        Edward K. Ream email: edreamleo@chart er.net
                        Leo: Literate Editor with Outlines
                        Leo: http://webpages.charter.net/edreamleo/front.html
                        --------------------------------------------------------------------


                        Comment

                        • Edward K. Ream

                          #27
                          Re: Leo + Python: the ultimate scripting tool: Conclusion

                          > Such a request is *NOT* a legitimate 'anti-junk-mail' measure.

                          Yes, it is. If you aren't willing to take 20 sec. to talk to me, then I'm
                          not willing to talk to you. Bye.

                          Edward
                          --------------------------------------------------------------------
                          Edward K. Ream email: edreamleo@chart er.net
                          Leo: Literate Editor with Outlines
                          Leo: http://webpages.charter.net/edreamleo/front.html
                          --------------------------------------------------------------------


                          Comment

                          • Edward K. Ream

                            #28
                            Re: Leo + Python: the ultimate scripting tool: gui issues

                            Several responses mentioned gui issues, which have little to do with
                            scripting. However, gui issues are near and dear to my heart, so I thought
                            I would summarize how things stand with Leo and guis:

                            1. Leo is in the midst of a global reorg that will separate gui-dependent
                            code from a gui-independent "core". This is going well, and will be
                            completed in about a week. This reorg is the basis for the 4.1 code base.
                            4.1 final is due out in December; there will be a lengthy period of testing
                            (including many new unit tests) while the code settles down.

                            A prototype plugin for Leo already exists that replaces Leo's default
                            Tk/tkinter gui with a wxPython gui. The original version of the prototype
                            had to duplicate a large amount of Leo's core to be even partly functional.
                            With the new code base the wxPython gui only needs to know how to do generic
                            wxPython kinds of stuff. All knowledge of Leo's core is handled by base
                            classes that are in Leo's core.

                            In another reply I misspoke: I said that anygui probably wasn't up to the
                            job. In fact, with the new organization it would be dead easy to create a
                            plugin to support anygui instead of, say, wxPython. Experimenting with
                            anygui would be virtually risk free: it would take an hour or so of time.
                            N.B. Gui plugins are _completely_ responsible for handling all gui-related
                            events. In particular, the interface between Leo's core and gui plugins
                            ignores event handling completely. This greatly simplifies the interface
                            and ensures that no assumptions at all about event handling "infect" Leo's
                            core. I was originally dubious about matching anygui's model of events.
                            Now I see the fear was unfounded.

                            2. One of my hopes in writing the original series of postings was that
                            there would be some discussion of integrating Leo with Emacs using pymacs.
                            This may have started to happen. I would really like help with this
                            project, and I'll support anyone working on such a project in any way I can.
                            I think running some kind of Leo window on Emacs would be totally cool.

                            3. Several people have mentioned that it would be good to start up Leo in
                            scripting mode. There are several easy ways to do this. It will happen
                            eventually: don't know when. Right now it seems more like a cute feature
                            than something vital...In any event, there will eventually be a way of
                            running Leo without any gui at all: a null-gui plugin probably could do this
                            with about 10 lines of code.

                            Edward
                            --------------------------------------------------------------------
                            Edward K. Ream email: edreamleo@chart er.net
                            Leo: Literate Editor with Outlines
                            Leo: http://webpages.charter.net/edreamleo/front.html
                            --------------------------------------------------------------------


                            Comment

                            • Edward K. Ream

                              #29
                              Re: Leo + Python: the ultimate scripting tool: gui issues

                              I forgot to mention that Leo already has plugins that allow fairly easy
                              integration with vim, xemacs and word. Another plugin allows full
                              customization of Leo's Open With menu. These plugins aren't perfect, and
                              they are a real good start.

                              These plugins were easy to write. For example, the guts of the xemacs
                              plugin is:

                              def open_in_emacs (tag,keywords):
                              if top():
                              top().frame.OnO penWith(("os.sp awnl",path,None ),)

                              Tighter integration with emacs/xemacs would take more work, and it would be
                              well worth it. I hope that it will happen soon. The Aha for me came when I
                              saw tetris in an xemacs window. xemacs probably can host Leo and make Leo
                              look good. I see hints about "foreign" windows in the emacs docs, and I
                              haven't followed up on that yet...

                              Edward
                              --------------------------------------------------------------------
                              Edward K. Ream email: edreamleo@chart er.net
                              Leo: Literate Editor with Outlines
                              Leo: http://webpages.charter.net/edreamleo/front.html
                              --------------------------------------------------------------------


                              Comment

                              • JanC

                                #30
                                Re: Leo + Python: the ultimate scripting tool: Conclusion

                                "Edward K. Ream" <edreamleo@char ter.net> schreef:
                                [color=blue][color=green]
                                >> Such a request is *NOT* a legitimate 'anti-junk-mail' measure.[/color]
                                >
                                > Yes, it is. If you aren't willing to take 20 sec. to talk to me, then
                                > I'm not willing to talk to you. Bye.[/color]

                                One question: why not let people do this "confirmati on" through email
                                instead of requiring them to start a separate program? Something like a
                                mailing list confirmation mail?

                                --
                                JanC

                                "Be strict when sending and tolerant when receiving."
                                RFC 1958 - Architectural Principles of the Internet - section 3.9

                                Comment

                                Working...