Include pysqlite2 into Python 2.5?

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Gerhard Haering

    #1

    Include pysqlite2 into Python 2.5?

    [This is a repost from a message to python-dev on the users' list,
    because python-dev-ers thought a broader discussion of this is a good
    idea.]

    Last December, we had a short thread discussing (in python-dev) the
    integration of PySQLite into Python 2.4. At the time, I was against
    inclusion, because I thought PySQLite was not ripe for it, mostly
    because I thought the API was not stable.

    Now, I have started writing a new PySQLite module, which has the
    following key features:

    - Uses iterator-style SQLite 3.x API: sqlite3_compile , sqlite3_step()
    etc. This way, it is possible to use prepared statements, and for
    large resultsets, it requires less memory, because the whole
    resultset isn't fetched into memory at once any longer.

    - Completely incompatible with the SQLite 0.x/1.x API: I'm free to
    create a much better API now.

    - "In the face of ambiguity, refuse the temptation to guess." -
    PySQLite 1.x tries to "guess" which Python type to convert to. It's
    pretty good at it, because it queries the column type information.
    This works for, I'd say 90 % of all cases at least. But as soon as
    you use anything fancy like functions, aggregates or expressions in
    SQL, the _typeless_ nature of SQLite breaks through and it will tell
    us nothing about the declared column type (of course, because the
    data is not coming from a database column).

    So I decided to change the default behaviour and make PySQLite
    typeless by default, too. Everything will be returned as a Unicode
    string (the default might be user-configurable per connection).

    Unless, unless of course the user explicitly activates the
    "guess-mode" ;-) But to do so, she must read the docs then she will
    be aware of the fact that it only works in 90 % of all cases.

    So why am I bothering you about this?

    I think that a simple embedded relational database would be a good
    thing to have in Python by default. And as Python 2.5 won't happen
    anytime soon, there's plenty of time for developing it, getting it
    stable, and integrating it.

    Especially those of you that have used PySQLite in the past, do you
    have any suggestions that would make the rewrite a better candidate
    for inclusion into Python?

    One problem I see is that even the new PySQLite will grow and try to
    wrap much of the SQLite API that are not directly related to the
    DB-API. If such a thing is too complicated/big for the standard
    library, then maybe it would be better to produce a much simpler
    PySQLite, especially for the Python standard library that leaves all
    the fancy stuff out. My codename would be "embsql".

    So, what would you like to see? "import sqlite", "import embsql", or
    "pypi.install(' pysqlite')" ?

    -- Gerhard

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.4 (GNU/Linux)

    iD8DBQFBd2/EdIO4ozGCH14RAh KpAJ9F33gxhA2j+ y/CeuKe0wvFpRZAKw CePMEL
    dzvLQjiR6GaRS8M 3eq2GXYY=
    =SEKQ
    -----END PGP SIGNATURE-----

  • Ganesan R

    #2
    Re: Include pysqlite2 into Python 2.5?

    >>>>> "Gerhard" == Gerhard Haering <gh@ghaering.de > writes:
    [color=blue]
    > Last December, we had a short thread discussing (in python-dev) the
    > integration of PySQLite into Python 2.4. At the time, I was against
    > inclusion, because I thought PySQLite was not ripe for it, mostly
    > because I thought the API was not stable.[/color]

    Thank you for PySQLite and congratulations for reaching the 1.0 milestone. I
    have only experimented with PySQLite so far but it's definitely in my
    toolbox for any future projects :-).
    [color=blue]
    > I think that a simple embedded relational database would be a good
    > thing to have in Python by default. And as Python 2.5 won't happen
    > anytime soon, there's plenty of time for developing it, getting it
    > stable, and integrating it.[/color]

    +1. bsddb is in there and SQLite definitely deserves to be included. A free
    license, no setup, easy to use - what more can you ask for :-).
    [color=blue]
    > Especially those of you that have used PySQLite in the past, do you
    > have any suggestions that would make the rewrite a better candidate
    > for inclusion into Python?[/color]

    I have not used it extensively to be able to get meaningful suggestions. My
    personal peeve is not with the PySQLite API but with DB-API 2.0. As some one
    (Alex Martelli?) mentioned in another thread, it's high time Python
    standardized on one (may be two) mandatory param styles.
    [color=blue]
    > One problem I see is that even the new PySQLite will grow and try to
    > wrap much of the SQLite API that are not directly related to the
    > DB-API. If such a thing is too complicated/big for the standard
    > library, then maybe it would be better to produce a much simpler
    > PySQLite, especially for the Python standard library that leaves all
    > the fancy stuff out. My codename would be "embsql".[/color]
    [color=blue]
    > So, what would you like to see? "import sqlite", "import embsql", or
    > "pypi.install(' pysqlite')" ?[/color]

    import sqlite

    Ganesan


    Comment

    • Istvan Albert

      #3
      Re: Include pysqlite2 into Python 2.5?

      Gerhard Haering wrote:
      [color=blue]
      > Especially those of you that have used PySQLite in the past, do you
      > have any suggestions that would make the rewrite a better candidate
      > for inclusion into Python?[/color]

      great idea!

      I'm currently in the planning phase of moving an application
      from a relational database to SQLite via PySQL. I think
      embedded, SQL capable databases are "the next big thing"
      there is just to much pain being felt by having to manage a
      big database when you don't really need them ....
      [color=blue]
      > One problem I see is that even the new PySQLite will grow and try to
      > wrap much of the SQLite API that are not directly related to the
      > DB-API. If such a thing is too complicated/big for the standard
      > library, then maybe it would be better to produce a much simpler
      > PySQLite, especially for the Python standard library that leaves all
      > the fancy stuff out. My codename would be "embsql".
      >
      > So, what would you like to see? "import sqlite", "import embsql", or[/color]

      I think import sqlite would be the sensible thing. Just as you said ...
      .... refuse the temptation to guess ...

      Istvan.

      Comment

      • John Fabiani

        #4
        Re: Include pysqlite2 into Python 2.5?

        Great idea and about time. Python needs some type of default SQL engine. I
        have not reviewed what you are doing but may I suggest you take a look at
        what VFP did with it's built-in data engine. VFP uses the concept of a
        cursor along with many built-in commands/functions/statements to work with
        the data.

        My only question is what happens when SQLLite becomes out of favor i.e. TK
        is today?

        John

        Comment

        • Alex Martelli

          #5
          Re: Include pysqlite2 into Python 2.5?

          John Fabiani <jfabiani@yolo. com> wrote:
          [color=blue]
          > Great idea and about time. Python needs some type of default SQL engine. I
          > have not reviewed what you are doing but may I suggest you take a look at
          > what VFP did with it's built-in data engine. VFP uses the concept of a
          > cursor along with many built-in commands/functions/statements to work with
          > the data.
          >
          > My only question is what happens when SQLLite becomes out of favor i.e. TK
          > is today?[/color]

          Same as what happens when BSDDB "becomes out of favor", or SAX, etc,
          etc. Python either keeps supporting it (likely), moves it to a separate
          download rather than bundling it (sensible), or drops it (unlikely but
          not impossible) at some major release. We're not talking about stuff
          that's particularly "fickle": Tkinter did its job excellently well for
          MANY years, after all, to take your same example.


          Alex

          Comment

          • Bengt Richter

            #6
            Re: Include pysqlite2 into Python 2.5?

            On Thu, 21 Oct 2004 18:14:51 +0200, aleaxit@yahoo.c om (Alex Martelli) wrote:
            [color=blue]
            >John Fabiani <jfabiani@yolo. com> wrote:
            >[color=green]
            >> Great idea and about time. Python needs some type of default SQL engine. I
            >> have not reviewed what you are doing but may I suggest you take a look at
            >> what VFP did with it's built-in data engine. VFP uses the concept of a
            >> cursor along with many built-in commands/functions/statements to work with
            >> the data.
            >>
            >> My only question is what happens when SQLLite becomes out of favor i.e. TK
            >> is today?[/color]
            >
            >Same as what happens when BSDDB "becomes out of favor", or SAX, etc,
            >etc. Python either keeps supporting it (likely), moves it to a separate
            >download rather than bundling it (sensible), or drops it (unlikely but
            >not impossible) at some major release. We're not talking about stuff
            >that's particularly "fickle": Tkinter did its job excellently well for
            >MANY years, after all, to take your same example.
            >[/color]
            ISTM there could be a useful compromise between inclusion and separate downloads.
            I.e., useful addons could be included in the core distribution as proxy stubs
            in an add-on package, so that you could interactively type e.g.,

            from addons import sqlite as sql

            and if you had not previously done that, the stub would ask you if you'd
            like to download and install the thing in question. You could cancel, or
            proceed, perhaps with various options. E.g. PIL might let you exclude jpg
            support if you didn't want it, and so forth. IIRC&UIAM the BSD ports system
            has this kind of functionality -- i.e., instead of full packages, there's
            automated download (or off CD) install info pre-packaged.

            Options just to download and view release notes or to download docs only
            for looking over before installing all might be useful sometimes also.

            Like wise, there could be an option to install under addons or site-packages.
            And I'm sure people would think of other handy stuff.

            There would have to be security checks with md5's or pgp signatures etc.,
            but IWT the aproach could be made to work. Official addons could live be
            anywhere, even with multiple mirrors, and different components used by
            higher level packages could have different urls, but the location urls, sizes, and
            hashes would be registered at python.org, so a stub would go there to find
            addon info for a particular version of python. This would also guarantee
            compatibility (or a notice that no compatible set of required parts is available).

            Just a thought.

            Regards,
            Bengt Richter

            Comment

            • Robert

              #7
              Re: Include pysqlite2 into Python 2.5?

              Gerhard Haering wrote:[color=blue]
              > [This is a repost from a message to python-dev on the users' list,
              > because python-dev-ers thought a broader discussion of this is a good
              > idea.]
              >
              > Last December, we had a short thread discussing (in python-dev) the
              > integration of PySQLite into Python 2.4. At the time, I was against
              > inclusion, because I thought PySQLite was not ripe for it, mostly
              > because I thought the API was not stable.
              >
              > Now, I have started writing a new PySQLite module, which has the
              > following key features:
              >
              > - Uses iterator-style SQLite 3.x API: sqlite3_compile , sqlite3_step()
              > etc. This way, it is possible to use prepared statements, and for
              > large resultsets, it requires less memory, because the whole
              > resultset isn't fetched into memory at once any longer.
              >
              > - Completely incompatible with the SQLite 0.x/1.x API: I'm free to
              > create a much better API now.
              >
              > - "In the face of ambiguity, refuse the temptation to guess." -
              > PySQLite 1.x tries to "guess" which Python type to convert to. It's
              > pretty good at it, because it queries the column type information.
              > This works for, I'd say 90 % of all cases at least. But as soon as
              > you use anything fancy like functions, aggregates or expressions in
              > SQL, the _typeless_ nature of SQLite breaks through and it will tell
              > us nothing about the declared column type (of course, because the
              > data is not coming from a database column).
              >
              > So I decided to change the default behaviour and make PySQLite
              > typeless by default, too. Everything will be returned as a Unicode
              > string (the default might be user-configurable per connection).
              >
              > Unless, unless of course the user explicitly activates the
              > "guess-mode" ;-) But to do so, she must read the docs then she will
              > be aware of the fact that it only works in 90 % of all cases.
              >
              > So why am I bothering you about this?
              >
              > I think that a simple embedded relational database would be a good
              > thing to have in Python by default. And as Python 2.5 won't happen
              > anytime soon, there's plenty of time for developing it, getting it
              > stable, and integrating it.
              >
              > Especially those of you that have used PySQLite in the past, do you
              > have any suggestions that would make the rewrite a better candidate
              > for inclusion into Python?
              >
              > One problem I see is that even the new PySQLite will grow and try to
              > wrap much of the SQLite API that are not directly related to the
              > DB-API. If such a thing is too complicated/big for the standard
              > library, then maybe it would be better to produce a much simpler
              > PySQLite, especially for the Python standard library that leaves all
              > the fancy stuff out. My codename would be "embsql".
              >
              > So, what would you like to see? "import sqlite", "import embsql", or
              > "pypi.install(' pysqlite')" ?
              >
              > -- Gerhard[/color]
              "import sqlite"

              Do we really have to wait unitil Python 2.5 to get it? ;-)

              Robert

              Comment

              • Dave Cook

                #8
                Re: Include pysqlite2 into Python 2.5?

                In article <ZFQdd.8495$6q2 .7811@newssvr14 .news.prodigy.c om>, John Fabiani
                wrote:
                [color=blue]
                > Great idea and about time. Python needs some type of default SQL engine.[/color]

                <AOL>
                Me, too!
                </AOL>

                I use sqlite heavily (via SQLObject and the current pysqlite).

                Dave Cook

                Comment

                • Gerhard Haering

                  #9
                  Re: Include pysqlite2 into Python 2.5?

                  On Thu, Oct 21, 2004 at 09:57:19PM -0400, Robert wrote:[color=blue]
                  > Do we really have to wait unitil Python 2.5 to get it? ;-)[/color]

                  It's too late for Python 2.4, which is in beta stage now. Also,
                  PySQLite 2.0, the one we're talking about here, is still pre-alpha.

                  -- Gerhard

                  -----BEGIN PGP SIGNATURE-----
                  Version: GnuPG v1.2.4 (GNU/Linux)

                  iD8DBQFBeMg/dIO4ozGCH14RAl7 YAKC106WWYUDLxW e+3YBq3ndRm4y+o wCfUtjb
                  R3ARGUytJ5KU0eL G3kpKf0U=
                  =/Uj/
                  -----END PGP SIGNATURE-----

                  Comment

                  • Wilk

                    #10
                    Re: Include pysqlite2 into Python 2.5?

                    Gerhard Haering <gh@ghaering.de > writes:
                    [color=blue]
                    > So, what would you like to see? "import sqlite", "import embsql", or
                    > "pypi.install(' pysqlite')" ?[/color]

                    apt-get install python-sqlite ;-)

                    I would like to see sqlite in python, the inclusion of sqlite in php was
                    greatly apreciated. I often use it for prototyping.

                    But for new users who will learn with sqlite, they will be disapointed
                    when they will want to use an other database, because of different
                    paramstyle, different datetime object... So i think before it could be
                    better to improve dbapi for user application side, and make pysqlite a
                    reference of implementation to follow (one paramstyle, return python
                    datetime, fetchdict...). If not it shall not be include imho.

                    We have the same problem with the web frameworks and server. I would
                    like to see a wsgi equivalent for database. To can plug any framework on
                    any database.

                    But when we look at the db-sig ml, it seems that there is a lot of work
                    to achieve this... btw guido was pushing to return python datetime.

                    --
                    Wilk - http://flibuste.net

                    Comment

                    • Aahz

                      #11
                      Re: Include pysqlite2 into Python 2.5?

                      In article <ZFQdd.8495$6q2 .7811@newssvr14 .news.prodigy.c om>,
                      John Fabiani <jfabiani@yolo. com> wrote:[color=blue]
                      >
                      >My only question is what happens when SQLLite becomes out of favor
                      >i.e. TK is today?[/color]

                      Tkinter isn't precisely out of favor, even now.
                      --
                      Aahz (aahz@pythoncra ft.com) <*> http://www.pythoncraft.com/

                      WiFi is the SCSI of the 21st Century -- there are fundamental technical
                      reasons for sacrificing a goat. (with no apologies to John Woods)

                      Comment

                      • ralobao

                        #12
                        Re: Include pysqlite2 into Python 2.5?

                        I think Python should expand its library of modules. It needs to
                        become more concise, its horrible to search the web for a module...

                        Sorry but its my thought.

                        Comment

                        Working...