transforming a list into a string

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Peter Otten

    #16
    Re: transforming a list into a string

    Roy Smith wrote:
    [color=blue]
    > I'm also not particularly happy about the choice of "it" as a variable
    > name. The "izip (it, it)" construct makes me think of Dr. Evil :-)[/color]

    Using "it" is just my convention (contradictio in adiecto :-) for iterators
    used in a pure algorithm rather than with a meaning determined by a
    concrete use case. It's similar to the traditional i in counting loops,
    having grown an additional "t" to disambiguate it. If you have a better
    name for the purpose, don't hesitate to tell me...
    [color=blue]
    > It's not the izip bit that bothers me in the original, it's the deeply
    > nested construct of
    >
    > ",".join(["{%s,%s}" % i for i in izip(it, it)])
    >
    > There's too much going on in that one line to get your head around
    > easily. I suppose people who are really into functional programming
    > might find it understandable, but I find it rather obtuse.[/color]

    I don't think three levels can be called "deep nesting". In particular the
    "somestr".join( ) construct is so ubiquitous that your brain will "optimize"
    it away after reading a small amount of Python code. But I see my oneliner
    meets serious opposition. Well, sometimes a few self-explanatory names and
    a helper function can do wonders:

    import itertools

    def pairs(seq):
    it = iter(seq)
    return itertools.izip( it, it)

    coords = ['1','2','7','8' ,'12','13']
    points = []
    for xy in pairs(coords):
    points.append(" {%s, %s}" % xy)

    print ", ".join(poin ts)

    That should be clear even to someone who has never heard of generators. Note
    that pairs() is only called once and therefore does not affect the speed of
    execution. Personally, I'd still go with the list comprehension instead of
    the above for-loop.

    By the way - expanding on Michele Simionato's chop(),

    I've written a generalized version of pairs():

    _missing = object()

    def ntuples(seq, N=2, filler=_missing ):
    """ Yield a sequence in portions of N-tuples.
    [color=blue][color=green][color=darkred]
    >>> list(ntuples("a bcdefg", 3))[/color][/color][/color]
    [('a', 'b', 'c'), ('d', 'e', 'f')]
    [color=blue][color=green][color=darkred]
    >>> list(ntuples("a bc", filler="x"))[/color][/color][/color]
    [('a', 'b'), ('c', 'x')]
    """
    if filler is _missing:
    it = iter(seq)
    else:
    it = itertools.chain (iter(seq), itertools.repea t(filler, N-1))
    iters = (it,) * N
    return itertools.izip( *iters)

    Enjoy :-)

    Peter

    Comment

    • Bengt Richter

      #17
      Re: transforming a list into a string

      On Sun, 1 Aug 2004 11:00:32 +1000, Andrew Bennetts <andrew-pythonlist@puzz ling.org> wrote:
      [...][color=blue]
      >
      >It's a two-liner, not a one-liner (although it could be made into a
      >one-liner with enough contortions...) .
      >[/color]
      Another contortion ;-)
      [color=blue][color=green][color=darkred]
      >>> items = ['1','2','7','8' ,'12','13'][/color][/color][/color]
      assumed defined as before, then one line:
      [color=blue][color=green][color=darkred]
      >>> ''.join([i%2 and c+'},' or '{'+c+',' for i,c in enumerate(items )])[:-1][/color][/color][/color]
      '{1,2},{7,8},{1 2,13}'

      Regards,
      Bengt Richter

      Comment

      • Steven Rumbalski

        #18
        Re: transforming a list into a string

        Peter Otten wrote:

        Peter's solution:[color=blue][color=green][color=darkred]
        >>>> from itertools import izip
        >>>> items = ['1','2','7','8' ,'12','13']
        >>>> it = iter(items)
        >>>> ",".join(["{%s,%s}" % i for i in izip(it, it)])[/color][/color]
        > '{1,2},{7,8},{1 2,13}'[/color]

        My first thought was:
        [color=blue][color=green][color=darkred]
        >>> items = ['1','2','7','8' ,'12','13']
        >>> ",".join(["{%s,%s}" % i for i in zip(items[::2], items[1::2])])[/color][/color][/color]
         '{1,2},{7,8},{1 2,13}'

        Two lines less, but it creates three unnecessary lists.  I like
        Peter's better.

        --Steven Rumbalski

        Comment

        • Christopher T King

          #19
          Re: transforming a list into a string

          On Sat, 31 Jul 2004, Tim Peters wrote:
          [color=blue]
          > Absolutely. Note that Peter Otten previously posted a lovely O(N)
          > solution in this thread, although it may be too clever for some
          > tastes:
          >[color=green][color=darkred]
          > >>> from itertools import izip
          > >>> items = ['1','2','7','8' ,'12','13']
          > >>> it = iter(items)
          > >>> ",".join(["{%s,%s}" % i for i in izip(it, it)])[/color][/color]
          > '{1,2},{7,8},{1 2,13}'[color=green][color=darkred]
          > >>>[/color][/color][/color]

          A bit too clever for mine, mostly because neither izip() nor zip() is
          guaranteed to process its arguments in a left-to-right order (although
          there's no reason for them not to). I'd rather do this:
          [color=blue][color=green][color=darkred]
          >>> from itertools import izip, islice
          >>> items = ['1','2','7','8' ,'12','13']
          >>> it1 = islice(items,0, None,2)
          >>> it2 = islice(items,1, None,2)
          >>> ",".join(["{%s,%s}" % i for i in izip(it1, it2)])[/color][/color][/color]
          '{1,2},{7,8},{1 2,13}'

          Although it doesn't improve efficiency any (or have that 'slick' feel), it
          does prevent needless head-scratching :)

          Curious, why isn't slicing of generators defined, using islice(), so "it1
          = iter(items)[0::2]" is valid?

          Comment

          • Tim Peters

            #20
            Re: transforming a list into a string

            [Christopher T King][color=blue]
            > ...
            > Curious, why isn't slicing of generators defined, using islice(), so "it1
            > = iter(items)[0::2]" is valid?[/color]

            The real question then is why iterators don't, because a
            generator-function returns a generator-iterator, and the latter
            supplies only the methods in the general iterator protocol (next() and
            __iter__()). That protocol was deliberately minimal, to make it easy
            for all kinds of objects to play along. islice() was invented long
            after. Now that islice() exists, it may indeed make sense to use it
            to give a meaning to slice notation applied to iterators. But doing
            so requires that iterators implement the appropriate type slots to
            "look like" they're also "sequences" (that's how dispatching for slice
            notation works), and that's a lot more to ask of objects. If only
            some iterators implement it (like generator-iterators), then the
            general interchangeabil ity of iterable objects we enjoy today would be
            damaged too.

            Comment

            • Terry Reedy

              #21
              Re: transforming a list into a string


              "Tim Peters" <tim.peters@gma il.com> wrote in message
              news:1f7befae04 0731171654c11ec 6@mail.gmail.co m...[color=blue]
              > Absolutely. Note that Peter Otten previously posted a lovely O(N)
              > solution in this thread, although it may be too clever for some
              > tastes:
              >[color=green][color=darkred]
              > >>> from itertools import izip
              > >>> items = ['1','2','7','8' ,'12','13']
              > >>> it = iter(items)
              > >>> ",".join(["{%s,%s}" % i for i in izip(it, it)])[/color][/color]
              > '{1,2},{7,8},{1 2,13}'[/color]

              While this usage of izip(it,it) is clever (and zip(it,it) behaves the
              same), it *breaks* the documented behavior of zip and hence of izip, and
              depends on seemingly inessential implementation details of zip/izip (which
              is only documented for izip).

              The Lib Manual 2.1 zip entry calls the args 'sequences' whereas it should
              say 'iterables'. It goes on "This function returns a list of tuples, where
              the i-th tuple contains the i-th element from each of the argument
              sequences. ... The returned list is truncated in length to the length of
              the shortest argument sequence." In fact, even back in 2.2:[color=blue][color=green][color=darkred]
              >>> it = iter([1,2,3,4])
              >>> zip(it,it)[/color][/color][/color]
              [(1, 2), (3, 4)]

              I believe the definition of zip would allow iterator inputs to be handled
              by list()ing them (for instance, left to right). A reason to do this might
              be to fix the otherwise unknown lengths so that the min could be determined
              so that the output list could be preallocated. If this were done, however,
              zip(it,it) would output [] instead.

              If zip were to build output tuples from the end, which the definition would
              also seem to allow, then zip(it,it) above would be [(2,1), (4,3)] instead.
              Zip's behavior seems to me undefined for this corner case.

              So the doc needs to be updated to specify the args as iterables, not
              restricted to sequences, and qualify the 'usual' behavior as depending on
              having distinct iterator inputs.

              The 5.14.1 Itertool functions izip entry says simply "Like zip() except
              that it returns an iterator instead of a list." It also give 'equivalent'
              Python code which happens to pin down the behavior for this corner case. I
              wonder if this code should really be taken as determinative documentation
              rather than as illustrative of possible implementation. (I may ask RH if
              need be.) If the former, then the zip doc could reference the izip
              equivalent code as specifying its behavior.

              Terry J. Reedy



              Comment

              • Christopher T King

                #22
                Re: transforming a list into a string

                On Sun, 1 Aug 2004, Tim Peters wrote:
                [color=blue]
                > [Christopher T King][color=green]
                > > ...
                > > Curious, why isn't slicing of generators defined, using islice(), so "it1
                > > = iter(items)[0::2]" is valid?[/color]
                >
                > If only some iterators implement it (like generator-iterators), then the
                > general interchangeabil ity of iterable objects we enjoy today would be
                > damaged too.[/color]

                Ah, I see your point. But most functions that expect iterators use iter()
                on them first (to iterize sequences), do they not? So long as iter()
                supplies the necessary __getslice__ implementation, the world would be
                happy. This situation would mirror the situation with list(): though a
                user-defined sequence might not implement __getslice__ (and still be
                usable as a list), the object returned by list() is guaranteed to.

                Comment

                • Tim Peters

                  #23
                  Re: transforming a list into a string

                  [Christopher T King][color=blue]
                  > Ah, I see your point. But most functions that expect iterators use iter()
                  > on them first (to iterize sequences), do they not? So long as iter()
                  > supplies the necessary __getslice__ implementation, the world would be
                  > happy. This situation would mirror the situation with list(): though a
                  > user-defined sequence might not implement __getslice__ (and still be
                  > usable as a list), the object returned by list() is guaranteed to.[/color]

                  The difference is that list() creates a concrete list object from its
                  argument, but there's no such thing as "a concrete iter object".
                  Iteration is a protocol, not a type. iter(obj) invokes
                  obj.__iter__(), and I don't know of any existing __iter__
                  implementation that returns an object that supports slicing. The only
                  thing required of __iter__ is that it return an object that supports
                  the iteration protocol (meaning an object that supports next() and
                  __iter__() methods). So again, adding the ability to slice too would
                  mean requiring more of __iter__ methods -- or changing the
                  implementation of iter() to ignore __iter__ methods and make something
                  up itself. It's A Visible Change no matter how you cut it.

                  Comment

                  • Christopher T King

                    #24
                    Re: transforming a list into a string

                    On Sun, 1 Aug 2004, Tim Peters wrote:
                    [color=blue]
                    > The difference is that list() creates a concrete list object from its
                    > argument, but there's no such thing as "a concrete iter object".[/color]

                    You've got me there.
                    [color=blue]
                    > Iteration is a protocol, not a type. iter(obj) invokes
                    > obj.__iter__(),[/color]

                    Only if obj.__iter__() is defined; otherwise it makes something up using
                    __getitem__ as appropriate.
                    [color=blue]
                    > and I don't know of any existing __iter__ implementation that returns an
                    > object that supports slicing.[/color]

                    True. Built-in classes (such as list & tupleiterator) would have to be
                    extended with this function (trivial if they all subclass from a common
                    base class). As to user classes, I'm proposing this on the assumption
                    that a programmer would implement their own __getitem__ (I've been saying
                    __getslice__ previously, __getitem__ is what I should be saying) if the
                    functionality is so desired, which can be as trivial as setting
                    __getitem__=isl ice, provided islice is extended to accept slice objects.
                    Though, this would break __getitem__ in the case of getting a single
                    item (I can see where this is heading)...
                    [color=blue]
                    > The only thing required of __iter__ is that it return an object that
                    > supports the iteration protocol (meaning an object that supports next()
                    > and __iter__() methods). So again, adding the ability to slice too
                    > would mean requiring more of __iter__ methods -- or changing the
                    > implementation of iter() to ignore __iter__ methods and make something
                    > up itself. It's A Visible Change no matter how you cut it.[/color]

                    You've made your point. I guess this will just have to go on the "it
                    would be neat if it worked this way, but it just doesn't" list for now ;)

                    Comment

                    • Bruce Eckel

                      #25
                      The term &quot;Protocol& quot;

                      Sunday, August 1, 2004, 6:51:15 PM, Tim Peters wrote:
                      [color=blue]
                      > Iteration is a protocol, not a type.[/color]

                      I know the term "protocol" has been used to describe a language
                      feature in a number of languages, but since we have no official
                      "protocol" support in Python I'm interested in what "we" mean by this
                      term. I'm going to guess that a protocol is like an interface in Java,
                      except that it doesn't have a concrete definition anywhere, but it is
                      implied through convention and use. Thus a protocol is a "latent
                      interface." Am I close? I'd like to understand this term better.

                      Bruce Eckel


                      Comment

                      • Phil Frost

                        #26
                        Re: The term &quot;Protocol& quot;

                        "Protocol" in python has no meaning beyond normal English. Specifically
                        the iteration protocol says the iterable must have a __iter__ method
                        which returns an object that has an __iter__ which returns self, and a
                        next() method that returns the next thing in the thing being iterated.
                        When there are no more things, next() raises StopIteration. All of this
                        is a simple protocol defined by the python language. It doesn't
                        introduce anything new to the language besides the protocol itself;
                        __iter__ and next are regular methods and StopIteration is raised just
                        as any other class can be raised as an exception.

                        On Sun, Aug 01, 2004 at 07:25:23PM -0600, Bruce Eckel wrote:[color=blue]
                        > Sunday, August 1, 2004, 6:51:15 PM, Tim Peters wrote:
                        >[color=green]
                        > > Iteration is a protocol, not a type.[/color]
                        >
                        > I know the term "protocol" has been used to describe a language
                        > feature in a number of languages, but since we have no official
                        > "protocol" support in Python I'm interested in what "we" mean by this
                        > term. I'm going to guess that a protocol is like an interface in Java,
                        > except that it doesn't have a concrete definition anywhere, but it is
                        > implied through convention and use. Thus a protocol is a "latent
                        > interface." Am I close? I'd like to understand this term better.
                        >
                        > Bruce Eckel[/color]

                        Comment

                        • kosh

                          #27
                          Re: transforming a list into a string

                          On Saturday 31 July 2004 6:44 am, Peter Otten wrote:[color=blue]
                          > jblazi wrote:[color=green]
                          > > Let us assume I have a list like
                          > >
                          > > ['1','2','7','8' ,'12','13]
                          > >
                          > > and would like to transoform it into the string
                          > >
                          > > '{1,2},{7,8},{1 2,13}'
                          > >
                          > > Which is the simplest way of achiebing this? (The list is in fact much
                          > > longer and I may have to cut the resulting strings into chunks of 100 or
                          > > so.)
                          > >[color=darkred]
                          > >>> from itertools import izip
                          > >>> items = ['1','2','7','8' ,'12','13']
                          > >>> it = iter(items)
                          > >>> ",".join(["{%s,%s}" % i for i in izip(it, it)])[/color][/color]
                          >
                          > '{1,2},{7,8},{1 2,13}'
                          >
                          >
                          > Peter[/color]

                          This way also works I am not sure how it compares on large sets for memory
                          usage, cpu time etc.

                          items = ['1','2','7','8' ,'12','13']
                          if len(items)%2 == 0:
                          #the len check is to make sure that the length of the set is evenly
                          #divisible by 2
                          output = "{%s,%s}" * (len(items)/2)% tuple(items)

                          Comment

                          • Duncan Booth

                            #28
                            Re: transforming a list into a string

                            Christopher T King <squirrel@WPI.E DU> wrote in
                            news:Pine.LNX.4 .44.04080118140 20.21160-100000@ccc4.wpi .edu:
                            [color=blue]
                            > On Sat, 31 Jul 2004, Tim Peters wrote:
                            >[color=green]
                            >> Absolutely. Note that Peter Otten previously posted a lovely O(N)
                            >> solution in this thread, although it may be too clever for some
                            >> tastes:
                            >>[color=darkred]
                            >> >>> from itertools import izip
                            >> >>> items = ['1','2','7','8' ,'12','13']
                            >> >>> it = iter(items)
                            >> >>> ",".join(["{%s,%s}" % i for i in izip(it, it)])[/color]
                            >> '{1,2},{7,8},{1 2,13}'[color=darkred]
                            >> >>>[/color][/color]
                            >
                            > A bit too clever for mine, mostly because neither izip() nor zip() is
                            > guaranteed to process its arguments in a left-to-right order (although
                            > there's no reason for them not to).[/color]

                            You should read an earlier thread on this topic:



                            I make exactly that point, that the order isn't guaranteed, and was
                            refuted fairly convincingly by Peter Otten. The documentation says that
                            izip is equivalent to a particular reference implementation. Any
                            implementation which didn't preserve the left to right ordering wouldn't
                            match the reference:

                            Peter Otten wrote:
                            [color=blue][color=green]
                            >> Passing the same iterator multiple times to izip is a pretty neat
                            >> idea, but I would still be happier if the documentation explicitly
                            >> stated that it consumes its arguments left to right.[/color]
                            >
                            > From the itertools documentation:
                            >
                            > """
                            > izip(*iterables )
                            >
                            > Make an iterator that aggregates elements from each of the iterables.
                            > Like zip() except that it returns an iterator instead of a list. Used
                            > for lock-step iteration over several iterables at a time. Equivalent
                            > to:
                            >
                            > def izip(*iterables ):
                            > iterables = map(iter, iterables)
                            > while iterables:
                            > result = [i.next() for i in iterables]
                            > yield tuple(result)
                            > """
                            >
                            > I'd say the "Equivalent to [reference implementation]" statement
                            > should meet your request.[/color]



                            Comment

                            • Aahz

                              #29
                              Re: The term &quot;Protocol& quot;

                              [cc'ing Alex so he can jump in if he wants]

                              In article <mailman.1042.1 091409923.5135. python-list@python.org >,
                              Bruce Eckel <BruceEckel@Mai lBlocks.com> wrote:[color=blue]
                              >
                              >I know the term "protocol" has been used to describe a language feature
                              >in a number of languages, but since we have no official "protocol"
                              >support in Python I'm interested in what "we" mean by this term. I'm
                              >going to guess that a protocol is like an interface in Java, except
                              >that it doesn't have a concrete definition anywhere, but it is implied
                              >through convention and use. Thus a protocol is a "latent interface." Am
                              >I close? I'd like to understand this term better.[/color]

                              Alex Martelli gave an excellent presentation on Design Patterns at OSCON,
                              where he made the point that "interface" is roughly equivalent to syntax,
                              whereas "protocol" is roughly equivalent to syntax plus semantics. In
                              other words, computer langauges rarely (if ever -- although I suppose
                              Eiffel comes close) enforce protocols in any meaningful way.

                              I'm hoping Alex posts his slides soon.
                              --
                              Aahz (aahz@pythoncra ft.com) <*> http://www.pythoncraft.com/

                              "To me vi is Zen. To use vi is to practice zen. Every command is a
                              koan. Profound to the user, unintelligible to the uninitiated. You
                              discover truth everytime you use it." --reddy@lion.aust in.ibm.com

                              Comment

                              • Tom B.

                                #30
                                Re: transforming a list into a string


                                "jblazi" <jblazi@hotmail .com> wrote in message
                                news:pan.2004.0 7.31.12.27.23.5 47000@hotmail.c om...[color=blue]
                                > Let us assume I have a list like
                                >
                                > ['1','2','7','8' ,'12','13]
                                >
                                > and would like to transoform it into the string
                                >
                                > '{1,2},{7,8},{1 2,13}'
                                >
                                > Which is the simplest way of achiebing this? (The list is in fact much
                                > longer and I may have to cut the resulting strings into chunks of 100 or
                                > so.)
                                >
                                > TIA,
                                >
                                > jb
                                >[/color]
                                Heres a one-liner,[color=blue][color=green][color=darkred]
                                >>>items = ['1','2','7','8' ,'12','13']
                                >>>[(items[x],items[x+1]) for x in range(0,len(ite ms)-1,2)][/color][/color][/color]
                                [('1', '2'), ('7', '8'), ('12', '13')]

                                Tom


                                Comment

                                Working...