[perl-python] generic equivalence partition

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Xah Lee

    [perl-python] generic equivalence partition

    another functional exercise with lists.

    Here's the perl documentation. I'll post a perl and the translated
    python version in 48 hours.

    =pod

    parti(aList, equalFunc)

    given a list aList of n elements, we want to return a list that is a
    range of numbers from 1 to n, partition by the predicate function of
    equivalence equalFunc. (a predicate function is a function that
    takes two arguments, and returns either True or False.)

    Note: a mathematical aspect: there are certain mathematical constraints
    on the a function that checks equivalence. That is to say, if a==b,
    then b==a. If a==b and b==c, then a==c. And, a==a. If a equivalence
    function does not satisfy these, it is inconsistent and basically give
    meaningless result.

    example:
    parti([['x','x','x','1'],
    ['x','x','x','2'],
    ['x','x','x','2'],
    ['x','x','x','2'],
    ['x','x','x','3'],
    ['x','x','x','4'],
    ['x','x','x','5'],
    ['x','x','x','5']], sub {$_[0]->[3] == $_[1]->[3]} )

    returns
    [[1],['2','3','4'],['5'],['6'],['7','8']];

    =cut

    In the example given, the input list's elements are lists of 4
    elements, and the equivalence function is one that returns True if the
    last item are the same.

    Note that this is a generic function. The input can be a list whose
    elements are of any type. What "parti" does is to return a partitioned
    range of numbers, that tells us which input element are equivalent to
    which, according to the predicate given. For example, in the given
    example, it tells us that the 2nd, 3rd, 4th elements are equivalent.
    And they are equivalent measured by the predicate function given, which
    basically tests if their last item are the same integer. (note that if
    we want to view the result as indexes, then it is 1-based index. i.e.
    counting starts at 1.)

    PS if you didn't realize yet, nested lists/dictionaries in perl is a
    complete pain in the ass.

    PS note that the code "sub {$_[0]->[3] == $_[1]->[3]}" is what's called
    the lambda form, in Perl.

    Xah
    xah@xahlee.org


  • Bryan

    #2
    Re: [perl-python] generic equivalence partition

    Xah Lee wrote:[color=blue]
    > another functional exercise with lists.
    >
    > Here's the perl documentation. I'll post a perl and the translated
    > python version in 48 hours.
    >
    > =pod
    >
    > parti(aList, equalFunc)
    >
    > given a list aList of n elements, we want to return a list that is a
    > range of numbers from 1 to n, partition by the predicate function of
    > equivalence equalFunc. (a predicate function is a function that
    > takes two arguments, and returns either True or False.)
    >
    > Note: a mathematical aspect: there are certain mathematical constraints
    > on the a function that checks equivalence. That is to say, if a==b,
    > then b==a. If a==b and b==c, then a==c. And, a==a. If a equivalence
    > function does not satisfy these, it is inconsistent and basically give
    > meaningless result.
    >
    > example:
    > parti([['x','x','x','1'],
    > ['x','x','x','2'],
    > ['x','x','x','2'],
    > ['x','x','x','2'],
    > ['x','x','x','3'],
    > ['x','x','x','4'],
    > ['x','x','x','5'],
    > ['x','x','x','5']], sub {$_[0]->[3] == $_[1]->[3]} )
    >
    > returns
    > [[1],['2','3','4'],['5'],['6'],['7','8']];
    >
    > =cut
    >
    > In the example given, the input list's elements are lists of 4
    > elements, and the equivalence function is one that returns True if the
    > last item are the same.
    >
    > Note that this is a generic function. The input can be a list whose
    > elements are of any type. What "parti" does is to return a partitioned
    > range of numbers, that tells us which input element are equivalent to
    > which, according to the predicate given. For example, in the given
    > example, it tells us that the 2nd, 3rd, 4th elements are equivalent.
    > And they are equivalent measured by the predicate function given, which
    > basically tests if their last item are the same integer. (note that if
    > we want to view the result as indexes, then it is 1-based index. i.e.
    > counting starts at 1.)
    >
    > PS if you didn't realize yet, nested lists/dictionaries in perl is a
    > complete pain in the ass.
    >
    > PS note that the code "sub {$_[0]->[3] == $_[1]->[3]}" is what's called
    > the lambda form, in Perl.
    >
    > Xah
    > xah@xahlee.org
    > http://xahlee.org/PageTwo_dir/more.html
    >[/color]

    this is the first thing that came to my mind. i'm sure there are more clever
    ways to do this.

    elements = [['x', 'x', 'x', '1'],
    ['x', 'x', 'x', '2'],
    ['x', 'x', 'x', '2'],
    ['x', 'x', 'x', '2'],
    ['x', 'x', 'x', '3'],
    ['x', 'x', 'x', '4'],
    ['x', 'x', 'x', '5'],
    ['x', 'x', 'x', '5']]
    pos = {}

    for i, element in enumerate(eleme nts):
    pos.setdefault( element[-1], []).append(i+1)

    p = pos.values()
    p.sort()
    [[1], [2, 3, 4], [5], [6], [7, 8]]


    bryan

    Comment

    • John Machin

      #3
      Re: [perl-python] generic equivalence partition

      On Thu, 24 Feb 2005 17:48:47 -0800, Bryan <belred@gmail.c om> wrote:
      [color=blue]
      >Xah Lee wrote:[color=green]
      >> another functional exercise with lists.
      >>
      >> Here's the perl documentation. I'll post a perl and the translated
      >> python version in 48 hours.
      >>
      >> =pod
      >>
      >> parti(aList, equalFunc)
      >>
      >> given a list aList of n elements, we want to return a list that is a
      >> range of numbers from 1 to n, partition by the predicate function of
      >> equivalence equalFunc. (a predicate function is a function that
      >> takes two arguments, and returns either True or False.)[/color][/color]

      [snip]
      [color=blue][color=green]
      >> example:
      >> parti([['x','x','x','1'],
      >> ['x','x','x','2'],[/color][/color]
      [snip][color=blue][color=green]
      >> ['x','x','x','5']], sub {$_[0]->[3] == $_[1]->[3]} )
      >>
      >> returns
      >> [[1],['2','3','4'],['5'],['6'],['7','8']];
      >>
      >> =cut
      >>
      >> In the example given, the input list's elements are lists of 4
      >> elements, and the equivalence function is one that returns True if the
      >> last item are the same.[/color]
      >[/color]
      [snip]
      [color=blue][color=green]
      >>[/color]
      >
      >this is the first thing that came to my mind. i'm sure there are more clever
      >ways to do this.
      >
      >elements = [['x', 'x', 'x', '1'],[/color]
      [snip][color=blue]
      > ['x', 'x', 'x', '5']]
      >pos = {}
      >
      >for i, element in enumerate(eleme nts):
      > pos.setdefault( element[-1], []).append(i+1)
      >
      >p = pos.values()
      >p.sort()
      >[[1], [2, 3, 4], [5], [6], [7, 8]]
      >
      >[/color]

      Bryan: Bzzzt. Xah was proposing a GENERAL function. You have HARDWIRED
      his (simplistic) example.

      Xah: Bzzzt. Too close to your previous exercise.




      Comment

      • David Eppstein

        #4
        Re: [perl-python] generic equivalence partition

        In article <1109245733.261 643.219420@f14g 2000cwb.googleg roups.com>,
        "Xah Lee" <xah@xahlee.org > wrote:
        [color=blue]
        > parti(aList, equalFunc)
        >
        > given a list aList of n elements, we want to return a list that is a
        > range of numbers from 1 to n, partition by the predicate function of
        > equivalence equalFunc. (a predicate function is a function that
        > takes two arguments, and returns either True or False.)[/color]

        In Python it is much more natural to use ranges from 0 to n-1.
        In the worst case, this is going to have to take quadratic time
        (consider an equalFunc that always returns false) so we might as well do
        something really simple rather than trying to be clever.

        def parti(aList,equ alFunc):
        eqv = []
        for i in range(len(aList )):
        print i,eqv
        for L in eqv:
        if equalFunc(aList[i],aList[L[0]]):
        L.append(i)
        break;
        else:
        eqv.append([i])

        If you really want the ranges to be 1 to n, add one to each number in
        the returned list-of-lists.

        --
        David Eppstein
        Computer Science Dept., Univ. of California, Irvine

        Comment

        • David Eppstein

          #5
          Re: [perl-python] generic equivalence partition

          In article <eppstein-CDFABC.20594724 022005@news.ser vice.uci.edu>,
          David Eppstein <eppstein@ics.u ci.edu> wrote:
          [color=blue]
          > def parti(aList,equ alFunc):
          > eqv = []
          > for i in range(len(aList )):
          > print i,eqv
          > for L in eqv:
          > if equalFunc(aList[i],aList[L[0]]):
          > L.append(i)
          > break;
          > else:
          > eqv.append([i])[/color]

          Um, take out the print, that was just there for me to debug my code.

          --
          David Eppstein
          Computer Science Dept., Univ. of California, Irvine

          Comment

          • Michael Spencer

            #6
            Re: [perl-python] generic equivalence partition

            David Eppstein wrote:[color=blue]
            > In article <1109245733.261 643.219420@f14g 2000cwb.googleg roups.com>,
            > "Xah Lee" <xah@xahlee.org > wrote:[color=green]
            >>given a list aList of n elements, we want to return a list that is a
            >>range of numbers from 1 to n, partition by the predicate function of
            >>equivalence equalFunc.[/color][/color]
            [color=blue]
            > In the worst case, this is going to have to take quadratic time
            > (consider an equalFunc that always returns false) so we might as well do
            > something really simple rather than trying to be clever.
            >
            > def parti(aList,equ alFunc):
            > eqv = []
            > for i in range(len(aList )):
            > print i,eqv
            > for L in eqv:
            > if equalFunc(aList[i],aList[L[0]]):
            > L.append(i)
            > break;
            > else:
            > eqv.append([i])
            >
            >[/color]
            Unless we can inspect the predicate function and derive a hash function such
            that hash(a) == hash(b) => predicate(a,b) is True. Then the partition can take
            linear time
            i.e.,[color=blue][color=green][color=darkred]
            >>> def equal(a,b):[/color][/color][/color]
            ... return a[-1] == b[-1]
            ...[color=blue][color=green][color=darkred]
            >>> def hashFunc(obj):[/color][/color][/color]
            ... return hash(obj[-1])
            ...[color=blue][color=green][color=darkred]
            >>> def parti(aList, hashFunc):[/color][/color][/color]
            ... eqv = {}
            ... for i,obj in enumerate(aList ):
            ... eqv.setdefault( hashFunc(obj),[]).append(i)
            ... return eqv.values()
            ...

            In the case where the predicate is a "black box", would a logistic regression
            over a sample of inputs enable a hash function to be derived experimentally?

            Michael

            Comment

            • Paul Moore

              #7
              Re: [perl-python] generic equivalence partition

              David Eppstein <eppstein@ics.u ci.edu> writes:
              [color=blue]
              > In article <1109245733.261 643.219420@f14g 2000cwb.googleg roups.com>,
              > "Xah Lee" <xah@xahlee.org > wrote:
              >[color=green]
              >> parti(aList, equalFunc)
              >>
              >> given a list aList of n elements, we want to return a list that is a
              >> range of numbers from 1 to n, partition by the predicate function of
              >> equivalence equalFunc. (a predicate function is a function that
              >> takes two arguments, and returns either True or False.)[/color]
              >
              > In Python it is much more natural to use ranges from 0 to n-1.
              > In the worst case, this is going to have to take quadratic time
              > (consider an equalFunc that always returns false) so we might as well do
              > something really simple rather than trying to be clever.[/color]

              As you say, with the spec as it stands, you can't do better than
              quadratic time (although it's O(n*m) where m is the number of
              partitions, rather than O(n^2)).

              You can do a lot better if you can use a "key" function, rather than
              an "equivalenc e" function, much as list.sort has a "key" argument, and
              itertools.group by (which is pretty close in function to this
              partitioning problem) uses a key argument.

              In fact, I'd have difficulty thinking of an example where I'd want a
              partition function as specified, in Python. In Perl, it makes a lot of
              sense, as Perl's array indexing operations lend themselves to slinging
              round lists of indices like this. But in Python, I'd be far more
              likely to use list.sort followed by itertools.group by - sort is stable
              (so doesn't alter the relative order within equivalence classes), and
              groupby then picks out the equivalence classes:
              [color=blue][color=green][color=darkred]
              >>> elements = [['x', 'x', 'x', '1'],[/color][/color][/color]
              .... ['x', 'x', 'x', '2'],
              .... ['x', 'x', 'x', '2'],
              .... ['x', 'x', 'x', '2'],
              .... ['x', 'x', 'x', '3'],
              .... ['x', 'x', 'x', '4'],
              .... ['x', 'x', 'x', '5'],
              .... ['x', 'x', 'x', '5']]
              [color=blue][color=green][color=darkred]
              >>> # No need to sort here, as the elements are already sorted![/color][/color][/color]
              [color=blue][color=green][color=darkred]
              >>> from pprint import pprint
              >>> pprint([(k, list(v)) for k, v in groupby(element s, itemgetter(3))])[/color][/color][/color]
              [('1', [['x', 'x', 'x', '1']]),
              ('2', [['x', 'x', 'x', '2'], ['x', 'x', 'x', '2'], ['x', 'x', 'x', '2']]),
              ('3', [['x', 'x', 'x', '3']]),
              ('4', [['x', 'x', 'x', '4']]),
              ('5', [['x', 'x', 'x', '5'], ['x', 'x', 'x', '5']])]

              If you avoid the sort, the whole thing is highly memory efficient, as
              well, because by using iterators, we don't ever take a copy of the
              original list.

              Having cleverly redefined the question so that it fits the answer I
              wanted to give, I'll shut up now :-)

              Paul.
              --
              To attain knowledge, add things every day; to attain wisdom, remove
              things every day. -- Lao-Tse

              Comment

              • Xah Lee

                #8
                Re: generic equivalence partition

                # the following solution is submitted by
                # Sean Gugler and David Eppstein independently
                # 20050224.

                @def parti(aList, equalFunc):
                @ result = []
                @ for i in range(len(aList )):
                @ for s in result:
                @ if equalFunc( aList[i], aList[s[0]] ):
                @ s.append(i)
                @ break
                @ else:
                @ result.append( [i] )
                @ return [[x+1 for x in L] for L in result] # add 1 to all numbers
                @
                @---------------

                as for my original perl code, i realized it is written to work on a
                sorted input. Here it is and the translated Python code.

                # perl
                sub parti($$) {
                my @li = @{$_[0]};
                my $sameQ = $_[1];

                my @tray=(1);
                my @result;

                for (my $i=1; $i <= ((scalar @li)-1); $i++) {
                if (&$sameQ($li[$i-1], $li[$i])) {
                push @tray, $i+1}
                else {
                push @result, [@tray]; @tray=($i+1);
                }
                }
                push @result, [@tray];
                return \@result;
                }


                @#python
                @def parti(li,sameQ) :
                @ tray=[1];
                @ result=[];
                @
                @ for i in range(1, len(li) ):
                @ if sameQ(li[i-1],li[i]):
                @ tray.append(i+1 )
                @ else:
                @ result.append(t ray)
                @ tray=[i+1]
                @ result.append(t ray)
                @ return result
                @



                Xah
                xah@xahlee.org


                Comment

                • Xah Lee

                  #9
                  Re: generic equivalence partition

                  folks:

                  when using google to post a reply, it sometimes truncates the subject
                  line. i.e. [perl-python] is lost. This software error is obvious, they
                  could not have not noticed it.

                  another thing more egregious is that google _intentionally_ edit with
                  people's posts. (e.g. they change email address lines without author's
                  permission, and they also change program codes so it no longer run).
                  Please spread these google irresponsibilit y to all related forums on
                  software responsibility and online forum issues.

                  Ostensible incorrect behavior like these by google is egregious enough
                  to generate a law suit and if such company do not take software
                  correctness seriously, we must punish them.

                  Please spread this awareness.

                  Xah
                  xah@xahlee.org


                  Comment

                  • Xah Lee

                    #10
                    Re: generic equivalence partition

                    People,

                    .... sorry for the latching on on this broadside issue, but it is
                    impotant ...

                    here's are some germane points from another online discussion:

                    the bug-reporting issue has came up so many times by so many people i
                    thought i'd make a comment of my view.

                    when a software is ostensibly incorrect, and if it is likely in
                    connection to egregious irresponsibilit y as most software companies are
                    thru their irresponsible licensing, the thing one should not do is to
                    fawn up to their ass as in filing a bug report, and that is also the
                    least effective in correcting the software.

                    the common attitude of bug-reporting is one reason that contributed to
                    the tremendous egregious irresponsible fuckups in computer software
                    industry that each of us have to endure daily all the time. (e.g.
                    software A clashed, software B can't do this, C can't do that, D i
                    don't know how to use, E download location currently broken, F i need
                    to join discussion group to find a work-around, G is all pretty and
                    dysfunctional.. . )

                    when a software is ostensibly incorrect and when the company is
                    irresponsible with their licensing, the most effective and moral
                    attitude is to do legal harm to the legal entity. This one an do by
                    filing a law suit or spreading the fact. Filing a law suit is
                    appropriate in severe and serious cases, and provided you have such
                    devotion to the cause. For most cases, we should just spread the fact.
                    When a company see facts flying about their incompetence or
                    irresponsibilit y, they will immediately mend the problem source, or
                    cease to exist.

                    Another harm sprang from the fucking bug-reporting attitude rampant
                    among IT morons is the multiplication of pop-ups that bug users for
                    bug-reporting, complete with their privacy intrusion legalese.



                    Xah
                    xah@xahlee.org



                    Xah Lee wrote:[color=blue]
                    > folks:
                    >
                    > when using google to post a reply, it sometimes truncates the subject
                    > line. i.e. [perl-python] is lost. This software error is obvious,[/color]
                    they[color=blue]
                    > could not have not noticed it.
                    >
                    > another thing more egregious is that google _intentionally_ edit with
                    > people's posts. (e.g. they change email address lines without[/color]
                    author's[color=blue]
                    > permission, and they also change program codes so it no longer run).
                    > Please spread these google irresponsibilit y to all related forums on
                    > software responsibility and online forum issues.
                    >
                    > Ostensible incorrect behavior like these by google is egregious[/color]
                    enough[color=blue]
                    > to generate a law suit and if such company do not take software
                    > correctness seriously, we must punish them.
                    >
                    > Please spread this awareness.
                    >
                    > Xah
                    > xah@xahlee.org
                    > http://xahlee.org/PageTwo_dir/more.html[/color]

                    Comment

                    • Erik Max Francis

                      #11
                      Re: generic equivalence partition

                      Xah Lee wrote:
                      [color=blue]
                      > ... sorry for the latching on on this broadside issue, but it is
                      > impotant ...[/color]

                      You made a typo in that last word there. Obviously you meant to write
                      an _e_ instead of an _a_.

                      --
                      Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
                      San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
                      All bad poetry springs from genuine feeling.
                      -- Oscar Wilde

                      Comment

                      Working...