Parallel/distributed generator

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • George Sakkis
    #1

    Parallel/distributed generator

    I'm looking for any existing packages or ideas on how to implement the
    equivalent of a generator (in the Python sense, i.e.
    http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
    way. As a use case, imagine a function that generates a range of
    primes. I'd like to be able to do something along the following lines:

    def iterprimes(star t=1, end=None):
    # ...
    yield prime


    # rpc-related initialization
    ....
    rpc_proxy = some_rpc_lib(it erprimes, start=1e6, end=1e12)
    for prime in proxy:
    print prime

    Is there any module out there that does anything close to this ?

    George

    Tags: None
    • Diez B. Roggisch
      #2
      Re: Parallel/distributed generator

      George Sakkis wrote:
      I'm looking for any existing packages or ideas on how to implement the
      equivalent of a generator (in the Python sense, i.e.
      http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
      way. As a use case, imagine a function that generates a range of
      primes. I'd like to be able to do something along the following lines:
      >
      def iterprimes(star t=1, end=None):
      # ...
      yield prime
      >
      >
      # rpc-related initialization
      ...
      rpc_proxy = some_rpc_lib(it erprimes, start=1e6, end=1e12)
      for prime in proxy:
      print prime
      >
      Is there any module out there that does anything close to this ?
      Have you tried using pyro to just return a generator? After all, it's just
      an object with a next()-method, that raises a certain exception. I can't
      see why pyro shouldn't be able to handle that.

      diez

        Comment

        • Paul McGuire
          #3
          Re: Parallel/distributed generator

          On May 23, 2:58 pm, half.ital...@gm ail.com wrote:
          On May 23, 11:00 am, George Sakkis <george.sak...@ gmail.comwrote:
          >
          >
          >
          >
          >
          I'm looking for any existing packages or ideas on how to implement the
          equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
          way. As a use case, imagine a function that generates a range of
          primes. I'd like to be able to do something along the following lines:
          >
          def iterprimes(star t=1, end=None):
          # ...
          yield prime
          >
          # rpc-related initialization
          ...
          rpc_proxy = some_rpc_lib(it erprimes, start=1e6, end=1e12)
          for prime in proxy:
          print prime
          >
          Is there any module out there that does anything close to this ?
          >
          George
          >
          DOLT!- Hide quoted text -
          >
          - Show quoted text -
          I thought you were making a joke about parallel processing.
          I thought you were making a joke about parallel processing.
          I thought you were making a joke about parallel processing.
          I thought you were making a joke about parallel processing.

          -- Paul

            Comment

            • George Sakkis
              #4
              Re: Parallel/distributed generator

              On May 23, 4:22 pm, Paul McGuire <p...@austin.rr .comwrote:
              On May 23, 2:58 pm, half.ital...@gm ail.com wrote:
              >
              >
              >
              On May 23, 11:00 am, George Sakkis <george.sak...@ gmail.comwrote:
              >
              I'm looking for any existing packages or ideas on how to implement the
              equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
              way. As a use case, imagine a function that generates a range of
              primes. I'd like to be able to do something along the following lines:
              >
              def iterprimes(star t=1, end=None):
              # ...
              yield prime
              >
              # rpc-related initialization
              ...
              rpc_proxy = some_rpc_lib(it erprimes, start=1e6, end=1e12)
              for prime in proxy:
              print prime
              >
              Is there any module out there that does anything close to this ?
              >
              George
              >
              DOLT!- Hide quoted text -
              >
              - Show quoted text -
              >
              I thought you were making a joke about parallel processing.
              I thought you were making a joke about parallel processing.
              I thought you were making a joke about parallel processing.
              I thought you were making a joke about parallel processing.
              >
              -- Paul
              No, he probably sent it from Google groups as I did, which claimed
              that "your message has not been sent, please try again later". Go
              figure.

              George

                Comment

                • half.italian@gmail.com
                  #5
                  Re: Parallel/distributed generator

                  On May 23, 1:22 pm, Paul McGuire <p...@austin.rr .comwrote:
                  On May 23, 2:58 pm, half.ital...@gm ail.com wrote:
                  >
                  >
                  >
                  On May 23, 11:00 am, George Sakkis <george.sak...@ gmail.comwrote:
                  >
                  I'm looking for any existing packages or ideas on how to implement the
                  equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
                  way. As a use case, imagine a function that generates a range of
                  primes. I'd like to be able to do something along the following lines:
                  >
                  def iterprimes(star t=1, end=None):
                  # ...
                  yield prime
                  >
                  # rpc-related initialization
                  ...
                  rpc_proxy = some_rpc_lib(it erprimes, start=1e6, end=1e12)
                  for prime in proxy:
                  print prime
                  >
                  Is there any module out there that does anything close to this ?
                  >
                  George
                  >
                  DOLT!- Hide quoted text -
                  >
                  - Show quoted text -
                  >
                  I thought you were making a joke about parallel processing.
                  I thought you were making a joke about parallel processing.
                  I thought you were making a joke about parallel processing.
                  I thought you were making a joke about parallel processing.
                  >
                  -- Paul
                  Damn computers!

                    Comment

                    • Simon Brunning
                      #6
                      Re: Parallel/distributed generator

                      On 23 May 2007 11:00:56 -0700, George Sakkis <george.sakkis@ gmail.comwrote:
                      I'm looking for any existing packages or ideas on how to implement the
                      equivalent of a generator (in the Python sense, i.e.
                      http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
                      way.
                      Kamaelia? http://tinyurl.com/35fjbr

                      --
                      Cheers,
                      Simon B.
                      simon@brunningo nline.net
                      Small Values of Cool
                      http://www.brunningonline.net/simon/blog/

                      GTalk: simon.brunning | MSN: small_values | Yahoo: smallvalues

                        Comment

                        Previous template Next
                        Working...