a question regarding call-by-reference

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • enjoying the view

    #1

    a question regarding call-by-reference

    I am working on a school project, trying to build a simple RPC stub
    generator. The idea is that the generator takes a normal python file
    with functions defined in it and produces a client and server stub. The
    client stub has the same functions defined, but they just connect to
    the server and ask it to call the desired functions. The server stub is
    a server listening for incoming requests and dispatching them to the
    appropriate functions and sending back the results. Extremely simple
    and I've gotten it to mostly work, but I have one problem:
    call-by-reference parameters. An example:

    A function like this:
    def sum(a, b):
    return a + b

    would yield a client stub like this (simplified):
    def sum(a, b):
    send_message({' funcname': 'sum', 'args': (a, b)})
    result = receive_message ()
    return result

    Now, this works just fine. I get problems when I try to do something
    like this:
    def add_elm(list):
    list.append('el m')
    return

    Imported normally this would work fine. The list given as a parameter
    would be one element larger. But when the stubs are generated, the
    function doesn't return anything and the list is appended in the server
    and the client-side list is left untouched. At first I thought the
    solution was easy, just return the changed parameters and place them in
    the parameter variables in the client stub:
    def add_elm(list):
    send_message({' funcname': 'add_elm', 'args': (list)})
    response = receive_message ()
    (list) = response.get('a rgs')
    return response.get('r esult')

    The problem is, this doesn't work. The original list doesn't point to
    the changed list. I have been trying to figure this out and it seems
    that if I just assign to the list variable it just modifies the local
    (to the function) name space, and that those changes aren't reflected
    in the list in the original name space.

    I believe there are some ways around this, but I haven't found one that
    would not require any special handling in either the code calling the
    client stub or the original functions. I want to maintain transparency.

    etv

  • Paul McGuire

    #2
    Re: a question regarding call-by-reference

    "enjoying the view" <pparkkin@gmail .com> wrote in message
    news:1139250105 .744691.110440@ o13g2000cwo.goo glegroups.com.. .[color=blue]
    > I am working on a school project, trying to build a simple RPC stub
    > generator. The idea is that the generator takes a normal python file
    > with functions defined in it and produces a client and server stub. The
    > client stub has the same functions defined, but they just connect to
    > the server and ask it to call the desired functions. The server stub is
    > a server listening for incoming requests and dispatching them to the
    > appropriate functions and sending back the results.[/color]
    <snip>

    One thing you might try is iterating over the arguments and doing special
    handling for lists and dicts, which can be updated within the called module.
    Using your example (changing 'list' to 'listarg' to avoid confusion):

    def add_elm(listarg ):
    listarg.append( 'elm')
    return

    Generate code like:

    def add_elm(listarg ):
    send_message({' funcname': 'add_elm', 'args': (listarg)})
    response = receive_message ()
    listarg[:] = response.get('a rgs')[0][:]
    return response.get('r esult')

    If the listargs get long/complicated, you might be doing a lot of data
    copying at the cost of performance (especially since this is a defensive
    copy - you don't *know* that the list was updated in the server in the first
    place - I guess you could test first with "if response.get('a rgs')[0] !=
    listarg:" or something).

    For dicts, do:

    dictarg.update( response.get('a rgs')[n])

    This will be a little cheaper than just total replacement of the contents,
    which is what the list [:] slice will do.

    But this special-case handling only works for lists and dicts (don't know
    how you would handle tuples or scalar variables), you quickly run into other
    problems with passing objects. I'm taking you at your word that you are
    creating a *simple* RPC stub generator. At this point, you can generate
    stubs for methods that do not update their arguments, or for methods that
    pass lists or dicts. I've used CORBA in the past, and this is similar to
    using a subset of the full parameter functionality, such as using only "in"
    parameters (as opposed to "out" or "inout"). That is, instead of defining a
    method that modifies its parameter value, have the new value returned by the
    function. Python actually helps you here, since it is so easy to pass back
    a tuple of values from a function.

    This kind of parameter marshalling/unmarshalling gets complicated quickly.
    You are already very near the edge of non-intrusiveness on the client and
    server sides - to get more features, you'll need to start adding some
    aspects of parameter passing mechanisms (such as CORBA's in/out/inout
    keywords), and object marshalling conventions (such as requiring objects
    that are passed to implement compare() and update() or copy() methods).

    -- Paul


    Comment

    • Claudio Grondi

      #3
      Re: a question regarding call-by-reference

      enjoying the view wrote:[color=blue]
      > I am working on a school project, trying to build a simple RPC stub
      > generator. The idea is that the generator takes a normal python file
      > with functions defined in it and produces a client and server stub. The
      > client stub has the same functions defined, but they just connect to
      > the server and ask it to call the desired functions. The server stub is
      > a server listening for incoming requests and dispatching them to the
      > appropriate functions and sending back the results. Extremely simple
      > and I've gotten it to mostly work, but I have one problem:
      > call-by-reference parameters. An example:
      >
      > A function like this:
      > def sum(a, b):
      > return a + b
      >
      > would yield a client stub like this (simplified):
      > def sum(a, b):
      > send_message({' funcname': 'sum', 'args': (a, b)})
      > result = receive_message ()
      > return result
      >
      > Now, this works just fine. I get problems when I try to do something
      > like this:
      > def add_elm(list):
      > list.append('el m')
      > return
      >
      > Imported normally this would work fine. The list given as a parameter
      > would be one element larger. But when the stubs are generated, the
      > function doesn't return anything and the list is appended in the server
      > and the client-side list is left untouched. At first I thought the
      > solution was easy, just return the changed parameters and place them in
      > the parameter variables in the client stub:
      > def add_elm(list):
      > send_message({' funcname': 'add_elm', 'args': (list)})
      > response = receive_message ()
      > (list) = response.get('a rgs')
      > return response.get('r esult')
      >
      > The problem is, this doesn't work. The original list doesn't point to
      > the changed list. I have been trying to figure this out and it seems
      > that if I just assign to the list variable it just modifies the local
      > (to the function) name space, and that those changes aren't reflected
      > in the list in the original name space.
      >
      > I believe there are some ways around this, but I haven't found one that
      > would not require any special handling in either the code calling the
      > client stub or the original functions. I want to maintain transparency.
      >
      > etv
      >[/color]
      I am not an expert, so maybe someone else can provide here a nice tricky
      solution, but I am quite sure, that there is no simple way around the
      described problem.
      The only strategy which comes to my mind in this context is to use for
      all function arguments and return values only pickled objects which need
      to be unpickled on the other side. Probably some compression algorithm
      can help to keep the network traffic down in case large amount of data
      must be exchanged.

      Claudio

      Comment

      • Kent Johnson

        #4
        Re: a question regarding call-by-reference

        enjoying the view wrote:[color=blue]
        > Imported normally this would work fine. The list given as a parameter
        > would be one element larger. But when the stubs are generated, the
        > function doesn't return anything and the list is appended in the server
        > and the client-side list is left untouched. At first I thought the
        > solution was easy, just return the changed parameters and place them in
        > the parameter variables in the client stub:
        > def add_elm(list):
        > send_message({' funcname': 'add_elm', 'args': (list)})
        > response = receive_message ()
        > (list) = response.get('a rgs')
        > return response.get('r esult')
        >
        > The problem is, this doesn't work. The original list doesn't point to
        > the changed list. I have been trying to figure this out and it seems
        > that if I just assign to the list variable it just modifies the local
        > (to the function) name space, and that those changes aren't reflected
        > in the list in the original name space.[/color]

        Try
        list[:] = response.get('a rgs')
        this will change the value of the list passed in.

        Kent

        Comment

        • Bengt Richter

          #5
          Re: a question regarding call-by-reference

          On 6 Feb 2006 10:21:45 -0800, "enjoying the view" <pparkkin@gmail .com> wrote:
          [color=blue]
          >I am working on a school project, trying to build a simple RPC stub
          >generator. The idea is that the generator takes a normal python file
          >with functions defined in it and produces a client and server stub. The
          >client stub has the same functions defined, but they just connect to
          >the server and ask it to call the desired functions. The server stub is
          >a server listening for incoming requests and dispatching them to the
          >appropriate functions and sending back the results. Extremely simple
          >and I've gotten it to mostly work, but I have one problem:
          >call-by-reference parameters. An example:
          >
          >A function like this:
          >def sum(a, b):
          > return a + b
          >
          >would yield a client stub like this (simplified):
          >def sum(a, b):
          > send_message({' funcname': 'sum', 'args': (a, b)})
          > result = receive_message ()
          > return result
          >
          >Now, this works just fine. I get problems when I try to do something
          >like this:
          >def add_elm(list):
          > list.append('el m')
          > return
          >
          >Imported normally this would work fine. The list given as a parameter
          >would be one element larger. But when the stubs are generated, the
          >function doesn't return anything and the list is appended in the server
          >and the client-side list is left untouched. At first I thought the
          >solution was easy, just return the changed parameters and place them in
          >the parameter variables in the client stub:
          >def add_elm(list):
          > send_message({' funcname': 'add_elm', 'args': (list)})
          > response = receive_message ()[/color]
          if the response contains an updated list, you can update the content
          of the add_elm caller's list by slice assignment, e.g.,
          list[:] = response.get(<h owever you get the updated list>')

          But you are in really tricky territory here I think. There is any number
          of ways to mutate objects via function argument reference. Not just
          if they're lists. And lists can be deeply nested, so you could be sending
          deep copies of lists across a network, which could involve most of your memory
          even if the first level has only a few elements. And if there are side effects besides
          the direct mutations, you have additional problems. What if a function accesses
          arg.prop and that's a property that keeps count or otherwise tracks instances of something?

          The totally general solution is probably impossible. So if I were you I would go back to
          the specifier of this problem and negotiate some severe restrictions on assumptions
          about arguments (maybe just outlaw direct mutation and access side effects ;-)
          [color=blue]
          > (list) = response.get('a rgs')
          > return response.get('r esult')
          >
          >The problem is, this doesn't work. The original list doesn't point to
          >the changed list. I have been trying to figure this out and it seems
          >that if I just assign to the list variable it just modifies the local
          >(to the function) name space, and that those changes aren't reflected
          >in the list in the original name space.
          >
          >I believe there are some ways around this, but I haven't found one that
          >would not require any special handling in either the code calling the
          >client stub or the original functions. I want to maintain transparency.
          >
          >etv
          >[/color]
          Good luck ;-)

          Regards,
          Bengt Richter

          Comment

          • enjoying the view

            #6
            Re: a question regarding call-by-reference

            Thank you everyone for your helpful replies!

            I think the problems that arise with nested and overly large lists and
            dictionaries, and difficulties of handling other mutable datatypes will
            make my little assignment just too difficult. I'll just specify that
            call-by-reference isn't supported and leave it at that.

            Between floors. Going down.
            Enjoying the view.

            Comment

            • Casey Hawthorne

              #7
              Re: a question regarding call-by-reference

              Don't send the whole list or parts of the list to the server unless
              actually needed!

              If the server has to do an append, have the server send back the new
              elements to be appended and then do the appending on the client side!

              Or

              Does the homework specify that the entire mutable type be sent to the
              server?


              "enjoying the view" <pparkkin@gmail .com> wrote:
              [color=blue]
              >I am working on a school project, trying to build a simple RPC stub
              >generator. The idea is that the generator takes a normal python file
              >with functions defined in it and produces a client and server stub. The
              >client stub has the same functions defined, but they just connect to
              >the server and ask it to call the desired functions. The server stub is
              >a server listening for incoming requests and dispatching them to the
              >appropriate functions and sending back the results. Extremely simple
              >and I've gotten it to mostly work, but I have one problem:
              >call-by-reference parameters. An example:
              >
              >A function like this:
              >def sum(a, b):
              > return a + b
              >
              >would yield a client stub like this (simplified):
              >def sum(a, b):
              > send_message({' funcname': 'sum', 'args': (a, b)})
              > result = receive_message ()
              > return result
              >
              >Now, this works just fine. I get problems when I try to do something
              >like this:
              >def add_elm(list):
              > list.append('el m')
              > return
              >
              >Imported normally this would work fine. The list given as a parameter
              >would be one element larger. But when the stubs are generated, the
              >function doesn't return anything and the list is appended in the server
              >and the client-side list is left untouched. At first I thought the
              >solution was easy, just return the changed parameters and place them in
              >the parameter variables in the client stub:
              >def add_elm(list):
              > send_message({' funcname': 'add_elm', 'args': (list)})
              > response = receive_message ()
              > (list) = response.get('a rgs')
              > return response.get('r esult')
              >
              >The problem is, this doesn't work. The original list doesn't point to
              >the changed list. I have been trying to figure this out and it seems
              >that if I just assign to the list variable it just modifies the local
              >(to the function) name space, and that those changes aren't reflected
              >in the list in the original name space.
              >
              >I believe there are some ways around this, but I haven't found one that
              >would not require any special handling in either the code calling the
              >client stub or the original functions. I want to maintain transparency.
              >
              >etv[/color]
              --
              Regards,
              Casey

              Comment

              Working...