Asynchronous urllib (urllib+asyncore)?

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Jonathan Gardner

    #1

    Asynchronous urllib (urllib+asyncore)?

    So, I ran into a problem that I would like to write as little code as
    possible to solve.

    The problem is that I would like to send out a bunch of HTTP requests
    simultaneously, using asynchronous techniques, and then do stuff with
    the results in parallel. Think of something like Google's map-reduce.

    It sounds like this would be something someone else has done before. I
    searched google, but I don't think there is any discussion anywhere of
    anyone doing something like this.

    I am sure Twisted folks can code this up in 2 lines or less, or it
    probably already exists there by default. But I need this to exist in
    a non-asynchronous environment. I can't let Twisted take over
    everything.

    So, has someone done something like this? Is it something where I can
    bring the urllib and asyncore together? Or is it much more involved?
  • Terry Jones

    #2
    Re: European python developers

    Hi Jerry
    I am hoping one or two members of this list might help me locate in Europe
    Do you mean you want to re-locate here or that you're trying to locate
    (i.e., find) people already here?
    My personal first choice is Spain
    I'm in Barcelona (but not looking for work!). There are a couple of
    companies around that I know of who use Python and quite a lot of
    individual programmers.
    The point is that I can't really afford to run around and experiment, so I
    was hoping for some helpful comments or suggestions on how to research
    I'm not sure what you're wanting to research. See first comment above.

    Terry

    Comment

    Working...