programmatically navigate websites

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Tina
    #1

    programmatically navigate websites

    How can I programmaticall y navigate a website just as a user would? click
    on buttons, hyperlinks, etc. I know how to read nodes using XPath but I
    don't know what classes to use to navigate.

    I have to write a program to automatically read mail from an html mail
    system.
    Thanks,
    T

    Tags: None
    • bruce barker
      #2
      Re: programmaticall y navigate websites

      it will depend on the website. you can use webclient to get the page.
      fews sites render valid xml, so you will need an html parser to read the
      page. some pages require javascript, or have javascript modifiy the dom.
      this would be common with a mail system, but it may have a dumb mode
      (maybe for mobile devices). for instance with aspx any thing but an
      imagebutton or submit button require javascript to do postback. you will
      need emulate this.

      you should read the w3c http 1.1 protocol spec so you know how to get
      and post data. some site (say an aspx site with viewstate), will require
      you do a get and postback matching data.

      you should get a tool like fiddler, so you can see the raw data passed
      back and forth and what headers are required.



      -- bruce (sqlwork.com)



      Tina wrote:
      How can I programmaticall y navigate a website just as a user would?
      click on buttons, hyperlinks, etc. I know how to read nodes using XPath
      but I don't know what classes to use to navigate.
      >
      I have to write a program to automatically read mail from an html mail
      system.
      Thanks,
      T

        Comment

        • Joe Fawcett
          #3
          Re: programmaticall y navigate websites



          "Tina" <TinaMSeaburn@n ospamexcite.com wrote in message
          news:ubLQPu9MJH A.2348@TK2MSFTN GP05.phx.gbl...
          How can I programmaticall y navigate a website just as a user would? click
          on buttons, hyperlinks, etc. I know how to read nodes using XPath but I
          don't know what classes to use to navigate.
          >
          I have to write a program to automatically read mail from an html mail
          system.
          Thanks,
          T
          You have a few options:
          * Automate IE - you can navigate to a page and use the DOM to fill out text
          boxes, press buttons and read results
          * Use MSXML2.XmlHttp. 6.0 - you can make POST and GET calls and parse the
          returned results using the 'HtmlFile' COM class to create an HTML document
          * The same as above but using WebRequest and WebResponse from .NET
          * Use a tool like Visual Studio - you can record macros that move you
          through a site, then you refine them to use inputs from a database and make
          decisions according to returned data


          --

          Joe Fawcett (MVP - XML)
          http://joe.fawcett.name


            Comment

            Previous template Next
            Working...