JS Web Robot

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Paul Dennis
    #1

    JS Web Robot

    Hi,

    I'm trying to write a web robot using JavaScript.
    It's objective would be to surf around and look
    for patterns in the way web pages link to each
    other or in the text they contain. Data would be
    returned in a web box which could later be copied
    into another application.

    That's not to tough a challenge. I can make a
    JS application surf around my hard drive or
    web site with ease. I simply click an html into
    a second window and wait for the document
    readyState to be complete, then grab the
    document.links array and point the window
    at a new location. Off it goes.

    But when it tries to surf from my drive to
    my web site, or from my web site to another
    web site, it gets an error. It crashes the first
    time it tries to check the readyState of a
    document from a different server.

    I think that maybe JS has been designed to foil
    attempts to build web robots with it. If so, is there
    any way around it? Or maybe I'm just missing a
    critical JS detail or two. So, does anyone know
    what's going on here? Can anyone help me out?

    -Paul Dennis.


    Tags: None
    • Lasse Reichstein Nielsen
      #2
      Re: JS Web Robot

      "Paul Dennis" <pbdennis@telus .net> writes:
      [color=blue]
      > But when it tries to surf from my drive to
      > my web site, or from my web site to another
      > web site, it gets an error. It crashes the first
      > time it tries to check the readyState of a
      > document from a different server.
      >
      > I think that maybe JS has been designed to foil
      > attempts to build web robots with it.[/color]

      The browser security model has. If you try to access the content of a
      page from a different domain, you are stopped - the hard way.
      [color=blue]
      > If so, is there any way around it?[/color]

      Not in any browser, but if it is just your own browser you might be
      able to give it extended permissions. If the browser is IE, you can
      look into HTML Applications (google for "HTML application HTA").

      /L
      --
      Lasse Reichstein Nielsen - lrn@hotpop.com
      DHTML Death Colors: <URL:http://www.infimum.dk/HTML/rasterTriangleD OM.html>
      'Faith without judgement merely degrades the spirit divine.'

        Comment

        Previous template Next
        Working...