q: How to get search engines crawl data

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • =?Utf-8?B?SklNLkgu?=

    q: How to get search engines crawl data

    How to get search engines crawl data
    I have a web application that uses user controls and pulls data directly
    from database and shows it to users in the internet. So there is not html
    that has the data, if I need to get Google or other search engines to crawl
    these pages, what are my options?


  • ThatsIT.net.au

    #2
    Re: How to get search engines crawl data


    "JIM.H." <JIMH@discussio ns.microsoft.co mwrote in message
    news:B19B629B-022D-4E08-B070-D89A46491EC9@mi crosoft.com...
    How to get search engines crawl data
    I have a web application that uses user controls and pulls data directly
    from database and shows it to users in the internet. So there is not html
    that has the data, if I need to get Google or other search engines to
    crawl
    these pages, what are my options?
    >

    If the data is displayed as default then the search engine will read it as
    html, if your users have to click on a button for example to get the data
    then of cause it will not be indexed

    Comment

    • =?Utf-8?B?SklNLkgu?=

      #3
      Re: How to get search engines crawl data

      Thanks for the reply. Data will be displaying via page_load, it is not in
      html, is there any workaround?

      "ThatsIT.net.au " wrote:
      >
      "JIM.H." <JIMH@discussio ns.microsoft.co mwrote in message
      news:B19B629B-022D-4E08-B070-D89A46491EC9@mi crosoft.com...
      How to get search engines crawl data
      I have a web application that uses user controls and pulls data directly
      from database and shows it to users in the internet. So there is not html
      that has the data, if I need to get Google or other search engines to
      crawl
      these pages, what are my options?
      >
      >
      If the data is displayed as default then the search engine will read it as
      html, if your users have to click on a button for example to get the data
      then of cause it will not be indexed
      >

      Comment

      • Mark Rae [MVP]

        #4
        Re: How to get search engines crawl data

        "JIM.H." <JIMH@discussio ns.microsoft.co mwrote in message
        news:54403C58-754A-436E-88D2-ADCC150D8EE0@mi crosoft.com...
        >If the data is displayed as default then the search engine will read it
        >as
        >html, if your users have to click on a button for example to get the data
        >then of cause it will not be indexed
        >
        Data will be displaying via page_load
        Any default data which is displayed during Page_Load will be picked up by a
        crawler...
        it is not in HTML.
        Pretty much everything displayed on a web page is HTML in one form or
        another - exceptions obviously include things like Flash etc, but this
        doesn't apply in your case...


        --
        Mark Rae
        ASP.NET MVP


        Comment

        • =?Utf-8?B?SklNLkgu?=

          #5
          Re: How to get search engines crawl data

          Thanks for the reply. I am talking about the data coming from database, on
          the fly, I am still not clear how a search engine would crawl it while the
          data is simply on the client’s browser basically. Can you give me more detail
          please?

          "Mark Rae [MVP]" wrote:
          "JIM.H." <JIMH@discussio ns.microsoft.co mwrote in message
          news:54403C58-754A-436E-88D2-ADCC150D8EE0@mi crosoft.com...
          >
          If the data is displayed as default then the search engine will read it
          as
          html, if your users have to click on a button for example to get the data
          then of cause it will not be indexed
          Data will be displaying via page_load
          >
          Any default data which is displayed during Page_Load will be picked up by a
          crawler...
          >
          it is not in HTML.
          >
          Pretty much everything displayed on a web page is HTML in one form or
          another - exceptions obviously include things like Flash etc, but this
          doesn't apply in your case...
          >
          >
          --
          Mark Rae
          ASP.NET MVP

          >
          >

          Comment

          • =?Utf-8?B?UGV0ZXIgQnJvbWJlcmcgW0MjIE1WUF0=?=

            #6
            Re: How to get search engines crawl data

            You have to understand how search engine crawlers work. They load a page,
            index it. In the process they look for links ("A" tag) in the page. Then they
            proceed to load each linked page and they repeat the process. It does not
            matter *Where* your content is coming from. Hope that helps.
            -- Peter
            Site: http://www.eggheadcafe.com
            UnBlog: http://petesbloggerama.blogspot.com
            Short Urls & more: http://ittyurl.net


            "JIM.H." wrote:
            Thanks for the reply. I am talking about the data coming from database, on
            the fly, I am still not clear how a search engine would crawl it while the
            data is simply on the client’s browser basically. Can you give me more detail
            please?
            >
            "Mark Rae [MVP]" wrote:
            >
            "JIM.H." <JIMH@discussio ns.microsoft.co mwrote in message
            news:54403C58-754A-436E-88D2-ADCC150D8EE0@mi crosoft.com...
            >If the data is displayed as default then the search engine will read it
            >as
            >html, if your users have to click on a button for example to get the data
            >then of cause it will not be indexed
            >
            Data will be displaying via page_load
            Any default data which is displayed during Page_Load will be picked up by a
            crawler...
            it is not in HTML.
            Pretty much everything displayed on a web page is HTML in one form or
            another - exceptions obviously include things like Flash etc, but this
            doesn't apply in your case...


            --
            Mark Rae
            ASP.NET MVP

            Comment

            • =?Utf-8?B?SklNLkgu?=

              #7
              Re: How to get search engines crawl data

              Thanks Peter. I think it is getting more clear but just to make sure I do not
              miss anything; The data grid has the data shown to a user base on criteria in
              page_load in code behind, let’s say search engine somehow crawled this page
              successfully at that specific time while page is loaded just because somebody
              has access that page. When the user close the page, input criteria is lost,
              data is not available anymore on the screen. Now are we saying that since
              search engine has crawled this page it will be able to include that page into
              other people’s search result? If yes, the search engine should be keeping
              those data somewhere, where would that be?



              "Peter Bromberg [C# MVP]" wrote:
              You have to understand how search engine crawlers work. They load a page,
              index it. In the process they look for links ("A" tag) in the page. Then they
              proceed to load each linked page and they repeat the process. It does not
              matter *Where* your content is coming from. Hope that helps.
              -- Peter
              Site: http://www.eggheadcafe.com
              UnBlog: http://petesbloggerama.blogspot.com
              Short Urls & more: http://ittyurl.net
              >
              >
              "JIM.H." wrote:
              >
              Thanks for the reply. I am talking about the data coming from database, on
              the fly, I am still not clear how a search engine would crawl it while the
              data is simply on the client’s browser basically. Can you give me more detail
              please?

              "Mark Rae [MVP]" wrote:
              "JIM.H." <JIMH@discussio ns.microsoft.co mwrote in message
              news:54403C58-754A-436E-88D2-ADCC150D8EE0@mi crosoft.com...
              >
              If the data is displayed as default then the search engine will read it
              as
              html, if your users have to click on a button for example to get the data
              then of cause it will not be indexed

              Data will be displaying via page_load
              >
              Any default data which is displayed during Page_Load will be picked up by a
              crawler...
              >
              it is not in HTML.
              >
              Pretty much everything displayed on a web page is HTML in one form or
              another - exceptions obviously include things like Flash etc, but this
              doesn't apply in your case...
              >
              >
              --
              Mark Rae
              ASP.NET MVP

              >
              >

              Comment

              • ThatsIT.net.au

                #8
                Re: How to get search engines crawl data


                "JIM.H." <JIMH@discussio ns.microsoft.co mwrote in message
                news:615E8B18-B288-4AEE-84F4-A3846D2433CA@mi crosoft.com...
                Thanks Peter. I think it is getting more clear but just to make sure I do
                not
                miss anything; The data grid has the data shown to a user base on criteria
                in
                page_load in code behind, let’s say search engine somehow crawled this
                page
                successfully at that specific time while page is loaded just because
                somebody
                has access that page. When the user close the page, input criteria is
                lost,
                data is not available anymore on the screen. Now are we saying that since
                search engine has crawled this page it will be able to include that page
                into
                other people’s search result? If yes, the search engine should be keeping
                those data somewhere, where would that be?



                The crawler will load the page as if it is a client, it will simply follow a
                URL. what ever loads no mater where it comes from will be indexed. It does
                no matter what users are using the page at the time. the crawler is the
                user, how it loads when the crawler follows the links is what will be
                indexed



                >
                >
                >
                "Peter Bromberg [C# MVP]" wrote:
                >
                >You have to understand how search engine crawlers work. They load a page,
                >index it. In the process they look for links ("A" tag) in the page. Then
                >they
                >proceed to load each linked page and they repeat the process. It does not
                >matter *Where* your content is coming from. Hope that helps.
                >-- Peter
                >Site: http://www.eggheadcafe.com
                >UnBlog: http://petesbloggerama.blogspot.com
                >Short Urls & more: http://ittyurl.net
                >>
                >>
                >"JIM.H." wrote:
                >>
                Thanks for the reply. I am talking about the data coming from database,
                on
                the fly, I am still not clear how a search engine would crawl it while
                the
                data is simply on the client’s browser basically. Can you give me more
                detail
                please?
                >
                "Mark Rae [MVP]" wrote:
                >
                "JIM.H." <JIMH@discussio ns.microsoft.co mwrote in message
                news:54403C58-754A-436E-88D2-ADCC150D8EE0@mi crosoft.com...
                >
                If the data is displayed as default then the search engine will
                read it
                as
                html, if your users have to click on a button for example to get
                the data
                then of cause it will not be indexed

                Data will be displaying via page_load
                >
                Any default data which is displayed during Page_Load will be picked
                up by a
                crawler...
                >
                it is not in HTML.
                >
                Pretty much everything displayed on a web page is HTML in one form or
                another - exceptions obviously include things like Flash etc, but
                this
                doesn't apply in your case...
                >
                >
                --
                Mark Rae
                ASP.NET MVP

                >
                >

                Comment

                Working...