php and Googlebot

**Yttrium** · Jul 17 '05, 01:48 PM

Re: php and Googlebot

"somaboy mx" <nosuch@fakemai l.fk> a écrit dans le message de news:
42c3d3a1$1_2@x-privat.org...

[color=blue]
> From my server stats I can see that the "master view" page is being[/color]
indexed[color=blue]
> fine by Google, but the bot apparently doesn't follow the links to the
> detail pages.
>
> Does anyone have any idea why?[/color]

Be patient...

**ECRIA Public Mail Buffer** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

Bots don't like dynamic URLs.

ECRIA

HugeDomains.com

http://www.ecria.com

**Alvaro G Vicario** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

*** ECRIA Public Mail Buffer wrote/escribió (Thu, 30 Jun 2005 11:54:05
-0400):[color=blue]
> Bots don't like dynamic URLs.[/color]

Since nowadays most interesting info comes from dynamic URLs, that doesn't
say much about bots.

--
-- Álvaro G. Vicario - Burgos, Spain
-- http://bits.demogracia.com - Mi sitio sobre programación web
-- Don't e-mail me your questions, post them to the group
--

**defaultuserbr@yahoo.com** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

Alvaro G Vicario wrote:[color=blue]
> *** ECRIA Public Mail Buffer wrote/escribió (Thu, 30 Jun 2005 11:54:05
> -0400):[color=green]
> > Bots don't like dynamic URLs.[/color]
>
> Since nowadays most interesting info comes from dynamic URLs, that doesn't
> say much about bots.[/color]

What would you have the bots do, fill out forms, click buttons? Even if
they could, would you want a bunch webcrawler bots firing your cgis and
PHP stuff all the time?

Brian

**juglesh** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

ECRIA Public Mail Buffer wrote:[color=blue]
> Bots don't like dynamic URLs.[/color]

They dont seem to mind mine.

**www.douglassdavis.com** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

defaultuserbr@y ahoo.com wrote:[color=blue]
> Alvaro G Vicario wrote:[color=green]
> > *** ECRIA Public Mail Buffer wrote/escribió (Thu, 30 Jun 2005 11:54:05
> > -0400):[color=darkred]
> > > Bots don't like dynamic URLs.[/color]
> >
> > Since nowadays most interesting info comes from dynamic URLs, that doesn't
> > say much about bots.[/color]
>
>
> What would you have the bots do, fill out forms, click buttons? Even if
> they could, would you want a bunch webcrawler bots firing your cgis and
> PHP stuff all the time?[/color]

I would... And I wouldn't mind them firing my cgis and PHP stuff a
few times a day. Wouldn't really hurt anything.

And they don't have to fill out forms most of the time, just follow the
given links. As long as there are links from the home page.

--
Contact Us Script:

http://www.douglassdavis.com

**Yttrium** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

"ECRIA Public Mail Buffer" <ngsremovemail@ ecria.com> a écrit dans le message
de news: da14im$b5k$1@mu rdoch.acc.Virgi nia.EDU...[color=blue]
> Bots don't like dynamic URLs.
>[/color]
Completly wrong

**somaboy mx** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

"ECRIA Public Mail Buffer" <ngsremovemail@ ecria.com> wrote in message
news:da14im$b5k $1@murdoch.acc. Virginia.EDU...[color=blue]
> Bots don't like dynamic URLs.[/color]

I'd be surprised ifthat were the case. I've had pages with a couple of
variables in the urlstring which got excellent indexing.

This said, I believe there might be some benefit in "clean" url's, if only
for human readability.

..soma

**ECRIA Public Mail Buffer** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

Google for one generally only indexes dynamic URLs for news sites and
forums. The reason is simple: Dynamic content changes unpredictably and
cannot be indexed reliably without additional credible information about the
content.

This is a discussion-based newsgroup, so disagree if you must. Feel free to
give us an example of your dynamic non-forum/news/blog URL that appears in
web search results. Doing so is a much better way to make your point than
anonymously posting "You don't know what you're talking about".

We have been in the Design/SEO business for years, and we know exactly what
we are talking about.

ECRIA

HugeDomains.com

http://www.ecria.com

**Daniel Tryba** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

ECRIA Public Mail Buffer <ngsremovemail@ ecria.com> wrote:[color=blue]
> Google for one generally only indexes dynamic URLs for news sites and
> forums. The reason is simple: Dynamic content changes unpredictably and
> cannot be indexed reliably without additional credible information about the
> content.
>
> We have been in the Design/SEO business for years, and we know exactly what
> we are talking about.[/color]

Then you should know that there is no way to determine if an URL is
dynamically generated or not other than parsing it's contents. If a
spider/indexer decides to mark something as dynamic based on the form of
an URL the developers should be fired at once.

**Daniel Tryba** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

ECRIA Public Mail Buffer <ngsremovemail@ ecria.com> wrote:[color=blue]
> Google for one generally only indexes dynamic URLs for news sites and
> forums. The reason is simple: Dynamic content changes unpredictably and
> cannot be indexed reliably without additional credible information about the
> content.
>
> We have been in the Design/SEO business for years, and we know exactly what
> we are talking about.[/color]

Then you should know that there is no way to determine if an URL is
dynamically generated or not other than parsing it's contents. And even
that doesn't mean anything (an unchanged page can still be dynamically
generated and a changed page could be manually updated).

If a spider/indexer decides to mark something as dynamic based on the
form of an URL the developers should be fired at once.

**ECRIA Public Mail Buffer** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

" Then you should know that there is no way to determine if an URL is
dynamically generated or not other than parsing it's contents. And even that
doesn't mean anything (an unchanged page can still be dynamically generated
and a changed page could be manually updated)."

Agreed. Furthermore, a static page URL may actually be a dynamically
generated page. For example, there is not a single HTML file on
http://www.ecria.com - but HTML is all anyone will see.

However, the point is that if there are variables in a URL, robots assume
that the page is generated dynamically - which is a fair assumption. We're
not condoning this - it's just what happens.

There are ways to get around it, but they don't change the fact that this is
the way robots work.

Other than a blog/forum/news pages, how many web sites containing URL
variables show up in, say, a Google search?

See what I mean?

ECRIA

HugeDomains.com

http://www.ecria.com

**Daniel Tryba** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

ECRIA Public Mail Buffer <ngsremovemail@ ecria.com> wrote:[color=blue]
> However, the point is that if there are variables in a URL, robots assume
> that the page is generated dynamically - which is a fair assumption.[/color]

Bad assumption, they could be used clientside.
[color=blue]
> There are ways to get around it, but they don't change the fact that this is
> the way robots work.[/color]

That's why I say they should be fired. They are to lazy to fix bad
assumptions and even go out of their way to insert this kind of silly
behavior.
[color=blue]
> Other than a blog/forum/news pages, how many web sites containing URL
> variables show up in, say, a Google search?
>
> See what I mean?[/color]

No, all pages of eg http://www.amsterdamchinafestival.nl/ appear to be
listed in Google. All dynamically generated by hiddeous URL without
trying to hide it's dynamic and it's no blog, form or news page. Even if
it was how would a spider know it's on of the "special" sites?

**ECRIA Public Mail Buffer** · Jul 17 '05, 01:49 PM

Re: php and Googlebot

There's a difference between being LISTED and being RANKED -
http://www.amsterdamchinafestival.nl/ does not show up in Google results for
"Amsterdam china festival" - it's own title!

How does Google know it's a news/blog site? Beats me - ask them. They know.
Probably something to do with the Google Groups technology.

I think that will have to be my final word... this discussion is not going
anywhere.

ECRIA

HugeDomains.com

http://www.ecria.com

php and Googlebot

php and Googlebot

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment