Wanted: PHP Site Map Generator!

[Mik Foggin]

Hi,

Having spent my free time over the last few months converting several
hundred pages of mainly static (s)html into eight pages of data driven php
loveliness and a whopping MySQL database I'm faced with a bit of a dilemma.

Will search engines be able to crawl all the index.php?query paths so
people can find stuff? I'm currently at the top of the heap on Google as my
site has been around for years and is pretty niche and I don't want to lose
that advantage.

Which leads to the main part of the question; is there a 'site map'
generator which will vreate a single crawlable HTM page with all my
possible pages on iot which I can then link to from my home page which will
enable the search spiders to do their thing?

Or am I left with the unenviable task of having to put response/redirects
on all the old pages pointing to the equivalent dynamic pages?

Many thanks,

Mik Foggin


--
Remove capitals to reply :)

[Justin Koivisto]

Re: Wanted: PHP Site Map Generator!

Mik Foggin wrote:[color=blue]
> Having spent my free time over the last few months converting several
> hundred pages of mainly static (s)html into eight pages of data driven php
> loveliness and a whopping MySQL database I'm faced with a bit of a dilemma.
>
> Will search engines be able to crawl all the index.php?query paths so
> people can find stuff? I'm currently at the top of the heap on Google as my
> site has been around for years and is pretty niche and I don't want to lose
> that advantage.[/color]

Well, you don't actually have to change the URIs of the pages on your
site. In fact, doing so may get you into a bit larger problem since all
your inbound links and such will no longer be valid.

You can use apache's mod_rewrite (or isapi_rewrite for IIS) to make your
changes transparent to the client. For instance, say you have products
that were once accessed via the following:

/products/prodnum1.shtml
/products/prodnum2.shtml
/products/prodnum3.shtml

The way it sounds like you have it now is more like:

/products.php?pr odnum=prodnum1
/products.php?pr odnum=prodnum2
/products.php?pr odnum=prodnum3

If you used mod_rewrite, you can take care of that by putting the
following in a .htaccess file:

RewriteEngine On
RewritePath /
RewriteRule ^products/([a-zA-Z0-9_-])+\.shtml$ prod.php?prodnu m=$1 [L]

The uri /products/number.shtml will then tranlate to the page
prod.php?prodnu m=number

*NOTE:* the target php file name should not be the same as part of the
URI you want to translate. Dependind on the order that the modules are
loaded, this may not give you the results you expect.

Added bonus: no need to have webmasters of other sites fix links. No
need to worry about search engines finding the new content. No need to
worry about loosing points with Google's PR.

Just remember, sites that have URIs that don't change over time will
have a better chance at keeping higher rankings in the search engines.
[color=blue]
> Which leads to the main part of the question; is there a 'site map'
> generator which will vreate a single crawlable HTM page with all my
> possible pages on iot which I can then link to from my home page which will
> enable the search spiders to do their thing?[/color]

http://home.snafu.de/tilman/xenulink.html is a good piece of software...
very handy!
[color=blue]
> Or am I left with the unenviable task of having to put response/redirects
> on all the old pages pointing to the equivalent dynamic pages?[/color]

You should do this anyway, but use mod_rewrite or equivalent if you had
a strong structure to begin with. Creating new files to handle each one
of these is a pain in the arse.

There also used to be a place on the php website where you could
download their url search code. The concept was simple. Say you enter in
a URI that doesn't exist on the site..

http://www.php.net/preg_replace

you are automatically redirected to:

http://www.php.net/manual/en/function.preg-replace.php

I'm sure someone here still has the code or the url to get it from... ;)

--
Justin Koivisto - spam@koivi.com
PHP POSTERS: Please use comp.lang.php for PHP related questions,
alt.php* groups are not recommended.

[S.T.]

Re: Wanted: PHP Site Map Generator!

> Mik Foggin wrote:[color=blue][color=green]
> > Having spent my free time over the last few months converting several
> > hundred pages of mainly static (s)html into eight pages of data driven[/color][/color]
php[color=blue][color=green]
> > loveliness and a whopping MySQL database I'm faced with a bit of a[/color][/color]
dilemma.[color=blue][color=green]
> >
> > Will search engines be able to crawl all the index.php?query paths so
> > people can find stuff? I'm currently at the top of the heap on Google as[/color][/color]
my[color=blue][color=green]
> > site has been around for years and is pretty niche and I don't want to[/color][/color]
lose[color=blue][color=green]
> > that advantage.[/color]
>
> Well, you don't actually have to change the URIs of the pages on your
> site. In fact, doing so may get you into a bit larger problem since all
> your inbound links and such will no longer be valid.
>
> You can use apache's mod_rewrite (or isapi_rewrite for IIS) to make your
> changes transparent to the client. For instance, say you have products
> that were once accessed via the following:
>
> /products/prodnum1.shtml
> /products/prodnum2.shtml
> /products/prodnum3.shtml
>
> The way it sounds like you have it now is more like:
>
> /products.php?pr odnum=prodnum1
> /products.php?pr odnum=prodnum2
> /products.php?pr odnum=prodnum3
>
> If you used mod_rewrite, you can take care of that by putting the
> following in a .htaccess file:
>
> RewriteEngine On
> RewritePath /
> RewriteRule ^products/([a-zA-Z0-9_-])+\.shtml$ prod.php?prodnu m=$1 [L]
>
> The uri /products/number.shtml will then tranlate to the page
> prod.php?prodnu m=number[/color]
[snip]

This is exactly what I did (or rather, had to do) when converting our
well-indexed site from static to dynamic. Can't say it was the most fun I
ever had, but it was worth it.

http://www.devarticles.com/art/1/506 gives a nice intro to mod_rewrite if
you're on Apache. Also note if you find the page is coming up fine but your
images and links are broken, the <base href> tag is your friend.

[Mik Foggin]

Re: Wanted: PHP Site Map Generator!

Justin Koivisto <spam@koivi.com > wrote in
news:Kgncb.938$ ow5.40456@news7 .onvoy.net:

[color=blue]
>
> You can use apache's mod_rewrite (or isapi_rewrite for IIS) to make
> your changes transparent to the client. For instance, say you have
> products that were once accessed via the following:
>
> /products/prodnum1.shtml
> /products/prodnum2.shtml
> /products/prodnum3.shtml
>
> The way it sounds like you have it now is more like:
>
> /products.php?pr odnum=prodnum1
> /products.php?pr odnum=prodnum2
> /products.php?pr odnum=prodnum3
>
> If you used mod_rewrite, you can take care of that by putting the
> following in a .htaccess file:
>
> RewriteEngine On
> RewritePath /
> RewriteRule ^products/([a-zA-Z0-9_-])+\.shtml$ prod.php?prodnu m=$1 [L]
>
> The uri /products/number.shtml will then tranlate to the page
> prod.php?prodnu m=number
>
>[/color]

Thanks for the response, unfortunately the original structure wasn't as
well defined as your suggestion may require; I have 7 sub directories but
there is no simple structure to the file names, so rather than as in your
example above its more like:

/products/apples.shtml
/products/cheese.shtml
/products/cornedbeef.shtm l

need mapping to /products/index.php?prodn um=x

so I expect I can't write a simple rule but will have to set a rewrite
for each file. Labourious but undoubtedly worth it!

I'm checking with my web hosts to see whether or not I can actually use
the mod_rewrite function at all so we'll see how we go!

Mik.

--
Remove capitals to reply :)

[sk]

Re: Wanted: PHP Site Map Generator!

Sounds like it might be too late now, but if you'd planned ahead you
could have imported into your database both the page content itself and
its old URI (that is, the directory name and filename) as a separate
field. Then you could do a one-time query from your database to
generate a list of static redirects that you could paste into your
httpd.conf, e.g.:

/food/cabbage.html /article.php?id= 3
/drink/hot/tea.htm /article.php?id= 4
....

That's what I did years ago in the days before fancy geegaws like
mod_rewrite.

--
Steve Koppelman



Mik Foggin wrote:[color=blue]
> Justin Koivisto <spam@koivi.com > wrote in
> news:Kgncb.938$ ow5.40456@news7 .onvoy.net:
>
>
>[color=green]
>>You can use apache's mod_rewrite (or isapi_rewrite for IIS) to make
>>your changes transparent to the client. For instance, say you have
>>products that were once accessed via the following:
>>
>>/products/prodnum1.shtml
>>/products/prodnum2.shtml
>>/products/prodnum3.shtml
>>
>>The way it sounds like you have it now is more like:
>>
>>/products.php?pr odnum=prodnum1
>>/products.php?pr odnum=prodnum2
>>/products.php?pr odnum=prodnum3
>>
>>If you used mod_rewrite, you can take care of that by putting the
>>following in a .htaccess file:
>>
>>RewriteEngi ne On
>>RewritePath /
>>RewriteRule ^products/([a-zA-Z0-9_-])+\.shtml$ prod.php?prodnu m=$1 [L]
>>
>>The uri /products/number.shtml will then tranlate to the page
>>prod.php?prod num=number
>>
>>[/color]
>
>
> Thanks for the response, unfortunately the original structure wasn't as
> well defined as your suggestion may require; I have 7 sub directories but
> there is no simple structure to the file names, so rather than as in your
> example above its more like:
>
> /products/apples.shtml
> /products/cheese.shtml
> /products/cornedbeef.shtm l
>
> need mapping to /products/index.php?prodn um=x
>
> so I expect I can't write a simple rule but will have to set a rewrite
> for each file. Labourious but undoubtedly worth it!
>
> I'm checking with my web hosts to see whether or not I can actually use
> the mod_rewrite function at all so we'll see how we go!
>
> Mik.
>[/color]

[Mik Foggin]

Re: Wanted: PHP Site Map Generator!

sk <steve-no-spam@hatless-dot-com-without-the-spam.com> wrote in
news:Kgsdb.6063 68$o%2.285348@s ccrnsc02:
[color=blue]
> Sounds like it might be too late now, but if you'd planned ahead you
> could have imported into your database both the page content itself
> and its old URI (that is, the directory name and filename) as a
> separate field. Then you could do a one-time query from your database
> to generate a list of static redirects that you could paste into your
> httpd.conf, e.g.:
>
> /food/cabbage.html /article.php?id= 3
> /drink/hot/tea.htm /article.php?id= 4
> ...
>
> That's what I did years ago in the days before fancy geegaws like
> mod_rewrite.
>
> --
> Steve Koppelman
>[/color]

Seems like mod_rewrite is a no-go so the httpd.conf could be a real life
saver,

Many thanks,

Mik