Using fopen(), Curl or 'Other' to grab remote contents

[James]

Hi guys,

I have been building a search engine here - not because I have plans of
dethrowning Google but as a simple app upon which to develop a function
set that I can use for other things.

So I coded my little search engine, uploaded it and it didn't work with
my host (Dreamhost.com) :

Warning: fopen(): URL file-access is disabled in the server
configuration

There is no way around this. Maybe I am a fool but when I am creating
apps I like to have as little configuration requirements as possible --
in my mind it means more people can use my stuff. In this app I use
fopen to go off and grab the page contents from which I extract the
meta tags, strip away all the HTML tags, headers, and javascript and
store what I need to.

I did the above without using fragile weak get_meta_tags and have my
own version of get_headers - because its a PHP5 function only, and
because I dont like the idea of a page making multiple connections to a
site.

Essentially my functions will work if I can grab the entire page
contents and throw them into a variable which I in turn pass.

Is there a way of doing this without using fopen and curl??

I'm going to have to write a wrapper for my functions as it is so this
works seemlessly regardless of the method used - I will include Curl in
this however as mentioned earlier I prefer not to have to use extra
extensions.

Is it common for fopen to be disabled for remote content? Or do I just
have an over zealous admin staff.

I was planning on using these function with XML files down the road...
this is the core of everything I am doing.

Help/Advice appreciated

[Screwball]

Re: Using fopen(), Curl or 'Other' to grab remote contents

James wrote:[color=blue]
> Hi guys,
>
> I have been building a search engine here - not because I have plans of
> dethrowning Google but as a simple app upon which to develop a function
> set that I can use for other things.
>
> So I coded my little search engine, uploaded it and it didn't work with
> my host (Dreamhost.com) :
>
> Warning: fopen(): URL file-access is disabled in the server
> configuration
>
> There is no way around this. Maybe I am a fool but when I am creating
> apps I like to have as little configuration requirements as possible --
> in my mind it means more people can use my stuff. In this app I use
> fopen to go off and grab the page contents from which I extract the
> meta tags, strip away all the HTML tags, headers, and javascript and
> store what I need to.[/color]

<snip>
[color=blue]
> Is it common for fopen to be disabled for remote content? Or do I just
> have an over zealous admin staff.[/color]

Not sure about common, but the wespace provided by my ISP has the same
kind of restrictions on accessing off-site material.

http://www.blueyonder.co.uk/blueyonder/getContent.jspx?page=h_services_pwp_advanced_restrictions

Maybe there's an equivalent page for your provider?

--
Steve.

[Colin McKinnon]

Re: Using fopen(), Curl or 'Other' to grab remote contents

James wrote:
[color=blue]
>
> Hi guys,
>[/color]
<snip>[color=blue]
>
> Warning: fopen(): URL file-access is disabled in the server
> configuration
>[/color]

Yup - check you can use fsockopen() and roll your own HTTP stack. If this
sounds too scary you could try phpclasses.org or freshmeat.net as I'm sure
there may already be code available for PHP to do this already (Snoopy?).

BTW: why write your own search engine? Check out MnoGo.

C.

[James]

Re: Using fopen(), Curl or 'Other' to grab remote contents

As I mentioned :D so I have a foundation to build some functions -- not
grandeous plans, no de-throwning of google :D --