MIni search engine

[codexxx]

Hi,

I am going to make a mini search engine. Though I am in a deep sea but got some clue how to move on. As I came to know that robot or crawlers are the main things in a search engine but I think database is also playing an important role here.So please give me suggestion that how that database should be designed or give some website links for good tutorial on search engine.

Thanks and Regards
codexxx

[nathj]

Hi,

My first question is what is this mini search engine for? If yuo are developing a search for a web site that is not database driven I would look at implementing a Google search on your site. this is quick, easy and very effective.

If the site is powered by a database then the simple solution is to use fulltext indexes. I have just done something like this and while there are a few draw backs with fulltext searches they do work pretty well.

Let me know which of these is most appropriate for you and I'll try to help further.

Cheers
nathj

[codexxx]

Hi,

My first question is what is this mini search engine for? If yuo are developing a search for a web site that is not database driven I would look at implementing a Google search on your site. this is quick, easy and very effective.

If the site is powered by a database then the simple solution is to use fulltext indexes. I have just done something like this and while there are a few draw backs with fulltext searches they do work pretty well.

Let me know which of these is most appropriate for you and I'll try to help further.

Cheers
nathj

Hi nathj,

Thanks a lot for your reply.The word mini was ironical :).
Actually I wanted to say vary basic and not so called askjeev or dogpile type search engine. ( I dont want to mention GOOGLE :P ). Anyway actually I want to make an web search engine using robots or crawlers. Though I dont know anything much about it but I had come to know that they use meta tags and other things.
And you told that you have created a search engine . So do plz help me by giving me some suggestions and also give some links for tutorials.

Thanks and Regards
codexxx

[gulyan]

To make a very simple crawler you can use php. You begin with a web page (home page probably). You download the page using php sockets. You put the content in a variable and treat it as a string. You search for links to other pages in that string. What I mean is that you search for something like this:

<a href="www.link.com">link</a>

You then apply the same method with each link. I think you will nead to use a boolean vector so you don't process the same link more than one time (you risk ending in an infinte loop). And you will probably need to make sure that you only process internal links (to the same site). Hope this helps :P

[nathj]

Hi nathj,

Thanks a lot for your reply.The word mini was ironical :).
Actually I wanted to say vary basic and not so called askjeev or dogpile type search engine. ( I dont want to mention GOOGLE :P ). Anyway actually I want to make an web search engine using robots or crawlers. Though I dont know anything much about it but I had come to know that they use meta tags and other things.
And you told that you have created a search engine . So do plz help me by giving me some suggestions and also give some links for tutorials.

Thanks and Regards
codexxx

Sorry for the confusion, what I have deon is simply implement a search using fulltext indexing for a single site that is powered by a database. In this case I built the database ad so I know the structure and have access to it for searching.

The earlier answer sound like a good way forward though. I must admit I've not used the sockets stuff but string manipulation is fairly straightforward .

Cheers
nathj