word file searching

[shanmugamit]

hi,
Is it better to store large text data(10000 document) inside the database or to keep it in file and save its path in the database, and if I want to do search for a word in the text, which better and FASTER (the database or the file).now i using file system for search but it takes more time.plz give some idea for better performance

[pbmods]

Heya, shanmugamit.

grepping through the filesystem is likely to be much faster than searching a database.

The major benefit to using a database search, though, is that you get relevance sorting and query expansion free of charge.

If you just want a straight up text search, I'd look into using grep. Even with thousands of files, it's still pretty fast.

Either way, realize that you're going to be accessing your hard disk to do the search (even if you store the data in a database), and that's going to be slower than if you can keep your data small enough that you can fit it all in RAM.

Your best bet would be to create an index for best performance. I don't know a whole lot about it, but the Wikipedia has a decent introduction (http://en.wikipedia.org/wiki/Index_(search_engine)).