Python for search engine development

[corebump]

hi everybody,
i planinng develop a search engine and i think using the python. Python
performance is enough this project?

Best Regards

[wittempj@hotmail.com]

Re: Python for search engine development

Well, Google applies some Python in their implementation, see

http://www-db.stanford.edu/~backrub/google.html

[Mike Meyer]

Re: Python for search engine development

"corebump" <m.yalcin@gmail .com> writes:
[color=blue]
> hi everybody,
> i planinng develop a search engine and i think using the python. Python
> performance is enough this project?[/color]

If you're going to do the heavy lifting in Python, maybe. It depends
on what you're going to search, and the performance requirements for a
search.

On the other hand, if you use a Python wrapper around a full-text
search facility, so that python finds documents, takes queries and
formats results for the user, it should be just fine.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[gene tani]

Re: Python for search engine development

http://groups.google.com/group/comp.lang.python/browse_thread/thread/6e6dc84e68e25039/1436d0b3466e262a?q=lucene&rnum=1#1436d0b3466e262a

Mike Meyer wrote:[color=blue]
> "corebump" <m.yalcin@gmail .com> writes:
>[color=green]
> > hi everybody,
> > i planinng develop a search engine and i think using the python. Python
> > performance is enough this project?[/color]
>
> If you're going to do the heavy lifting in Python, maybe. It depends
> on what you're going to search, and the performance requirements for a
> search.
>
> On the other hand, if you use a Python wrapper around a full-text
> search facility, so that python finds documents, takes queries and
> formats results for the user, it should be just fine.
>
> <mike
> --
> Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
> Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.[/color]

[Alex Martelli]

Re: Python for search engine development

wittempj@hotmai l.com <martin.witte@g mail.com> wrote:
[color=blue]
> Well, Google applies some Python in their implementation, see
> http://www-db.stanford.edu/~backrub/google.html[/color]

"Some" is correct. As for writing a search engine in Python _only_,
hmmm -- I honestly don't know. You could surely develop a working
implementation, but then, to make it perform well, you'd most likely
want to profile it and retune some CPU-intensive parts using pyrex, or
C.

So, if during your program development process you can find good
open-source C or C++ libraries offering a fast implementation of some of
the CPU-bound stuff you know you'll need, you would probably be better
off by wrapping those libraries (again using pyrex, or maybe SWIG, or
Boost Python for C++, ...) rather than redoing them from scratch in
Python (and probably later having to do some tuning on those parts).
One notably strong point of Python is that it "plays well with others",
and I would advise you to leverage this fact.


Alex