summarize text

[robin]

hello list,

does anyone know of a library which permits to summarise text? i've
been looking at nltk but haven't found anything yet. any help would be
very welcome.
thank you all in advance,

robin

[Tim Chase]

Re: summarize text

> does anyone know of a library which permits to summarise text?[color=blue]
> i've been looking at nltk but haven't found anything yet. any
> help would be very welcome.[/color]

Well, summarizing text is one of those things that generally
takes a brain-cell or two to do. Automating the process would
require doing it either smartly (some sort of
neural-net/NLP/Markov-chain technology, which is a non-trivial
task--something one might consider braving in the 3rd or 4th-year
of a university computer-science program), or doing it fairly
dumbly. As an example of a "dumb" solution, you can use regexps
to trim off the first few words and the last few words and call
that a "summary":
[color=blue][color=green][color=darkred]
>>> import re
>>> r = re.compile(r'^( .{8}.*?\b)\s.*\ s(\b.{8}.*?)', re.DOTALL)
>>> s = """This is the first line[/color][/color][/color]
.... and it has a second line
.... and a third line
.... and the last line is the fourth line."""[color=blue][color=green][color=darkred]
>>> result = r.sub(r"\1...\2 ",s.strip() )
>>> result[/color][/color][/color]
'This is the...fourth line.'

You can adjust the "{8}" portions for more or less
leader/trailing context characters.

The regexp might need a bit of tweaking for somewhat short
strings, but if they're fairly short, one might not need to
summarize them ;)

-tkc

[gene tani]

Re: summarize text


robin wrote:[color=blue]
> hello list,
>
> does anyone know of a library which permits to summarise text? i've
> been looking at nltk but haven't found anything yet. any help would be[/color]

unclear what you're asking, maybe look at:

http://www.cs.waikato.ac.nz/~ml/weka/index.html

http://www.kdnuggets.com/software/suites.html

http://www.ailab.si/orange

http://mallet.cs.umass.edu/index.php/Main_Page

http://minorthird.sourceforge.net/

http://www.dia.uniroma3.it/db/roadRunner/

http://www.lemurproject.org/

[robin]

Re: summarize text

thanks for all your replies. lemur looks pretty interesting!
robin

gene tani wrote:[color=blue]
> robin wrote:[color=green]
> > hello list,
> >
> > does anyone know of a library which permits to summarise text? i've
> > been looking at nltk but haven't found anything yet. any help would be[/color]
>
> unclear what you're asking, maybe look at:
> http://www.cs.waikato.ac.nz/~ml/weka/index.html
>
> http://www.kdnuggets.com/software/suites.html
> http://www.ailab.si/orange
>
> http://mallet.cs.umass.edu/index.php/Main_Page
> http://minorthird.sourceforge.net/
> http://www.dia.uniroma3.it/db/roadRunner/
>
> http://www.lemurproject.org/[/color]

[Lawrence D'Oliveiro]

Re: summarize text

.... sorry, I thought you said "summarize Proust".

:)