Pattern Recognition

[Aaron]

This there a public program/algorithm that can tell me the key points of a
text?

For example I entered the following text:

Web logs, or blogs, the online personal diaries where big names and no names
expound on everything from pets to presidents, are going mainstream. While
still a relatively small piece of total online activity, blogging has caught
on with affluent young adults. As Forrester Research analysts recently
noted, blogging will become increasingly common as these consumers age.

The program should give me the main keywords such as: blog, online,people.. .

I know some spam filters does this and Google??
I don't need it to be super accurate, but just to demostrate that this is
possible.

Any help is greatly appreciated,
Aaron

[Chad Z. Hower aka Kudzu]

Re: Pattern Recognition

"Aaron" <kuya789@yahoo. com> wrote in
news:uMFZUv6EFH A.1392@tk2msftn gp13.phx.gbl:[color=blue]
> This there a public program/algorithm that can tell me the key points of
> a text?[/color]

Do you want to know the key points or just evaluate? If you just want to
evaluate you should look at Bayesian algolrithms.



--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Empower ASP.NET with IntraWeb

http://www.atozed.com/IntraWeb/

[Aaron]

Re: Pattern Recognition

key points

If the program were to process this post, it would output keywords like:
programming, internet...

"Chad Z. Hower aka Kudzu" <cpub@hower.org > wrote in message
news:Xns95FEBCA CFA190cpub@127. 0.0.1...[color=blue]
> "Aaron" <kuya789@yahoo. com> wrote in
> news:uMFZUv6EFH A.1392@tk2msftn gp13.phx.gbl:[color=green]
>> This there a public program/algorithm that can tell me the key points of
>> a text?[/color]
>
> Do you want to know the key points or just evaluate? If you just want to
> evaluate you should look at Bayesian algolrithms.
>
>
>
> --
> Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
> "Programmin g is an art form that fights back"
>
> Empower ASP.NET with IntraWeb
> http://www.atozed.com/IntraWeb/[/color]

[Aaron]

Re: Pattern Recognition

Nick ,thanks for the info, I'll keep looking and let you guys know.

The best example I can think of right now is Google's Adsense. It does a
very good job analyzing the webpage and pin pointing the central meaning.
Any idea what method they use?


"Nick Malik [Microsoft]" <nickmalik@hotm ail.nospam.com> wrote in message
news:vN2dnRh0Vs vkjI7fRVn-2g@comcast.com. ..[color=blue]
> Hello Aaron,
>[color=green]
>> This there a public program/algorithm that can tell me the key points of
>> a text?
>>
>> For example I entered the following text:
>>
>> Web logs, or blogs, the online personal diaries where big names and no
>> names expound on everything from pets to presidents, are going
>> mainstream. While still a relatively small piece of total online
>> activity, blogging has caught on with affluent young adults. As Forrester
>> Research analysts recently noted, blogging will become increasingly
>> common as these consumers age.
>>
>> The program should give me the main keywords such as: blog,
>> online,people.. .[/color]
>
> Fascinating. Simply keyword counting produces nearly nothing. The only
> words that occur more than once are "blogging" and "names." The word
> "people" that you produce in your list of keywords doesn't occur in the
> paragraph at all.
>
> You would need an algorithm that creates a contextual map through a
> lexical tree and produces, effectively, an "understand ing" of the key
> concept of the paragraph. Effectively, you are entering the field of
> Computational Linguistics.
>
> There is some fascinating research on Natural Language Processing that
> began in the late 80s (and continues today) that addresses many of these
> ideas. I'm sure that some of the current "search" research has raised
> interest further. Microsoft Research, IBM Research, and others are very
> much interested in these areas.
>
> One example would be the Text Mining project at IBM:
> http://www.trl.ibm.com/projects/textmining/index_e.htm
>
> A good link for coding systems that follow some of these practices is
> here: http://www.cl.cam.ac.uk/Research/NL/anlt.html
>
> There is WAY too much involved, morphologically , lexically, and
> linguistically, to demonstrate even the simplest of these algorithms in a
> newsgroup message. Start at your local college library and/or Google for
> "Natural Language Processing" Go from there.
>
> --
> --- Nick Malik [Microsoft]
> MCSD, CFPS, Certified Scrummaster
> http://blogs.msdn.com/nickmalik
>
> Disclaimer: Opinions expressed in this forum are my own, and not
> representative of my employer.
> I do not answer questions on behalf of my employer. I'm just a
> programmer helping programmers.
> --
>
>[/color]

[Predictor]

Re: Pattern Recognition


Aaron wrote:[color=blue]
> This there a public program/algorithm that can tell me the key points[/color]
of a[color=blue]
> text?
>
> For example I entered the following text:
>
> Web logs, or blogs, the online personal diaries where big names and[/color]
no names[color=blue]
> expound on everything from pets to presidents, are going mainstream.[/color]
While[color=blue]
> still a relatively small piece of total online activity, blogging has[/color]
caught[color=blue]
> on with affluent young adults. As Forrester Research analysts[/color]
recently[color=blue]
> noted, blogging will become increasingly common as these consumers[/color]
age.[color=blue]
>
> The program should give me the main keywords such as: blog,[/color]
online,people.. .[color=blue]
>
> I know some spam filters does this and Google??
> I don't need it to be super accurate, but just to demostrate that[/color]
this is[color=blue]
> possible.[/color]


It sounds like you're interested in text mining. Try KDnuggets
(http://www.kdnuggets.com) in the "Software" section.

-Will Dwinnell

http://will.dwinnell.com