scraping display to obtain all on-screen text using OCR

[Jonathan Epstein]

I would like to perform a more classical type of "screen scraping"
than what most people now associate with this term. I only want to
find all the text on the current screen, and obtain associated screen
coordinates. This probably must be done using OCR.

This need only run on Windows. A fairly-pure Python solution would be
ideal because most of the software which would use this functionality
is also written in Python.

The ideal output would consist of a list of tuples, where each tuple
consists of ("string found", a, b, c, d) where the latter four
constitute a bounding rectangle associated with the text that was
found. It might also be handy to throw in some font information.

Thanks in advance for any pointers.

Jonathan

[Paul Rubin]

Re: scraping display to obtain all on-screen text using OCR

jaepstein_63@ya hoo.com (Jonathan Epstein) writes:[color=blue]
> I would like to perform a more classical type of "screen scraping"
> than what most people now associate with this term. I only want to
> find all the text on the current screen, and obtain associated screen
> coordinates. This probably must be done using OCR.
> This need only run on Windows.[/color]

Usually you do that by intercepting the Windows text painting events,
rather than anything as horrendous as OCR'ing.

[Roger Binns]

Re: scraping display to obtain all on-screen text using OCR

Jonathan Epstein wrote:[color=blue]
> I would like to perform a more classical type of "screen scraping"
> than what most people now associate with this term. I only want to
> find all the text on the current screen, and obtain associated screen
> coordinates. This probably must be done using OCR.
>
> This need only run on Windows.[/color]

You can use the accessibility APIs to get that information. Start at

http://weblogs.asp.net/oldnewthing/archive/2004/04/23/118893.aspx

[color=blue]
> A fairly-pure Python solution would be
> ideal because most of the software which would use this functionality
> is also written in Python.[/color]

You may be able to do it using win32all, ctypes or worst case a
SWIG wrapper.

Roger