Web Crawler for Flash Applications

[df220m10]

I’m trying to extract information (people's names and other info.) from a public database into a spreadsheet for academic research. Unfortunately, info about the people in the dataset (like their name and occupation) is not visible in the source code. Instead, there is this application by Caspio done, from what I can tell, in Flash.

I have several related questions:

1. Is there a way for Python to scrape information from Flash applications?

2. I know that Python’s mechanize can perform automatic searches (for example, searching a lot of information from a list). However, can it also

- select options from a dropdown menu?
- click arrows on the bottom of a screen to proceed to the next page of a listing of information?

Thanks,
A

[micmast]

This sounds like a fun project! :)

First things first, to get the information from a flash application you need to take a look at the flash itself. Usually when the information is loaded into a flash application it makes a request to a site. Then the answer is inserted into the flash and the results appear.

To know which site the flash application is contacting you need to decompile it. There are multiple applications that can do that, you need to search for the actionscript that triggers the request.

If you have that, you can continue searching a solution with python