Fishing for a way to search a humongous database

[richard]

http://safer.fmcsa.dot.gov/CompanySnapshot.aspx

How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output form, just
the bare basics.
That is, trucking company name, city location, and zip code and a couple of
other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT number.
Using that field as a starting point, one might be able to extract the rest
of the information.
Going by a name search, the site only returns 500 matches and does not care
where in the name the field appears.

I'm open to suggestions.

[Paul Lautman]

Re: Fishing for a way to search a humongous database

richard wrote:

http://safer.fmcsa.dot.gov/CompanySnapshot.aspx

>
>
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output
form, just the bare basics.
That is, trucking company name, city location, and zip code and a
couple of other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT
number. Using that field as a starting point, one might be able to
extract the rest of the information.
Going by a name search, the site only returns 500 matches and does
not care where in the name the field appears.
>
I'm open to suggestions.

Create a fulltext index?

[richard]

Re: Fishing for a way to search a humongous database


"Paul Lautman" <paul.lautman@b tinternet.comwr ote in message
news:4lape8F129 7jU1@individual .net...

richard wrote:

>http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
>>
>>
>How could I go about automating a search of this site's database?
>I'm not interested in all the details they present on the output
>form, just the bare basics.
>That is, trucking company name, city location, and zip code and a
>couple of other fields
>With over a million records, manual searching is out of the question.
>Every trucking company is required to have, and display, a USDOT
>number. Using that field as a starting point, one might be able to
>extract the rest of the information.
>Going by a name search, the site only returns 500 matches and does
>not care where in the name the field appears.
>>
>I'm open to suggestions.

>
Create a fulltext index?

I have no clues as to what that is.

[richard]

Re: Fishing for a way to search a humongous database


"richard" <don@john.sonwr ote in message
news:ecp8e602o2 v@news2.newsguy .com...

http://safer.fmcsa.dot.gov/CompanySnapshot.aspx

>
>
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output form,
just the bare basics.
That is, trucking company name, city location, and zip code and a couple
of other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT number.
Using that field as a starting point, one might be able to extract the
rest of the information.
Going by a name search, the site only returns 500 matches and does not
care where in the name the field appears.
>
I'm open to suggestions.
>

PS:

One of my ideas was to automate a search simulating a manual search using
the usdot number as a base.
If the number was issued, then save the output page as a text file.
If it was not issued, skip it and go to the next. As an unissued number
results in a stock "error" message.

[Jerry Stuckle]

Re: Fishing for a way to search a humongous database

richard wrote:

http://safer.fmcsa.dot.gov/CompanySnapshot.aspx

>
>
How could I go about automating a search of this site's database?
I'm not interested in all the details they present on the output form, just
the bare basics.
That is, trucking company name, city location, and zip code and a couple of
other fields
With over a million records, manual searching is out of the question.
Every trucking company is required to have, and display, a USDOT number.
Using that field as a starting point, one might be able to extract the rest
of the information.
Going by a name search, the site only returns 500 matches and does not care
where in the name the field appears.
>
I'm open to suggestions.
>
>
>
>

Get a copy of the database from FMCSA?

--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attgl obal.net
=============== ===

[richard]

Re: Fishing for a way to search a humongous database


"Jerry Stuckle" <jstucklex@attg lobal.netwrote in message
news:_t6dnSfLTd _aAW3ZnZ2dnUVZ_ uidnZ2d@comcast .com...

richard wrote:

>http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
>>
>>
>How could I go about automating a search of this site's database?
>I'm not interested in all the details they present on the output form,
>just the bare basics.
>That is, trucking company name, city location, and zip code and a couple
>of other fields
>With over a million records, manual searching is out of the question.
>Every trucking company is required to have, and display, a USDOT number.
>Using that field as a starting point, one might be able to extract the
>rest of the information.
>Going by a name search, the site only returns 500 matches and does not
>care where in the name the field appears.
>>
>I'm open to suggestions.
>>
>>
>>
>>

>
Get a copy of the database from FMCSA?

Have asked, no reply yet.

[Jerry Stuckle]

Re: Fishing for a way to search a humongous database

richard wrote:

"Jerry Stuckle" <jstucklex@attg lobal.netwrote in message
news:_t6dnSfLTd _aAW3ZnZ2dnUVZ_ uidnZ2d@comcast .com...
>

>>richard wrote:
>>

>>>http://safer.fmcsa.dot.gov/CompanySnapshot.aspx
>>>
>>>
>>>How could I go about automating a search of this site's database?
>>>I'm not interested in all the details they present on the output form,
>>>just the bare basics.
>>>That is, trucking company name, city location, and zip code and a couple
>>>of other fields
>>>With over a million records, manual searching is out of the question.
>>>Every trucking company is required to have, and display, a USDOT number.
>>>Using that field as a starting point, one might be able to extract the
>>>rest of the information.
>>>Going by a name search, the site only returns 500 matches and does not
>>>care where in the name the field appears.
>>>
>>>I'm open to suggestions.
>>>
>>>
>>>
>>>

>>
>>Get a copy of the database from FMCSA?

>
>
Have asked, no reply yet.
>
>
>

You may have to dig around to find the right person(s) to ask. If it's
like other federal agencies, people just ignore questions not in their
job function. And if it is in their job function, they may or may not
answer - depending on whether the coin comes up heads or tails :-)

--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attgl obal.net
=============== ===