EPA Transportation and Air Quality (TAQ)

[hkuchampudi]

EPA Transportation and Air Quality (TAQ)

• Agency: Environmental Protection Agency
• Agency Division: EPA Transportation and Air Quality (TAQ)
• Data Type: Transport and Air Quality
• Data Format: Various

I have mined the metadata for the EPA's Transportation and Air Quality (TAQ) documents and have hosted the direct download links to the documents in my repository. I need help mining the documents themselves as I do not have the space to download them.

Downloading PDFs

You can execute the following command replacing the placeholders with the appropriate values to download files in bulk:

awk 'FNR>=[Starting_Line_Number] && FNR<=[Ending_Line_Number]' [Links_Location] | while read -r link; do curl --retry 10 -OL $(echo $link | tr -d '\r'); done

• [Starting_Line_Number] with the line number of the first link to download
• [Ending_Line_Number] with the line number of the last link to download
• [Links_Location] with the path to the downloaded Links.txt file from the repository

Download Information

Property	Value
Number links/documents	25690
Estimated total filesize	11 GB

[JeremiahCurtis]

Well, this is interesting. I ran downthemall (firefox) on the list of OTAQ links provided in the original post. Worked like a charm for the first half, but then I got thousands of 'file access errors'....everything on the list up to https://iaspub.epa.gov/otaqpub/display_file.jsp?docid=20729&flag=1 downloaded fine, but nothing after that

[HostileGranola]

I am pulling this data with wget --content-disposition --trust-server-names -i https://raw.githubusercontent.com/hkuchampudi/GovDataDump/master/EPA%20Transport%20and%20Air%20Quality/Links.txt. Will update with sizes and hashes once it is all downloaded.

[HostileGranola]

I have a copy of this data as of 12:00:00 UTC.

Hashes computed using hashdeep -erl are here.

Sizes computed using du -b --max-depth=1 --human-readable are here.

[x775]

I have a complete copy as of this posting.

md5: 4735e9dc629746010baca30e368046d1
sha256: 86122a2a39cbbd2b35fd702b9c4e05106c46791a8dab4f36626dfe2815258656

Individual checksums: https://gist.github.com/x775/8cf8445faed2c47fc7702ad898be055d

Size: 11.67764GB

Compressed name: EPA_Transportation_and_Air_quality.7z
Compressed md5: 3430c5799f66bcf248bae01771164938
Compressed sha256: 55fa39a1085c60c2611fc1a41577ed3eec5eb788a81e22f07a3623a54b3eb25c
Compressed size: 5.20261GB
Compressed download link: https://drive.google.com/open?id=0B6PlQrUTwL1PcmQ3Y29IOGhGY0U

[derpasaurusz]

Hello, I have a public mirror for this data located here

Hashes computed using hashdeep -erl located here

Size:12GB
Compressed name: EPA_Transportation_and_Air_Quality_TAQ.tar.bz2
Compressed md5: a1d68b6a2b280b1b5344774f8b62cac7
Compressed sha256: a28a5a4b9d6832a040219ddedfb748885160a79805327cfe7294c1d6a41b9514
Compressed size:9.3GB
Compressed download link can be found here

Still fairly new to this. Let me know if I missed anything.

[HostileGranola]

I no longer have this dataset due to storage space constraints. Apologies.

[annie-things]

All mirrors offline.