I'm trying to download all the occurrence data with COUNTRY=US
http://www.gbif.org/occurrence/search?COUNTRY=US
I've requested a download file, which is here http://www.gbif.org/occurrence/download/0039949-160910150852091
However, I've been having a lot of trouble getting the download to complete since it is so large(43.2GB). I've tried various browsers and also wget on both Linux and Mac. It typically fails with a network error in the browser. wget displayed strange behavior in that it claimed the download was successful after downloading 4.1GB. This was on 64 bit linux with ext4, so I don't think there was a filesystem limitation.
Any ideas on how to download this data? Does it make sense to write a script to use the API to request the datasets one by one to split it up?
Thanks for any help