American Red Cross projects


#1

Information about drone imagery sets collected by the American Red Cross is available here.

Image sets are made available for download with the intent that they can be used for purposes such as test datasets for development of tools like OpenDroneMap. We have (or will soon) uploaded processed scenes to OpenAerialMap; please do not upload orthomosiacs derived from our image sets to OpenAerialMap.

For questions or feedback please contact me.

All imagery is CC-BY 4.0 American Red Cross.


#2

This is absolutely awesome. Thanks for sharing these into the ODM ecosystem. They will be invaluable for software improvement and testing.


#3

The data I downloaded from Amazon S3 are always corrupted. How can I solve it? Is it a problem of my Internet connection?


#4

I’ve been able to download a couple datasets using the links with no problem. Is there a certain one that you can’t get or all of them end up corrupted?

You might try using wget to download the file.
There is a --continue parameter you can use if a file download doesn’t complete (although I’m not sure if it works with AWS S3 storage). Documentation on wget download parameters.


#5

Thank you for your response. 20170504-union.zip this one, I have downloaded it for 3 times, all of them are broken. I found that if you try to download a zip file larger that 5GB, it is always broken. I had tried the wget way, the file was corrupted too.


#6

@15_boood
I have downloaded the zip file using AWS Command Line Interface. Absolutely fine and able to extract all the images. Would you please provide more information on how you download and what version of the tools you are using to download the zip file (if any).


#7

You can find the AWS Command Line Interface install instructions at:
https://docs.aws.amazon.com/cli/latest/userguide/installing.html
and you’ll want to run a command something like:
aws s3 cp s3://arcimagery/drone_amcross/20170517-union.zip

You’ll need to reconfigure the URLs from the spreadsheet but all the pieces are there:
https://arcimagery.s3.amazonaws.com/drone_amcross/20170504-union.zip
becomes
s3://arcimagery/drone_amcross/20170517-union.zip


#8

I’m afraid I need the bucket access key first then I can use the AWS Command Line Interface to download these files.


#9

It’s a public file, and @webDev was able to do it without access credentials. It shouldn’t be required.


#10

Yes correct sir @danbjoseph.

@15_boood you dont need any credentials except your own credentials to setup/configure the AWS Command Line interface. And The above documentation/link of AWS will walk through that process. You just need to install the AWS Command Interface and configure that AWS Command Interface using your own AWS user credentials and those credentials you can download as a CSV file that includes your access key and secret or simply copy and paste.
Follow the link posted by @danbjoseph.
I will give a try with wget and post back.


#11

I had tried the s3 command line tool and it works like a charm. Thank you!!


pinned #12

#13

I’ve finally uploaded all our imagery from Canaan, Haiti. Images are zipped by collection day, which unfortunately does not always coincide with a contiguous area. But if you download everything you’ll have 28,231 images (~120GB) covering, I think, approximately 35 sq km.

Missing Maps blog post about the activities.

Note: The sensors were those on a Mavic Pro :neutral_face:. The overlap varies due to a lot of changes in elevation (however, mission launch points were generally at high points within an area, so overlap should generally be greater than needed not less). One of the zips is images collected by a Sony Nex5 carried by a Tuffwing and doesn’t cover any area not already covered by the Mavics imagery.

canaan-georef-0-pts


#14

This is awesome! I was just looking for larger datasets, having just processed the largest one from spreadsheet.

Thanks for sharing @danbjoseph! This is going to be good for testing split-merge.


#15

Dear @danbjoseph, an amazind dataset, thank you so much. I have successfully obtained the smallest dataset (i.e. 20170507-ttl_office_100.zip) and will start running it through WebODM.

I could not find any info on GCPs, is there any GCP file / visible GCP in any of the datasets? Sorry if I throw this request at you without looking through all the data first but I really don’t want to download all of the data just to find out.

Thanks!!


#16

The images are geotagged so the output will be geo-referenced but there are no deployed ground control point sets


#17

Is it just me, or are the majority of these zips corrupt? Tried them on two different connections, a Windows 10 and a Fedora 28 laptop, with several different zip applications.

The only zip program that works with them is Fedora’s CLI unzip, but it REALLY isn’t happy about it and heaps of files aren’t extracted:
[…]
file #2163: bad zipfile offset (local header sig): 14579858273
file #2164: bad zipfile offset (local header sig): 14583581168
file #2165: bad zipfile offset (lseek): 14587281408
file #2166: bad zipfile offset (lseek): 14591115264
[…]
file #2854: bad zipfile offset (lseek): 17172471808
file #2855: bad zipfile offset (lseek): 17176240128
inflating: 20171219/20171219_ROC101_DJI_0110.JPG
inflating: 20171219/20171219_ROC101_DJI_0111.JPG


#18

Had some issues as well. 7zip on Windows ended up working well for me.


#19

@ITWarrior, another user was having trouble with corrupted zip files, and i think it was because their downloads weren’t completing properly. using the s3 command line tools for the download seemed to resolve their problem, see post 7 above.