Great Lakes bathymetry about joerd

[iandees]

List files

There's only 5 of them, so here they are:

cat > mgg_greatlakes_listing.txt << EOF
https://www.ngdc.noaa.gov/mgg/greatlakes/erie/data/geotiff/erie_lld.geotiff.tar.gz
https://www.ngdc.noaa.gov/mgg/greatlakes/huron/data/geotiff/huron_lld.geotiff.tar.gz
https://www.ngdc.noaa.gov/mgg/greatlakes/michigan/data/geotiff/michigan_lld.geotiff.tar.gz
https://www.ngdc.noaa.gov/mgg/greatlakes/ontario/data/geotiff/ontario_lld.geotiff.tar.gz
https://www.ngdc.noaa.gov/mgg/greatlakes/superior/data/geotiff/superior_lld.geotiff.tar.gz
EOF

Mirror files

cat mgg_greatlakes_listing.txt \
    | xargs \
      -I {} \
      -P 24 \
      -n 1 \
      sh -c 'export f="{}"; export s3="s3://elevation-sources-prod/mgg_greatlakes/`basename $f`"; curl -s $f | AWS_DEFAULT_PROFILE=openterrain aws s3 cp - $s3; echo $s3' \
    | tee mgg_greatlakes_s3_objects.txt

Transcode files

cat mgg_greatlakes_s3_objects.txt | \
while read filename; do
    bn=$(basename $filename)
   make submit-job job=aws/transcode-job.json.hbs input=${filename} output=s3://elevation-sources-transcoded/mgg_greatlakes/$(sed 's/\.geotiff\.tar\.gz//' <<< $bn) name=$(sed 's/\.geotiff\.tar\.gz//' <<< $bn)
done

[iandees]

These jobs failed because the transcoder didn't properly handle the .tar.gz extension.

[iandees]

We finally got these in, but when the tiler tries to composite with them, it fails for some reason and 500s. I'll take a closer look later.

[iandees]

I just re-transcoded these and they don't break the tiler anymore, but they also aren't showing up over the top of ned13, which doesn't seem to have holes cut out for the Great Lakes.

[nvkelso]

change their priority to -1?

…

On Thu, Jul 13, 2017 at 2:00 PM, Ian Dees ***@***.***> wrote: I just re-transcoded these and they don't break the tiler anymore, but they also aren't showing up over the top of ned13, which doesn't seem to have holes cut out for the Great Lakes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#23 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA0EOyNqS3Cnj_VEnMQLP0bAvmaxuDX4ks5sNoVjgaJpZM4HWqm8> .

[iandees]

That helps. It looks like vertical measurements are unexpected though, because there's a hillshade border around the data:

(southern end of Lake Michigan)

[nvkelso]

Does it need a different clip? Where could we do that?

[iandees]

I'm going to disable this for now. I haven't figured out a way to get rid of the stripe and the resolution isn't all that great anyway.

[iandees]

footprints=> update footprints set enabled=false where source='mgg_greatlakes';
UPDATE 8

[nvkelso]

Re-opening for a rainy day.

[zerebubuth]

Info for rainy day. These DEMs each have individual vertical offsets, I think maybe their zero height datum is the mean surface level of each lake? The offset data is available from NOAA or pre-digested in Joerd v1.

[iandees]

Thanks!