microsoft / usbuildingfootprints

Computer generated building footprints for the United States

Introduction

Microsoft Maps is releasing country wide open building footprints datasets in United States. This dataset contains 129,591,852 computer generated building footprints derived using our computer vision algorithms on satellite imagery. This data is freely available for download and use.

License

This data is licensed by Microsoft under the Open Data Commons Open Database License (ODbL).

Data Vintage

The vintage of the footprints depends on the vintage of the underlying imagery. Bing Imagery is a composite of multiple sources with different capture dates. Each building footprint has a capture date tag associated if we were able to deduce the vintage of imagery source.

Footprints inside the highlighted region on the map are from 2019-2020. There are 73,250,745 such building footprints. This is the focal area where we rerun extraction for the latest release.

The rest of the footprints were extracted from older images, having wider range of capture dates, averaging 2012 year approximately. We have reused footprints from previous releases in this area.

FAQ

What the data include?

129,591,852 building footprint polygon geometries divided by 50 US states and the District of Columbia in GeoJSON format.

Why is the data being released?

Microsoft has a continued interest in supporting a thriving OpenStreetMap ecosystem.

What is the GeoJson format?

GeoJSON is a format for encoding a variety of geographic data structures. For Intensive Documentation and Tutorials, Refer to GeoJson Blog.

Should we import the data into OpenStreetMap?

Maybe. Never overwrite the hard work of other contributors or blindly import data into OSM without first checking the local quality. While our metrics show that this data meets or exceeds the quality of hand-drawn building footprints, the data does vary in quality from place to place, between rural and urban, mountains and plains, and so on. Inspect quality locally and discuss an import plan with the community. Always follow the OSM import community guidelines.

Will the data be used or made available in larger OpenStreetMap ecosystem?

Yes. Currently Microsoft Open Buildings dataset is used in ml-enabler for task creation. You can try it out at AI assisted Tasking Manager. The data will also be made available in Facebook RapiD.

What is the creation process for this data?

The building extraction is done in two stages:

1. Semantic Segmentation – Recognizing building pixels on the aerial image using DNNs
2. Polygonization – Converting building pixel blobs into polygons

Stage1: Semantic Segmentation

DNN architecture and training

The network backbone we used is EfficientNet described here. Although we have millions of labels at our disposal, we found that an effective combination of supervised and unsupervised training yields the best results.

Stage 2: Polygonization

Method description

We developed a method that approximates the prediction pixels into polygons making decisions based on the whole prediction feature space. This is very different from standard approaches, e.g. the Douglas-Peucker algorithm, which are greedy in nature. The method tries to impose some of a priori building properties, which is, at the moment, manually defined and automatically tuned. Some of these a priori properties are:

How good is the data?

Our metrics show that in the vast majority of cases the quality is at least as good as data hand digitized buildings in OpenStreetMap.

DNN model metrics

These are the intermediate stage metrics we use to track DNN model improvements and they are pixel based. Pixel recall/precision = 95.5%/94.0%

Polygon evaluation metrics

Match metrics:

Metric	Value
Precision	98.5%
Recall	92.4%

We evaluate following metrics to measure the quality of the output:

1. Intersection over Union – This is the standard metric measuring the overlap quality against the labels
2. Shape distance – With this metric we measure the polygon outline similarity
3. Dominant angle rotation error – This measures the polygon rotation deviation

On our evaluation set contains ~15k building. The metrics on the set are:

IoU	Shape distance	Rotation error [deg]
0.86	0.4	2.5

False positive ratio in the corpus

We estimate <1% false positive ratio in 1000 randomly sampled buildings from the entire output corpus.

What is the coordinate reference system?

EPSG: 4326

Will there be more data coming for other geographies?

They are already available.

External References

The building data are featured in NYTimes article.

A Vector Tile implementation of the data is hosted by Esri.

Download links

State or district	Number of Buildings	Unzipped size
Alabama	2,455,168	672.58 MiB
Alaska	111,042	30.00 MiB
Arizona	2,738,732	806.59 MiB
Arkansas	1,571,198	425.40 MiB
California	11,542,912	3.35 GiB
Colorado	2,185,953	619.88 MiB
Connecticut	1,215,624	324.20 MiB
Delaware	357,534	94.00 MiB
District of Columbia	77,851	22.52 MiB
Florida	7,263,195	2.01 GiB
Georgia	3,981,792	1.04 GiB
Hawaii	252,908	64.72 MiB
Idaho	942,132	259.43 MiB
Illinois	5,194,010	1.35 GiB
Indiana	3,379,648	920.20 MiB
Iowa	2,074,904	517.95 MiB
Kansas	1,614,406	428.38 MiB
Kentucky	2,447,682	663.98 MiB
Louisiana	2,173,567	600.69 MiB
Maine	758,999	187.84 MiB
Maryland	1,657,199	410.84 MiB
Massachusetts	2,114,602	566.87 MiB
Michigan	4,982,783	1.24 GiB
Minnesota	2,914,016	762.08 MiB
Mississippi	1,507,496	394.08 MiB
Missouri	3,190,076	840.28 MiB
Montana	773,199	200.45 MiB
Nebraska	1,187,234	302.72 MiB
Nevada	1,006,278	296.10 MiB
New Hampshire	577,936	146.40 MiB
New Jersey	2,550,308	681.55 MiB
New Mexico	1,037,096	291.54 MiB
New York	4,972,497	1.25 GiB
North Carolina	4,678,064	1.22 GiB
North Dakota	568,213	143.54 MiB
Ohio	5,544,032	1.42 GiB
Oklahoma	2,159,894	582.14 MiB
Oregon	1,873,786	545.94 MiB
Pennsylvania	4,965,213	1.23 GiB
Rhode Island	392,581	105.21 MiB
South Carolina	2,299,671	612.67 MiB
South Dakota	661,311	166.31 MiB
Tennessee	3,212,306	890.22 MiB
Texas	10,678,921	2.83 GiB
Utah	1,081,586	306.98 MiB
Vermont	351,266	87.92 MiB
Virginia	3,079,351	797.04 MiB
Washington	3,128,258	884.38 MiB
West Virginia	1,055,625	260.33 MiB
Wisconsin	3,173,347	817.06 MiB
Wyoming	386,518	99.32 MiB

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Legal Notices

Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.

Privacy information can be found at https://privacy.microsoft.com/en-us/

Microsoft and any contributors reserve all others rights, whether under their respective copyrights, patents, or trademarks, whether by implication, estoppel or otherwise.

usbuildingfootprints's Issues

It would be of great value to current disaster recovery efforts to add Puerto Rico and USVI.

A few topology errors detected

I used ogr2ogr to load buildings from southern Louisiana into SQL Server. That process detected a small number of topology errors. Here is a lightly edited version of the command and error messages:

ogr2ogr.exe -f "MSSQLSpatial" "MSSQL:server=mssql2016;database=Test;trusted_connection=yes;" "Louisiana.geojson" -t_srs "EPSG:4326" -a_srs "EPSG:4326" -lco "GEOM_TYPE=geography" -lco "GEOM_NAME=Geometry" -progress -clipsrc -93.911601 28.909162 -88.950615 30.485868

ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point -93.360444081030153 30.248059998115576 at -93.360444081030153 30.248059998115576 ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point -90.071937444444444 29.934727555555554 at -90.071937444444444 29.934727555555554 Warning 1: Ring Self-intersection at or near point -90.043338000000006 29.857880999999999 ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point -90.014427003208837 29.878139898386664 at -90.014427003208837 29.878139898386664 ERROR 1: TopologyException: Input geom 0 is invalid: Ring Self-intersection at or near point -93.468692000000004 30.358803999999999 at -93.468692000000004 30.358803999999999 ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point -92.034133058082276 30.273381288798021 at -92.034133058082276 30.273381288798021 ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point -90.104453000000007 29.898416000000001 at -90.104453000000007 29.898416000000001 ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point -91.989179968309486 30.283958535569884 at -91.989179968309486 30.283958535569884 ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point -90.33083200171086 29.532288062446536 at -90.33083200171086 29.532288062446536 ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point -90.768384909631834 30.408825994049835 at -90.768384909631834 30.408825994049835

The process was otherwise successful producing a table with 1145195 rows. Thanks for making these data available.

Height Data?

A quick extract for my county (Nevada County, CA) shows that there's no height data. Is there height data for other locations, or is it not present for anything?

Really a GeoJSON?

When converting the json with tippecanoe I'm getting an error for every line.

Code line:

tippecanoe -o Wyoming.mbtiles -zg --drop-densest-as-needed Wyoming.json

Error:

Wyoming.json:376881: feature without properties hash
In JSON object {"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[-104.4228260023874,41.151506633415309],[-104.4229350980483,41.151537892349722],[-104.4229579082815,41.151492748294707],[-104.4230375721159,41.151515570002118],[-104.4230714466938,41.151448528312251],[-104.4228826937695,41.151394455225308],[-104.4228260023874,41.151506633415309]]]}}

[information request] time period of source imagery

Hi All,
Thanks to the team for developing such a useful set of data. I'm interested in using building footprints for Santa Clara County, CA and need a rough year estimate of the underlying source Bing imagery used to generate the footprints. Does anyone have suggestions on how I can track down the source imagery to generate an estimate of the time period these footprints represent?
Any tips would be greatly appreciated.
Thanks!
Jenny

Boundary Effects

The team that produces these footprints should be congratulated for their efforts. A truely amazing piece of work.

There are, however, some boundary effects where there are missing footprints. These seems to tiles that have not gone through the analysis. You can see it in the Bay Area of San Francisco, and, for example, around Philadelphia in the following screenshots:

Is there a reason they are missing, and will they be added in the future?

Thanks in advace,
Simon.

Polygonization algorithm?

I would like to know if you are going to release the Polygonization algorithm? Really looking forward that.
Or could you give some brief description on how it is implemented?

typo in README.md

a priory should be a priori

Error to load California data

I am not able to open California data, is anyone else having this problem?

Required member "properties" missing from feature objects

Tremendous resource. Thank you for this ...

I just downloaded the DC example and, in looking at the GeoJSON, all feature objects are missing the member "properties" as a requirement (at least according to the GeoJSON spec 3.2).

A Feature object has a member with the name "properties". The value of the properties member is an object (any JSON object or a JSON null value)

I know at least http://geojson.io is throwing an error with the GeoJSON files:

Invalid JSON file: TypeError: Cannot convert undefined or null to object

Any possibility of adding "properties":{} to each feature object?

Trained DNN ResNet34 and/or labeled training data for academic use

Is it possible to gain access to the trained DNN ResNet34 model and or labeled training data used to detect building footprints for academic use. Is this something that I could ask for in a Microsoft AI for Earth training grant?

Boundary effects redux

Just resurfacing #13 - it looks like this is still happening in several metro areas. So far I've seen the tile edge artifacts in Denver, San Francisco, Los Angeles, Houston, and New York.

I pasted some snapshots on the previous issue.

Data Currency?

Less a issue, but more a question. Will we seen a continuous stream of refreshes from your group as imagery/data continues to expand and evolve?

Documentation: date of imagery

Hi, Awesome work!
Could you provide the data of imagery and / or capturing in the documentation?

Large gap in data around Chicago

I see a large gap in buildings around the City of Chicago, Illinois. Does anyone else see this?

Random shapes in the Upper Peninsula (MI)

There seems to be spurious content in the Michigan file.

What's odd is I can't say where they are actually from! It doesn't appear to be extra bits of Canada stuck in.

[Information Request] - Puerto Rico

Where can I find the building footprint for Puerto Rico?

Produce one giant GeoJSON

May you please also publish one giant GeoJSON (or any other format) for the whole US? Doing the merging on your own hardware is nearly impossible with a standard PC.

I have 32 GB of memory and was still unable to merge the geojsons with geojson-merge despite increasing the node heap limit to 30 GB.

Dates of imagery used

One thing that would be helpful in the ReadMe would be to tell us what the dates of imagery acquisition (when the plane flew) are. Are these datasets 2017 US buildings or 2018 US buildings? Or earlier?

geoJSON state datasets too large

Hello,

I see that the datasets are available on the state level – however, these datasets for each state are very large and I did not have any luck with trying to convert the geoJSON files to shapefiles (the process took way longer than usual and overheated my computer).

Besides suggesting a better computer to handle heavier processing - is it possible get any building footprint datasets for cities, so that the files are smaller and easier on my computer for converting?

I would prefer that the data are all from one source (for consistency), as opposed to going to multiple sources for the data. My team is searching for building footprints for the following cities:

• Minneapolis, MN
• Atlanta, GA
• Raleigh, NC
• St Paul, MN
• Charlotte, NC
• Winston-Salem, NC
• Chicago, IL
• Dallas, TX

Any other suggestions/solutions would be very much appreciated!

Thanks,
Elija

Douglas.... PEUCKER

C'mon folks.

https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm

Coordinate Reference System

What is the coordinate reference system these data are expressed in? I couldn't find this in the documentation. EPSG 4326? EPSG 4269?

Round buildings become angular

Are you aware of round buildings like storage tanks?
They become very ugly:

Missing buildings in some versions

When looking closely at Salt Lake City for example, there are whole strips of missing buildings. It seems that in the NYT article their version doesn't have these missing strips and in fact those exact areas seems to load first when panning around as if they are a separate layer. Any idea has to how I can get these missing strips?

The horizontal black in the attached image is an example of a strip of missing buildings and a second image with NAIP imagery.

some polygons invalid

after importing the California json set into PostGIS, testing for ST_Valid(geom) returns 193 invalids out of 10556550 rows.

msft_bldgs_invalids.tsv.zip

POSTGIS="2.4.3" PGSQL="100" GEOS="3.6.2-CAPI-1.10.2 4d2925d6" PROJ="Rel. 4.9.3, 15 August 2016" GDAL="GDAL 2.3.1, released 2018/06/22" LIBXML="2.9.3" LIBJSON="0.11.99" LIBPROTOBUF="1.2.1"

ogr2ogr -dim 2 -lco GEOMETRY_NAME=geom -lco SCHEMA=public -nln msft_ca_jun18 -nlt POLYGON -lco FID=gid -f PostgreSQL PG:dbname=msft_bldgs California.json -t_srs EPSG:4326

msft_bldgs-# COPY (select gid, st_asgeojson(geom) as geom from msft_ca_jun18 where not st_isvalid(geom)) to '/tmp/msft_bldgs_invalids.tsv' ;

timeline for open source of the segmentation models and polygonization algorithms?

We are working through the internal process to open source the segmentation models and polygonization algorithms.

Any idea (rough timeline) of when the model and algorithms will be open sourced?

Citation for publication

Hello,

I'm preparing to send a manuscript off for publication and I have included this layer as a variable in my analysis. I could not find metadata with a preferred citation for this layer. Do you have a preferred citation?

Thank you for your help,
Dan

Alaska looks totally wrong

I have take a look at the data in Alaska and could not find any really building (on esri-imagery, bing is most time only great pixels).
Most of the "buildings" are in nowhere [1]. Buildings with real positions have wrong geometry [2]

Please review.

[1]
or:

[2]

Building Height Calculation

It is amazing project. It'll be great if we can add building height to the polygon metadata. There is an existing script which detects building height based on the shadow of the building

It'll make this dataset rich for all sorts of analyses.

[information request] File encoding not supported by node.js?

I've attempted to parse the files with Node.js using all available encodings. What is the actual encoding of the file so that I can possibly manipulate it prior to parsing.

Too large files

Would it be possible to also create smaller .json files for smaller parts of the state or even cities? Because large files like CA, FL or OH are very difficult to read without special tools (that I'm not aware of).

Suggested Citation

I would like to cite this data source in a journal article. Do you have a recommended citation?

missing blobs for TN, TX

seems like the link is broken for these two states after I run wget I get this:

wget https://usbuildingdata.blob.core.windows.net/usbuildings/Texas.zip --2018-06-28 14:12:01-- https://usbuildingdata.blob.core.windows.net/usbuildings/Texas.zip Resolving usbuildingdata.blob.core.windows.net (usbuildingdata.blob.core.windows.net)... 13.93.168.80 Connecting to usbuildingdata.blob.core.windows.net (usbuildingdata.blob.core.windows.net)|13.93.168.80|:443... connected. HTTP request sent, awaiting response... 404 The specified blob does not exist. 2018-06-28 14:12:01 ERROR 404: The specified blob does not exist..

Off nadir Correction

Thanks for releasing such awesome data.

I wanted to know if you are doing any correction for predictions if some of the images are off-nadir. Or the model itself predicts results with the correction?

trimming and conversion to .osm

I wrote a little program to convert a bounding box's worth of data into a .osm file.
https://gist.github.com/RussNelson/a2da4104eb1b83b3a66fc6bc32a82ec4

Typo in one of the image

Please note that you have a typo in the image here:
https://github.com/Microsoft/USBuildingFootprints/raw/master/images/segmentation.PNG
We can read sematic rather than semantic ;-)

Extensive false positives in rocky desert areas

I'm seeing a large number of false positives, numbering at least 5,000 polygons, in the rocky desert areas of southern and eastern Utah. It appears that large rocks in certain types of geologic formations are being extracted as buildings. I have attached screenshots from the Navajo Nation area in southeastern Utah. All of these images are north of the Utah-Arizona and south of the San Juan River.

In area shown in the image below, building footprints are represented with a yellow outline and a transparent fill. This image shows approximately 1,500 individual footprints:

The area shown is very sparsely populated, with likely no more than a hundred actual buildings. When examined in detail, the errors are apparent:

Large rectangular rock features are being extracted as buildings footprints. Also, areas with no discernible features are being extracted as buildings. For example, this is the area near the Monument Valley Airport has no rocks, but large irregular footprints are present:

Errors like these are present over hundreds of square miles in the rocky desert and mountainous areas of the eastern half of the state. The total number is easily more than 5,000 false footprint polygons. I imagine it's similar in other desert states.

Classification Issues in forested areas

Thank you for publishing this data. It is really wonderful.

You might want to consider more training for forested areas. I'm seeing a high false positive rate in stands of ponderosa pine on flat terrain. Here are some examples:
https://binged.it/2zordYk
https://binged.it/2ufYOOa

Suggestions for converting to format usable in ArcGIS?

Hi, and thanks for posting all this data! It should be very useful as described, but I'm having trouble opening/converting the JSON file (I'm trying to use the Georgia.json file). I've tried opening it in QGIS, but I think the file is so large that it crashes the program. In ArcMap, I've tried the JSON to Features tool, but I get "unexpected error" - not sure if this is due to file size or what. I also tried the ArcGIS Data Interoperability Quick Import tool, but it doesn't list JSON as an input file type.

It would be great if this data were broken up into smaller pieces, instead of an entire state in one big chunk...but since that's not currently available, any suggestions on how to use this huge file would be appreciated!

training dataset information

Hello,

Could you please confirm if you use the labelled training data of same geographic location (US) to train your FCN network ? Second, as you mentioned that the training data was having resolution of 1ft/pixel, do you have to use same resolution images for predictions also or this trained work is able to generate building footprints on any resolution image ?

Thanks,
Amardeep

Strange data omissions

What could explain the missing building data in Denver, as shown in the photo (missing buildings around inside of the square)? It must be an artifact of something? Anything I can do to fix that? Happy to try to use your models.

[not an issue, an information request] Entrance coordinates and splitting of combined buildings

Hi to all,

Thanks to everyone in this project for publishing and sharing such useful data.

I have two simple questions about this project.

1-) Do we have a chance to get the front-door (entrance) coordinates of the buildings in this data set? I particularly need the coordinates of the connection points between the roads (streets) and the buildings.

2-) Do this data set provide (split) the coordinates of the sub-properties in the combined buildings (including multiple properties inside themselves) individually (independently)?

Thanks in advance for any kind of responses.

Regards,
Mustafa

Will height information be available in the future?

Great to see so many more buildings have been added in the 2nd release!!

It seems that in this version, building height data is not available. Is there any plan to add building height data in the future?

Thanks.

California issues

I've been processing the California data and I'm seeing a lot of large "buildings" that really aren't buildings. Many of them are in the desert, south of the Salton Sea. Example:

Others are in farm land:

(I realize that sometimes farms acreage is covered in tents, which might appear to be a building. But in this case, it's just a field.)

Of the 100 largest buildings in California, it looks like about 50 of them aren't really buildings. I can provide a list if that will help.

a priori

https://en.wikipedia.org/wiki/A_priori_and_a_posteriori

actual count of building footprints

The readme says:

This dataset contains 124,885,597 computer generated building footprints in all 50 US states.

However, when I load all the files, I find 122,608,100 total, which exactly matches the sum of the state-level counts in the table toward the bottom of the readme. So it appears the dataset actually contains 122,608,100 footprints, correct?

Release the labeled training data?

Super awesome to see so much data released.

What would be really cool is to release the labeled training data - the verified buildings, along with the imagery that was used to create it. Then others could make their own computer vision models.

Related to #9 - the data would be much more useful if it could be a commons for others to do similar things. But definitely appreciated what you've done so far!

enhancement: apply rounding to computer generated decimal precision

unbelievably cool announcement!

if there's anywhere in your pipeline that you could rough up the subatomic coordinate precision, consumers everywhere would be much obliged.

{
  "type": "Feature",
  "geometry": {
    "type": "Polygon",
    "coordinates": [[
      [-162.29951646880551, 67.065383815817825],
      [-162.29952962664339, 67.065303662945269],
      [-162.29931058186409, 67.065298202629734],
      [-162.29929742402621, 67.065378355520338],
      [-162.29951646880551, 67.065383815817825]
    ]]
  }
} 

file this one under: 😉

Data Currency?

Just curious if you see your project doing a refresh of this data. This is a few years old now and where some folks would have a interest in running this on a regular/annual basis the horsepower isn't cheap.

detial or reference of the polygonization method

Is there anyone know the detial or reference of the polygonization method？
@jharpster @nitrif Is that possible tell us some details?