retroplasma / earth-reverse-engineering Goto Github PK

Awesome work!

Could you provide .proto files that decode bulkMetadata and nodeData?

The coordinates have their physical meanings. The origin is earth center and the length unit is meter.

Maybe it's better to keep the original coordinates, or add an option to let the user choose how to normalize the output mesh.

I wonder whether I can get a model of a place in octant level 17 or above without having mountains and buildings included? My model is full of hills and I want to use a model without terrain to know where the mean sea level is at, so that I can get the height of the hills above sea level.

This might be more of a Blender related question, but maybe it relates to the way the .obj file is put together.

My problem is that the verts do not line up from one octant to the next. This is particularly visible because I am working with outlines and freestyle, and the grid is popping up quite noticeably:

I've been trying merging by distance or even remeshing, but both alter the original shapes way too much. Any ideas on how I could mitigate this problem?

I have ran the python command and same thing happens with non textured meshes and in 3DS max, no clue how to fix it as I want to use this as a base to model from.
https://gfycat.com/AgonizingMeatyLeveret

It looks as if sometimes the textures get mixed up.

node index.js 21 5

Output:
https://skfb.ly/6Gqow

What other info would be helpfull?

Hi,

All models are slightly rotated. Even model that model from readme have incorrect rotation. I don't know is this a bug or I'm doing something wrong.

It looks like that the mesh is on an ECEF coordinate system, but on a pure sphere not an ellipsoid in usual GIS systems.

Originally posted by @LexSong in #4 (comment)

Greetings LexSong

Following your insight into the use of ECEF projection I tried firing-up BlueMarble Geographic Calculator and started experimenting. My workflow is to:

1. Copy out the object file's "v: X Y Z" coordinates into a plain CSV file using Notepad++.
2. In BlueMarble set source projection as "Geocentric WGS 84"
3. Set output projection as "Geodetic WGS 84"
4. Import into ArcGIS and check the positioning.

Tried on two areas in the UK and the Bahamas.
So far area extent and Longitude positions of the results looks okay
HOWEVER Latitude in both test cases is about 20 km off in a Northerly direction.

So...really great if you could please explain a little bit more what you mean when you say the exact ECEF projection used is "a pure sphere not an ellipsoid in usual GIS systems."?
And any suggestion on

how to handle that??

Thanks!

Ian

Example showing projection error for the Bahama test:

I tried to decode the mesh using the information from Readme.md, as well as dump_obj.js, but nothing worked. It seems to me that during this time, the layout of the mesh vertices has changed.
I've tried node https://kh.google.com/rt/earth/NodeData/pb=!1m2!1s214272516372616353!2u852!2e1!4b0
There are 3 mesh in this file. Each of them contains 18744, 3678 and 1809 bytes of vertices. Second and third not even a multiple of 8.

I can download the a small area in 3d with the program, thank you for that. But I want to download a city, so I need to manually merge each piece of 3d area in blender. Is there a method to download large area? For example, typing left top corner coordinates, left bottom corner coordinates, right top corner coordinates and right bottom corner coordinates to form a large rectangle area to download.

Not sure if this is the right place for it, but I've spent hours trying to figure this one out and have gotten nowhere. Here's my issue, importing the generated obj and rendering with either Eevee or Blender Internal renderer works fine. However, when I try to render with Cycles, I get one of two results:

1. No textures (this I can fix, by reassigning materials/textures/uv maps)
2. What appears to be randomly garbled textures.

Has anyone gotten this to render nicely with Cycles?

E.g.:

Sorry it's so dark.

Hi!

Holy shit, I got so happy when I saw this, I tried to reverse google earth during a week or two this summer... However I did not succeed further than playing around with the textures a lil bit, and also learning how to use a debugger properly :P

However... I got something you might be interested in! @retroplasma

syntax = "proto2";

package geo_globetrotter_proto_rocktree;

message BulkMetadataRequest {
    optional NodeKey node_key = 1;
}

message NodeDataRequest {
    optional NodeKey node_key = 1;
    optional Texture.Format texture_format = 2;
    optional uint32 imagery_epoch = 3;
}

message NodeKey {
    optional string path = 1;
    optional uint32 epoch = 2;
}

message CopyrightRequest {
    optional uint32 epoch = 1;
}

message TextureDataRequest {
    optional NodeKey node_key = 1;
    optional Texture.Format texture_format = 2;
    optional Texture.ViewDirection view_direction = 3;
}

message BulkMetadata {
    repeated NodeMetadata node_metadata = 1;
    optional NodeKey head_node_key = 2;
    repeated double head_node_center = 3 [packed = true];
    repeated float meters_per_texel = 4 [packed = true];
    optional uint32 default_imagery_epoch = 5;
    optional uint32 default_available_texture_formats = 6;
    optional uint32 default_available_view_dependent_textures = 7;
    optional uint32 default_available_view_dependent_texture_formats = 8;
}

message NodeMetadata {
    optional uint32 path_and_flags = 1;
    optional uint32 epoch = 2;
    optional uint32 bulk_metadata_epoch = 5;
    optional bytes oriented_bounding_box = 3;
    optional float meters_per_texel = 4;
    repeated double processing_oriented_bounding_box = 6 [packed = true];
    optional uint32 imagery_epoch = 7;
    optional uint32 available_texture_formats = 8;
    optional uint32 available_view_dependent_textures = 9;
    optional uint32 available_view_dependent_texture_formats = 10;
    
    enum Flags {
        RICH3D_LEAF = 1;
        RICH3D_NODATA = 2;
        LEAF = 4;
        NODATA = 8;
        USE_IMAGERY_EPOCH = 16;
    }
}

message NodeData {
    repeated double matrix_globe_from_mesh = 1 [packed = true];
    repeated Mesh meshes = 2;
    repeated uint32 copyright_ids = 3;
    optional NodeKey node_key = 4;
    repeated double kml_bounding_box = 5 [packed = true];
    optional Mesh water_mesh = 6;
    repeated Mesh overlay_surface_meshes = 7;
}

message Mesh {
    optional bytes vertices = 1;
    optional bytes vertex_alphas = 9;
    optional bytes texture_coords = 2;
    optional bytes indices = 3;
    optional bytes octant_ranges = 4;
    optional bytes layer_counts = 5;
    repeated Texture texture = 6;
    optional bytes texture_coordinates = 7;
    repeated float uv_offset_and_scale = 10 [packed = true];
    optional bytes layer_and_octant_counts = 8;
    optional bytes normals = 11;
    optional bytes normals_dev = 16;
    optional uint32 mesh_id = 12;
    optional bytes skirt_flags = 13;
    
    enum Layer {
        OVERGROUND = 0;
        TERRAIN_BELOW_WATER = 1;
        TERRAIN_ABOVE_WATER = 2;
        TERRAIN_HIDDEN = 3;
        WATER = 4;
        WATER_SKIRTS = 5;
        WATER_SKIRTS_INVERTED = 6;
        OVERLAY_SURFACE = 7;
        OVERLAY_SURFACE_SKIRTS = 8;
        NUM_LAYERS = 9;
    }
    
    enum LayerMask {
        TERRAIN_WITH_OVERGROUND = 7;
        TERRAIN_WITH_WATER = 28;
        TERRAIN_WITHOUT_WATER = 14;
    }
}

message Texture {
    repeated bytes data = 1;
    
    optional Format format = 2;
    enum Format {
        JPG = 1;
        DXT1 = 2;
        ETC1 = 3;
        PVRTC2 = 4;
        PVRTC4 = 5;
        CRN_DXT1 = 6;
    }
    
    optional uint32 width = 3 [default = 256];
    optional uint32 height = 4 [default = 256];
    
    optional ViewDirection view_direction = 5;
    enum ViewDirection {
        NADIR = 0;
        NORTH_45 = 1;
        EAST_45 = 2;
        SOUTH_45 = 3;
        WEST_45 = 4;
    }
    
    optional uint32 mesh_id = 6;
}

message TextureData {
    optional NodeKey node_key = 1;
    repeated Texture textures = 2;
}

message Copyrights {
    repeated Copyright copyrights = 1;
}

message Copyright {
    optional uint32 id = 1;
    optional string text = 2;
    optional string text_clean = 3;
}

message PlanetoidMetadata {
    optional NodeMetadata root_node_metadata = 1;
    optional float radius = 2;
    optional float min_terrain_altitude = 3;
    optional float max_terrain_altitude = 4;
}

Hello! Sorry if this is the wrong place to ask, but I'm trying to implement this myself so I can get a better understanding of how the Google Earth protocol works and possibly implement it in other languages than C(++) and JS. When I import and use exporter/lib/decode-resource.js it works fine, but when I use protoc or protobufjs with a DXT texture the data is too short and can't be decoded, and some data seems to be missing. Do you have any idea why? Thank you

Hello, i've tried with node.js v8 and v16 on windows 10, please help me identify the cause of this problem. I've tried with 2 different internet connections (proxy and no proxy).

npm config set proxy http://{url}:{port}
npm config set https-proxy http://{url}:{port}

command run:
node lat_long_to_octant.js 37.420806884765625 -122.08419799804688

Gave up after 5 tries.
Error: connect EACCES 142.250.184.110:443
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1133:16) {
errno: -4092,
code: 'EACCES',
syscall: 'connect',
address: '142.250.184.110',
port: 443
}

Did you find out a (logical) way on how to get the correct octants of a given coordinate? the positioning seems rather random and it's nearly impossible to get an obj from an POI without trial & error or loading all (>30) octants of any given viewport on google maps into the nodejs script.

Greetings,

Many thanks for the absolutely awesome tool.
Just a quick suggestion for format of the lat_long_to_octant.js output:
Please include appropriate +/- symbol for Lat/Lon.
That might be particularly helpful at some stage for Qctants close to either the Equator Greenwich Meridian.

For example:

Octant Level: 17

{ n: 51.50115966796875,
s: 51.4984130859375,
w: 0.120849609375,
e: 0.1263427734375 }
30604240604272700 (Close to but west of the Greenwich Meridian.)

Should actually have negative coordinates for both west & east :

Octant Level: 17

{ n: 51.50115966796875,
s: 51.4984130859375,
w: -0.120849609375,
e: -0.1263427734375 }
30604240604272700

P.S. Even better a flat CSV output format such as: the following would allow even
easier GIS import:

Level_ID, Latitude, Longitude
LVL17_NW, 51.50115966796875,-0.120849609375
LVL17_NE, 51.50115966796875,-0.1263427734375
LVL17_SW, 51.4984130859375,-0.120849609375
LVL17_SE, 51.4984130859375,-0.1263427734375
LVL18_NW....
etc

Cheers,

Ian

Hi there:

i was downloading an area of 0.2*0.4 degree. In the process, i got error like this. And i can;t connect to Google Earth.

Retrying https://kh.google.com/rt/earth/PlanetoidMetadata in 1 second.
Retrying https://kh.google.com/rt/earth/PlanetoidMetadata in 2 seconds.
Retrying https://kh.google.com/rt/earth/PlanetoidMetadata in 4 seconds.
Retrying https://kh.google.com/rt/earth/PlanetoidMetadata in 8 seconds.
Gave up after 5 tries.
Error: HTTP status code 403 for https://kh.google.com/rt/earth/PlanetoidMetadata

Is it becasue my computer was forbidden due to my downloading large volume data?

Thanks

Greetings All,

The following is a useful link to a map showing places with 3D Coverage in GE

https://earth.app.goo.gl/VWZbtz

Hi
Thanks for your research on this great tool!
Though, I have a problem using it. For some reason, my model is coming up totally messed. I am importing OBJ into 3dsmax, after centering-scaling script, and it looks like this: https://dl.dropboxusercontent.com/s/xb8r3gh32b6m9ko/shot_190909_165318.png
Can you please let me know what the issue can be?
Thanks a lot!

Hi, can somebody explain how to get higher LOD for meshes and higher textures when using dump_obj.js if it possible.
Settings with max octant level 18 and dump obj level 20 provide for me those results:

(also tried to play with different values which gave me same data)
Left - blender, Right - GEarth

This repo inspire me to write small experimental app which allows to easy download large chunks of 3d imagery. It uses earth-reverse-engineering/exporter code library as local web service to get octants and download obj files.
Some work in progress screens:

I have lot of troubles doing the manipulation described in BLENDER.MD to make the obj with the correct textures (like setting shader view to rendered and running the python script) I know this is probably linked to Blender being overly complicated but maybe the steps could be explained in a better way?

Thanks

Instead of forcing people to use Node 8, you can simply replace two lines in decode-texture.js to solve the problem:

1. return { extension: 'jpg' , buffer: new Buffer(tex.bytes) }; => return { extension: 'jpg' , buffer: Buffer.from(tex.bytes) };

2. const buf = new Buffer(bytes); => const buf = Buffer.from(bytes);

Originally posted by @desmondCTNG in #3 (comment)

Hey all,

Amazing project @retroplasma, some great stuff. Wanted to know if anyone knows of any way to exclude non building data, specifically trees? This would be a major help to what I'm working on. I saw that the trees are blended with roof textures often, but wanted to see if someone with more exp in this project knows of a way. Thanks!

Hi, thanks for this work.
This is very useful.

If you happen to need to project back the 3D x,y,z coordinates to geographic (lat,lon) coordinates, be aware that Google seems to model the earth as a perfect sphere (rather than an ellipsoid).

Hence, you can get lat, lon from basic trigonometry (here in Python) :

lon = atan2(y, x) * 180 / pi
lat = atan2(z, sqrt(x*x + y*y)) * 180 / pi
height = sqrt(x * x + y * y + z * z)

Those coordinates are then compliant with WGS84 / EPSG:4326

Can someone create a script or what that can combine the textures downloaded into one or few texture atlas?

May I request for a script that helps me to resize the object created, so that 1 meter or the object is equal to 1 meter of the real world. It is true that Blender has a resizing function, but sometimes the result is slightly inaccurate.

It will also be great if you can provide tips on how to make that script.

What applications on Linux/Mac OSX can be used to view the files with textures? I tried FXB Review and Photoshop and neither show textures. Thanks

Hello,
when trying to center/rescale an obj composed of numerous octants (retrieved by LexSong script) we get this:

node center_scale_obj.js

<--- Last few GCs --->

[212756:0000011DE7FADED0]    61064 ms: Mark-sweep 1380.3 (1406.8) -> 1380.3 (1406.8) MB, 644.5 / 0.0 ms  (average mu = 0.105, current mu = 0.020) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 000003A4A7B041C1]
Security context: 0000027D3FE1E6C9 <JSObject>
    1: clearBuffer(aka clearBuffer) [000000C26EBA96D1] [_stream_writable.js:~495] [pc=000003A4A7B642E5](this=000003A4D67022E1 <undefined>,stream=00000198F5C831E1 <WriteStream map = 000003C60DBD71E1>,state=0000034FB0C03C09 <WritableState map = 000003C60DBD0071>)
    2: onwrite(aka onwrite) [000000C26EBA9611] [_stream_writable.js:465] [bytecode=0000021A344...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF642E982F5
 2: 00007FF642E74156
 3: 00007FF642E75890
 4: 00007FF64336AD5E
 5: 00007FF64336AC93
 6: 00007FF643249CB4
 7: 00007FF643240797
 8: 00007FF64323ED1C
 9: 00007FF64323E1C5
10: 00007FF6432479F6
11: 00007FF64314BB21
12: 00007FF64314C5E2
13: 00007FF64317834C
14: 00007FF643177FE4
15: 00007FF643178560
16: 00007FF64315F245
17: 00007FF6434D0FF0
18: 000003A4A7B041C1

Look like the computer either ran out of ram or there is some kind of limitation in the node javascript engine, is there any possible fixes for this you can think of ? like storing on hard disk ?
will try tomorrow with a 32 gb ram server, here i only have 16gb

Hello, latest master broke the cross-platform compatibility (it was working yesterday) here are the logs:

node.exe lat_long_to_octant.js 48.856086, 2.297932
48.856086, 2.297932
-------------
30 { n: 90, s: 0, w: 0, e: 90 }
306 { n: 90, s: 45, w: 0, e: 90 }
3060 { n: 67.5, s: 45, w: 0, e: 45 }
30604 { n: 56.25, s: 45, w: 0, e: 22.5 }
306040 { n: 50.625, s: 45, w: 0, e: 11.25 }
3060406 { n: 50.625, s: 47.8125, w: 0, e: 5.625 }
30604060 { n: 49.21875, s: 47.8125, w: 0, e: 2.8125 }
306040607 { n: 49.21875, s: 48.515625, w: 1.40625, e: 2.8125 }
3060406071 { n: 48.8671875, s: 48.515625, w: 2.109375, e: 2.8125 }
30604060716 { n: 48.8671875, s: 48.69140625, w: 2.109375, e: 2.4609375 }
306040607163 { n: 48.8671875, s: 48.779296875, w: 2.28515625, e: 2.4609375 }
3060406071636 { n: 48.8671875, s: 48.8232421875, w: 2.28515625, e: 2.373046875 }
30604060716362 { n: 48.8671875,
  s: 48.84521484375,
  w: 2.28515625,
  e: 2.3291015625 }
306040607163624 { n: 48.856201171875,
  s: 48.84521484375,
  w: 2.28515625,
  e: 2.30712890625 }
3060406071636243 { n: 48.856201171875,
  s: 48.8507080078125,
  w: 2.296142578125,
  e: 2.30712890625 }
30604060716362436 { n: 48.856201171875,
  s: 48.85345458984375,
  w: 2.296142578125,
  e: 2.3016357421875 }
306040607163624362 { n: 48.856201171875,
  s: 48.854827880859375,
  w: 2.296142578125,
  e: 2.29888916015625 }
3060406071636243627 { n: 48.856201171875,
  s: 48.85551452636719,
  w: 2.297515869140625,
  e: 2.29888916015625 }
30604060716362436276 { n: 48.856201171875,
  s: 48.855857849121094,
  w: 2.297515869140625,
  e: 2.2982025146484375 }
306040607163624366 { n: 48.856201171875,
  s: 48.854827880859375,
  w: 2.296142578125,
  e: 2.29888916015625 }
3060406071636243663 { n: 48.856201171875,
  s: 48.85551452636719,
  w: 2.297515869140625,
  e: 2.29888916015625 }

node dump_obj.js 3060406071636243663 20
TypeError: Cannot read property '0' of null
    at Fd.T.Ja (C:\Users\rakia\gitlab\tools\earth-reverse-engineering\lib\decode-resource.js:13:111839)
    at Fd.T.Fa (C:\Users\rakia\gitlab\tools\earth-reverse-engineering\lib\decode-resource.js:13:109954)
    at rf.a (C:\Users\rakia\gitlab\tools\earth-reverse-engineering\lib\decode-resource.js:13:114798)
    at Object.c.onmessage (C:\Users\rakia\gitlab\tools\earth-reverse-engineering\lib\decode-resource.js:13:114410)
    at C:\Users\rakia\gitlab\tools\earth-reverse-engineering\lib\decode-resource.js:29:9
    at new Promise (<anonymous>)
    at decode (C:\Users\rakia\gitlab\tools\earth-reverse-engineering\lib\decode-resource.js:27:16)
    at decode (C:\Users\rakia\gitlab\tools\earth-reverse-engineering\lib\utils.js:73:23)
    at process._tickCallback (internal/process/next_tick.js:68:7)
octants: 0

The idea is to stream earth data into game engines. As a background for deeper analysis or whatever useful.
I mind if anyone has tried to start the process.

Hey guys, as stated in another issue i've been working on a process to dump large areas that are actually usable in 3d engine or possible to stream over a sig server, but what lack my system is actually the altitude

I can display my tiles next to each other and this blend kind of nicely except when there is an altitude difference between the tiles

would you think it'd be possible to actually the actual elevation of a tile ?

May I ask the what coordinate system is this program using, is it WGS84? Where should I edit to get the lat long coordinates and height of every downloaded octant?

First off: wonderful project.

Second: Not sure if you'll include one. But if so, what license do you plan on using for this project?

I have used this a number of times with no problems but suddenly I am getting an "EEXIST" error when attempting to run dump_obj.js. I have confirmed the file is definitely not in the folder so I am unsure why I am getting this error.

I'm new with this. I download the model following the instructions, but when I import it on blender I can't show the model. Do I something wrong?

(node:56060) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.

Hello,

First off I will say that this is a great plugin - definitely the best I have found online. I was wondering if it is possible to download more than one octant at a time? Is this related to LexSong's script? If so I received this error when running it -

File "find_overlaps.py", line 10
URL_PREFIX = f"https://kh.google.com/rt/{PLANET}/"
^
SyntaxError: invalid syntax

If that is not the case - don't worry about the error. All that I'm trying to do is to download multiple octants at the same time. Thank you in advance!!!

-Lukas

I just update to the latest version and get this error:

> node index.js 205172436353455 17

found: 205172436353455
found: 2051724363534550
found: 20517243635345503
TypeError: Cannot read property 'length' of null
    at search (C:\Users\LexSong\earth-reverse-engineering\index.js:133:28)
    at process.internalTickCallback (internal/process/next_tick.js:77:7)

I tried to git bisect, and found that the commit 2197458 caused the error.

When using your example octants (or others) I will get the following error thrown:

arch linux

node --version
v10.10.0

[…]
found: 20527061605273514374
found: 20527061605273514375
found: 20527061605273514376
found: 20527061605273514377
octants: 149
(node:3681) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(node:3681) UnhandledPromiseRejectionWarning: Error: Input file is missing or of an unsupported image format
(node:3681) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:3681) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I can see the files beeing created into downloaded_files, they have a size but will not open correctly (tried to import with blender & open with g3dviewer)

Greetings,

Given that the data is geospatial in nature and 3d object files are not designed to handle or retain important geopatial information such as the projection an option to output at least some of the data to a more appropriate Geospatial format would be awesome,

I suggest XYZ vertex position data could be better held in LAS file.
https://en.m.wikipedia.org/wiki/LAS_file_format

Could somebody please write a dump_las.js script?
An even simpler dump_XYZ_csv.js script would also be great.

Thanks!

Ian

I add a python script to find overlapping octants by bounding box, instead of a single point:
https://github.com/LexSong/earth-reverse-engineering-utils

The script will stop early if it finds too many octants at the same level, so it can be used to dump large areas. I don't know javascript well enough to add the function to the main repo.

Hopefully, people will find this useful.

Example output:

> python find_overlaps.py 37.419714, -122.083275 37.420626, -122.085045
LatLonBox(north=37.420626, south=37.419714, west=-122.085045, east=-122.083275)
[Octant level 1]
2
[Octant level 2]
20
[Octant level 3]
205
[Octant level 4]
2052
[Octant level 5]
20527
[Octant level 6]
205270
[Octant level 7]
2052706
[Octant level 8]
20527061
[Octant level 9]
205270616
[Octant level 10]
2052706160
[Octant level 11]
20527061605
[Octant level 12]
205270616052
[Octant level 13]
2052706160527
[Octant level 14]
20527061605273
[Octant level 15]
205270616052735
[Octant level 16]
2052706160527351
[Octant level 17]
20527061605273514
[Octant level 18]
205270616052735141
205270616052735140
[Octant level 19]
2052706160527351416
2052706160527351417
2052706160527351415
2052706160527351414
2052706160527351405
2052706160527351407
2052706160527351406
2052706160527351404
[Octant level 20]
20527061605273514162
20527061605273514160
....

It would be great to cache the downloaded map data.

@retroplasma would you be interested in merging a PR with this feature if someone delivers?
What's the simplest and most robust way to add a map data caching mechanism?

I just stumbled across this project and I'm very intrigued by it. But I'm struggling to understand how the octants work. In particular, I'm using @LexSong's script find_overlaps.py (#25) to find which octants I need for a given area described by Lat/Lon corners.

That script gives me several octants for various levels. However, when I pass these octants to dump_obj.js, it sometimes is not able to download any data. Why is this?

Even more confusing, sometimes it only fails for some of the octants of a given level returned by find_overlaps.py but others of the same level work.

In the tests that I have done so far, the octants that did work actually covered the bounding box I asked for and I did not seem to be missing data, even though some octants find_overlaps.py listed for that level did not download anything. Is it safe to assume that for any level, the subset of the octants listed by find_overlaps.py that successfully download will actually cover the bounding box of interest?

Lastly, a question regarding the parameters passed to dump_obj.js: What does the second number passed do exactly? Is it not the same as the octant level? If so, why do we need to specify it again, is it not already defined by the octant we also pass to dump_obj.js?

Thanks any help you can provide!

In my testing fork, I add two scripts to handle mesh normalization.

In normalize_obj.py, the script normalize the mesh with a reference XYZ point instead of the mesh center. Therefore we could normalize several meshes with the same origin and keep their relative position. The script also rotate the mesh with latitude and longitude of the reference point, to align north/south and east/west axes.

To find the reference point, in another script find_obj_mid_point.py, I compute the mesh midpoint by convert mesh vertices to latitude/longitude/altitude and calculate the midpoint's location. The trick here is that we don't have to calculate the midpoint of high resolution mesh. We should use a low resolution mesh instead to avoid unnecessary computation.

Nice work! It might be useful to indicate which version of Node people should be running via an .nvmrc file.

https://github.com/creationix/nvm

C:\Users\LexSong\earth-reverse-engineering>node dump_obj.js 204162437140 12
found: 204162437140
octants: 1

C:\Users\LexSong\earth-reverse-engineering>node dump_obj.js 204162437140 13
found: 204162437140
Error: HTTP status code 404 for https://kh.google.com/rt/earth/BulkMetadata/pb=!1m2!1s204162437140!2u823

Not sure about why there is a 404.

I want to dumb a large zone but want to be divided in many files, using >Find_overlaps.py helped me to get the list of the octants I need, but now I cannot find a fast way to dump all those octants in different files each

pos: 1329866.230289 -4643494.267515 4154677.131562, dir: 0.387660 0.739359 0.550516
buffered: 0, tot_n: 0, tot_b: 1, pot_n: 0, pot_b: 1, obs n: 0, obs b: 0
pos: 1329866.230289 -4643494.267515 4154677.131562, dir: 0.387660 0.739359 0.550516
buffered: 0, tot_n: 0, tot_b: 1, pot_n: 0, pot_b: 1, obs n: 0, obs b: 0
pos: 1329866.230289 -4643494.267515 4154677.131562, dir: 0.387660 0.739359 0.550516
buffered: 0, tot_n: 0, tot_b: 1, pot_n: 0, pot_b: 1, obs n: 0, obs b: 0
pos: 1329866.230289 -4643494.267515 4154677.131562, dir: 0.387660 0.739359 0.550516
buffered: 0, tot_n: 0, tot_b: 1, pot_n: 0, pot_b: 1, obs n: 0, obs b: 0
pos: 1329866.230289 -4643494.267515 4154677.131562, dir: 0.387660 0.739359 0.550516
buffered: 0, tot_n: 0, tot_b: 1, pot_n: 0, pot_b: 1, obs n: 0, obs b: 0
...

no data loaded
Visual Studio 2019 :x64-windows