Coder Social home page Coder Social logo

vcityteam / py3dtilers Goto Github PK

View Code? Open in Web Editor NEW
162.0 14.0 44.0 66.62 MB

Tilers accepting various input formats (OBJ, 3DCity databases, GeoJson, IFC) and producing 3DTiles tilesets.

License: Other

Python 100.00%
3d-tiles 3dcitydb 3dtiles citygml geojson ifc open-source opensource python b3dm obj

py3dtilers's Introduction

Python 3DTiles Tilers

Build Status Documentation Status

Python 3.8 Python 3.9 Python 3.10 Python 3.11

Py3DTilers is a Python tool and library allowing to build 3D Tiles tilesets out of various geometrical formats e.g. OBJ, GeoJSON, IFC or CityGML through 3dCityDB databases.

Py3DTilers uses py3dtiles python library (forked from Oslandia's py3dtiles) for its in memory representation of tilesets.

Py3DTilers can only produce Batched 3D Models (B3DM). If you want to produce Point Clouds (PNTS), see Oslandia's py3dtiles CLI.

An article that present the tool is available in the ISPRS annals here.

Demo

Find 3D Tiles created with Py3DTilers in this online demo.

CLI Features

  • Common features: features shared by all tilers
  • ObjTiler: converts OBJ files to a 3D Tiles tileset
  • GeojsonTiler: converts GeoJson files to a 3D Tiles tileset
  • IfcTiler: converts IFC files to a 3D Tiles tileset
  • CityTiler: converts CityGML features (e.g buildings, water bodies, terrain...) extracted from a 3dCityDB database to a 3D Tiles tileset
  • TilesetReader: read, merge or transform 3DTiles tilesets

Installation from sources

See supported Python versions

For Unix

Install binary sub-dependencies with your platform package installer e.g. for Ubuntu use

apt-get install -y libpq-dev       # required usage of psycopg2 within py3dtilers
apt-get install python3 python3-dev

Install Py3DTilers in a safe python virtual environment (not mandatory yet quite recommended)

apt-get install virtualenv git
git clone https://github.com/VCityTeam/py3dtilers
cd py3dtilers
virtualenv -p python3 venv
. venv/bin/activate
(venv)$ pip install -e .

For Windows

In order to install Py3DTilers from sources use:

git clone https://github.com/VCityTeam/py3dtilers
cd py3dtilers
python3 -m venv venv
. venv/Scripts/activate
(venv)$ pip install -e .

Usage

In order to access to the different flavors of tilers, refer to the corresponding readmes to discover their respective usage and features:

Useful tutorials:

Develop with Py3DTilers

Before commiting, please run tests and make sure coding style is respected.

Running the tests

After the installation, if you additionally wish to run unit tests, use

(venv)$ pip install -e .[dev,prod]
(venv)$ pytest

To run CityTiler's tests, you need to install PostgreSQL and Postgis.

To setup PostgreSQL with Postgis on Windows, follow the first step (1. Download PostgreSQL/PostGIS) of 3DCityDB tutorial.
For Ubuntu, follow this tutorial.

Coding style

First, install the additional dev requirements

(venv)$ pip install -e .[dev]

To check if the code follows the coding style, run flake8

(venv)$ flake8 .

You can fix most of the coding style errors with autopep8

(venv)$ autopep8 --in-place --recursive py3dtilers/

If you want to apply autopep8 from root directory, exclude the venv directory

(venv)$ autopep8 --in-place --exclude='venv*' --recursive .

Developing Py3DTilers together with py3dtiles

By default, the Py3DTilers' setup.py build stage uses github's version of py3dtiles (as opposed to using Oslandia's version on Pypi. When developing one might need/wish to use a local version of py3dtiles (located on host in another directory e.g. by cloning the original repository) it is possible

  1. to first install py3dtiles by following the installation notes
  2. then within the Py3DTilers (cloned) directory, comment out (or delete) the line reference to py3dtiles.

This boils down to :

$ git clone https://github.com/VCityTeam/py3dtiles
$ cd py3dtiles
$ ...
$ source venv/bin/activate
(venv)$ cd ..
(venv)$ git clone https://github.com/VCityTeam/py3dtilers
(venv)$ cd py3dtilers
(venv)$ # Edit setup.py and comment out py3dtiles reference
(venv)$ pip install -e .
(venv)$ pytest

Concerning CityTiler

  • For developers, some design notes
  • Credentials: CityTiler original code is due to Jeremy Gaillard (when working at LIRIS, University of Lyon, France)

Configuring your IDE

When configuring your IDE to run a specific tiler, you must indicate the module you want to run (e.g. py3dtilers.CityTiler.CityTiler) and not the path to the file (i.e. not ${workspace_root}/py3dtilers/CityTiler/CityTiler.py), otherwise python will not be able to resolve the relative import of the Tilers to the Common package of Py3DTilers. An example of launch configuration in VSCode:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "<launch_config_name>", // e.g. "CityTiler" or "bozo"
      "type": "python",
      "request": "launch",
      "module": "<tiler_module>", // e.g. py3dtilers.CityTiler.CityTiler
      "args": [
        "--db_config_path",
        "${workspaceRoot}/py3dtilers/CityTiler/<my_config_file.yml>"
      ],
      "console": "integratedTerminal"
    }
  ]
}

Profiling

Python standard module cProfile allows to profile Python code.

In code

Import modules:

import cProfile
import pstats

Profile the code between enable() and disable():

cp = cProfile.Profile()
cp.enable()  # Start profiling

# code here

cp.disable()  # Stop profiling
p = pstats.Stats(cp)
p.sort_stats('tottime').print_stats()  # Sort stats by time and print them

In command line

cProfile can be run in the shell with:

python -m cProfile script.py

py3dtilers's People

Contributors

andreiveselov avatar autra avatar cbessot avatar charly-perspectives avatar clementcolin avatar diegovinasco avatar elemoine avatar ericboix avatar jailln avatar jeremy-gaillard avatar ldgeo avatar lorenzomarnat avatar nikosaul avatar pblottiere avatar peppsac avatar wesley-petit avatar yannlaurickabe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

py3dtilers's Issues

Apply flake8

Oslandia last updates integrates the usage of flake8.

  • Update setup.py accordingly (done in PR #12 )
  • fix the coding style errors/warnings...
  • Document it's usage in the readme.md (after running test sessions? )
  • Have CI apply flake8

IfcTiler unit convertions should be located in a single place

If you look for occurrences of the 100 integer in the code located in the IfcTiler module
(e.g. in ifcObjectGeom.py) chances are they concern conversions from centimeters to
meters (used in 3dTiles). Such conversions:

  • should not be scattered in the code but localized in a unique place (because of DRY. That location can now document it's choices and address many units.
  • should not hardwire the unit choice of some input. The code actually works because the considered
    input IFC files were using centimeters. What about other input possibilities ? They should be dealt with
    automatically
  • should be documented (why a hundred a not a thousand) ?

Attribute Error --> tileset-merger

Hi
I'm having problems with TilesetMerger. When I launch it, the result is:

Traceback (most recent call last):
File "/home/user/py3dtilers/venv/bin/tileset-merger", line 11, in
load_entry_point('py3dtilers', 'console_scripts', 'tileset-merger')()
File "/home/user/py3dtilers/py3dtilers/TilesetReader/TilesetMerger.py", line 104, in main
tilesets = merger.reader.read_tilesets(paths)
AttributeError: 'TilesetReader' object has no attribute 'read_tilesets'

My Debian specification is:

cat /etc/os-release

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

uname -a

Linux vm-lxosgeo-test 5.10.0-10-amd64 #1 SMP Debian 5.10.84-1 (2021-12-08) x86_64 GNU/Linux

python3 --version

Python 3.9.2

Could anyone help me?
Thank you

Improvements for the CityTiler/GeojsonTiler rework

This issue lists the possible improvements of the CityTiler/GeojsonTiler rework.

  • Split LODs creation from GLTF creation in lod_tree
  • Let the user choose if he wants splitted surfaces when using the CityTiler with buildings (splitted surfaces is commented code for now)
  • Add a flag to create (or not create) LOD1 when using a tiler
  • Improve extensions' architecture
  • Write more comments in code ?

Extension : property definition

When considering a TileSet having a batch table nothing is said about the semantics of such data (what the data represents). This fails to respect the linked open data principle (how so?).
In order to integrate a semantic description of data (some would dubiously call it metadata) present in the batch table, we could use a new specific extension as placeholder.
The work involved by the creation of a new extension is similar to the one realized for thee temporal extension, that is

  • define the data model (UML level)
  • transpose that model to a json schema in py3dtiles
  • integrate the usage of that extension in Py3dtilers (add a description, URI, for each semantic data)

Then on the client side

  • encode the data model in JS
  • realize the associated extension reader
  • integrate that extension support in UD-Viz
  • register the Extension

Eventually, disseminate that extension :

  • publish the extension in itowns (when accepted promote the client side code to iTowns out of UD-Viz)
  • publish the extension in Cesium 3DTiles repository

[CityTiler] 3D Tiles with textures are almost always bigger than the input CityGML

I've observed that in most cases, the 3D Tiles created is bigger than the input CityGML by a factor of 2 of 3 (only happens for conversions with textures). It seems odd since the CityGML is based on XML and not optimized while 3D Tiles is mainly binary and a bit of json and is meant to be optimized. Sometimes though, the 3D Tiles produced is smaller than the input CityGML but I have not been able to find out why yet. I think this issue may be related to #65 ? but it needs to be investigated more thoroughly.

Error when running the citygmltiler

I get the following error when running the CityGMLTiler:

Traceback (most recent call last):
  File "/home/vincent/Documents/dev/py3dtilers/venv/bin/citygml-tiler", line 33, in <module>
    sys.exit(load_entry_point('py3dtilers', 'console_scripts', 'citygml-tiler')())
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/CityTiler/CityTiler.py", line 181, in main
    tileset = city_tiler.from_3dcitydb(cursor, objects_type, split_surfaces)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/CityTiler/CityTiler.py", line 129, in from_3dcitydb
    cityobjects = CityMCityObjects.retrieve_objects(cursor, objects_type)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/CityTiler/citym_cityobject.py", line 97, in retrieve_objects
    result_objects = objects_type()
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/CityTiler/citym_building.py", line 45, in __init__
    super().__init__(objects)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/CityTiler/citym_cityobject.py", line 55, in __init__
    super().__init__(cityMCityObjects)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/Common/object_to_tile.py", line 99, in __init__
    ObjectsToTile.default_mat = ColorConfig().get_default_color()
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/Color/color_config.py", line 19, in __init__
    with open(config_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: './py3dtilers/Color/default_config.json'

Wrong filepath in last PR or missing default config ?

Reduce memory footprint of CityTiler by working tile by tile?

I think we could reduce the memory footprint of the CityTiler by working tile by tile instead than with the entire tileset at once and therefore scale up to converting big datasets.

From what I understand, the current main steps of the CityTiler are:

  • Retrieve city objects from the database (just the metadata, not the geometry nor the texture) in retrieve_objects()
  • Retrieve geometries (and textures) in get_surfaces_with_texture() (same process without textures)
  • Create the tileset in create_tileset_from_geometries():
    • scale, translate and reproject geometries
    • create the tree structure (one of):
      • kd tree (only uses the centroid of the cityobjects and not the geometries) + create lods (optional) (uses geometries)
      • create loa (only uses cityobjects centroid)
    • create and write textures atlases on disk (tile by tile) (optional)
    • create all tiles and tiles contents
  • write the full tileset (geometry and attributes) on disk

I propose the following workflow:

  • Retrieve city objects (just the metadata, not the geometry nor the texture) and their centroid from the database
  • create the tree (one of):
    • kd-tree + one prepare one LOD level in the tree (without actually computing this LOD)
    • loa
  • Tile by tile:
    • load geometries and textures
    • scale, translate and reproject geometries
    • create and write texture atlas on disk
    • create the lod tiles content (optional)
    • create and write tiles and tiles contents on disk

What do you think ? Could this proposal not be possible due to other tilers or other processes that I missed ?

Conflicting CityTiler and CityTemporalTiler db_config_path command line argument

When using the temporal tiler the argparse has two entries for db_config_path:

usage: citygml-tiler-temporal [-h] [--obj [OBJ]] [--loa [LOA]] [--lod1] [--offset [OFFSET [OFFSET ...]]] [--scale [SCALE]] [--crs_in [CRS_IN]]
                              [--crs_out [CRS_OUT]] [--with_texture] [--with_BTH] [--split_surfaces] [--db_config_path DB_CONFIG_PATH [DB_CONFIG_PATH ...]]
                              [--time_stamps TIME_STAMPS [TIME_STAMPS ...]] [--temporal_graph TEMPORAL_GRAPH [TEMPORAL_GRAPH ...]]
                              [db_config_path] [{building,relief,water,bridge}]

A small utility that build a 3DTiles tileset out of data

positional arguments:
  db_config_path        path to the database configuration file
  {building,relief,water,bridge}
                        identify the object type to seek in the database

optional arguments:
  -h, --help            show this help message and exit
  --obj [OBJ]           When defined, also create an .obj model of the features. The flag must be followed by the name of the obj that will be created.
  --loa [LOA]           Creates a LOA when defined. The LOA is a 3D extrusion of polygons. Objects in the same polygon are merged together. Must be followed by
                        the path to directory containing the polygons .geojson
  --lod1                Creates a LOD1 when defined. The LOD1 is a 3D extrusion of the footprint of each object.
  --offset [OFFSET [OFFSET ...]]
                        Substract an offset to all the vertices.
  --scale [SCALE]       Scale geometries by the input factor.
  --crs_in [CRS_IN]     Input projection.
  --crs_out [CRS_OUT]   Output projection.
  --with_texture        Adds texture to 3DTiles when defined
  --with_BTH            Adds a Batch Table Hierarchy when defined
  --split_surfaces      Keeps the surfaces of the cityObjects split when defined
  --db_config_path DB_CONFIG_PATH [DB_CONFIG_PATH ...]
                        Path(es) to the database configuration file(s)
  --time_stamps TIME_STAMPS [TIME_STAMPS ...]
                        Time stamps (corresponding to each database)
  --temporal_graph TEMPORAL_GRAPH [TEMPORAL_GRAPH ...]
                        GraphML-Json temporal data filename(s)

There is a positional db_config_path argument from CityTiler and an optional --db_config_path argument from the temporal tiler.

Because of this I am having some issues declaring the database configurations for the temporal tiler as shown in the documentation because everything declared with the --db_config_path flag is overwitten by the defaut path from db_config_path

For example:

(venv) diego@diego-IdeaPad-L340-17IRH-Gaming:~/Desktop/py3dtilers$ citygml-tiler-temporal \
  --db_config_path ~/Desktop/UD-Reproducibility/Computations/3DTiles/GratteCielTemporal2009-2018/CityTilerDBConfig2009.yml \
    ~/Desktop/UD-Reproducibility/Computations/3DTiles/GratteCielTemporal2009-2018/CityTilerDBConfig2012.yml \
    ~/Desktop/UD-Reproducibility/Computations/3DTiles/GratteCielTemporal2009-2018/CityTilerDBConfig2015.yml \
    ~/Desktop/UD-Reproducibility/Computations/3DTiles/GratteCielTemporal2009-2018/CityTilerDBConfig2018.yml \
  --time_stamps 2009 2012 2015 2018 \
  --temporal_graph ../Datasets/Villeurbanne_GratteCiel_2009-2018/3-diff/2009-2012/DifferencesAsGraph.json \
    ../Datasets/Villeurbanne_GratteCiel_2009-2018/3-diff/2012-2015/DifferencesAsGraph.json \
    ../Datasets/Villeurbanne_GratteCiel_2009-2018/3-diff/2015-2018/DifferencesAsGraph.json \

Mismatching number of databases vs time-stamps:
 - databases (configurations):  py3dtilers/CityTiler/CityTilerDBConfig.yml
 - timestamps:  ['2009', '2012', '2015', '2018']
Exiting.

Could we change the CityTemporalTiler flag to something unique such as --db_config_paths for example ? This should fix the conflict

Explictly express that the nodes of a GeometryTree are GeometryNodes

Some attentive reading of usages of GeometryTree (one must look at derived classes) shows that this is tree of GeometryNodes (in fact derived classes of GeometryNodes).
Provide some hints for a developer to easily get that understanding, just by reading geometry_tree.py.
Maybe the comments could be stating thits ?
Or maybe the usage of type hinting would be even clearer ?
The constructor could then be written something like

def __init__(self, root_nodes: list[GeometryNode]):

Provide tiler computation-footprint and resulting tileset feature-characterics

It would be interesting, when computing a tileset, to display some data about the tiling process itself as well as the created tileset characteristics. We distinguish two sub-sets:

  • What concerns the process (what is lost when only given the result of process: tracability):
    • Type of input data (filename of input files)
    • Compression mode and characteristics/indicators
    • Computation time
    • Number of lods
  • What concerns the result of the process (what can be retrieved when given the result). This part should be implemented in Py3DTiles by enhancing the info command (see this issue):
    • Number of tiles
    • Size of each tile, along with the number of object per tile
    • Number of features (e.g. buildings, trees...)

Provide the ability to configure geometric error

The geometric error defines when a tile should be displayed by the client (depending on the distance to the camera). This should be configurable (and editable with the TilesetReader). It would also allow to define when to change between LOD1 or LOA and the most detailed tiles for instance.

[CityTiler] Error when using textures together with LOD1

When using textures together with LOD1 I hit the following error:

Traceback (most recent call last):
  File "/home/vincent/Documents/dev/py3dtilers/venv/bin/citygml-tiler", line 33, in <module>
    sys.exit(load_entry_point('py3dtilers', 'console_scripts', 'citygml-tiler')())
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/CityTiler/CityTiler.py", line 183, in main
    tileset = city_tiler.from_3dcitydb(cursor, objects_type, split_surfaces)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/CityTiler/CityTiler.py", line 148, in from_3dcitydb
    return self.create_tileset_from_geometries(objects_to_tile, extension_name=extension_name)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/Common/tiler.py", line 100, in create_tileset_from_geometries
    return create_tileset(objects_to_tile, self.args.lod1, create_loa, self.args.loa, extension_name, self.args.with_texture)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/Common/tileset_creation.py", line 13, in create_tileset
    lod_tree = LodTree(objects_to_tile, create_lod1, create_loa, polygons_path, with_texture)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/Common/lod_tree.py", line 24, in __init__
    lod1_node = Lod1Node(group.objects_to_tile, 5)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/Common/lod_node.py", line 40, in __init__
    extruded_polygon = ExtrudedPolygon(object_to_tile)
  File "/home/vincent/Documents/dev/py3dtilers/py3dtilers/Common/polygon_extrusion.py", line 38, in __init__
    points = hull.exterior.coords[:-1]
AttributeError: 'LineString' object has no attribute 'exterior'

CityTiler process gets killed when converting CityGML with too many/heavy textures

When running the CityTiler with too many / too heavy textures, the process gets killed by my OS (Ubuntu 20.04). According to the kernel logs, it seems to be due to memory usage:

[267574.260974] Out of memory: Killed process 1606130 (citygml-tiler) total-vm:7268524kB, anon-rss:6205768kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:12756kB oom_score_adj:0

It happens with the 1st borough of Lyon 2018 vintage for instance.

Do you have the same issue ?

May be some memory is not correctly released or all textures are loaded at once ?

Embed material in b3dm

Currently, It seems that there is no way to add a specific material to an object in the B3DM produced. Looking at the specification, it should be possible to do so.
It would be interesting to add such feature.
For example, we could then add a color for each building, using their texture to be a little more realist than a white building.

Maybe pg2b3dm have this feature

[CityTiler] Weird vertices

The 3D Tiles created with the CityTiler have weird vertices: it seems that all buildings are connected with a vertex, while they are not in the original CityGML:

lyon-1-wireframe

image

I reproduced it on different input CityGML and the visualisation is the same in iTowns and Cesium. It happens whether the transformation is ran with or without textures. I reproduce this issue before and and after #71 so it does not seem related to this PR and neither to issue #77. Note that it has no visual impact when not in wireframe mode.

Have the Tilers produce separate styling data

Short term work

Make the break down in sub-task e.g.

  • survey of available solutions
  • technical evaluation of possible solution
  • ...

Discussion

The GeoJson Tiler can already produce TileSets than converts/encode some input attribute as a rendering color.
On the client side Ud-Viz can be configured with a rendering color.

Systematize/generalize this separation between data and presentation and, when necessary (that is when some coloring data gets extracted/produced), have the tilers produce separate presentation/styling data. By separate we here mean that the styling data should not be included in the tileset files but in a separate file.

Notes

  • Cesium already proposes a Styling data model that we could adopt (both on the upstream py3dTilers side and the UD-Viz rendering side)
  • Check if iTowns already proposes a styling mechanism ?

Wrong Bounding box calculation

If you read the documentation, it explains that how a bounding box works.
Currently, when we try and look at bounding boxes in UD-Viz, it shows that west and south extent at 0 for any created tileset.
Maybe I'm wrong but I think it is because of the notion of half-length presented in the documentation : for example, if i look at the tileset for Limonest, I have the following bounding box :
918977,
2590516,
161,
918977,
0.0,
0.0,
0.0,
2590516,
0.0,
0.0,
0.0,
161

The three first elements should be the bounding box center. The next three shows the x-axis direction and half-length.
If I understand the half-length notion correctly, it means that the bounding box does the following math to obtain the min and max for the x axis, using bounding-box[0] and bounding-box[3] : min = bounding-box[0] - bounding-box[3] and max = bounding-box[0] + bounding-box[3]. It gives us the following bounding box for x : [0,1837954].

If I take the extent for the x-axis for the background, it looks like this [1833742,1842872]. As our tileset is included in the extent's background, I think that the min and max x should be included in this range.

Extend tilers tests

Provide tests for the following tilers

  • IfcTiler

Extend the testing data set

Import alternate example files (license free) from various datasets and data types (GeoJson, Ifc,...)
and integrate them in the test suite.
When the data diversity is concerned, and in order to keep the repository history/footprint as light as possible, consider manually reducing the imported data file.
Yet, when specific tests really require extensive data to be meaningful (e.g when testing memory footprint, algorithm complexity) you might consider commiting heavy files to the repository.

Possible sources of data (check the license and cite the source):

Tilers requiring a DB for testing

The CityTiler and CityTemporalTiler are already be CI-tested (on Travis). But doing so on the user/developer desktop require a manual pre-installation and set up (with psql) of a testing database.

Limitations:

  • the testing database installation instructions are only available for Windows: provide their equivalent for OSX and Ubuntu
  • the tests will currently try to run the CityTiler/CityTemporalTiler tests and thus implictly expect a local Postgres/postgis installation (and will fail otherwise). Implement a pre-condition for the availability of such database and have pytest skip the tests requiring that condition when it is not fulfilled.

Already realized tests

As mentioned below by @LorenzoMarnat , tests have been provide for the following tilers

Clarify what GeometryTree.leaf_nodes are for (and what is their type).

The following concerns the geometry_tree.py file.

If we consider the graph theory terminology, a (tree) leaf is a node (vertex) without sub-nodes. Now, when given a root (node) of a tree one can navigate to the leafs of the tree.
Hence storing them again in leaf_nodes seems only justified on optimization purposes.
But then leaf_nodes must be kept updated when the tree gets edited, which places some constraint on the code.
Are leaf_nodes and leafs (of the root_nodes) (where leafs is a "function" of a node/root_node) the same thing ?
Document/clarify what lead_nodes are for...

In addition, there is no setter for lead_nodes. Having a setter would be a good place to locate/document the type of leaf_nodes. What are leaf_nodes types ? ObjectsToTile ? GeometryNodes ? Derived classes of GeometryNodes ?

Invalid tiles when using Batch Table Hierarchy in CityTiler

When creating tiles with batch table hierarchy with the CityTiler (in its current state on Master), the resulting tileset can't be loaded in UD-Viz due to this error:
image

I can't tell if the Batch Table Hierarchy is broken on the creation side (CityTiler) or on the loading side (UD-Viz). Did this feature even work one day ? or is it unfinished job ?

To create a tileset with BTH:

citygml-tiler --with_BTH

Provide some structural diagrams

  • Update and complete the documentation
  • Create some structural diagrams (eg class diagrams)
    • a class diagram with GeometryNode, GeometryTree and their derived classes LOD1, LOA respectively LodTree, TileHierarchy as well as they py3dTiles counterparts Tile and TileSet
    • a instance diagram illustrating a geometryTree (Nodes and leafs) a Tileset hierarchy (annotated with the convertion functions?)

Rename ObjectsToTile to something with better semantics

The name (of the type) "ObjectToTile" doesn't carry the right semantics (which is confusing when reading the code).
Change that type name.
Maybe a clearer name would be TilableObject ?
In fact ObjectToTile is an interface (abstract class) to which some derived class must comply in order to be integrated into a Tile. Because "object" is a very broad term that doesn't add much semantics when used in TilableObject maybe we could rename "ObjectToTole" to "TilableInterface" ?

[CityTiler] "Dancing buildings"

The geometries of the 3D Tiles produced with the CityTiler move alone when moving the camera:

2022-03-07.10-21-27.mp4

This happens for conversions with and without textures and since PR #71 . I reproduce this on iTowns and Cesium and I don't reproduce it with tilesets produced before #71 with the same iTowns/Cesium apps.

[CityTiler] Tiling with textures creates many more tiles

Running the city tiler with the --with-texture option results in a tileset with a lot more tiles. From my tests, from 32x to 64x more tiles are generated with the --with-texture option for the same input dataset. Do you know where this come from and if it is an intended behavior ?

I think this has an impact on the tileset size and on the visualization performances.

Remove hardwired geographical coordinates system code

Within IfcTiler/ifcObjectGeom.py code like

transformer = Transformer.from_crs("EPSG:27562", "EPSG:3946")

hard-wires

  • the knowledge projection system of the input data (we could have another system for the input)
  • the projection system of the resulting 3dTiles output (the user might like to specify another projection system)

Such a rewriting/conversion should be within the Tiler but outside of it. Let the Tiler produce a 3dTiles using the projection system of it's input. Then write a 3dTiles converter utility (CLI) that converts the projection system. This would separate the concerns.

Add support for textures to ObjTiler

Currently, material textures specified in a .mtl file next to the .obj are lost when convertinf between .obj (Wavefront) files and 3DTiles.
It would be nice to embed the specified texture coordinates and the textures files in the generated 3DTiles.

Limit the size of the tiles and their atlas

To balance our tiles, we put an arbitrary number of features in each tiles (with a kd-tree). Since some features are way bigger than others (more triangles, heavier texture images, ...), the size of the different tiles and their atlas varies a lot. Some of the textures are way to big to be rendered correctly.

To have more balanced tiles, we should be able to limit the size of a tile depending on different parameters:

  • the number of triangles in a tile.
  • the geographic area/extent of a tile. This will ensure that all our tiles are the same length and that we have a balanced decomposition of space.
  • if the tiles are textured, we could balance the tiles with the size of their atlas. Since the textures are bigger in memory than the geometries, it would be usefull to set a maximum atlas size. Some work has been done to limit the size of the atlases #35.

LODs and reprojection

When reprojecting geometries with --crs_out flag while using the LODs flags (--lod1 or --loa), the tiles corresponding to the LODs have wrong positions and bounding boxes.

[CityTiler] Color the city objects depending on their type

Provide the ability the apply a color on the city objects. The color depends on the citygml objectclass of the object. The cityobjects can be buildings, relief, surfaces (roof, wall, etc), ...

WIP on citytiler_colors branch

  • Create and save materials in glTF
  • Configure colors with a JSON file
  • Query the objectclass of each cityobject
  • Optional flag to add the colors

CRS from ID and proj4

crs_in and crs_out should handle IDs (crs_in 3946) and proj4 (crs_in "+proj=latlon") with pyproj

We could also define some useful CRS (3946, 4978, etc) as constants

py3dtilers is not referenced by google

It's maybe a little bit secondary but py3dtilers is not referenced by google: searching py3dtilers in google does not yield any result for this repository on none of the two pages. It yields results for py3dtiles, or for py3dtilers on githubmemory, but not the github repository. This may have a strong impact on its visibility.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.