jaminzoda / terra-antiqua Goto Github PK

This module should be more flexible. Give the option to add several layers in a list and the tool would compile them.
This still needs discussion!

The tool Clip from another raster could be enhanced to allow to cut a piece of a raster and paste it somewhere else in the same raster. We could use a polygon (mask) to define the area we will move.

Some thoughts about this:

• Steps suggested from Gerold Zeilinger: Extract by mask, shift raster, merge

• When moving a block, the gap it leaves behind can be filled with interpolation. The edge will be too sharp. This could be solved with smoothing along that edge (the polygon border), within a buffer.

There should be a new tool that creates mountains ranges.

The problem: The Topography modifier can't create topography where there isn't. So it'd be good to have a tool that allows the user to create a mountain range where there is a flat topography.
In any case first you need to define an area with a polygon. Inside it, all values are changes to NaN.
From that polygon the tool creates Voronoi polygons and extracts the vertices from them. Then it keeps only the ones inside the polygon originally defined.

1. Automatic creation of topo / bathy: The user enters a maximum value (depth or height) and that gets assigned to the vertices following a gradient depending on the distance these have to the polygon's edge. The further the point is from the edge, the closest its value will be from the maximum (depth or height).
   Then the empty cells will be interpolated between those vertices and the surroundings to create a natural-looking pattern.

2. Manual data input: In case we want to define manually the constraining points, we add them as a point shp-file. This way we should be able to add as many files as we want with different values to use as fixed points. For this we don't need the Voronoi polygons.

3. Hybrid method: These two possibilities could be mixed, sing the vertices from the Voronoi polygons as well as the points entered by the user (if he/she has data available). The vertices from the Voronoi polygons are given the maximum value (height or depth) and the points entered by the user have their own altitude value.
   This would allow to create more complex pattern where there isn't enough data. So in those areas we would still be able to create a broader ocean/mountainous region and just define few points.

We should find/design new icons for the tools.

For the user it is important to know what exactly has been modified by an algorithm and try to quantify the modification. It would be good to trace all the modifications and highlight them in a separate raster and return it to user. Provide an option to return only the modified part of the raster.

For the tool Create TB, the advanced options would be: Width of the mountain slope, ruggedness, Width of the continental slope; Widht of the shelf; Maximum depth of shelf
For the Paleoshorelines: Maximum and minimum ranges for emerging and submerging

The colouring of the maps is created as a temporary colour ramp and it should be a permanent one that can be found and edited by the user in the symbology properties.

While creating a sea, the algorithm takes into account the preexisting bathymetry. If there is some, the algorithm doesn't assign any value to that area and then interpolates between the created values inside the polygon and the preexisting ones to create an open ocean.

This idea is about creating realistic drainage patterns (e.g. stream power low) to the relief created.
K.A^m .S^n (?)
To be developped

Currently the plugin reconstructs a DEM for a single time step. An enhancement of the plugin would be to add a batch reconstruction option so that the reconstruction can be performed for several time slices at the same time. Two options could be implemented:

1. Providing a batch input interface so that users input data and parameters for several time steps simultaneously and get several outputs. An alternative to inputting multiple data and parameters, is to store parameters for different time slices in the attribute table of a single shapefile marked with the time of reconstruction. Using these parameters plugin will need to reconstruct several DEMs (for different time slices) with a single input data-set.

2. Use ready, reconstructed, DEM for two (or more) time slices to interpolate between them and produce a reconstructed map(s) for a time snapshot between the inputed ones. For interpolating between time snapshots the position of each pixel in the input rasters should be traced through time. For this, using PyGplates, the plugin should rotate masks of continental blocks (that are used to cut blocks of topography in GPlates) and keep track of each geographic feature and its pixels. Then, we can use instances of the same pixel in different input rasters (for different time slices) to interpolate a value for the pixel in the raster of the time of reconstruction.

Guillaume changes some help texts. The corresponding fields form the interfaces have to be updated.

When we don't specify a name for the output from the MaskMaker we get an error because it doesn't overwrite the previous result. This works fine for the rasters in other tools but not for the vector layers here.

For Removing artefacts to function properly we now need to store user preference and track the tool to see if it is enabled or disabled. This is used to enable and disable it. Now, the settings are stored in a .txt file, and temporary setting for track the tools state is stored in a Class under TaSettings name. These settings and other preferences should be moved to a QSettings class, which automatically stores all the settings inside an .ini file inside users folder.

Formula modification of topography or bathymetry that is performed by Modify Topography/Bathymetry algorithm requires entering a correct formula. For example a formula like 0.2x will not work because the multiplication operator is not specified. Need to validate formula input before running the algorithm. Also, to provide feedback for the user, in case if a wrong formula is entered, need to check the formula in modFormula function (utils.py) and raise an exception, if it is wrong. The exception then should be caught by a try and except statement, and displayed in the log tab.

There is a new feature to remove artefacts, a new help text is needed for that.

Rename all the modules according to the new conventions.
An excel file has been created with these new names and the new order for the icons in the UI.
The title of the dialogs should also correspond to the ¡name of the tool
@jaminzoda you should check it and make the changes you consider.
the file: e7e209d

After drawing the polygon and entering a formula, the tool isn't working. This python error shows up:

We need to use sphinx for the documentation. This is the tool that creates the html help page and maybe a pdf documentation too.
The help html should look like a Wikipedia page. I think that is the format sphinx creates anyway.
There should be one page where the whole workflow is explained, with different chapters/sections and links to it from the content section at the beginning ("Wikipedia" format).
Then there could be a second page with known issues / errors and the way to solve them.
A way to give feedback should be there too. A link to GitHub, an e-mail...

This is just to summarise all the tools where we think a step by step "wizard" UI could be useful. There will be links to issues where we mentioned this already.
Modify TB #50
Create TB #128
Prepare masks
Compile TB (#104 somehow related)

The idea is to make something similar to the software installation wizards, where the user has a succession of windows that guide him step by step through the installation process. Since the interface of these tools is becoming more and more complex, this kind of approach would be useful.
The "tree" in each case is still to be defined.
I put this for v0.3, but maybe it is too much...

It should be possible to make a low resolution paleogeographic map within minutes.
This without tracing continental-ocean boundaries and shorelines that both take forever.
To do that I suggest the following:

1. Cut and move topo with continental blocks and create/import bathymetry simply.
2. Make the resolution low enough so as to fill in small gaps between blocks*.
3. Fix topo and bathymetry with masks**.
4. Interpolate and smooth heavily, voilà!

*This is to avoid tracing the Continent-Ocean boundary which took a week.
**Here you can either do nothing or do a lot depending how realistic you want it.

It would be good to be able to take the parameters necessary to "Create t/b" from the attribute table of the polygons. This way the user could create different features at the same time.

If this makes the interface too complicated, it could also be remade in the form of a "wizard" with several steps. (#129)

Some things from the html code in the help texts aren't displayed correctly in the text box. The div and the h4 ,for example.
This could be also an issue with images, once we start putting some as examples.

The issue: There is that problem now with pieces of ocean getting in-between continental blocks while doing the compilation. It happens because the algorithm takes topography data inside the blocks and bathymetry data outside of them. So when there is a piece of the bathymetry raster underneath the continent, if the blocks on top have gaps between them (because they moved during the reconstruction), then the algorithm will take the values from the bathymetry raster.

The temporary solution: This requires to draw the COB. That is a single line which marks clearly for the compiler where it has to take info from the topography or the bathymetry raster. Then these lines are merged into polygons which are used while compiling. The problem is that drawing these lines is very time consuming.

Proposed enhancement: When taking into account the tectonic blocks, the algorithm should draw a buffer (5 pixels would be ok) around each polygon so that when it cuts the bathymetry raster it takes into account that new area. This would mean the gaps between tectonic blocks (eg. Andes) are covered and the bathymetry values aren’t taken there. Of course we would lose 50km of bathymetry data around each block and they would be filled with NaN values. The gaps will be filled with interpolation later, creating a 50km slope between continental shelf and deep ocean. But it doesn’t matter (because in most cases the slope isn't shorter than that) and makes the whole even more realistic, by adding a slope at the end of the continental shelves.

A windows with a couple of sentences should be shown when clicking on the icon, so the user knows what to do.

Should we take out the option to create sea with voronoi polygons? They aren't mentioned in the new version of the help text 081efe1. But I can add it.
And there is a bug in the ui. The ruggedness of the mountain isn't hiding when switching from "mountain range" back to "sea".

The gaps in rasters are now filled by interpolation. But sometimes users need to fill them with some arbitrary values. In this case, we need an option to fill gaps with specific values. This can be done either in Modify Topography, Remove Artefacts or Standard Processing.

There should be the option to perform the smoothing or the interpolation to a mask.
Since the interpolation is only filling gaps, it would be perhaps better to use the smoothing (which is actually a sort of interpolation) for this.

When smoothing globally, there might be some areas that need intensive smoothing and some others that do not need to be intensive. As an example, passive margins would need several rounds of smoothing to make the difference between deep ocean and the land minimal. To solve this, we may introduce smoothing by masks. For this the smoothing tool should have an option to introduce the masks file. And each mask can have its own smoothing parameters, such as the radius in pixels to consider while smoothing. Also, the smoothing tool is too slow. There should be different approaches for smoothing. Need to explore smoothing techniques.

The help text could contain some examples such as images (sketches), or the correct format of the formula in the TopoModif.

The standard processing tools are still using the old dialog that is designed in QtDesigner. Should be rebased on the new base_dialog and use new feedback object for logging, showing progress and maintaining communication between the algorithms and the dialog. When rebased, the TaFeedbackOld should be removed from utils.py and smoothing.py

It would be good to show a welcome page for users when starting the plugin for the first time. On the welcome page the user will be given general idea about the work of the plugin and providing a link to a manual and perhaps an example qgis project with example data.

The feature creator dialog did not work in earlier versions of Qgis (<3.10), and raised QgsProcessingException - incorrect parameter value for expression.
This is related to a change in random points creating algorithm that was introduced with Qgis version 3.10. The initial version of feature creator used the latest version of this algorithm. Need to adapt the feature creator to the old version of the algorithm.

New options to create plains, active / passive margins and Mid-ocean ranges.
To be developped

We should start creating the manual. We're often referring to it in the help texts.
It should contain all the details about how the tools work that we're not writing in the help texts.
This is related to the shpynx documentation #55

All the tools in Terra Antiqua manipulate on 2 or more data sources, which sometimes are of different types (i.e. Raster and vector). These data sources, depending on their source, may have different Coordinate Reference Systems (CRS). However, for producing unbiased results, the input layers must have the same CRS. Therefore, before running any algorithm, the plugin must first check if all the input layers have the same CRS. If it is not the case, it should prompt transformation of layers' CRS to match each other, or to match the project's CRS.

The smoothing is too slow because it uses the primitive averaging of values in a window. Instead, filtering techniques such as Gaussian filter or convolution technique should be used.

Can the readme file be changed?
I have suggestions for these two lines:
6a9b541#r37473583
6a9b541#r37473636

Interpolation inside polygon should be Combined with the interpolation.

The polygon that is drawn by the tool to remove artefacts could be somewhat transparent. This would help drawing it

The "layer" field in merged masks (such as coastlines, Continental shelves and shallow seas) layer gets populated with the name of merging layers (e.g. continents). However, sometimes, when the same fields of merging layers have different data types, they can't be merged. Therefore we refactor fields before merging to make the data types of the same fields in two merging layers the same. During refactoring the refactored layer gets a name "Refactored_layer", which then is populated in the "layer" field of the merged layer. During compilation of topography and bathymetry, the features, which have values, in their "layer" field , different from "Continents", "Continental shelves" and Shallow sea" will not be rasterized and hence in resulting compiled DEM the areas of these features will be empty.

The idea of saving settings is already started in v0.2 with the artefact remover. But we can take it further. In each tool where the user has to enter some parameters, we can keep them in that settings file so that next time the user wants to use that tool it shows the parameters he used before.
Then we should add a button on the interface to "reset settings" that would put back the default settings.

1. Every time we ask the user to choose a mask layer, we give the option to select only one mask through the attribute table. There should be a way to select more than just one mask. A sort of field where we could add features based on their attributes one by one so that it's possible to use 2, 3, 4 etc... instead of the whole layer or just 1 feature.

2. Fill from another raster should have this option too. It might be useful to be able to select one (or several) mask(s) from a layer to perform this operation. Otherwise the user needs to create one extra layer with just the mask(s) he (she) wants for that particular case. It'd be faster and easier to be able to get that mask from any layer he (she) has already.

I have to work on a better colour ramp.
It should be symmetrical to look nice and still display the map values in a clear way.
The current colour ramp is a good step in this direction.
Most of the values are concentrated between -500 and +2000 and there is a spike also around -5000.
I'll need to see the histogram to work on this.

The dialog windows of the standart Processing tools doesn't display the right help text. It get stuck at the first one (interpolation) and doesn't show the new ones when selecting other tools. Screenshot added here.A Python error is shown (screenshot).
Maybe it's related to the recent reorganization of all folders inside the plugin?

Sometimes one map can be smoothed several times, for example when creating several versions of one map or when fixing a part of the map and smoothing again. This leads to different shorelines in different versions of the map, if different somoothing factor is used. To fix this problem two enhancements should be introduced in smoothing tool:

1. Smooth inside polygons - for the case when only part of the map should be smoothed multiple times;
2. Introducing an optional parameter for keeping the shorelines fixed and changing the algorithm to include this functionality - for the case when the whole should be smoothed multiple times.

We need to implement rescaling in the tool.
This could be 3 options:

1. Modification with formula
   Here we should ask for:

• Bounding values (bounding min / bounding max)
• Formula

2. Heightening / flattening
   Here we should ask for:

• Final maximum (the user chooses the highest value of the resulting topography)
  The minimum value should be taken automatically from the raster. And we shouldn't give the possibility to choose a minimum here, it's to confusing.

3. Rescaling
   Here we should be asking for:

• Bounding values (bounding min / bounding max)
  --> If there is an empty field here, the original min and max values from the raster should be taken.
• Final max and final min (the desired values for the resulting topography)

For now, the log and progress functionality is provided from inside the QThread objects. To make consistent and to avoid duplicating of the code, need to use the TaFeedback object that is now created. It should be based on the python logging module and should provide the debugging capability.

The paleoshorelines tool looks inside the shorelines mask and if the values are below sea level it brings them up above it, and if outside the shorelines mask values are above sea level, they are set below it. The values are set below and above sea level with a rescaling technique, which preservers the initial pattern of the relief, rescaling the elevation values between new minimum and maximum values. This approach has a con: when setting the values above zero we define new maximum (e.g 5 m); in some cases there might be a need to set it higher (e.g. 100 m); In this case we may end up with an artifact with high values close to the shoreline and low elevation behind it. To solve this issue, we can instead of rescaling set the values to NAN and then interpolate them.

The tool isn't able to merge two vector files that contain fields with the same name and different data types.
It should convert the data type before merging.

If the window with the tooltip is closed instead of clicking OK, the code is broken. The closing button should be removed so there is only OK to be rpessed.
The windows shoul also be made dependent, so the user can't interact with the main interface until it's cosed.