tudelftgeodesy / doris Goto Github PK

Makefile in doris_core refers to construct_dem.sh (l. 124) in bin directory, but such file does not exist. For this reason make install fails.

I want to install Doris 5.0, but I cant do make step.
I sent all details.
Doris.pdf
INSTALL.txt
README.txt
I installed python 2.7, g++7.5, gcc7.5, gfortran 7.5.
Ubuntu 20.04

• Changes in create_dem.py:
– Applied patch on in the _srtm_listing() function to change
the source for DEMs
– Uploaded the missing WW15MGH.DAC file and
provided https://github.com/anurag-kulshrestha/
geoinformatics/raw/master/WW15MGH.DAClink
– Patched to correctly geo-register the downloaded EGM file
• Changes in download_sentinel_data_orbits.py
– Changed the link for orbit download to https://scihub.
copernicus.eu/gnss
– Streamlined the download process by updating the web-
crawler.

Hello everyone, I am encountering several issues while processing two scenes of Sentinel-1A data using Doris, and I am quite puzzled. I would like to seek some advice.

1. Why does the DEM data automatically downloaded by Doris either consist entirely of zeros or a single negative number? I am unsure how to convert the DEM I downloaded into a format that Doris can recognize.

2. Regarding warning messages: During the execution of Job3, a WARNING is displayed: "File: 2021-05-29.raw does not seem to exist (may not be a problem)"; for Job13, it shows "WARNING: RS_DBOW: min. line < overlap (writing: 6 lines with zeros before first resampled line)"; for Job23, it warns "WARNING: DEM does not cover entire interferogram."; and for Job30, it states "File: 'cint.raw' has wrong fileformat or dimensions: bytesperpix=8, #l=1455, #p=23692; size_computed=275774880B v. size_ondisk=16936200B," as well as a warning during the final phase of unwrapping that "File: unwrapped.raw does not seem to exist (may not be a problem)." I am uncertain about the impact of these warnings on the final results and have not been able to find relevant information.

Hi, Im trying to process a pair of CSK and Im stuck in resample step. The images sizes are ~1.5GB each but the rsmp_orig_slave_line.raw and rsmp_orig_slave_pixel.raw are growing indefinitely (in PROGRESS 0% they reach ~40GB each)

Im using the following input file for reample

# **********************************************************************
# ***  Doris inputfile generated by: run at: Sep 02, 2004 (Thursday) *****
# **********************************************************************
# ***
# ***  Filename:        Inputfiles/input.resample
# ***  Author:          Doris User
# ***  Master:          master
# ***  Slave :          slave
# ***  
# **********************************************************************

PROCESS         resample                // slave
RS_METHOD       cc6p                    // good choice
RS_OUT_FILE     Outdata/slave.resampled
RS_OUT_FORMAT   ci2                     //
RS_SHIFTAZI     on


STOP                                    // mandatory

Python 2 is will be officially supported up to 2020 [1].
It would be nice to have the full compatibility with Python 3 of all the python modules and scripts bundled in the doris distribution.

[1] https://www.python.org/dev/peps/pep-0373/

In resample step, i find log file every burst very large about 31.4Gb. And rsmp files no data. I can't find some bugs.

It seem that the Python module doris_stack/functions/create_image.py is truncated for some reason.

Hello, as shown in the figure, when I run the fourth step (run -s4) in Doris, the following error appears. How can I solve it? Looking forward to your reply!

when i run python prepare_datastack_main.py,and input full path to the shapefile,it recalls me The path is incorrect, use another path.i don't know what's the matter.thanks much

How to use doris5.0 to process sentinel 1 data,

Hope to get a detailed answer.

thanks !

When i tried to install DORIS 5 with "pip install doris " the OSR file could not be downloaded therefor installation stoped there.
i tried to download OSR and installed it with pip ... but it not worked too and froze at processing step.
how can I solve this problem ?

root@suresh:/home/suresh/Downloads/doris/datastack/stack# bash doris_stack.sh
Traceback (most recent call last):
File "/home/suresh/Downloads/Doris-master/doris_stack/main_code/doris_main.py", line 4, in
from doris.doris_stack.main_code.doris_sentinel_1 import DorisSentinel1
ImportError: No module named doris.doris_stack.main_code.doris_sentinel_1

Am getting the following issue when I try to run bash doris_stack.sh.
Kindly help me solve this issue

I've been trying to install sartools on cygwin. I entered into the file and typed "make". Here is what popped up after it makes:

$ make
g++ -O3 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o dateconv.o dateconv.cc
dateconv.cc: In function ‘int main(int, char**)’:
dateconv.cc:51:35: error: cannot convert ‘tm*’ to ‘const char*’
51 | strftime(indatestring,informat,&tm_tmp);
| ^~~~~~~
| |
| tm
In file included from /usr/include/pthread.h:14,
from /usr/lib/gcc/x86_64-pc-cygwin/10/include/c++/x86_64-pc-cygwin/bits/gthr-default.h:35,
from /usr/lib/gcc/x86_64-pc-cygwin/10/include/c++/x86_64-pc-cygwin/bits/gthr.h:148,
from /usr/lib/gcc/x86_64-pc-cygwin/10/include/c++/ext/atomicity.h:35,
from /usr/lib/gcc/x86_64-pc-cygwin/10/include/c++/bits/ios_base.h:39,
from /usr/lib/gcc/x86_64-pc-cygwin/10/include/c++/ios:42,
from /usr/lib/gcc/x86_64-pc-cygwin/10/include/c++/ostream:38,
from /usr/lib/gcc/x86_64-pc-cygwin/10/include/c++/iostream:39,
from dateconv.cc:9:
/usr/include/time.h:67:49: note: initializing argument 3 of ‘size_t strftime(char*, size_t, const char*, const tm*)’
67 | size_t _maxsize, const char *__restrict _fmt,
| ~~~~~~~~~~~~~~~~~~~~~~~^~~~
make: *** [Makefile:161: dateconv.o] Error 1

Hope you can help me out.

When I lunch the make -f Makefile command, terminal return the following error and fails:
bk_messages.hh:214:26: error: invalid conversion from ‘char’ to ‘const char*’ [-fpermissive] strcat(name_,'\0');// terminate id
What can this error depend on? How I can solve it?

I'm starting to work with StaMPS and I use the ALOS2 data for my case study and running the Stamps software. unfortunately, I have a error after running the step_coreg_simple script as following:

"ERROR : [coregistration.cc[4058]]: code ???: No data found > threshold.
Caught error of EXCEPTION class!
It is: specific error
total cpu: 1 min 49.3288 sec"

In your setup.py you define a package named install which can be quite problematic to install at system level.

Please consider to rename that package into e.g. doris_install to avoid possible name conflicts with other packages.

Alternatively you could consider to move the three python packages:

• install
• prepare_stack
• doris_stack

under a common root package e.g. doris or pydoris

Having a single root package would make much easier to package Doris to be included in common Linux distributions.

Traceback (most recent call last):
File "/home/prosper/Desktop/Doris/Doris-master/doris_stack/main_code/doris_main.py", line 28, in
doris_sentinel_1.run(args.parameterfilepath, start_date, end_date, master_date)
File "/home/prosper/Desktop/Doris/Doris-master/doris_stack/main_code/doris_sentinel_1.py", line 88, in run
processing.initialize()
File "/home/prosper/Desktop/Doris/Doris-master/doris_stack/main_code/single_master_stack.py", line 175, in initialize
self.create_full_swath()
File "/home/prosper/Desktop/Doris/Doris-master/doris_stack/main_code/single_master_stack.py", line 195, in create_full_swath
self.full_swath[date][dat_type] = copy.deepcopy(self.stack[date][bursts[0]][dat_type])
KeyError: 'master'

Has anyone run into this error yet after running 'bash doris_stack.sh'

Hello, as stated in the title the Doris core directory won't compile after using the configure executable; this appears to be due the fact that during the definition of some friend functions in the orbitbk.hh file (namely the coordinates change ones) they are given fixed values for some arguments (such as MAXITER and CRITTERTIM or CRITTERPOS). I tried changing the code and the only thing that made compiling possible was not giving the friend functions their fixed arguments but at the same time re-defining out of the orbit class in the same header file the same functions, no longer as friend ones, with fixed arguments, as shown here: https://stackoverflow.com/questions/22533511/friend-declaration-specifying-a-default-argument-must-be-a-definition This made compiling possible but causes segmentation fault with sigsev. This issue seems to be present both in the version 4 and 5 of Doris. See attached files for what exactly was shown on my terminal. I'm using currently a macbook pro late 2013 15" with the latest version of MacOs Big Sur at the time (11.2.3) and with Xcode 12D4e version.

I just wonder when the new stable version will come out?

Hello,

I am using Mac systems (macOS High Sierra). The TERRASAR-X data are processed using STAMPS 4.1 and Doris 5.0.3. When I run " make_coreg ", it caught an error shown below . I am getting the same error on ubuntu 18.04 operating system. Thank you in advance.

WARNING : It is highly recommended to use FC_PLOT to plot the computed FINE offset vectors.

WARNING : It is higly recommended to use CPM_PLOT to review the estimated model.

PROGRESS: Interpretation inputoptionsfile finished.

total cpu: 0 min 0.002551 sec

PROGRESS: Finished initialization

[orbit.cc] orbvector_type: 30 vs velo prm fixed: 31

PROGRESS: Orbit: interpolation coefficients computed.

[orbit.cc] orbvector_type: 30 vs velo prm fixed: 31

PROGRESS: Orbit: interpolation coefficients computed.

ERROR : [ioroutines.cc[770]]: readres: could not find pattern: "Slope_CoarseCorr_pixels:" in file: coreg.out

Caught error of EXCEPTION class!

It is: file error

total cpu: 0 min 0.011175 sec

Hello,
I installed Doris 4.0.8 but I didn't understand how to process multiple Sentinel 1 images. Does it provide us with multi-process and visualization? Could you please help me with this issue?

when l run make following error appers,

g++ -O -Wno-deprecated -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D__USE_FFTW_LIBRARY__ -I../../fftw-3.2.1/include -c -o processor.o processor.cc
In file included from processor.cc:45:
orbitbk.hh:144:18: error: friend declaration of ‘int32 lp2xyz(real8, real8, const input_ell&, const slcimage&, orbit&, cn&, int32, real8)’ specifies default arguments and isn’t a definition [-fpermissive]
144 | friend int32 lp2xyz(
| ^~~~~~
orbitbk.hh:155:18: error: friend declaration of ‘int32 xyz2orb(cn&, const slcimage&, orbit&, const cn&, int32, real8)’ specifies default arguments and isn’t a definition [-fpermissive]
155 | friend int32 xyz2orb(
| ^~~~~~~
orbitbk.hh:164:18: error: friend declaration of ‘int32 xyz2t(real8&, real8&, const slcimage&, orbit&, const cn&, int32, real8)’ specifies default arguments and isn’t a definition [-fpermissive]
164 | friend int32 xyz2t(
| ^~~~~
orbitbk.hh:174:18: error: friend declaration of ‘int32 xyz2lp(real8&, real8&, const slcimage&, orbit&, const cn&, int32, real8)’ specifies default arguments and isn’t a definition [-fpermissive]
174 | friend int32 xyz2lp(
| ^~~~~~
orbitbk.hh:184:18: error: friend declaration of ‘int32 ell2lp(real8&, real8&, const input_ell&, const slcimage&, orbit&, real8, real8, real8, int32, real8)’ specifies default arguments and isn’t a definition [-fpermissive]
184 | friend int32 ell2lp(
| ^~~~~~
orbitbk.hh:197:18: error: friend declaration of ‘int32 lp2ell(real8, real8, const input_ell&, const slcimage&, orbit&, real8&, real8&, real8&, int32, real8)’ specifies default arguments and isn’t a definition [-fpermissive]
197 | friend int32 lp2ell(
| ^~~~~~
processor.cc: In function ‘int main(int, char**)’:
processor.cc:363:15: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result]
363 | system(cmd);// this does the work
| ~~~~~~^~~~~
processor.cc:1204:15: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result]
1204 | system(cmd);// This does the actual work
| ~~~~~~^~~~~
processor.cc:2208:41: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result]
2208 | if (input_i_fine.plotoffsets) system(cmd);
| ~~~~~~^~~~~
processor.cc:2399:37: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result]
2399 | if (input_i_coregpm.plot) system(cmd);
| ~~~~~~^~~~~
processor.cc: In function ‘void handleinput(int, char**, input_gen&)’:
processor.cc:3641:17: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result]
3641 | system("helpdoris");
| ~~~~~~^~~~~~~~~~~~~
processor.cc:3650:17: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result]
3650 | system(cmd);
| ~~~~~~^~~~~
processor.cc: In function ‘void preview(int32, int32, int32, const char*, const string&, const string&)’:
processor.cc:4170:11: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result]
4170 | system(DEBUG.get_str());
| ~~~~~~^~~~~~~~~~~~~~~~~
processor.cc:4175:11: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result]
4175 | system(INFO.get_str());
| ~~~~~~^~~~~~~~~~~~~~~~
make: *** [Makefile:158: processor.o] Error 1

system information
uname -a
Linux panicker 5.4.0-51-generic #56-Ubuntu SMP Mon Oct 5 14:28:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal

g++ --version
g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I installed ubuntu 20.04.
I installed fftw-3.2.1
I cant install doris.
my problem when I install Doris-5.0.3Beta is bk_messages.hh:214:26: error: invalid conversion from ‘char’ to ‘const char*’ [-fpermissive].
i tried various type of doris, but not installed.

While installing the sar_tools I get that
dateconv.cc: In function ‘int main(int, char**)’:
error: invalid conversion from ‘char*’ to ‘size_t’ {aka ‘long unsigned int’} [-fpermissive]
51 | strftime(indatestring,informat,&tm_tmp);
| ^~~~~~~~
| |
| char*
dateconv.cc:51:34: error: cannot convert ‘tm*’ to ‘const char*’
51 | strftime(indatestring,informat,&tm_tmp);
| ^~~~~~~
| |
| tm*
In file included from /usr/include/pthread.h:14,
from /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/include/c++/x86_64-pc-cygwin/bits/gthr-default.h:35,
from /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/include/c++/x86_64-pc-cygwin/bits/gthr.h:148,
from /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/include/c++/ext/atomicity.h:35,
from /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/include/c++/bits/ios_base.h:39,
from /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/include/c++/ios:42,
from /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/include/c++/ostream:38,
from /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/include/c++/iostream:39,
from dateconv.cc:9:
/usr/include/time.h:67:49: note: initializing argument 3 of ‘size_t strftime(char*, size_t, const char*, const tm*)’
67 | size_t _maxsize, const char *__restrict _fmt,
| ~~~~~~~~~~~~~~~~~~~~~~~^~~~
make: *** [: dateconv.o] Error 1

and it doesn't run the make command for the sar_tools.
Does anyone know how to fix it?

When I lunch this script, at the end of it execution, it crashes with the follow message error:
Traceback (most recent call last): File "prepare_datastack_main.py", line 18, in <module> prepare_data_stack.prepare(args.doris_input_file) File "/home/giovanni/doris-work/prepare_stack/prepare_datastack.py", line 51, in prepare password=srtm_password, username=srtm_username) File "/home/giovanni/doris-work/prepare_stack/create_dem.py", line 63, in create latlim, lonlim = self._kml_shp_2_bb(shape_filename) File "/home/giovanni/doris-work/prepare_stack/create_dem.py", line 419, in _kml_shp_2_bb dat = shapes[0]['geometry']['coordinates'] IndexError: list index out of range
How I can resolve it and start the processing of my images?

Hi DORIS Users,

When I tried to execute DORIS for Sentinel data, I am facing the following issue.

RAW_DATA_ABSOLUTE_PATH= /home/suresh/Downloads/doris/stack/20180325/swath_3/burst_4/rsmp_orig_slave_pixel.raw
Warning 4: Unable to open rsmp_orig_slave_pixel.r00 ... skipping.

ERROR 4: MFF header file read successfully, but no bands
were successfully found and opened.
Traceback (most recent call last):
File "/home/suresh/Downloads/Doris_master/doris_stack/functions/do_reramp_SLC.py", line 71, in
ChirpFilt = get_ramp(resFilename, resampled=1, type='chirp')
File "/home/suresh/Downloads/Doris_master/doris_stack/functions/get_ramp.py", line 82, in get_ramp
PixRgGrid = freadbk(Path_MFF_HDR,1, 1,int(Naz_res),int(Nrg_res))
File "/home/suresh/Downloads/Doris_master/doris_stack/functions/get_ramp.py", line 263, in freadbk
print 'Could not open'+Path_MFF_HDR
NameError: global name 'Path_MFF_HDR' is not defined
RAW_DATA_ABSOLUTE_PATH= /home/suresh/Downloads/doris/stack/20180325/swath_3/burst_5/rsmp_orig_slave_pixel.raw
Warning 4: Unable to open rsmp_orig_slave_pixel.r00 ... skipping.

stack folder is /home/suresh/Downloads/doris/stack
burst is swath_3_burst_5_swath_3_burst_6
ps select is 1
max baseline is 200
master date is 2018-02-17
type ESD is ps
reading metadata from /home/suresh/Downloads/doris/stack/20180325/swath_3
/home/suresh/Downloads/doris/stack/20180325/swath_3
Traceback (most recent call last):
File "/home/suresh/Downloads/Doris_master/doris_stack/functions/ESD_ps_ds.py", line 470, in
save_overlapping(stack_folder, master_date, dates, overlap)
File "/home/suresh/Downloads/Doris_master/doris_stack/functions/ESD_ps_ds.py", line 69, in save_overlapping
slave_1 = freadbk(burst1 + 'slave_rsmp_reramped.raw', line_start, first_pixel_this, line_length, pixel_length , 'complex64', this_nr_oflines, this_nr_ofpixels)
File "/home/suresh/Downloads/Doris_master/doris_stack/functions/ESD_functions.py", line 84, in freadbk
file_dat = np.memmap(path_file, dtype=dt, mode='r', shape=(lines, pixels))
File "/usr/lib/python2.7/dist-packages/numpy/core/memmap.py", line 221, in new
fid = open(filename, (mode == 'c' and 'r' or mode)+'b')
IOError: [Errno 2] No such file or directory: 'burst_5/slave_rsmp_reramped.raw'
Traceback (most recent call last):
File "/home/suresh/Downloads/Doris_master/doris_stack/main_code/doris_main.py", line 28, in
doris_sentinel_1.run(args.parameterfilepath, start_date, end_date, master_date)
File "/home/suresh/Downloads/Doris_master/doris_stack/main_code/doris_sentinel_1.py", line 122, in run
processing.esd()
File "/home/suresh/Downloads/Doris_master/doris_stack/main_code/single_master_stack.py", line 1288, in esd
diff_m = np.load(os.path.join(f, esd_type + '_diff_matrix.npy'))
File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 370, in load
fid = open(file, "rb")
IOError: [Errno 2] No such file or directory: '/home/suresh/Downloads/doris/stack/esd/swath_3_burst_4_swath_3_burst_5/ps_diff_matrix.npy'

Kindly help me with the issue.

Thank you in advance.
Regards,
Suresh Devaraj

Hi,
when I install sar_tool
I face the error
cpxfiddle.cc:1857:21: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
if (argv[optind]=='\0')