aerospaceresearch / directdemod Goto Github PK

The PLL may go out of sync and/or may rotate the constellation or flip it etc. This also changes the output bitstream. Even if this happens, we must ensure that the syncs are detected.

To ensure that, all possible "needles" or "syncs" have to be initialized and used for correlation. The purpose of this issue is to just verify that all possibilities have been considered.

Bug, feature request, or proposal:

Feature Request

What is the expected behaviour?

We should be able to execute the main.py file from anywhere in the system/terminal. For example:

[email protected] ~/ $ DirectDemod -c 137000000 -f 137100000 -d noaa “file.wav”

What is the current behavior?

Right now, we need to be inside the repository to execute the command. For example:

[email protected] ~/home/DirectDemod $ python main.py -c 137000000 -f 137100000 -d noaa “file.wav”

What is the use-case or motivation for changing an existing behaviour?

It will allow the program to be executed from anywhere in the system, and other programs can also leverage this tool more easily then. It will also make this tool more finished and systematic.

Is there anything else we should know?

This feature won't affect the current execution of program, it will just make it accessible throughout the system. so, eventually this step will make this project more ready for production.

most filters need more samples. these samples also range outside the current chunk into the next chunk. please regard it here and in further similar implementations.

the .wav files we imported so far were recorded by SDR# programme.

the rtlsdr community uses also .dat files with the raw iq baseband output from the rtlsdr dongle.
DirectDemod shall be able to also import raw iq baseband. Then we also need to have either files with meta data of the recording, or the user need to provide the input data like center frequency, sample rate and so on that is stored in the .wav file.

The Sync flags are reference markers within the signals. All following pixel informations are alligned and distrubted in between them.
Due to two reasons the the pixels "size" is not equally distrbuted with respect to time.

1. due to doppler shift
2. sample drops

by knowing the syn c flag positions, the doppler shift effect can be compensated by having difference bin sizes per pixel.
and also in case of drops of samples the effect can be mildered by this method.

So, an optional adaptive pixel size would be an enhancement.

pixel_size_per_step = pixel_count / (flagB_position - flagA_position)

Note:
This cna be done AFTER the previous issue with knowing the position of the syn flag with respect to the image grid. these are the known positions!

When this is used as an input to the program.
python main.py -f 137100000 -d noaa SDRSharp_20170830_073907Z_145825000Hz_IQ_autogain.wav
we get an error which is hard to understand.
ERROR An error occured during decoding of frequency 1 of 1 frequencies
ERROR The error is: zero-size array to reduction operation minimum which has no identity
It should provide an error like "No NOAA data found".

When executing

sudo docker run --rm -ti -v 'pwd':/opt/code direct_demod python main.py sam
ples/SDRSharp_20170830_073907Z_145825000Hz_IQ_autogain.wav

following error occurs

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    from scipy.io.wavfile import write as sc_write
ModuleNotFoundError: No module named 'scipy'

After adding 'scipy' to requirements.txt, a new error occurs:

Traceback (most recent call last):
  File "main.py", line 282, in <module>
    plt.plot(binary_filter, label="binary filter")
  File "/usr/local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 3250, in plot
    ax = gca()
  File "/usr/local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 962, in gca
    return gcf().gca(**kwargs)
  File "/usr/local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 592, in gcf
    return figure()
  File "/usr/local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 539, in figure
    **kwargs)
  File "/usr/local/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 171, in new_figure_manager
    return cls.new_figure_manager_given_figure(num, fig)
  File "/usr/local/lib/python3.6/site-packages/matplotlib/backends/backend_tkagg.py", line 1049, in new_figure_manager_given_figure
    window = Tk.Tk(className="matplotlib")
  File "/usr/local/lib/python3.6/tkinter/__init__.py", line 2017, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

I tried adding the following lines to main.py (which uses AGG as a backend for matplotlib instead of default Xwindows) but the above error still persists.

import matplotlib
matplotlib.use('Agg')

Please look into this.

I have not seen any tests written for the code in this project.

Tests help us ensure nothing breaks that when we fix anything but also ensure everything is working as required.

@hornig if you think this is a good Idea, I will work on these during the application period of the the ESA Summer of Code in Space.

While launching from the binder badge, at https://gke.mybinder.org/v2/gh/aerospaceresearch/DirectDemod/Vinay_dev ,
the console shows:

Found built image, launching...
Launching Server...
Launch Attempt 1 failed, retrying...
Launch Attempt 2 failed, retrying...
Launch Attempt 3 failed, retrying...

There is no more info available from Binder.

adding an output (to json or other form) for the APT telemetry blocks.
would that be usefull?

https://sourceforge.isae.fr/projects/weather-images-of-noaa-satellites/wiki/Apt_

importing files with selectable samplerate (for .dat files,...).
peraps it is also usefull for wav in case the wav doesn't contain the meta data. perhaps possible when not recorded with SDRsharp.

I'm getting an error when running trying to decode noaa signals with this command:
main.py -c 137000000 -f 137100000 -d noaa "audio.wav"

I believe the error comes from line 623 in decode_noaa.py

Here is the output on the console:

INFO     Offset for this frequency was determined to be 100000.000000 Hz
INFO     Decoding NOAA data
INFO     Beginning FM demodulation to get audio in chunks
INFO     Processing chunk 1 of 1 chunks
ERROR    An error occured during decoding of frequency 1 of 1 frequencies
ERROR    The error is: too many indices for array```

currently, only Sync A is used for syncing the pixels of one line to the start of the image grid.
The Sync B in the middle of the image grid is not used so far.

In case of a sample frop during the first half of the line, the following half can be alligned as well. By this, distoration will only be effecive in the first half.

Signals with narrow bandwidth like CW and also 1200 BPSK (Funcube) are harder to decode due to the doppler shift and the rather high change of center frequency in relation to the bandwidth.

So it is suggested to add an automatic doppler shift correction in post processing. This shall allow to keep the signal within the bandwidth arround the expected center frequency.

It should be an optional feature due to not all signals are facing higher troubles when decoding the uncorrected signal.

The NOAA recordings of different stations should be mergable.
It will provide two advantages:

1. it will increase the image size and mage it bigger, because overlapping parts of each station adds up.
2. the pixels with several sources will be "stronger". This means that of one image line is noisy in the recording on station 1, it can be with lower noise at station 2 and decreasing the signal to noise level of the combined solution.

synchronization of the images can be achieved with 3 methods:

1. by taking the minute marker (only possible with longer recordings).
2. by using line matching. taking one line and find this pattern by correlation in the other recoring
3. by the dgsn gps synch method. this is only possible with DGSN stations.

please add more conversion tables for NOAA composite images. Currently on one differential IR image can be done with the right IR channels activated on the NOAA.

Please also include an official specification document, where it is explained (if possible). The current one is homebrew and does not neccesarily reflect the way it is officially been done.