Coder Social home page Coder Social logo

drowe67 / codec2 Goto Github PK

View Code? Open in Web Editor NEW
145.0 145.0 25.0 17.32 MB

Open source speech codec designed for communications quality speech between 700 and 3200 bit/s. The main application is low bandwidth HF/VHF digital radio.

License: GNU Lesser General Public License v2.1

CMake 1.92% C 89.79% MATLAB 6.03% Python 0.36% Shell 0.83% Assembly 0.54% Makefile 0.52%

codec2's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

codec2's Issues

WP4000 C Port of Modem

When the Octave simulations and OTA testing have shown the new modem waveform is useful, port the modem to C. Note that for a complete Low SNR FReeDV mode we also need a candidate speech codec (WP2000). This WP deals only with the C port of the modem code alone

  • Shall we still use the Octave->ofdm_xxx->freedv_xx workflow?
  • define ctests we need to pass
  • obtain/develop a new LPDC code that includes payload and txt bits, so that txt bits are now protected (alternative to reliable text system).
  • perform port
  • review code
  • review corrections
  • PR is merged

Reviewers: David and Mooneer

Deliverables:

  • list of ctests and pass criteria
  • ofdm_xx.[ch], ofdm_xxx.c ported and passing ctests
  • merged PR

WP2030 - sine.c cleanup

  • review source code and comments
  • Hamming -> Hann window
  • some tests, e.g. estimate amplitude on sine wave, analysis/synthesis test with contrived pitch
  • check variable names match codec2.pdf
  • consider making trapezoidal window function a triangular function, as I think with TW == 0.05 that is effectively what we are doing.
  • consider making notation a bit more consistent, e.g. n for time index, k for DFT index, m for harmonic index
  • In v est function distinguish Am from model->A
  • decide between phi (source code) for phase or theta (latex), it should be consistent

WP4000 - ch tool upgrade

ch is a C channel simulation tool. To simulate multipath channel is requires Doppler spreading samples. It currently uses float files of these samples generated by Octave "off line" at build time.
Add on the fly generation of Doppler samples
Use doppler_spread Octave code in freedv_low study, as this fixes some bugs
in current codec2/octave version.
Port Octave doppler_spread function to C. Will require run time generation of filter coeffs (see misc/rect_ft), or a fixed lowFs and fixed filter coeffs. Fixed lowFs might be OK for C.
Place C code in codec2_ch.[ch]. Part of move to a channel simulation library.
Write ctest that compares Octave and C version, e.g. same power spectrum for same input parameters. We wouldn't expect the time domain samples to be identical. Test doppler spectra for MPG, MPP, MPD.
Refactor ch to use on the fly Doppler sample generation.
Check that it can generate samples on the fly (in real time) with modest use of CPU on a typical laptop (few % of CPU).
Modify ctests to remove off line doppler spread sample generation

WP4000 - COTS Radio Tuning

FreeDV development is performed using simulated channel that assume an "ideal" HF radio with flat frequency response and a linear PA. In practice radios and how they are connected to FreeDV varies widely. In this WP we will work out how to maximize the Tx and Rx performance of FreeDV signals using a selection of COTS radios.

We have some general guidelines that we have developed over the years (e.g. just tickle AGC). We are not sure if additional Tx or RX filtering helps of hinders FreeDV. Should AGC be on or off? What is the effect of over driving the Tx, is more better or worse? Can we damage radios with over driving FreeDV, or do they shut down gracefully? Do common radios have a flat Tx spectrum so power is allocated evenly over all carriers?

  • Write a test plan for evaluating freedv-gui with COTS radios, for example how will radio be tested and how we will measure performance? Maximum RMS power and min PER for a given noise of channel. Can we simulate a HF channel in the Rx, e.g. the effect of impulse noise of AGC? Consider separate Tx and Rx tests, e.g. optimise Tx for best RMS power/spectral flatness.
  • Select a range of radios to test (5?).
  • Build test bench (hardware/software) required to test a radio with freedv-gui, for example dummy loads, power sampler/ Can we simulate a HF channel in the Rx, for example using a SDR to generate a simulated HF signal for Rx testing.
  • Is this something we could collaborate on with other test labs, e.g. ARRL?
  • Acquire radios for test. Evaluate performance of selected radios.
  • Debug and improve performance if possible by adjusting radio. Log any issues that could be improved by FreeDV work.
  • Document how best to configure radio for maximum performance
  • Share results e.g. blog posts/conference presentation

WP2040 - phase.c clean up

  • align codec2.pdf and phase.c variable names
  • move postfilter.[ch] into phase.c
  • new name for background noise postfilter to avoid confusion with spectral amplitude postfilter. This will mean some refactoring in c2sim.c and codec2.c

WP2010 - Codec 2 Algorithm Unit Tests

Following on from #20, some ctests to help explain/illustrate algorithm, and support refactoring/code clean up, and test ports to other (non C99) compilers, fixed point, FPGA etc.

See also WP #32 #33 for ideas on tests. There may be existing ctests that cover some of these topics (especially quantisation)

  • sinusoidal analysis
  • pitch estimation and refinement
  • overlap add synthesis
  • quantisation, e.g. LSP/LPC & 700C (newamp1)

Deliverables

  1. ctests

WP2007 - Review Codec 2 Algorithm Description

Review the document created in #20

  • typos
  • spelling
  • Optional: math, general layout, missing information on algorithms in Section 2 & 3, need for more figures or better explanation
  • get ctest -E test_codec2_doc to work as a GitHub action

WP2060 - Subjective Testing

For WP4000 Low SNR mode, we need a speech codec with improved quality compared to Codec 2 700C (note the FreeDV 700D/E modes using Codec 2 700E vocoder mode for the codec). In this WP, the speech quality of the new candidate 700 bits/s mode will be tested.

  • Decide on scope of test (e.g. number of samples)
  • Collect samples from on line speech databases
  • Collect samples from Hams in multiple languages (e.g. English, German, Japanese)
  • Perform Subjective tests using #24
  • Evaluate results
  • Decide if new candidate should proceed to integration with WP4000 modem

Using the fsk modem in external projects? fsk_api.c / fsk_api.h needed?

Hey team,

I've been playing with using the fsk.h and fsk library from libcodec2 for my selcall project. It's been working great as a modem for the project - many thanks for providing it.

I'm aware that this library likely isn't intended to be used in other projects like this. At the moment the usage is quite clunky, getting the user to compile libcodec2 from source and then setting CFLAGS="-I/path/to/codec2/src - this is because kiss_fftr.h isn't included in the installed header files when doing make install

I am wondering about the best way to approach this problem. Some options I see working are:

  • codec2 add kiss_fftr.h to the files exported to /usr/local/include/codec2
  • codec2 could provide an _api version of fsk, similar to the freedv_api approach which seems to work ok
  • I could vendor'ize the fsk specific components into my project

Thoughts?

WP2005 - Codec 2 Algorithm Description

A document describing how Codec 2 works

  • High level description using block diagrams and figures, average Ham level audience. This could be broken out for articles.
  • Maths based description
    • pitch estimation
    • sinusoidal analysis
    • voicing estimator
    • sinusoidal overlap add synthesis
    • LPC/LSP quantisation
    • newamp1 quantisation
    • post filtering
    • further work for document
  • Makefile or script to build doc, figures
  • github action to make sure we can keep building doc (should this be part of ctests?)

Deliverables

  1. Latex doc
  2. Merge PR

pkg-config includedir and source code includes do not match

Summary

If the codec2 library is not installed in the default library/include path, other projects cannot properly include it via pkg-config. E.g. I get include errors in my application on MacOS when having codec2 installed via brew.

  • using #include <config2/config2.h leads to an error, the file cannot be found.
  • using #include <config2.h> leads to an error inside config2.h: it cannot find <config2/version.h>

the pkg-config file defines the include path as
includedir=@CMAKE_INSTALL_PREFIX@/@CMAKE_INSTALL_INCLUDEDIR@/codec2
But config2.h itself refers to other files using the codec2/XY.h.

This mismatch does not cause a problem on most Linux distress, as the lib will be installed inside the default include dir (/usr/include), so together with pkg-config, codec2.h and codec2/codec2.h and codec2/version.h are in the include path

Steps to reproduce
Install codec2 in a non-default path. E.g. $HOME/.
Try to compile this application.

#include <codec2/codec2.h>
int main() {
  return 0;
}

Proposed fix

  1. in codec2.pc, includedir could be set to @CMAKE_INSTALL_PREFIX@/@CMAKE_INSTALL_INCLUDEDIR@. Then, all applications must include codec2/codec2.h. This would be a breaking change to the existing include behaviour, some applications currently could include codec2.h

  2. We could add both paths to the include dir: Cflags: -I${includedir}:${includedir}/codec2. This keeps the existing include behaviour and fixes the issue for non-standard install paths.

I created a pull request for the second option.

New Dataset

I want to compress the test dataset of librispeech through codec2 to complete multiple bit rates, such 1.2kbps 2.4kbps 0.6kbps. How can I achieve this?

Possible LDPC VLA issues

Possible issues outlined in: #11

No action required at this time from FreeDV Team, as they relate to FreeDV features (e.g. 2020B/C, reliable text) that are likely to be deleted during the course of the ARDC funded project. Review in 12 months, or when new waveforms developed.

Data synchronization

I'm testing the data API right now to check if codec2 fits my intended purpose. I wrote a simple program similar to freedv_datac1_tx, that successfully generates a frame that freedv_datac1_rx can decode. So far, so good!

Now, to prepare attaching the code to data coming from a sound card, I added some random "noise" in front of the frame with random length. Fed with such data, however, freedv_datac1_rx can not always decode the frame. It depends on the number of bytes that I add as "noise". It can deal with a few bytes, or with a number of bytes that is divisible by 200 (other patterns may be possible, but that's what I saw), then the decoder can synchronize to the start of the frame and decode it. Other random numbers of bytes lead to the decoder seemingly not being able to find the frame, and it doesn't get decoded.

I wonder if this has to do with freedv_nin? How would you attach the decoding methods to a real-world sound card, where you don't know where a frame begins? Can FreeDV reliably synchronize to such a random stream?

modem_stats speed up

In modem_stats.c we are using a FFT size of 2*MODEM_STATS_NSPEC , I think we could use a real fft (fftr) and halve that, reducing CPU by at least half.

Help

(base) rian0532@DESKTOP-PQV8NDO:/mnt/e/000/codec2/build_linux$ aplay -f S16_LE hts1a_c2_2400.raw
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5701:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM default
aplay: main:831: audio open error: No such file or directory

WP4000 - Study task list

Place holder for misc detailed tasks in #19

Deliverable is freedv_low study (Latex doc equalisation.m, acquisition.m):

  • Acquisition - simulate additive sine waves
  • Simulate speech (SSB)
  • supplementary filtering (does filtering help or hinder), test acquisition, equalisation.
  • complete fine frequency section, establish target/threshold std dev, try combining information from two pilots, table of worst case for a few channels, probability of successful acquisition for std dev. Is it significant?
  • Modify equalisation.m sim to use any pilot value (e.g. Barker), (like acquisition.m). Currently assumes all pilots 1's. Update MSE expressions in Latex doc for any pilot value.
  • build up one or more of the nastier channel models from DRM. Add perf curves for these. We'd like to get a feel if our new waveform can handle long distance paths OK.
  • some automated tests to make sure simulation code doesn't break. A few checks from equalisation.m and acquisition.m would be enough, add GitHub action.
  • set up correct Ts, Tcp, Nuw etc to get Rb=1400 bit/s over the channel with 4ms CP
  • A few days looking into single carrier modems, now FCC has removed 300 baud limit. Pros and cons? PAPR reduction. How to deal with ISI. Any reduction in Lcp? Maybe a first pass simulation. Its need RN filtering, what is PAPR of RN filtered QPSK?
  • discussion of wingman pilots. Decn - do we keep them? They add 2Rs to the bandwidth. I imagine we can get away without them. They add annoying (Nc+2) factors everywhere.
  • document new waveform parameters (e.g. in spreadsheet or latex doc). Diagram of modem frame showing pilots, wingman pilots.

WP9000 - Optimisation

Place holder for the WP9000 CPU Load Optimsations WP, which we can extend to other optimisations such as run-time memory.

We are currently designing against a minimum spec of a generic stm32f405 like we use on the SM1000, which IIRC is 192k RAM/512kbytes flash.

Via email @silseva has provided some fine guidelines for RAM optimisation, something I hadn't considered until now:

  1. The current codec2 implementation takes around 32kB of heap and 15kB of stack; this means we have to dedicate 48kB of RAM only for the codec itself. (Note this is probably Codec2 3200)
  2. Flash usage is not of great concern because MCUs tend to have way more flash space than RAM space (Note this is good - as we would like to use large VQs in new Codec modes)
  3. If possible, it'd be good to have the encode and decode functions to use as less stack as possible. This because some RTOSes does not allow for dynamic allocation of the threads stack, which means that we'd need to reserve a 16kB chunk in the .bss section (which is statically allocated at compile time) for the codec2 thread stack and this leads to an inefficient usage of the RAM space.

Enabling libasan causes failures during tests

@barjac reported an AddressSanitizer error during freedv-gui build:

[100%] Built target freedv_datac0c1_rx
make[1]: Leaving directory '/home/baz/build/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/build_linux'
/usr/bin/cmake -E cmake_progress_start /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/build_linux/CMakeFiles 0
+ cd src
+ export LD_LIBRARY_PATH=/home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/LPCNet/build_linux/src
+ LD_LIBRARY_PATH=/home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/LPCNet/build_linux/src
+ ./freedv_tx 2020 /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/LPCNet/wav/wia.wav -
+ ./freedv_rx 2020 - /dev/null
=================================================================
==36243==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7ffefa65e990 at pc 0x000000402ffb bp 0x7ffefa6593d0 sp 0x7ffefa6593c8
READ of size 8 at 0x7ffefa65e990 thread T0
    #0 0x402ffa in my_get_next_tx_char /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/src/freedv_tx.c:46
    #1 0x7f66ff6b3c02 in freedv_comptx_2020 /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/src/freedv_2020.c:174
    #2 0x7f66ff6a575c in freedv_comptx /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/src/freedv_api.c:389
    #3 0x7f66ff6a5ea5 in freedv_tx /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/src/freedv_api.c:329
    #4 0x40284d in main /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/src/freedv_tx.c:205
    #5 0x7f66ff47c736 in __libc_start_call_main (/lib64/libc.so.6+0x23736)
    #6 0x7f66ff47c7f4 in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x237f4)
    #7 0x402ea0 in _start (/home/baz/build/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/build_linux/src/freedv_tx+0x402ea0)

Address 0x7ffefa65e990 is located in stack of thread T0 at offset 240 in frame
    #0 0x40231f in main /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/src/freedv_tx.c:59

  This frame has 3 object(s):
    [32, 36) 'opt_idx' (line 97)
    [48, 128) 'f2020' (line 66)
    [160, 256) 'my_cb_state' (line 187) <== Memory access at offset 240 is inside this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-use-after-scope /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864/codec2/src/freedv_tx.c:46 in my_get_next_tx_char
Shadow bytes around the buggy address:
  0x10005f4c3ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005f4c3cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005f4c3d00: cb cb cb cb 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005f4c3d10: 00 00 00 00 f1 f1 f1 f1 04 f2 00 00 00 00 00 00
  0x10005f4c3d20: 00 00 00 00 f2 f2 f2 f2 f8 f8 f8 f8 f8 f8 f8 f8
=>0x10005f4c3d30: f8 f8[f8]f8 f3 f3 f3 f3 00 00 00 00 00 00 00 00
  0x10005f4c3d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005f4c3d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005f4c3d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005f4c3d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005f4c3d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==36243==ABORTING
frames decoded: 0  output speech samples: 0
+ cd /home/baz/BLD/BLD_MgaX_FREEDV/freedv-git/SOURCES/freedv-1.9.9-202403221300-3e864

Additionally, it appears a large number of ctests fail with AddressSanitizer enabled. Building with the following on an aarch64 Ubuntu VM:

$ CFLAGS=-fsanitize=address LDFLAGS=-fsanitize=address cmake -DLPCNET_BUILD_DIR=../../LPCNet/build_linux -DUNITTEST=1 ..
$ make -j6
$ ASAN_OPTIONS=detect_odr_violation=0 make test
....
61% tests passed, 60 tests failed out of 155

Total Test time (real) = 252.27 sec

The following tests FAILED:
          1 - test_clang_format (Failed)
         13 - test_ch_papr (Failed)
         19 - test_COHPSK_modem_octave_port (Failed)
         20 - test_COHPSK_modem_AWGN_BER (Failed)
         21 - test_COHPSK_modem_freq_offset (Failed)
         23 - test_OFDM_modem_octave_port (Failed)
         24 - test_OFDM_modem_octave_port_Nc_31 (Failed)
         34 - test_OFDM_modem_700D (Failed)
         35 - test_OFDM_modem_700D_ldpc (Failed)
         36 - test_OFDM_modem_2020_ldpc (Failed)
         37 - test_OFDM_modem_AWGN_BER (Failed)
         38 - test_OFDM_modem_fading_BER (Failed)
         39 - test_OFDM_modem_phase_est_bw (Failed)
         41 - test_OFDM_modem_time_sync_2020 (Failed)
         42 - test_OFDM_modem_700E_AWGN (Failed)
         43 - test_OFDM_modem_2020B_AWGN (Failed)
         47 - test_OFDM_modem_datac1 (Failed)
         51 - test_OFDM_modem_datac13_octave (Failed)
         52 - test_OFDM_modem_datac4_ldpc_burst (Failed)
         53 - test_OFDM_modem_datac13_ldpc_burst (Failed)
         66 - test_freedv_api_1600 (Failed)
         68 - test_freedv_api_700D_backwards_compatability (Failed)
         69 - test_freedv_api_700D_speech (Failed)
         71 - test_freedv_api_700D_AWGN_BER (Failed)
         72 - test_freedv_api_700D_AWGN_BER_USECOMPLEX (Failed)
         76 - test_freedv_api_2020_to_ofdm_demod (Failed)
         77 - test_freedv_api_2020_from_ofdm_mod (Failed)
         78 - test_freedv_api_2020_awgn (Failed)
         79 - test_freedv_api_2020B_mpp (Failed)
         80 - test_freedv_api_2400A (Failed)
         81 - test_freedv_api_2400B (Failed)
         82 - test_freedv_api_800XA (Failed)
         83 - test_freedv_api_rawdata_800XA (Failed)
         84 - test_freedv_api_rawdata_2400A (Failed)
         85 - test_freedv_api_rawdata_2400B (Failed)
         86 - test_peak_levels (Failed)
         87 - test_peak_levels_lpcnet (Failed)
         98 - test_freedv_reliable_text_ideal_2020 (Failed)
        100 - test_freedv_reliable_text_fade_2020 (Failed)
        101 - test_memory_leak_FreeDV_1600_tx (Failed)
        102 - test_memory_leak_FreeDV_1600_rx (Failed)
        103 - test_memory_leak_FreeDV_700D_tx (Failed)
        104 - test_memory_leak_FreeDV_700D_rx (Failed)
        105 - test_memory_leak_FreeDV_700C_tx (Failed)
        106 - test_memory_leak_FreeDV_700C_rx (Failed)
        107 - test_memory_leak_FreeDV_FSK_LDPC_tx (Failed)
        108 - test_memory_leak_FreeDV_DATAC0_tx (Failed)
        109 - test_memory_leak_FreeDV_DATAC1_tx (Failed)
        110 - test_memory_leak_FreeDV_DATAC3_tx (Failed)
        111 - test_memory_leak_FreeDV_DATAC4_tx (Failed)
        112 - test_memory_leak_FreeDV_DATAC13_tx (Failed)
        113 - test_memory_leak_FreeDV_700E_tx (Failed)
        114 - test_memory_leak_FreeDV_2020_tx (Failed)
        115 - test_memory_leak_FreeDV_2020_rx (Failed)
        116 - test_memory_leak_FreeDV_2020B_tx (Failed)
        117 - test_memory_leak_FreeDV_2020B_rx (Failed)
        125 - test_vq_mbest (Failed)
        145 - test_freedv_data_raw_ofdm_datac4_burst_file (Failed)
        146 - test_freedv_data_raw_ofdm_datac13_burst_file (Failed)
        153 - test_demo_700d_python (Failed)
Errors while running CTest
Output from these tests are in: /home/parallels/freedv-gui/codec2/build_linux/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
make: *** [Makefile:91: test] Error 8

(More tests were failing without disabling the ODR detection. I disabled above since it seemed like a false alarm.)

Example of one of the ctest failures:

test 69
    Start 69: test_freedv_api_700D_speech

69: Test command: /usr/bin/sh "-c" "cd /home/parallels/freedv-gui/codec2/build_linux/src;
                            ./freedv_tx 700D ../../raw/ve9qrp_10s.raw - |
                            ./ch - - --No -20 |
                            ./freedv_rx 700D - /dev/null --squelch -2 -vv"
69: Test timeout computed to be: 1500
69: ch: Fs: 8000 NodB: -20.00 foff: 0.00 Hz fading: 0 nhfdelay: 0 clip: 32767.00 ssbfilt: 1 complexout: 0
69: =================================================================
69: ==49821==ERROR: AddressSanitizer: stack-use-after-scope on address 0xffffce666d80 at pc 0xaaaab4232d98 bp 0xffffce662bb0 sp 0xffffce662bc0
69: READ of size 8 at 0xffffce666d80 thread T0
69:     #0 0xaaaab4232d94 in my_get_next_tx_char /home/parallels/freedv-gui/codec2/src/freedv_tx.c:46
69:     #1 0xffff841c6a40 in freedv_comptx_ofdm /home/parallels/freedv-gui/codec2/src/freedv_700.c:266
69:     #2 0xffff841bd2a4 in freedv_comptx /home/parallels/freedv-gui/codec2/src/freedv_api.c:375
69:     #3 0xffff841bd984 in freedv_tx /home/parallels/freedv-gui/codec2/src/freedv_api.c:329
69:     #4 0xaaaab42328f0 in main /home/parallels/freedv-gui/codec2/src/freedv_tx.c:205
69:     #5 0xffff83f273f8 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
69:     #6 0xffff83f274c8 in __libc_start_main_impl ../csu/libc-start.c:392
69:     #7 0xaaaab4232c2c in _start (/home/parallels/freedv-gui/codec2/build_linux/src/freedv_tx+0x2c2c)
69: 
69: Address 0xffffce666d80 is located in stack of thread T0 at offset 128 in frame
69:     #0 0xaaaab42321ac in main /home/parallels/freedv-gui/codec2/src/freedv_tx.c:59
69: 
69:   This frame has 3 object(s):
69:     [32, 36) 'opt_idx' (line 97)
69:     [48, 144) 'my_cb_state' (line 187) <== Memory access at offset 128 is inside this variable
69:     [176, 256) 'f2020' (line 66)
69: HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
69:       (longjmp and C++ exceptions *are* supported)
69: SUMMARY: AddressSanitizer: stack-use-after-scope /home/parallels/freedv-gui/codec2/src/freedv_tx.c:46 in my_get_next_tx_char
69: Shadow bytes around the buggy address:
69:   0x200ff9cccd60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9cccd70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9cccd80: cb cb cb cb 00 00 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9cccd90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9cccda0: f1 f1 f1 f1 04 f2 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
69: =>0x200ff9cccdb0:[f8]f8 f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9cccdc0: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9cccdd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9cccde0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9cccdf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
69:   0x200ff9ccce00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
69: Shadow byte legend (one shadow byte represents 8 application bytes):
69:   Addressable:           00
69:   Partially addressable: 01 02 03 04 05 06 07 
69:   Heap left redzone:       fa
69:   Freed heap region:       fd
69:   Stack left redzone:      f1
69:   Stack mid redzone:       f2
69:   Stack right redzone:     f3
69:   Stack after return:      f5
69:   Stack use after scope:   f8
69:   Global redzone:          f9
69:   Global init order:       f6
69:   Poisoned by user:        f7
69:   Container overflow:      fc
69:   Array cookie:            ac
69:   Intra object redzone:    bb
69:   ASan internal:           fe
69:   Left alloca redzone:     ca
69:   Right alloca redzone:    cb
69:   Shadow gap:              cc
69: ==49821==ABORTING
69: ch: SNR3k(dB):      nan  C/No....:      nan
69: ch: peak.....:     0.00  RMS.....:      nan   CPAPR.....:   nan 
69: ch: Nsamples.:        0  clipped.:      nan%  OutClipped:   nan%
69: frames decoded: 0  output speech samples: 0
69: 
69: =================================================================
69: ==49822==ERROR: LeakSanitizer: detected memory leaks
69: 
69: Direct leak of 9 byte(s) in 1 object(s) allocated from:
69:     #0 0xffffb5da8e30 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:454
69:     #1 0xaaaac0d02888 in main /home/parallels/freedv-gui/codec2/src/ch.c:152
69:     #2 0xffffb59473f8 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
69:     #3 0xffffb59474c8 in __libc_start_main_impl ../csu/libc-start.c:392
69:     #4 0xaaaac0d0442c in _start (/home/parallels/freedv-gui/codec2/build_linux/src/ch+0x442c)
69: 
69: SUMMARY: AddressSanitizer: 9 byte(s) leaked in 1 allocation(s).
1/1 Test #69: test_freedv_api_700D_speech ......***Failed  Required regular expression not found. Regex=[frames decoded: 62  output speech samples: 7
]  1.11 sec

A quick look seems to indicate that it's due to the following definition for my_cb_state (in both freedv_rx and freedv_tx):

  } else {
    /* set up callback for txt msg chars */
    struct my_callback_state my_cb_state;
    sprintf(my_cb_state.tx_str, "cq cq cq hello world\r");
    my_cb_state.ptx_str = my_cb_state.tx_str;
    my_cb_state.calls = 0;
    freedv_set_callback_txt(freedv, NULL, &my_get_next_tx_char, &my_cb_state);
  }

This definition should be moved to the top of main() to eliminate this error.

How to test VHF data packet mode using command line?

$ ./src/freedv_data_tx 2400A - --frames 15 | ./src/freedv_data_rx 2400A - as mentioned in the example on README_data.md simply returns :
frames decoded: 0

What is the source of the data that goes into freedv_data_tx?

The freedv_data_raw_tx and freedv_data_raw_rx work more or less according to the description.

WP4000 - Top Level Plan

graph TD;
    A[<a href='https://github.com/drowe67/codec2/issues/19'>#19</a> Study]-->B[<a href='https://github.com/drowe67/codec2/issues/19'>#17</a> Study Task List];
    B-->C[<a href='https://github.com/drowe67/codec2/issues/19'>#18</a> Octave Simulation];
    C-->D[<a href='https://github.com/drowe67/codec2/issues/15'>#18</a> Controlled OTA Testing];
    E[<a href='https://github.com/drowe67/codec2/issues/15'>#16</a> ch tool upgrade]-->F;
    D-->F[<a href='https://github.com/drowe67/codec2/issues/15'>#23</a> C Port of Modem];

Embedded Codec2 warnings

I see the following compiler warnings when compiling ezDV:

[1221/1349] Building C object _deps/codec2-build/src/CMakeFiles/codec2.dir/fmfsk.c.obj
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fmfsk.c: In function 'fmfsk_demod':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fmfsk.c:186:9: warning: unused variable 'eye_max' [-Wunused-variable]
  186 |   float eye_max;
      |         ^~~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fmfsk.c:185:7: warning: unused variable 'neyeoffset' [-Wunused-variable]
  185 |   int neyeoffset;
      |       ^~~~~~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fmfsk.c:184:7: warning: unused variable 'neyesamp' [-Wunused-variable]
  184 |   int neyesamp;
      |       ^~~~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fmfsk.c:165:13: warning: unused variable 'k' [-Wunused-variable]
  165 |   int i, j, k;
      |             ^
[1229/1349] Building C object _deps/codec2-build/src/CMakeFiles/codec2.dir/dump.c.obj
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_Sn':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:124:60: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  124 |   for (i = 0; i < m_pitch / 2; i++) fprintf(fsn, "%f\t", Sn[i]);
      |                                                          ~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:126:66: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  126 |   for (i = m_pitch / 2; i < m_pitch; i++) fprintf(fsn, "%f\t", Sn[i]);
      |                                                                ~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_Sw':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:144:18: warning: implicit conversion from 'float' to 'double' to match other operand of binary expression [-Wdouble-promotion]
  144 |             10.0 * log10(Sw[i].real * Sw[i].real + Sw[i].imag * Sw[i].imag));
      |                  ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_Sw_':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:163:14: warning: implicit conversion from 'float' to 'double' to match other operand of binary expression [-Wdouble-promotion]
  163 |         10.0 * log10(Sw_[i].real * Sw_[i].real + Sw_[i].imag * Sw_[i].imag));
      |              ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_Ew':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:181:18: warning: implicit conversion from 'float' to 'double' to match other operand of binary expression [-Wdouble-promotion]
  181 |             10.0 * log10(Ew[i].real * Ew[i].real + Ew[i].imag * Ew[i].imag));
      |                  ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_softdec':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:197:60: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  197 |   for (i = 0; i < n; i++) fprintf(fsoftdec, "%f\t", softdec[i]);
      |                                                     ~~~~~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_model':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:214:36: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  214 |   sprintf(line, "%12f %12d ", model->Wo, model->L);
      |                               ~~~~~^~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:216:33: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  216 |     sprintf(s, "%12f ", model->A[l]);
      |                         ~~~~~~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:221:25: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  221 |     sprintf(s, "%12f ", 0.0);
      |                         ^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_quantised_model':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:244:36: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  244 |   sprintf(line, "%12f %12d ", model->Wo, model->L);
      |                               ~~~~~^~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:246:33: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  246 |     sprintf(s, "%12f ", model->A[l]);
      |                         ~~~~~~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:251:25: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  251 |     sprintf(s, "%12f ", 0.0);
      |                         ^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_phase':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:273:57: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  273 |   for (l = 1; l <= L; l++) fprintf(fphase, "%f\t", phase[l]);
      |                                                    ~~~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:274:62: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  274 |   for (l = L + 1; l <= MAX_AMP; l++) fprintf(fphase, "%f\t", 0.0);
      |                                                              ^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_phase_':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:290:59: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  290 |   for (l = 1; l <= L; l++) fprintf(fphase_, "%f\t", phase_[l]);
      |                                                     ~~~~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:291:62: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  291 |   for (l = L + 1; l < MAX_AMP; l++) fprintf(fphase_, "%f\t", 0.0);
      |                                                              ^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_snr':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:322:25: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  322 |   fprintf(fsnr, "%f\n", snr);
      |                         ^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_lpc_snr':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:336:28: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  336 |   fprintf(flpcsnr, "%f\n", snr);
      |                            ^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_Pwb':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:353:62: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  353 |   for (i = 0; i < FFT_ENC / 2; i++) fprintf(fpwb, "%f\t", Pwb[i]);
      |                                                           ~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_Pw':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:369:60: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  369 |   for (i = 0; i < FFT_ENC / 2; i++) fprintf(fpw, "%f\t", Pw[i]);
      |                                                          ~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_Rw':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:385:60: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  385 |   for (i = 0; i < FFT_ENC / 2; i++) fprintf(frw, "%f\t", Rw[i]);
      |                                                          ~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_weights':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:401:58: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  401 |   for (i = 0; i < order; i++) fprintf(fweights, "%f\t", w[i]);
      |                                                         ~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_lsp':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:417:53: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  417 |   for (i = 0; i < 10; i++) fprintf(flsp, "%f\t", lsp[i]);
      |                                                  ~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_lsp_':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:433:55: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  433 |   for (i = 0; i < 10; i++) fprintf(flsp_, "%f\t", lsp_[i]);
      |                                                   ~~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_mel':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:449:56: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  449 |   for (i = 0; i < order; i++) fprintf(fmel, "%f\t", mel[i]);
      |                                                     ~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_ak':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:481:55: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  481 |   for (i = 0; i <= order; i++) fprintf(fak, "%f\t", ak[i]);
      |                                                     ~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_ak_':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:497:57: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  497 |   for (i = 0; i <= order; i++) fprintf(fak_, "%f\t", ak_[i]);
      |                                                      ~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_Fw':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:513:55: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  513 |   for (i = 0; i < 256; i++) fprintf(ffw, "%f\t", Fw[i].real);
      |                                                  ~~~~~^~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_e':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:529:57: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  529 |   for (i = 0; i < 500 / 2; i++) fprintf(fe, "%f\t", e_hz[i]);
      |                                                     ~~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:531:59: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  531 |   for (i = 500 / 2; i < 500; i++) fprintf(fe, "%f\t", e_hz[i]);
      |                                                       ~~~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_sq':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:547:60: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  547 |   for (i = 0; i < m_pitch / 2; i++) fprintf(fsq, "%f\t", sq[i]);
      |                                                          ~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:549:66: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  549 |   for (i = m_pitch / 2; i < m_pitch; i++) fprintf(fsq, "%f\t", sq[i]);
      |                                                                ~~^~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_dec':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:565:60: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  565 |   for (i = 0; i < 320 / 5; i++) fprintf(fdec, "%f\t", Fw[i].real);
      |                                                       ~~~~~^~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_bg':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:580:32: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  580 |   fprintf(fbg, "%f\t%f\t%f\n", e, bg_est, percent_uv);
      |                                ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:580:35: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  580 |   fprintf(fbg, "%f\t%f\t%f\n", e, bg_est, percent_uv);
      |                                   ^~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:580:43: warning: implicit conversion from 'float' to 'double' when passing argument to function [-Wdouble-promotion]
  580 |   fprintf(fbg, "%f\t%f\t%f\n", e, bg_est, percent_uv);
      |                                           ^~~~~~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c: In function 'dump_E':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/dump.c:594:28: warning: implicit conversion from 'float' to 'double' to match other operand of binary expression [-Wdouble-promotion]
  594 |   fprintf(fE, "%f\n", 10.0 * log10(E));
      |                            ^
[1234/1349] Building C object _deps/codec2-build/src/CMakeFiles/codec2.dir/fsk.c.obj
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c: In function 'fsk_demod_core':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c:643:17: warning: unused variable 'neyeoffset' [-Wunused-variable]
  643 |   int neyesamp, neyeoffset;
      |                 ^~~~~~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c:643:7: warning: unused variable 'neyesamp' [-Wunused-variable]
  643 |   int neyesamp, neyeoffset;
      |       ^~~~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c:642:28: warning: unused variable 'eye_max' [-Wunused-variable]
  642 |   float meanebno, stdebno, eye_max;
      |                            ^~~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c: In function 'stats_init':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c:968:7: warning: unused variable 'M' [-Wunused-variable]
  968 |   int M = fsk->mode;
      |       ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c:967:7: warning: unused variable 'P' [-Wunused-variable]
  967 |   int P = fsk->P;
      |       ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c:966:13: warning: unused variable 'm' [-Wunused-variable]
  966 |   int i, j, m;
      |             ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c:966:10: warning: unused variable 'j' [-Wunused-variable]
  966 |   int i, j, m;
      |          ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/fsk.c:966:7: warning: unused variable 'i' [-Wunused-variable]
  966 |   int i, j, m;
      |       ^
[1250/1349] Building C object _deps/codec2-build/src/CMakeFiles/codec2.dir/modem_stats.c.obj
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/modem_stats.c: In function 'modem_stats_open':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/modem_stats.c:37:7: warning: unused variable 'i' [-Wunused-variable]
   37 |   int i;
      |       ^
[1270/1349] Building C object _deps/codec2-build/src/CMakeFiles/codec2.dir/lpcnet_freq.c.obj
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/lpcnet_freq.c: In function 'lpcnet_compute_band_energy':
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/lpcnet_freq.c:92:21: warning: implicit conversion from 'float' to 'double' to match other operand of binary expression [-Wdouble-promotion]
   92 |     bandE[i] = 10.0 * log10(sum[i]);
      |                     ^
[1271/1349] Building C object _deps/codec2-build/src/CMakeFiles/codec2.dir/ldpc_codes.c.obj
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/ldpc_codes.c:34:6: warning: missing initializer for field 'ldpc_data_bits_per_frame' of 'struct LDPC' [-Wmissing-field-initializers]
   34 |      (uint16_t *)HRA_112_112_H_rows, (uint16_t *)HRA_112_112_H_cols},
      |      ^
In file included from /Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/ldpc_codes.h:15,
                 from /Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/ldpc_codes.c:9:
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/mpdecode_core.h:34:7: note: 'ldpc_data_bits_per_frame' declared here
   34 |   int ldpc_data_bits_per_frame;
      |       ^~~~~~~~~~~~~~~~~~~~~~~~
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/ldpc_codes.c:39:6: warning: missing initializer for field 'ldpc_data_bits_per_frame' of 'struct LDPC' [-Wmissing-field-initializers]
   39 |      (uint16_t *)HRA_56_56_H_rows, (uint16_t *)HRA_56_56_H_cols},
      |      ^
/Users/mooneer/ezDV/firmware/build/_deps/codec2-src/src/mpdecode_core.h:34:7: note: 'ldpc_data_bits_per_frame' declared here
   34 |   int ldpc_data_bits_per_frame;
      |       ^~~~~~~~~~~~~~~~~~~~~~~~

This is likely due to compiling a couple of files that aren't necessary for 700D/E/1600 support (the SM1000 build explicitly enumerates each file brought in from Codec2 instead of just using CMake's project include functionality, so no warnings there). No action needed for now, just tracking in case we decide to fix this later.

memory affect in decoder

Using single codec2 instance, while trying to decode two streams of encoded voice from two different transmitters, decoded voice gets jitters/jerks/clutters, but if we setup two codec2 instances like this at receiver end

struct CODEC2 *c2_1 = codec2_create(CODEC2_MODE_3200);
struct CODEC2 *c2_2 = codec2_create(CODEC2_MODE_3200);

it solves the problem,
Is this expected behavior or I'm missing something here
decoder-problem

WP1000 - Top Level Plan

From our ARDC grant project plan, top level workflow to show how the planned work fits together. If anyone is interested, we can provide more details on each Work Package (WP).

FreeDV-028 ARDC Workflow Figure

WP4000 - Controlled OTA Testing

We'd like to find out how our prototype new waveform performs compared to (a) earlier FreeDV waveforms (b) SSB (c) on a range of channels. We'd like to evaluate performance as early as possible, as the next stages of C porting and FreeDV integration involve quite a lot of work. We need to know if we are on the right track.

In this WP we will transmit the new waveform over real HF channels, at the same time as samples of SSB and earlier waveforms, then evaluate the results.

  • Consider splitting this into several WP (it's getting big)
  • Decide what the metrics are - e.g. PER or actual Codec 2 voice? Any acquisition stats we want to measure?
  • Use Octave implementation of modem from #18
  • Transmit candidate waveform(s) OTA
  • Concatenate with control waveforms (e.g. 700D, 700E, SSB) so they are sent at approx the same time, which will mean the same channel.
  • Comply with station ID rules, e.g, have a station ID header, try not to Tx on top of any other stations (can we listen before we Tx?)
  • Script based, e.g. ota* at https://github.com/drowe67/codec2-dev/tree/master/unittest was used for similar testing, write up here Controlled FreeDV Testing
  • Design so the experiment can be repeated by other Hams, in order together data from different geographical locations. This means some documentation on setup, running, common issues. Perhaps reception by receivers other than KiwSDRs
  • Design so we can accept manually collected samples.
  • Consider (perhaps another WP) repeating experiment at different locations to capture any local issues, for example run in Japan and the UK to evaluate performance on NVIS channels common in those locations.
  • Test over HF, LEO, GEO channels
  • HF channels should include NVIS, 800-2000km, and multihop international (half way around globe), and polar paths. Can we break the modem on any channel?
  • Rx by one of more (Kiwi) SDRs
  • Automate, run over 1 week, 100s of samples
  • Collect data, measure PER, process, present for interpretation
  • Code (and data?) in it's own repo or sub-dir (not codec2/)

Deliverables:

  • code, documentation on how to set up and run in README, results.
  • Write up on Gdoc and/or FreeDV blog: summary of results, discussion, conclusions, further work, modem issues found (that need to be fixed)
  • Reviewed

FSK demodulator issue

Hello David!

I have been forwarded here from the wenet repository. I am having problems with the FSK demodulator.

Here's my original message from the wenet issue. I would appreciate it if you could shed some light onto this problem!

Hello!

Thank you for your wonderful software!

I am having problems with integrating the FSK demodulator from codec2 into my software. I have referenced your scripts multiple times, but I am unable to make any progress.

I am using the command nc -l -u localhost 7355 | ./fsk_demod --stats=100 -b 1 -u 23500 2 48000 100 - - 2> >(python3 test_read_1.py) | python3 test_read_2.py to receive audio from GQRX and then demodulate.

I'm expecting the stderr (i.e info from the demodulator) to go to test_read_1.py and the actual FSK data to go into test_read_2.py. Is that not right? I'm seeing nothing from test_read_1.py and only the stderr from test_read_2.py.

Here are the files, by the way:

test_read_1.py:

import sys

while True:
    data = sys.stdin.readline()
    print(data)
    print("RECEIVED DATA NO 1!")

test_read_2.py:

import sys

while True:
    data = sys.stdin.buffer.read(256) #Read 256 bytes
    print("RECEIVED DATA NO 2!")

I have been struggling for a few weeks now, and I would extremely appreciate any insight into this issue!

WP2050 - Subjective Testing System

Develop a subjective testing system, e.g. paired comparison tests, P808, or MUSHRA, in order to evaluate the speech quality of new Codec 2 modes.

  • Decide on test methodology, "P.808 : Subjective evaluation of speech quality with a crowdsourcing approach" looks promising.
  • Can it be run online, e.g. off a web site? It should at least be easily reproducible by groups of Ham in other countries, to perform language specific testing.
  • Can we develop and run such a system with modest resources available?
  • Develop software to run test and statistical post processing
  • Document how to install and run the tests (e.g. for Hams in other countries)
  • Practice run

WP2025 - static codebook files

Currently several codebook files are generated at build time from .txt templates. This works well on Linux like systems, and means we "don't repeat ourselves) (DRY principal) but its a pain point for builds on other operating systems. It means that not all of the C files are present in the repo.

Lets consider/discuss having all C files present in the repo, and do the .txt -> .c conversion at "development time". The files are only updated very rarely (some haven't changed for a decade).

Might be worth waiting until the new modes have settled, or include this in other refactoring work.

Take into account the PLT policy "We have standardized on C99 and develop and test using gcc on a Linux platform". Does this proposal conflict with that? The main purpose of that policy to to prevent open ended maintenance of non C99/non-Linux builds that distract resources away from core work that only we can do.

This should be discussed by the development team, and a consensus reached, before any work is started.

STM32 unit test failures

The following failures were discovered on main during work on #12:

The following tests FAILED:
	  5 - tst_ofdm_mod_plain (Failed)
	  6 - tst_ofdm_mod_ldpc (Failed)
	  7 - tst_ofdm_demod_quick (Failed)
	  8 - tst_ofdm_demod_ideal (Failed)
	  9 - tst_ofdm_demod_AWGN (Failed)
	 10 - tst_ofdm_demod_fade (Failed)
	 11 - tst_ofdm_demod_ldpc (Failed)
	 14 - tst_codec2_enc_1300 (Failed)
	 15 - tst_codec2_enc_700C (Failed)
	 21 - tst_api_mod_700E_CODEC (Failed)

(Note: these were run on a NUCLEO-F446RE dev board.)

More detailed test output at stm32-ctest-log.txt.

WP4000 - Octave Simulation

Using the recommendations from the freedv_low study, modify the OFDM modem Octave simulation to run the new waveform. This should be an option, i.e. don't break support for existing waveforms.

  • Decide on waveform name. We might need a series, e.g. rc1,rc2
  • Is Octave->ofdm_xx->freedv_xx still the best development process to use? Might depend on other help we get.
  • fix codec2/octave/doppler_spread (see changes made to freedv_low) version
  • ofdm_tx, ofdm_rx functional test working with new waveform (e.g. AWGN at Eb/No=4dB), spot check of BER
  • ofdm_tx, ofdm_rx performance test working with new waveform on AWGN, MPP, MPD. BER/PER curves. Similar BER/PER to freedv_low study.
  • support Nd1, Nd=2. Show improved performance with Nd=2.
  • ofdm_ldpc_tx,ofdm_ldpc_rx
  • new ctests to sopt check he new waveform

WP2020 - NLP cleanup

Clean up nlp.c clean up and add unit tests (ctests) to support

  • remove todos
  • review comments
  • remove profiling marcos (I don't think they are used any more)
  • remove 8-16 kHz resampling functions and 16 kHz support (we can discuss but I don't think it's needed any more)
  • come up with some unit tests, e.g. to support ports to other platforms. Consider tests that have a statistical pass criteria, a floating point error may cause non-exact output on different machines. For example we could count gross errors over a sample file against had estimates, and/or use contrived signals such as codec2-dev/estn0
  • unit test for pitch refinement. Make sure it really is refining. These days we probably have enough CPU for the full MBE cost function method. Optional - determine if we still need refinement.
  • align variable names with codec.pdf
  • bring plnlp.m back from codec2-dev, edit it, add instructions for it's use

WP1000 - Codec 2 Policies

Write up Policies that have been approved at FreeDV PLT level.

  • Decide where to document (e.g. in codec2/README.md?)
  • Please discuss with David first any work before effort is put into a PR - needs to be focused on core project aims
  • See also PLT meeting notes on Polices - what goes into the codec2 repo (libcodec2) and what doesn't
  • We are only interested in C99/gcc compilable code - support and maintenance for other compilers (e.g. MSVC) should go in forked repos that are maintained by others. We have finite need to focus on what's unique about this project (the DSP)
  • It's not codec2 if the ctests don't pass/aren't ported.

References: David's Sep 2023 report

Deliverable: Document
Reviewers: David & Mooneer

CMake and Ninja - "missing and no known rule to make it"

I was trying to import libcodec2 into my project. After importing, there was an error:
ninja: error: 'E:/Android/codec2_talkie-master/libcodec2-android/build/imported-lib/arm64-v8a/libcodec2.so', needed by 'E:/Android/codec2_talkie-master/libcodec2-android/build/intermediates/cxx/Debug/4gx1kb3u/obj/arm64-v8a/libCodec2JNI.so', missing and no known rule to make it

I was confused because I run 3 projects(Codec2Talkie、CodecAndroidMaster and my project) on my laptop , all of them have the ninja error, anyone know why it happend? Thanks alot

WP4000 - Study

Deep dive into OFDM modem algorithms, use math and simulation to seek improvements over FreeDV 700D/E. After the initial iteration, seek out a suitable reviewer for review.

  • Outline key requirements
  • Model for voice link
  • Equalisation
  • Multipath
  • Intersymbol interference
  • Timing estimation
  • PAPR reduction
  • Acquistion
    • Coarse timing
    • Coarse freq
    • UW post processing
    • state machine
    • Fine freq estimation and tracking
    • CPU load estimates
  • Consideration of single carrier (removal of 300 baud limit)
  • Automated tests to prevent bit rot of simulations

Refs

  1. #17
  2. freedv_low repo for the study

Deliverables

  • Latex document
  • Octave simulations
  • README summarizing how to run simulations and build Latex doc

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.