Coder Social home page Coder Social logo

ETI output in SYNCH about eti-stuff HOT 25 CLOSED

jvankatwijk avatar jvankatwijk commented on June 14, 2024
ETI output in SYNCH

from eti-stuff.

Comments (25)

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi @JvanKatwijk ,

Please read my disection of the ETI-NI standard regarding this at
Opendigitalradio/dabtools#16 (comment)

I think the documentation explicitly explains what needs to be done to send NULL transmissions.
See section 5.9 of Page 27: http://www.etsi.org/deliver/etsi_i_ets/300700_300799/300799/01_30_9733/ets_300799e01v.pdf

In fact, this it's required to deliver correct ETI-NI frames. And it's quite easy to implement as I exposed (more or less, catch the exception of "no signal" [aka, no lock, impossible to decode, etc.] and generate a NULL packet).

I hope you'll like to implement it! 😄

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

basicmaster avatar basicmaster commented on June 14, 2024

@lars18th:
I indeed forgot about § 5.9 and the null transmission, and therefore added support for it to DABlin, to prevent any issues in case it occurs.

However I still don't see the actual reason why generating such ETI frames should be generated here or by the dabtools.

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi @JvanKatwijk ,

You generate a lot of messages, what is the objective?

The objective it's have the tools to decouple the DAB demodulator from the DAB decoder. So I need to have ETI source bitstreams (from satellite, from ODR-DabMod, or from "eti-cmdline-*"); and ETI consumers (like DABlin).

The main problem at time it's to standardise (in the open source community for DAB) the ETI as an interchange format. For example, at time onle two ETI players exists: DABlin and "eti-backend".

Hi @basicmaster ,

However I still don't see the actual reason why generating such ETI frames should be generated

The reason it's to improve the compatibility with any DAB decoder. If some LI frames are missed from the ETI bitstream, then the decode process will be more complex.

Hi @JvanKatwijk ,

It would require a redesign since generating eti data only takes place
when/if we can interpret the signal

OK. Now I know about the problem. So, I see two "simple" solutions that can be applyed:

  1. If a discontinuity is created then increment the COUNTER of the next delivered frame. This will permit to the decoder to see the discontinuity. And in order to detect the discontinuity you can use a simple var with last timestamp (aka previous frame), and compare with the current timestamp (aka current frame). Then compare if more than 24ms (*N) time is ellapsed.

  2. The second option it's wait to interpret the signal with a timeout. If more than 24ms are passed, then generate an empty frame and continue waiting for reading with the same timeout.

I know that this change isn't trivial. But it enables the use of the ETI format as an interchange format.
What you think?

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi @JvanKatwijk ,

As I described here http://github.com/AlbrechtL/welle.io/issues/217,
at time, I'm using a brute-force-workaround to do the decoupling of the DAB demodulator and the DAB decoder. The trick is use the ODR-DabMod tool for generating the IQ samples and a custom RTL_TCP server that reads these IQ samples and sends to the DAB player.

However, I have a problem: The implementation of the ODR-DabMod requires no missing ETI-NI frames. So, I found that extending the eti-cmdline with a safe output (empty or NULL frames if no signal or huge errors) can be a solution.

What you think about this?

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

How would you recognize that frames are missing

These are the three cases that I detected during my tests related to troubles with missing frames:

  • Discontinuity detection by the ETI parser.
  • Error in the ETI bitstream.
  • Silence added to the output of the ETI parser.

Please, note that the only reason to propose to generate NULL frames isto solve these particular cases!

Then how would we recognize that there are missing frames?

I'm not a DAB expert. But here is my proposal:

  • Case 1: Errors in the Input. In case of the demodulator detects uncorrectable errors, then it needs to try to resynch until a good signal is detected. In the meantime, the output of the demodulator should be NULL frames.
  • Case 2: Discontinuity in the Input. When the demodulator detects that a hole exists in the input, it should "repair" the hole sending a NULL frame to the output in between the other correct frames.
  • Case 3: Input missing. When no input is detected, the output will be NULL until a signal is found.

I don't know if this it's hard to implement or not. But I like to comment with you about if this helps or not to solve the problems.

What you think?

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

apart from errors in the eti bitstream, is there a clock in the accepting
software or is it a counter for the eti frames

This varies according to the implementation. However, as far as I can deduce, the DAB Player mainly uses a clock for the audio output, and a counter for the incoming bitstream. So, as the ETI-NI has a counter (the FSYNC 24bit counter generated by the sender, that is included in the SYNC header, the 4 initial bytes of each ETI-NI frame) the only requirement is to generate the NULL frames with the correct counter mark. If this is done correctly the DAB decoder (aka Player) can have a deterministic behaviour even if the ETI frames have errors.

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

are you sure that the only thing then that is to be included in inserted
frames is the "correct" frame counter? and all other fields set to zero?

I explained here: Opendigitalradio/dabtools#16 (comment)

As a summary for NULL frames:

  • 0x 0nnn FFFF FFFF FFFF FFFF ... 5555 5555 5555 (In HEX separated each 4 digits with spaces)
    -- The first '0' byte is the STAT field indicating a "full error";
    -- Where nnn is the SEQ number generated by the sender;
    -- At leat 16 bytes with '0xF' values in the LIDATA;
    -- And rest until completing the size of the ETI-NI packet setting to '0x55'.

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi @JvanKatwijk ,

I'm sorry! I want to be a gentleman.
I've probably misinterpreted what you wanted.

In order to understand your request: You like to know the exact structure and content of a NULL frame? Is this or something else?

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi @JvanKatwijk ,

I see you got the idea and started an experimental version:
d8d3826

Because I would like to collaborate (as much as I can)... please can you explain something about the changes?
In the code I can see: "It will try to insert missing frames as null frames, starting with an error code 0 (i.e. non repairable)."
So, this version will generate one NULL frame each time it detects "non repairable" data?
This change grants a CBR in the output? That is 6144bytes each 24ms?

My idea is do some tests and report the results.
You agree?

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi,

Regarding the SYNC output I have this new idea based on only two changes:

  • First, the ETI generator will be modified to generate NULL frames when the input is NULL. The input is the data comming from the input driver. If this data is zero, null, have uncorrectable errors or are out of synch, then is detected as NULL and correspondingly one NULL ETI frame is output.

  • The second change is to add and internal clock to the input driver. That's more simple than sounds: the thread that reads samples from the driver saves the timestamps of last reads. If any drift passes a limit then it flush the input driver with zeroes. This will ensure that the input driver will output data in synch.

So, the concept is: the ETI generator works all the time in a push mode. The constant bitrate in the output is granted when the input driver is running, even if the incoming data is lost or erroneous.

You feel this is a feasible solution?

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi,

For the first goal (NULL frames from the ETI generator) my testing environment will be this:

  • From an error free capture of IQ samples of a DAB ensamble, save around of 120 seconds. Then open the ".iq" file and edit it. In around the middle insert a lot of zeroes, until the size of the file increases around 10%. This will create a recording of around 132 seconds with a silence around 60' with a duration of ~12 seconds, and after it continues as regular.

Then when this file be processed by the eti-cmdline-sdr the expected output would be: Regular ETI frames at start, and when the "silence" appears a sequence of NULL ETI frames. And after the silence finishes the ensemble continues without any interruption. So, the output ETI-NI has a CBR and it's playable without errors in the backend (or DABlin).

I'm right?

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi @JvanKatwijk ,

It means writing the ETI generator as a separate clock driven thread...

I feel I'm not explained well for the first goal: The ETI generator will continue to be the same as now, working in push mode. The only difference is when it receives bad data... and not when it not receives data. I'm not an expert of the current code (as you can imagine), and it's possible I miss something. However, if the lower layers that send data to the ETI generator will pass zero data (all zeroes) when bad data is detected, then is possible to generate the NULL ETI frame.

Have you considered the example I mentioned?

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi @JvanKatwijk ,

Thank you for continuing to help me! 😄

Just look at the code

I'm doing it, but I feel it difficult. Some issues for clarifying to me:

  • https://github.com/JvanKatwijk/eti-stuff/blob/master/eti-cmdline/src/eti-handling/eti-generator.cpp#L147
    • Is the "etiGenerator" now not an independent thread?
    • And the other working thread is not the "ofdmProcessor"?
  • The current architecture is not like this?
    • OFDM-processor: gets data from driver and pass it to the ETI-generator.
    • ETI-generator: controls the FIC handler for obtaining the FIBs.
    • FIC-handler: controls the FIB processor for decoding the FIBs.
    • ETI-generator: when "cifVector" (≈CIF data) and "fibVector" (≈FIB data) is filled, it generates an ETI frame.

It's this correct?

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

lars18th avatar lars18th commented on June 14, 2024

Hi @JvanKatwijk ,

If you think it is difficult then do not start with a major rewrite

I feel I can handle it! 😄
Regarding the first goal (generation of ETI NULL frames when the incoming data is invalid):

  • After review the "ETI generator" https://github.com/JvanKatwijk/eti-stuff/blob/master/eti-cmdline/src/eti-handling/eti-generator.cp, I discover that:
    • Initial note: A block (in Mode I) is 32bytes, and 76 blocks are used in one Transmission Frame. The ETI generator consumes blocks from the ring buffer (where the OFDM processor has put the data), and when 76 in total are reached [1 SYNC + 3 FIB + (numberofblocksperCIF*4) CIF; where numberofblocksperCIF=18], then it generates an ETI frame.
    • The reading point is at #L197.
    • The part processing the FIB blocks is at #L216.
    • The part processing the CIF blocks is at #L246.
    • Locations where an error can appear, and the data is discarded:
      • #L198: If the "next" block isn't the expected, then the code resumes to start (flush the buffer, and waits for the first FIC).
        • When changing to NULL ETI frames, this case needs to be trapped as a discontinuity and process it in consequence, but only reinitialize as a last resort.
      • #L209: Block counter overflow.
        • This case here is treated with a simple get-back-counter. I'm not sure if it's the best solution.
      • #L267: Continue reading when CIF counters be invalid.
        • This case is uncommon, except when FIB blocks contains invalid data. So, an specific treatment is required.
    • The location when a new ETI frame is created is at #L270.
      • This point is reached when FIC data is completed (FIB blocks completed) and MSC data is completed too (the last CIF block of a Transmission Frame comes).
    • The part when the ETI frame is filled is at #L278.
    • And the location when the ETI frame is written is at #L303.

So assuming that the OFDM processor doesn’t miss packets (that isn’t true now, for exemple when the SYNC packet 0 isn’t detected or SYNC is lost)...

  • etiGenerator::init_eti(): This function has to really fill out NULL frames. So, the needed changes are enable the error level (#L410, partially implemented, but not finished). And handle incorrect internal values (like CIFCounts). However, the first goal is a simple flag for a full NULL frame output when needed.
  • etiGenerator::handle_discontinuity(): This will be a new function for processing the three detected cases when invalid data can be discarded. In fact, it is likely that it will not become a real function, but a block of code.

I’m right? I’m missing something?
Remember: The first goal is to process input RAW files with silences (block of zeroes in the samples) and output the correct ETI-NI bitstream with the corresponding NULL frames (so the CBR is granted).

from eti-stuff.

JvanKatwijk avatar JvanKatwijk commented on June 14, 2024

from eti-stuff.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.