zh199225 / astericxx

A C++11 Implementation of the ASTERIX data format

astericxx - A C++11 Implementation of the ASTERIX data format

The ASTERIX data format

The EUROCONTROL ASTERIX data format is the quasi standard in the ATC domain for exchanging surveillance-related information between Air Traffic Service providers and participating clients.

The underlying data format is a rather generic byte-oriented container format that defines structures, data types, how to put all of them together (Datablocks) and specifies the byte sequencing and encoding.

As space efficient data encoding seemed to be a design criteria there is no metadata contained in the resulting byte message that could give away how to interpret the message on the receiving side.

Therefore, ASTERIX defines standardized profiles for specific use cases (for instance, track reports) called "Categories", making use of the generic container format. The first byte in an ASTERIX datablock specifies the category used to create the following byte stream and the receiver has to apply the corresponding profile to de-serialize the data correctly.

If you are familiar with Google's protobuf... ASTERIX is taking the same approach, but 5 years earlier.

Furthermore, ASTERIX definitely has also communication protocol properties as some catagories also define communication patterns, algorithms or are used for monitoring and control.

Main idea of this library

I have seen (and extended/maintained) a hand written implementation of ASTERIX that was implementing all items of all categories on the lowest level (masking bytes, case-switching FSPECs) resulting in literally ten thousands of lines of code. All of them written by a human.

Maintaining those implementations is not exactly fun. After 10 years of existence there were still bugs in the library, typical C&P artifacts because the work was so repetitive and boring.

So, the main idea of this library is pretty obvious:

• Really implement just the structure and data types of the ASTERIX container format
• Store the profile information in a machine readable format (XML)
• Generate the code of the library by applying XSL Transformations on the XML profiles

The polymorphism and templating system of C++ comes in really handy here, and so at the time of writing the library code that has actually been hand written is by far less than 1K LoC:

 find framework/  | egrep "(.hpp$|.cpp$)" | xargs wc -l
   52 framework/ISerializeDeserialize.hpp
   25 framework/Spare.hpp
   17 framework/ISerializeDeserialize.cpp
  111 framework/RepetitiveDataItem.hpp
  175 framework/IDatablock.hpp
  116 framework/IFSPEC.cpp
   47 framework/FixedLength.cpp
   21 framework/Spare.cpp
   34 framework/IFSPEC.hpp
   91 framework/FixedLength.hpp
  689 total

Dependencies and bootstrapping

You will need CMake, XSLTPROC and BoostUT >=1.55.

The code generator runs during the CMake configuring step and should create all source files below the "autogenerated" subdirectory:

  cmake <path to astericxx repo>
Generating Item source files from templates: 
  item34-000.xml
  item34-010.xml
  item34-020.xml
  item34-030.xml
  item34-041.xml
  item34-050.xml
  item34-060.xml
  item34-070.xml
  item34-090.xml
  item34-100.xml
  item34-110.xml
  item34-120.xml
Generating Category source files from templates: 
  cat34.xml
Generating top level library files; 
-- Boost version: 1.55.0
-- Found the following Boost libraries:
--   system
--   filesystem
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/build

Example

See the "demo" subdir for examples and the unit tests below "tests" for more detailed examples of the implementation.

In a nutshell, Datablock objects have vector like properties. That is, you can iterate the records and you can push_back new records:

astericxx::Datablockcat34 Cat34;
astericxx::cat34 northMarkerRecord;
northMarkerRecord.Item010.setSAC(12);
northMarkerRecord.Item010.setSIC(13);
northMarkerRecord.Item000.setMessageType(astericxx::item000MessageType::NorthMarker);
northMarkerRecord.Item030.setTimeOfDay(5678);
Cat34.push_back(northMarkerRecord);

Stream operators are implemented, so one can serialize directly to files:

std::ofstream myfile;
myfile.open("test.raw");
myfile << Cat34;
myfile.close();

Or you can choose to serialize to and de-serialize from vectors:

astericxx::Datablockcat34 Cat34out;
  
astericxx::cat34 testRecord;
testRecord.Item010.setSAC(12);
testRecord.Item010.setSIC(13);
testRecord.Item000.setMessageType(astericxx::item000MessageType::NorthMarker);
testRecord.Item070[0].setType(astericxx::item070CounterType::FilterPSR);
testRecord.Item070[0].setCounter(1234);
testRecord.Item070[1].setType(astericxx::item070CounterType::SingleSSRReport);
testRecord.Item070[1].setCounter(4321);
Cat34out.push_back(testRecord);
  
astericxx::Datablockcat34 Cat34in(Cat34out.serialize());