llnl / adiak

Standard interface for collecting HPC run metadata

Adiak: Recording HPC Run Metadata

Adiak is a library for collecting metadata from HPC application runs, and distributing that metadata to subscriber tools. Examples of Adiak metadata are:

• What user performed this run
• At what time did the run launch
• How long did the run spend in system time
• What MPI hosts are part of this run
• Application-provided parameters, such as peak velocity in an N-body problem

This metadata is provided as name/value pairs. For application-provided data, an example name might be "peak_velocity" and and have a value of 5.6. For other data, such as what user ran the code, there are adiak-managed standard names like "username".

Adiak has a tool interface, which allows tools to subscribe to this metadata. Example tools might include performance analysis tools, workflow tracking tools, or anything else that needs this metadata. Tools can iterate receive metadata name/values as they are provided by the application, or examine existing metadata values.

Documentation

More information can be found in https://llnl.github.io/Adiak/.

Contributing

Contributions to Adiak are welcome, and should be made under the MIT license. Please submit pull requests to the github repository.

SPDX-License-Identifier: MIT

LLNL-CODE-792240

adiak's Issues

Swap "[]

Adiak records sets as "[]" and lists as "{}". Should be swapped to align it with Python and JSON syntax for lists and sets.

Memory leak

When running the clang sanitizers in a code using Adiak, I got a bunch of leaked memory errors. While they are all different stack traces, at the root they are all coming from adiak_type_to_string. This was with version f6787c0 of the code. It seems like whover is calling adiak_type_to_string needs to free the buffer when it's done.

0: Direct leak of 13 byte(s) in 1 object(s) allocated from:
0:     #0 0x10dae6d in malloc (/usr/WS1/brunner6/buildSan/bin/some_code+0x10dae6d)
0:     #1 0x451e508 in adiak_type_to_string /usr/WS1/brunner6/buildSan/libraries/adiak_f6787c0/src/adiak.c:1199:22
0:     #2 0x1c0b005 in sina::adiakSinaCallback(char const*, int, char const*, adiak_value_t*, adiak_datatype_t*, void*) /usr/WS1/brunner6/buildSan/libraries/sina-cpp-1.10.0/src/AdiakWriter.cpp:201:31
0:     #3 0x4517848 in adiak_raw_namevalue /usr/WS1/brunner6/buildSan/libraries/adiak_f6787c0/src/adiak.c:125:10
0:     #4 0x451888b in adiak_namevalue /usr/WS1/brunner6/buildSan/libraries/adiak_f6787c0/src/adiak.c:195:11
0:     #5 0x451dedb in adiak_job_size /usr/WS1/brunner6/buildSan/libraries/adiak_f6787c0/src/adiak.c:1041:11
0:     #6 0x2aabca6199d6 in adiak::jobsize() /usr/WS1/brunner6/buildSan/include/adiak.hpp:235:14
0:     #7 0x2aabca6199d6 in codemetrics::initialize_adiak(int) /usr/WS1/brunner6/SomeCodeStandby/src/infrastructure/codemetrics/Adiak.cc:41:5

Mac OS X build error (template/types)

Hi there,

I have been able to build Adiak on Mac OS X fine, but when I try to use it as third party library in another code, I am getting a template type conversion error. Here is the console output for the build:

In file included from /Users/j0529096/Documents/code/thirdPartyLibs/for_caliper_install/adiak/include/adiak.hpp:52: /Users/j0529096/Documents/code/thirdPartyLibs/for_caliper_install/adiak/include/adiak_internal.hpp:153:40: error: cannot initialize a parameter of type 'void *' with an lvalue of type 'long long' element_type<T>::set(*val, obj); ^~~ /Users/j0529096/Documents/code/thirdPartyLibs/for_caliper_install/adiak/include/adiak.hpp:129:48: note: in instantiation of member function 'adiak::internal::parse<long long>::make_value' requested here bool result = adiak::internal::parse<T>::make_value(value, avalue, datatype); ^ /Users/j0529096/Documents/code/GEOSX/src/coreComponents/managers/initialization.hpp:155:10: note: in instantiation of function template specialization 'adiak::value<long long>' requested here adiak::value( name + " sum", total ); ^ /Users/j0529096/Documents/code/GEOSX/src/coreComponents/managers/initialization.cpp:184:3: note: in instantiation of function template specialization 'geosx::pushStatsIntoAdiak<long long>' requested here pushStatsIntoAdiak( "numThreads", numThreads ); ^ /Users/j0529096/Documents/code/thirdPartyLibs/for_caliper_install/adiak/include/adiak_internal.hpp:37:50: note: passing argument to parameter 'p' here static void set(adiak_value_t &v, void *p) { v.v_ptr = p; } ^ 1 error generated.

This is a Release build, with AppleClang 11.0.3.11030032 and Adiak-0.2.0.

Thanks for your help on this,

Herve

adiak_cmdline() should record a list, not a set

adiak_cmdline() currently records a set of strings. It should really be a list because the order of the arguments matters and duplicates are allowed.

Swap "[]" and "{}" for denoting sets and lists in typestrings

Python and JSON use "[]" for lists and "{}" for sets/dicts. Adiak does the opposite, which is confusing. Let's consider swapping it to align Adiak with existing practices.

Version Info Mismatch

The header file for adiak states the 0.2.2 release as version 0.3.0:

Adiak/include/adiak.h

Lines 15 to 17 in 054d269

	#define ADIAK_VERSION 0
	#define ADIAK_MINOR_VERSION 3
	#define ADIAK_POINT_VERSION 0

In other places, the version is still 0.2.1:

Adiak/CMakeLists.txt

Lines 2 to 4 in 2127ccf

	project (Adiak
	LANGUAGES CXX C
	VERSION 0.2.1)

Adiak/scripts/make_release_tarball.sh

Line 9 in 054d269

VERSION=0.2.1