Payloads in python events? about nvtx

[shwina]

Hi @jfaust - thanks for reporting. I suppose we can include passing a payload by introducing two more arguments to annotate:

• payload_type: a Python enum representing the type of the payload value
• payload_value: a Python int or float representing the payload value

Would that suffice for your needs?

[jcohen-nvidia]

@shwina -- before we do that, we should consider the Python API we want to expose for the new "extended payloads" feature we're about to ship in the C API. This allows users to pass more than a single value for the payload. In C, we give the user a way to define a schema for an arbitrary binary data layout, and then very efficiently pass blobs of binary data via just a pointer/size/typeID. The intent is that no data format conversion is necessary -- you just describe the data as it is in memory, and pass it to the tool as-is. This is way better for performance than e.g. converting a bunch of structured data to something like JSON and passing it as the message string, especially in the case where no tool is attached and all work to prepare arguments for NVTX calls is waste. I'm working on getting that feature published for the C API right now, and I'd like to find a way to make the Python API expose it in a Pythonic way, and with as good performance as possible. For simple things like dataclasses, we may be able to define a schema for the exact binary layout Python uses and do no conversion work to prepare the arguments for the C API, but for complex structures like maps, we'll probably need to do some kind of format conversion to flatten it, and that will take some investigation.

So, all that said, let's check to see if there's a way to keep the simple cases simple, but not make it awkward to use extended payloads. For simple payloads, do we really need the user to specify the type, like in the payload_type and payload_value idea above? Couldn't we just have payload (the value) and have our implementation detect what type it is? Also, the extended payloads C API takes a list of blobs, so maybe the easiest way to use the extended payloads feature is a different argument to annotate, e.g. payloads=[...] (note the plural). If we can easily have a lookup table from a Python variable's dynamic type to a pre-registered type ID, then we may not need the user to include a field like payload_type for each payload. I think we should have some design discussion about all this, because I'd like the barrier-to-entry for this multiple-payload API to be as low as possible.

[shwina]

Python does have a struct module that should allow constructing byte blobs like this quite easily:

https://docs.python.org/3/library/struct.html

What does the schema look like in the C API?

[jfaust]

I think passing just payload and detecting if it's an int or float would work fine for most use cases. If someone needs to force to the right type they can wrap it in int() or float().

I don't have a need for more structured payloads at the moment so take this with a grain of salt, but I like the idea of automatically generating serialization for e.g. dataclasses, with some way to register custom serialization for a class for other use cases.

[jcohen-nvidia]

Thanks @jfaust -- I think we can work incrementally here, then. My vote is adding a payload=... argument to annotate with detection of the appropriate type, and then adding a different argument for structured payloads later. @shwina -- I'll slack you a link to one of the presentations RobertD or I gave internally about how schema registration looks in the C API. It was designed to be friendly to auto-generation, so hopefully we can do some nice things for Python where the user gives us any object, and we can either register a new schema on the spot or reuse an existing registration if we've already seen an object of that type.

[shwina]

OK -- so if someone passes payload=int_value, we automatically assume int64? Given that all Python has is an int type. NumPy has more specific integer data types, but I don't want NVTX to depend on NumPy.

[jfaust]

I'd say that's good default behavior - there's always the option of adding payload_type (that defaults to autodetect) to force types if necessary.

[shwina]

OK - I'll be able to get to this in the next week. But, if this is something you'd be able/interested in contributing, that's extremely welcome!

[jfaust]

Absolutely! See #90

[shwina]

NVTX 0.2.10 on PyPI should contain the feature introduced in #90. Please let me know if it works as expected. Closing this issue - thanks so much @jfaust for the contribution!

[jfaust]

Whoa, thanks for the quick turnaround @shwina!