MQTT: Support for format parameter about streamsx.messaging

[chanskw]

In the old MQTT operators, we support a format parameter: bin / block.
Since we are now implementing operators in Java, we can no longer have a parameter that uses custom literals. In thinking about this more, I think we should avoid having "format" parameters in adapters in general. Format should be done externally in Parse / Format operators.

Therefore, I propose that we remove the "format" parameter from both of these operators and only support the "block" format. The adapters will always take a blob for MQTTSink and send out a blob for MQTTSource. If the user wants to use "bin" format, they can use the Parse and Format operators to achieve the same effect. I think most people have been using "block" format anyway since the operators are used for interfacing with external systems... does not really make sense to send out Streams binary format.

@mikespicer , @ddebrunner , @petenicholls - what do you think about this proposal?

[ddebrunner]

I think it makes sense to have formatting separate.

[mikespicer]

I prefer separating the data transport from the data format with separate operators for parse/format which can be used in conjunction with source/sink to provide many combinations.

I realise that you are asking about the bin format which means the data format is the binary serialization of a stream tuple, and I think it makes sense to separate that out to be parse/format operators. I presume that different message formats like text or binary with XMS and JMS etc. need to be handled differently as it determines if the messaging system will read a message or not. Is that the case?

[chanskw]

@mikespicer Yes, that is correct, we will have to handle message format for Text / Binary for XMS and JMS differently. For XMS, I believe the message format is handled by XMS client. We need to look at how we are interfacing with the client. For JMS, the message formats are handled by the operator, which we will need to review and figure out if we want to or how to separate those out.

[mikespicer]

OK good. On data format I completely agree that it should be handled using parse/format operators and not to have format parameters on the source/sink operators themselves. For message format we would handle it in the source/sink as its part of the configuration/parameters for interacting with messaging systems.

[chanskw]

Understood.... Will add this as part migration information about going from the product messaging toolkit to open-source messaging toolkit. Thanks!