ibmstreams / streamsx.microservice Goto Github PK

Following up previous pull request to update the guideline

Initial work to create sample implementation of Streams microservices

I created a "develop" branch for our development...

Development should go on develop branch. When we are ready, we can make releases and push the changes to master.

I will also make "develop" our default branch.

@ddebrunner Sorry, I ran the command again without thinking. I should have consulted before I set this up.

Please let me know if you have concern.

Create some SPL (main) composites that can act as services and support:

• Convert an SPL schema published stream to an JSON stream with the same topic.
• Subscribe to a published stream and submit message hub messages as JSON for each tuple
  • This would allow Stream Designer flows (just beta'ed) to consume published streams.
• Consume a MessageHub queue and publish messages as a stream
  • This would allow Stream Designer flows (just beta'ed) to produce data to be consumer by Streams subscribers.

It's been suggested to describe the microservices as "streaming microservices" to help differentiate from request/response ones.

not saying we rename the project, but in text descriptions use of streaming microservices would be encouraged.

• From a customer case with the health care toolkit a service should probably set resource tags for any operators that might require placement, eg. binding a TCP server to a know machine. Use of tags allows flexibility and Job Config overlays can be used to map tags (if required) for the service. E.g. for the health care Hapi ingest, have a resource tag of hapiIngest that might be mapped to eastWingIngest in a customer environment.

• A microservice must not hardcode its submission, e.g. only supporting DISTRIBUTED. In some ways this goes to being able to support a runner style where the runner takes a definition of the service and supports submitting it to any context, supporting submission parameters, job names etc. etc. Like the streamsx-runner script.

Do we need some user stories that will help guide us?

Claudia the microservice developer would like to easily create a microservice:

• in Python, Java, Scala or SPL
• with runtime properties such as job name automatically set when the microservice is submitted.
• that can be packaged and easily consumed by Alec the application composer as a single entity.

Alec the application composer would like to:

• discover microservices
  • running in an instance
  • available from a catalog
    • generally - ie. general list of what is available
    • or that subscribe to or publish a specific topic. (e.g. what microservices exist to analyze this stream of medical data)
• understand which topics a microservice consumes and produces (meta-data for a service)
• easily submit a microservice with no knowledge of its internals or dependencies.