Add support for sending the payloads using event channel about appscope

[michalbiesek]

Currently decision of payloads are done in following way:

We decide if payloads are enabled via :

• at least single protocol definition in scope configuration file used protocol definition mechanism
  or
• payloads are enabled see e.g.: SCOPE_PAYLOAD_ENABLE)

The decision where to put payloads are done in following way (if payloads are enabled via above):

• if cribl is enabled and SCOPE_PAYLOAD_TO_DISK environment variable is not set to true we will send the payloads via cribl transport channel (via event transport channel)
• if cribl is enabled and SCOPE_PAYLOAD_TO_DISK environment variable is set to true we will send the payloads to disk
• if cribl is not enabled we will send the payloads to disk

I need to modify mechanism above.
Initial support is done in #1595.
Proposal (waiting for feedback):
This will set payload to event channel:

scope run --payloads -e tcp://localhost:9999 run -- nc -lp 10001

This will set payload to disk:

SCOPE_PAYLOAD_TO_DISK=true scope run --payloads -e tcp://localhost:9999 run -- nc -lp 10001

My doubts are if we should introduced another env variable configruation option since event channel is enabled by default there therefore it requried to often use SCOPE_PAYLOAD_TO_DISK=true in case of saving payloads on disk

[michalbiesek]

TODO:

• add support for payloaddest in cli arguments
• extend the payload configuration in scope.yml with destinanation file/event

[michalbiesek]

I have modified my solution to following model:

This will set payload to disk:

scope run --payloads -e tcp://localhost:9999 -- nc -lp 10001

This will set payload to event transport type:

scope run --payloads --payloadsdest="event" -e tcp://localhost:9999 -- nc -lp 10001
# Alternatively You can use 
SCOPE_PAYLOAD_DEST=event scope run --payloads -e tcp://localhost:9999 -- nc -lp 10001

This will set payload to cribl transport type:

scope run --payloads --payloadsdest="event" -c tcp://localhost:9999 -- nc -lp 10001
# Alternatively You can use 
SCOPE_PAYLOAD_DEST=event scope run --payloads -c tcp://localhost:9999 -- nc -lp 10001

The payloads destination can be modified in the configuration file as well:

# Settings for the `payloads` feature
#
payload:

  # Enable payload capture
  #   Type:     boolean
  #   Values:   true, false
  #   Default:  false
  #   Override: $SCOPE_PAYLOAD_ENABLE
  #
  # This can produce large amounts of data from I/O-intensive programs and
  # should be considered carefully before being enabled.
  #
  # See `protocol` for a way to enable this for specific protocols instead of
  # all traffic.
  #
  enable: false

  # Determine the payload type destination
  #   Type:     string
  #   Values:   "dir", "event"
  #   Default:  "dir"
  #   Override: $SCOPE_PAYLOAD_DEST
  #
  #
  # This allows to specify the payload destination
  # - "event" allows to send the payloads to same location as events
  # - "dir" allows to use directory to store payload files
  #
  type: "dir"

  # Directory for payload files
  #   Type:     string
  #   Values:   (directory path)
  #   Default:  /tmp
  #   Override: $SCOPE_PAYLOAD_DIR
  #
  # Consider using a performant filesystem to reduce I/O performance impacts.
  #
  # Applies when dest is "dir".
  #
  dir: '/tmp'

Important

The change which is worth to get feedback:
This will set payload to disk:

scope run --payloads -c tcp://localhost:9999 -- nc -lp 10001

While previously the payloads were sended to events (cribl) in this case

WIth current changes the following commands will send payloads to cribl:

scope run --payloads -c tcp://localhost:9999 --payloadsdest="event" -- nc -lp 10001
SCOPE_PAYLOAD_DEST=event scope run --payloads -c tcp://localhost:9999  -- nc -lp 10001