circleci-archived / sumologic-orb

SumoLogic Orb < ARCHIVED >

Easily capture analytics from your CircleCI jobs in your Sumologic dashboard!

New Orb

The new SumoLogic orb and repository can be found here:

• repo: https://github.com/SumoLogic/sumologic-orb
• orb: https://circleci.com/developer/orbs/orb/sumologic/sumologic

Ensure to update the orb reference in your config with the orb now under the SumoLogic namespace.

orbs:
  sumologic: sumologic/[email protected]

sumologic-orb's Issues

Find no jobs in workflow, exits with status 1, other jobs running

Orb version

1.0.0

What happened

Followed setup instructions, getting an error message running the workflow collector:

Jobs in Workflow: 0

Only a single job has been found in the workflow, indicating this reporter is the only job in the pipeline.
Please add other jobs to the Workflow you wish to collect data on to send to Sumologic
Exited with code 1

I talked with support and they suggested I file a ticket here. They reviewed my config and confirmed it matches the instructions. Link to it failing:

https://circleci.com/gh/AdStage/adstage-platform-v2/1123?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

Expected behavior

Runs successfully and sends to Sumo Logic.

Orb runs forever + blocks subsequent phases from running

Orb version 1.0.2

What happened

• Included orb at the beginning of a workflow
• All orbs dependent on tasks running parallel to the sumologic orb were blocked until sumologic orb was cancelled
• Even when other jobs run successfully, sumologic orb hangs forever

Expected behavior

Sumologic orb reports workflow / job details to Sumo.

Orb triggers end of workflow process when workflow is still running

Orb version

1.0.6

What happened

The orb thinks that the workflow is either blocked or on hold (due to, for example, a manual quality gate) and then sends the end of workflow data to Sumo Logic. However, the workflow is fully automated

Expected behavior

The workflow is fully automated, so we do not expect a blocked or on_hold status to be returned from the workflow get-jobs API. It appears there may be a case where one of these two statuses are returned even for fully automated worfklows. Here is the code that assumes these statuses are not returned for automated workflows:

              if [ "$JOB_STATUS" == '"success"' ] || [ "$JOB_STATUS" == '"failed"' ];
              then
                echo "Job $JOB_NAME $JOB_NUMBER is complete - $JOB_STATUS"
              elif [ "$JOB_STATUS" == '"on_hold"' ] || [ "$JOB_STATUS" == '"blocked"' ];
              then
                # The condition to not block metrics sending when workflow use manually approved steps or is blocked.
                echo "Job $JOB_NAME $JOB_NUMBER need manual approval - $JOB_STATUS - skipping"
              else
                # If it is still running, then mark WF_FINISHED as false.
                WF_FINISHED=false
                echo "Setting status of WF_FINISHED to false"
              fi
            fi

The current thinking is the API may be returning "on_hold" or "blocked" in certain cases for automated workflows (potentially even as designed). If that's the case, we would need to understand those cases so that we do not assume the workflow is finished prematurely.

Feature Request: Attach arbitrary metadata to workflow messages sent to Sumo

Orb version

1.0.6

What happened

Not a bug. The ask is to allow support for arbitrary metadata to be attached to messages sent to Sumo.

Expected behavior

There are a few ways to go about this, but an initial pass could just allow for an arbitrary set of key value pairs (potentially as environment variables as well).

Blocked jobs are treated as complete, causing workflow-collector to attempt to send data before workflow is complete

Orb version

1.0.1

What happened

I added the orb to my workflow, which has several jobs that depend on earlier jobs being completed. I can watch the output of workflow-collector and see it checking the status of each job. I believe what is happening is after the intial checkout job completes, there's a moment before subsequent jobs have begun, where no jobs have a running status. They are all either successful or blocked. This causes the workflow collector to attempt to send data to Sumologic prematurely, and hits an error when trying to parse the JSON.

Expected behavior

Expected behavior would be that workflow-collector would recognize that the workflow is not complete, and continue to iterate through the jobs and check status.

Feature Request: Ability to rerun workflows and have data output of workflow appear in Sumo Logic

Orb version

1.0.6

What happened

It appears as if data only flows into Sumo Logic at the onset of a workflow running, but not for workflows that have failed and then are rerun from the beginning.

Let's say a workflow fails and you're alerted of this failure. If you try to rerun the workflow (including the job sumologic/workflow-collector) in CircleCI, no data is ingested in Sumo Logic.

So only the initial running of a workflow is ingested into Sumo Logic and not any reruns thereafter.

Expected behavior

Ideally, you should be able to rerun the entire workflow (including the sumologic/workflow-collector job) and all log output should be ingested into Sumo Logic, but again, this is not the case currently.

Remove config.yml data from Sumologic payload

config.yml is included in the sumologic payload by default. This should be removed to drastically reduce the size of the payload.

Orb Version 1.0.6/Pending Jobs

Orb version 1.0.6

What happened

When adding the sumologic/workflow-collector job to either the beginning or end of a workflow, the job will skip over jobs in STATUS: "on_hold" and STATUS: "blocked", resulting in the orb's workflow-collector job ending its collection and shipping incomplete logs to sumologic. The logs of the inflight (on_hold, blocked, etc.) jobs are never collected, as the orb's job does not wait for them to complete, nor circle back to check on them.

Below is (partial) sample output of the orb skipping two jobs then shipping workflow logs to sumologic. Neither the logs for JOB: "approve-tf" nor those for JOB: "publish-docker-image-1", below, are included in the logs sent to sumologic, though the orb does appear to consider this a successful run of its workflow-collector job.

Expected behavior

The orb would have logic to gracefully handle blocked, on_hold, or otherwise pending jobs and to include, not ignore, those jobs' completed build logs to sumologic.

Error response from daemon: manifest for cimg/base:latest not found

Orb version

1.0.0

What happened

So from the looks of it the version of the circleci/[email protected] at https://circleci.com/orbs/registry/orb/circleci/sumologic#executors-default
has an executor defined as

description: >
  Default reporting environment for Sumologic. This is a small Ubuntu based
  Docker image with a low resource class. Designed to load fast and use few
  credits.
docker:
  - image: cimg/base
resource_class: small

This is problematic, as the default tag if not defined is latest which is not defined for the cimg/base image; however, your master branch, https://github.com/CircleCI-Public/sumologic-orb/blob/master/src/executors/default.yml, defines the executor like:

description: >
  Default reporting environment for Sumologic. This is a small Ubuntu based Docker image with a low resource class. Designed to load fast and use few credits.
parameters:
  image:
    type: string
    default: "cimg/base"
    description: Docker image name
  tag:
    type: string
    default: "stable"
    description: Docker image tag

docker:
  - image: <<parameters.image>>:<<parameters.tag>>

resource_class: small

which would be great (defining the tag as stable which DOES exist), if only that is what the orb registry had. Can you update the orb registry? Perhaps bump the version.

Expected behavior

There shouldn't be a docker hub error ;)

Workflow with type: approval - JOB NUM is null

Orb version

1.0.2

What happened

When there is a job with type: approval, sumologic-orb fails. JOB NUM: null

Expected behavior

Runs successfully and sends to Sumo Logic.

Feature Request: Process workflow status

Currently, only job information is captured and displayed within the sumo logic dashboard. Workflow data at the end of the job must be processed before it can be displayed on sumo logic. Because the sumo-logic job runs within the workflow it is tracking, it must first determine if the workflow has failed based on the results of the other jobs and then modify the data before sending. this is already done for job data.

https://github.com/CircleCI-Public/sumologic-orb/blob/master/src/jobs/workflow-collector.yml#L142

Determine the correct status for the workflow before sending. Once a PR has been merged for this change, a change will need to be made to the official Sumo Logic dashboard.

Feature Request: Add parameterizable image for default executor

The current default executor used in the workflow-collector job has a hard-coded docker image "cimg/base". As per best practice, the image or tag version should be parameterized to protect the executor against potential volatile changes to the docker image.

Issue with Sumo Logic CircleCI Insights orb and manual approval steps in workflow

Orb version

1.1.0

What happened

We are using the circleci/sumologic orb (latest version) to forward CircleCI-related metrics to Sumo Logic, as outlined here .

Discovered an issue with the sumologic/workflow-collector: if a CircleCI workflow has a manual approval step, the workflow collector will gracefully terminate and not resume once approval is granted - as a result we will lose metrics for the rest of the workflow run.

Is there any way to re-trigger the workflow collector to run after approval has been granted, or any other workaround/solution? Thanks.

Expected behavior

Re-trigger the workflow collector to run after approval has been granted.

Successful workflows do not send stopped_at timestamp (cannot measure workflow duration)

Orb version

1.2.0

What happened

When the orb sends workflow json (see example below) for a successful run, it never includes stopped_at. Which makes sense because from the Insights API standpoint, the workflow hasn't finished yet.

{
  "pipeline_id": "1c055742-7ead-44e1-9407-958cc2ba8bcf",
  "id": "79ad735b-fa0e-4539-8db5-89b09adb425f",
  "name": "stage-deploy",
  "project_slug": "gh/xyz/xyz",
  "status": "running",
  "started_by": "d87d420e-cb2c-4b42-ace7-db839d0ec256",
  "pipeline_number": 17132,
  "created_at": "2021-09-24T21:31:33Z",
  "stopped_at": null
}

Expected behavior

While existing behavior makes sense from Insights API standpoint, it makes it impossible to calculate workflow duration. Which is a very valuable piece of info since that generally is what your build duration is. Measuring individuals jobs is useful, but it doesn't tell you how long the overall build takes.

Suggested solution

We can assume that the workflow is done when the orb sends the data (by definition, that's the logic within). We can either append another field to json or override the value in stopped_at with current system time.

P.S. I am open to making a contribution assuming we agree on the direction for the fix.