honeycombio / buildevents Goto Github PK

View Code? Open in Web Editor NEW

222.0 222.0 37.0 667 KB

A small helper to generate Honeycomb traces from CI builds

License: Apache License 2.0

Go 100.00%

buildevents's People

Contributors

Stargazers

Watchers

buildevents's Issues

Build for darwin arm64 to support Apple Silicon build hosts

Is your feature request related to a problem? Please describe.

Folks can't run buildevents on M1/M2 build hosts without building buildevents themselves.

Describe the solution you'd like

A darwin_arm64 build!

Describe alternatives you've considered

Users building it themselves. Seems silly for them to have a build pipeline of the build pipeline instrumentor when we already do.

Update for E&S

Buildevents currently requires a dataset and hardcodes service names for events.

Changes to consider:

change build event steps to a different field than service name
set dataset based on a configured service name (as is done with beelines) instead of dataset config
add service.name field in addition to service_name
trim whitespace from dataset derived from service name
provide default unknown_dataset service name / dataset

Feature Request - Google Cloud Build provider

Request for a GCB BUILDEVENT_CIPROVIDER value to use within Google Cloud Build.

buildevents cmd returns 0 exit status even if the subcommand returns non-zero

looks like the cobra-fication change dropped proxying the error status of the child process.

https://github.com/honeycombio/buildevents/blob/master/cmd_cmd.go#L59 will ensure that if the subcommand fails, err is filled in, but that method returns after adding the error to the span (and doesn't exit with an error status).

can `buildevents cmd` capture stdout / stderr on failed spans?

Is your feature request related to a problem? Please describe.
I've been trying out buildevents in GitHub actions and have been using buildevents cmd to run a number of shell commands, and send them as spans to Honeycomb. When a span has an error, it seems that while the exit code is captured as a span attribute, the actual error output (stdout or stderr) is not.

Describe the solution you'd like
From using other opentelelemtry and honeycomb libraries it seems useful to include the error message on the span. I would love to see the cmd command not just pass through stdout and stderr but also include them on a the span if the command exits with an error.

Describe alternatives you've considered
I've very briefly considered using tee to output errors to a file, detecting non-zero exit codes and including the output as an attribute via the BUILDEVENT_FILE. This would add a lot of boilerplate to my workflows, though - which are already a bit bloated through the instrumentation as it is.

Additional context
I've been enjoying this tool otherwise, thanks for making a great product and helping to instrument CI 🚀

buildevents adds extra "" to commands being passed to bash -c

This means that passing pipeline e.g. find | xargs will fail because there is no way to avoid wrapping the | in quotes, causing it to be interpreted as an argument ("|") to find rather than a shell pipe. And passing the entire "find | xargs" command as $6 doesn't work because then bash sees it wrapped in "" and tries to execute the command names "find|xargs".

Make `buildevents cmd` NAME parameter optional

The name is hand if the command itself is not very expressive. In my pipeline I'd be fine with always just using the command itself as name for a cmd span. Instead I have to add a redundant name to each invocation.

Spaces in dataset name leads to wrong URL on `build`

The URL printed by buildevents build translates spaces in the dataset name to %20, while the honeycomb UI uses dashes. So a dataset called litmus tests should be linked as litmus-tests but is linked as litmus%20tests, which does not work.

Occasional errors returned by buildevents

We recently added buildevents to our Gitlab CI pipelines to get a better understanding of where we're spending time in them but we do occasionally get errors from the buildevents command itself which then causes our CI pipelines to fail.

The error we see looks like this:

/usr/local/bin/buildevents: line 1: syntax error near unexpected token `newline'
/usr/local/bin/buildevents: line 1: `<!DOCTYPE html>'

For full reference we're triggering buildevents via the following command to minimise some boilerplate stuff in each project:

curl https://gitlab.example.com/pub/buildevents-boilerplate/raw/master/buildevents.sh | bash -s step

and that script looks like this:

#!/bin/sh

set -e

BUILDEVENTS_BIN_URL=https://github.com/honeycombio/buildevents/releases/latest/download/buildevents-linux-amd64
BUILDEVENTS_BIN=/usr/local/bin/buildevents

if [ -z "$BUILDEVENT_APIKEY" ]; then
    echo "error: Missing BUILDEVENT_APIKEY environment variable"
    echo "Check the CI/CD variables on Gitlab"
    exit 1
fi

if [ -z "$BUILDEVENT_DATASET" ]; then
    echo "warning: Missing BUILDEVENT_DATASET environment variable"
    echo "\`buildevents\` will be used by default"
fi

curl -L -o $BUILDEVENTS_BIN $BUILDEVENTS_BIN_URL
chmod +x $BUILDEVENTS_BIN

case "$1" in
    "init")
        echo "Initialising buildevents..."
        mkdir -p buildevents && date +%s > buildevents/buildstart
        ;;
    "start")
        echo "Logging step start..."
        mkdir -p buildevents && date +%s > buildevents/stepstart
        ;;
    "step")
        echo "Adding step span..."
        "$BUILDEVENTS_BIN" step "${CI_PIPELINE_ID}" "${CI_JOB_ID}" "$(cat buildevents/stepstart)" "${CI_JOB_NAME}"
        ;;
    "build")
        "$BUILDEVENTS_BIN" build "${CI_PIPELINE_ID}" "$(cat buildevents/buildstart)" success
        ;;
    *)
        echo "error: Unexpected <"$1"> argument for buildevents.sh"
        echo "Valid values are: init, start, step, and build"
        exit 1
        ;;
esac

Full output for that command looks like this:

$ curl https://gitlab.example.com/pub/buildevents-boilerplate/raw/master/buildevents.sh | bash -s start
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  1305  100  1305    0     0  14711      0 --:--:-- --:--:-- --:--:-- 14829
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  9269    0  9269    0     0  18919      0 --:--:-- --:--:-- --:--:-- 18955
Logging step start...
$ if [ "$SHELL" = "/bin/bash" ]; then shopt -s expand_aliases; fi && alias buildevents-cmd="/usr/local/bin/buildevents cmd ${CI_PIPELINE_ID} \"${CI_JOB_ID}\""
$ buildevents-cmd storybook-build -- yarn run storybook:build
/usr/local/bin/buildevents: line 1: syntax error near unexpected token `newline'
/usr/local/bin/buildevents: line 1: `<!DOCTYPE html>'
Running after script...
$ curl https://gitlab.example.com/pub/buildevents-boilerplate/raw/master/buildevents.sh | bash -s step
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  1305  100  1305    0     0  13885      0 --:--:-- --:--:-- --:--:-- 14032
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  9269    0  9269    0     0  48459      0 --:--:-- --:--:-- --:--:-- 48528
Adding step span...
/usr/local/bin/buildevents: line 1: syntax error near unexpected token `newline'
/usr/local/bin/buildevents: line 1: `<!DOCTYPE html>'
ERROR: Job failed: command terminated with exit code 1

I only happen to have seen this on step commands so far.

I'm not sure if this is an error being returned by the Honeycomb API and not handled correctly here? Or are we doing something wrong?

Add a `quiet` flag

I ran into an issue when I was trying to use terraform and terragrunt with this tool. I have aliased terraform so that terragrunt calls the following command:

buildevents cmd $CI_PIPELINE_ID $STEP_SPAN_ID $TF_NAME -- terraform "$@"

This was tripping up terragrunt though, because it was adding extra information to the output:

e.g.
running /bin/bash -c "terraform" "--version"

I was able to get around it by piping the output to awk '{if(NR>1)print}', but it would have been helpful if I could have used a --quiet flag.

Add ability to override trace.parent_id

What is the problem you're trying to solve

I'm tracing builds on buildkite. When using the step command, I would like
the ability to change the trace.parent_id so that I can nest spans underneath
other spans so that the waterfall graph is easier to read.

I've tried setting the trace.parent_id in the $BUILDEVENT_FILE, but the
step command always sets trace.parent_id to the same value as the
trace.trace_id and doesn't allow me to override that value. This makes sense
for children of the root trace, but does not allow for nested spans.

Describe the solution you'd like

I would like the ability to override the trace.parent_id by adding it to the
tags file:

BUILDEVENT_FILE=tags-list.txt
STEP_START=$(date +%s)
STEP_SPAN_ID=$(echo before_script | sum | cut -f 1 -d \ )
echo "trace.parent_id=${BUILDKITE_JOB_ID}" >> $BUILDEVENT_FILE
buildevents step $BUILDKITE_BUILD_ID $STEP_SPAN_ID $STEP_START name

Provide the option to use a different shell

Many CI jobs use alpine images which don't come with bash. Instead they have a minimal bin/sh installation. Is there anyway we can allow a user to choose their shell?

If you think this is a good idea I would love to help implement it :)

buildevents cmd: Forward stdin

Currently buildevents cmd does not forward on stdin to the subprocess, making it impossible to use in a pipeline

Windows support

A bunch of CI jobs out there run on Windows, especially for folks using .NET. We should ensure this works in a Windows environment, and if so, publish an executable for it.

describe data structures

Could you document the data structures (and their semantics) of the events that are being sent for each buildevents command?

I'm asking because I'd like to run a similar instrumentation without buildevents actually, and just looking at the code... it's not easy.

A simple example of unfortunate semantics: README say buildevents cmd $TRAVIS_BUILD_ID $STEP_SPAN_ID ... and code goes traceID := strings.TrimSpace(args[0]); stepID := strings.TrimSpace(args[1]) ... "trace.parent_id": stepID, "trace.span_id": spanID so step-span-id is step-id is parent-id ?! 🙄

Get SpanID within step

Is your feature request related to a problem? Please describe.

I am using go-test-trace, and would like to integrate it with buildevents.
The plugin provides me with a TraceID, but not seemingly not a SpanID. It would be nice to have that provided, so I could rebuild the traceparent supported by that plugin.

Describe the solution you'd like

I would like to be able to run go-test-trace -traceparent=00-$TRACE_ID-$SPAN_ID-01 so my traces from both tools share the same context.

Describe alternatives you've considered

Additional context

Add the ability to override the duration_ms

I'm running buildevents from outside the pipeline because Bitbucket pipelines is not so friendly in introspecting himself. I have a cron job that queries all the finished pipelines and pushes information to honeycomb but in this setup I cannot rely on buildevents default duration_ms value as I make the call long after the build/pipeline has finished.

Returned URL is missing environment

Versions

Go: (unknown, using precompiled version from release)
Buildevents: v0.10.0

Steps to reproduce

Run buildevents build with a correct API key etc.

Additional context

API key represents a nondefault environment called ci.

Observed result

To stdout is printed a URL in format

http://ui.honeycomb.io/<team>/datasets/<dataset>/trace?trace_id=<traceId>&trace_start_ts=<ts>&trace_end_ts=<ts>

When I click it, it redirects me to the default environment test which is empty for me, with the home screen shown instead of the trace. Instead, I have my buildevents stuff in an environment ci.

Expected result

To stdout is printed a URL in format

http://ui.honeycomb.io/<team>/environments/<env>/datasets/<dataset>/trace?trace_id=<traceId>&trace_start_ts=<ts>&trace_end_ts=<ts>

(including the environment).

When I click it, I see the trace.

Additional question

Is it possible to make a different environment default, or delete an environment?

Create markers via CLI

Is your feature request related to a problem? Please describe.

I'd like to mark from our CI when we're deploying

Describe the solution you'd like

I'd like to be able to create and end a marker through the CLI

Describe alternatives you've considered

Additional context

shouldn't output usage when a subcommand exits with non-zero status

seeing this on circleci:

buildevents cmd $CIRCLE_WORKFLOW_ID $BUILDEVENTS_SPAN_ID doodle-yarn -- yarn --frozen-lockfile --prefer-offline
running /bin/bash -c "yarn" "--frozen-lockfile" "--prefer-offline"
yarn install v1.19.0
[1/4] Resolving packages...
[2/4] Fetching packages...
error Incorrect integrity when fetching from the cache
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Error: exit status 1
Usage:
  buildevents cmd [flags] BUILD_ID STEP_ID NAME -- [shell command to execute]

Flags:
  -h, --help   help for cmd

Global Flags:
  -a, --apihost string    [env.BUILDEVENT_APIHOST] the hostname for the Honeycomb API server to which to send this event (default "https://api.honeycomb.io")
  -k, --apikey string     [env.BUILDEVENT_APIKEY] the Honeycomb authentication token
  -d, --dataset string    [env.BUILDEVENT_DATASET] the name of the Honeycomb dataset to which to send these events (default "buildevents")
  -f, --filename string   [env.BUILDEVENT_FILE] the path of a text file holding arbitrary key=val pairs (multi-line-capable, logfmt style) to be added to the Honeycomb event
  -p, --provider string   [env.BUILDEVENT_CIPROVIDER] if unset, will inspect the environment to try and detect Travis-CI, CircleCI, or GitLab-CI.

pretty minor, but that usage message makes me think there's a problem with my usage of the buildevents CLI itself. I have to scan up to find the actual error.

[Feature request] Add support for GitHub Actions

Missing in the list of CI providers that are supported out of the box is GitHub Actions. It's a fairly new player, but it's quickly gaining a lot of adoption.
So I'd like to ask if you can add support for GitHub Actions as well.

The available environment variables are documented here: Using environment variables
Most notably they use GITHUB_ACTIONS to indicate that the process is run within the GitHub Actions environment.

I've been able to pass these fields manually using the BUILDEVENT_FILE. But since I wasn't familiar with the logfmt format, this cost me more time that I would've liked.
Example: https://github.com/kvrhdn/tfe-run/blob/master/.github/workflows/integration.yaml#L27

Update to cimg/go image and go 1.16.x

buildevent step - Not working in gitlab-ci

I am trialling pushing buildpipelines to honeycomb. I have followed the guide and I can see the overall build event appearing in the data set. However none of the step events arrive.

I have tested this locally in isolation using docker to replicate the pipeline, set BUILDEVENT_APIKEY with the same API as the pipeline and running the exact same command as the pipeline works. Does anyone have any idea why this is not working with step but works with build

OUTPUT

Downloading artifacts
Downloading artifacts for setup-buildevents (6412303482)...
Downloading artifacts from coordinator... ok        host=storage.googleapis.com id=6412303482 responseStatus=200 OK token=glcbt-65
Executing "step_script" stage of the job script
Using docker image sha256:3cff1c6ff37e0101a081[19](https://gitlab.com/cdoyle27/cicd-observability-example/-/jobs/6412303484#L19)d0743ba95c7f505d00298f5a169129e4e9086cde9e for ubuntu:[20](https://gitlab.com/cdoyle27/cicd-observability-example/-/jobs/6412303484#L20).04 with digest ubuntu@sha256:80ef4a44043dec4490506e6cc4289eeda2d106a70148b74b5ae91ee670e9c35d ...
$ echo "export STEP_START=$(date +%s)" >> ./env
$ echo "export STEP_SPAN_ID=${CI_JOB_ID}" >> ./env
$ source <(cat ./env ${BUILDEVENTS_DIR}/env)
$ touch ${BUILDEVENT_FILE}
$ echo "validating..."
validating...
$ sleep 2
$ echo "validated..."
validated...
Running after_script
Running after script...
$ cat ./env ${BUILDEVENTS_DIR}/env
export STEP_START=1710698317
export STEP_SPAN_ID=6412303484
export BUILD_START=1710698302
export PATH="/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/builds/cdoyle27/cicd-observability-example/buildevents/bin/"
$ source <(cat ./env ${BUILDEVENTS_DIR}/env)
$ echo "buildevents step ${CI_PIPELINE_ID} ${STEP_SPAN_ID} ${STEP_START} ${CI_JOB_NAME}"
buildevents step 1216781316 6412303485 1710698332 some_test_job
$ buildevents step ${CI_PIPELINE_ID} ${STEP_SPAN_ID} ${STEP_START} ${CI_JOB_NAME}
Cleaning up project directory and file based variables
Job succeeded

CI_FILE

.default_before_script: &default_before_script
  - echo "export STEP_START=$(date +%s)" >> ./env
  - echo "export STEP_SPAN_ID=${CI_JOB_ID}" >> ./env
  - source <(cat ./env ${BUILDEVENTS_DIR}/env)
  - touch ${BUILDEVENT_FILE}

.default_after_script: &default_after_script
  - cat ./env ${BUILDEVENTS_DIR}/env
  - source <(cat ./env ${BUILDEVENTS_DIR}/env)
  - echo "buildevents step ${CI_PIPELINE_ID} ${STEP_SPAN_ID} ${STEP_START} ${CI_JOB_NAME}"
  - buildevents step ${CI_PIPELINE_ID} ${STEP_SPAN_ID} ${STEP_START} ${CI_JOB_NAME}


default:
  image: ubuntu:20.04
  before_script:
    - *default_before_script
  after_script:
    - *default_after_script

variables:
  BUILDEVENTS_DIR: buildevents
  BUILDEVENT_FILE: ./buildeventsfile

stages:
  - .pre
  - validate
  - test
  - build
  - .post


###
# Observability setup
###


setup-buildevents:
  stage: .pre
  image: golang:latest
  before_script:
    - mkdir -p ${BUILDEVENTS_DIR}/bin
    - echo "export BUILD_START=$(date +%s)" >> ${BUILDEVENTS_DIR}/env
    - echo "export PATH=\"$PATH:${PWD}/buildevents/bin/\"" >> ${BUILDEVENTS_DIR}/env
  after_script: []
  script:
    - curl -L -o main https://github.com/honeycombio/buildevents/releases/latest/download/buildevents-linux-amd64
    - chmod 755 main
    - mv main ${BUILDEVENTS_DIR}/bin/buildevents

  artifacts:
    paths:
      - ${BUILDEVENTS_DIR}/


send_success_trace:
  stage: .post
  image: golang:latest
  before_script:
    - source <(cat ./env ${BUILDEVENTS_DIR}/env)
  after_script: []
  script:
    - echo " error=false" >> ${BUILDEVENT_FILE}
    - "traceURL=$(buildevents build $CI_PIPELINE_ID $BUILD_START success)"
    - "echo \"Honeycomb Trace: $traceURL\""
  rules:
    - when: on_success

send_failure_trace:
  stage: .post
  image: golang:latest
  before_script:
    - source <(cat ./env ${BUILDEVENTS_DIR}/env)
  after_script: []
  script:
    - echo " error=true" >> ${BUILDEVENT_FILE}
    - "traceURL=$(buildevents build $CI_PIPELINE_ID $BUILD_START failure)"
    - "echo \"Honeycomb Trace: $traceURL\""
  rules:
    - when: on_failure


some_validate_job:
  stage: validate
  script:
    - echo "validating..."
    - sleep 2
    - echo "validated..."

some_test_job:
  stage: test
  script:
    - echo "testing..."
    - sleep 1
    - echo "tested..."

some_build_job:
  stage: build
  script:
    - echo "building..."
    - sleep 2
    - echo "building..."

[CircleCI] watch job takes too long on successful build

The watch job will (correctly) poll to ensure a failed build isn't going to start any more jobs. But when the watch job is the only job left, that means the build is finished, since there are no more jobs available to run. In that case it should exit immediately instead of polling, allowing the build to finish quickly. In the current state the watch job extends the time it takes to build by an additional few minutes on every successful build and that's annoying.

The watch job should be updated to know why it thinks the build is finished and gain some subtlety on when it should accept that it really is and when it should distrust the finished state and poll.

[CircleCI] `watch` aborts on job failure, underreporting workflow duration

Apparently when buildevents watch encounters a failed job, it finishes the root span of the workflow run prematurely while other jobs are still running.

This leads to undercounting the workflow duration.

Log output from buildevents watch $CIRCLE_WORKFLOW_ID:

Oct  3 17:22:53.785: polling for jobs: 3 blocked, 1 failed, 2 running, 26 success
Oct  3 17:22:58.785: polling for jobs: 3 blocked, 1 failed, 1 running, 27 success
Build failed!
https://ui.honeycomb.io/sym-bee-ont/datasets/buildevents/trace?trace_id=d7bb7e39-5d6a-4afe-9900-a42bd71df865&trace_start_ts=1570121965&trace_end_ts=1570123979

In this workflow, the remaining blocked jobs actually ran to completion, but the root spans duration does not reflect that.

[CircleCI] Update data structures to reflect changed keys in Job object

https://github.com/CircleCI-Public/api-preview-docs/blob/master/docs/breaking.md

The key names in the Job object changed. Update to use the new key names.

Note that this likely needs to happen in https://github.com/maplebed/go-circleci and then this project updated to use the newer library.

why doesn't this project use itself?

Seems like an easy way to provide an example of how it can be used.

Buildevent API key failure is vague

When setting up buildevents, if the BUILDEVENT_APIHOST environment variable has JUST a hostname, and not a scheme + hostname, the failure message is "unable to verify API key" which is misleading.

Add public documentation for buildevents

Is your feature request related to a problem? Please describe.

Currently, in the main docs site, there is only a single link out to buildevents. This makes discovery hard.

Describe the solution you'd like

Document buildevents in the main docs site.

Describe alternatives you've considered

Additional context

Suggestions on instrumenting travisCI with parallel stages/jobs

Hi, I was wondering if you had some suggestions on how to instrument a travisCI configuration that has multiple stages/jobs that are running in parallel. The way I'm currently thinking makes each job its own trace.

[feature request] some way to add additional fields to buildevents

I would like to add the size of built artifacts to the data buildevents sends to Honeycomb. There is currently no way to augment any of the spans sent with custom fields. Beyond specifically adding the size of artifacts, I think buildevents will greatly benefit from a method of adding relevant fields in a generic fashion. (eg maybe you could add the name of the test that failed the build! or the github commit that triggered the build! the possibilities are endless!)

Generate 8-byte span IDs to conform with OpenTelemetry spec (instead of 16-byte currently)

Thank you for making this tool! It made it super easy to instrument our CircleCI builds and deploys.

Is your feature request related to a problem? Please describe.
tl;dr of problem: buildevents cmd generates 16-byte span IDs, but OpenTelemetry specifies (and the Python library implements) an 8-byte limit for span IDs. This blocks us from connecting Python/OpenTelemetry spans to a buildevents trace.

Longer version: I am using buildevents to instrument CircleCI workflows that call many Python scripts. I am using OpenTelemetry to instrument some of those Python scripts. I am trying to include the spans generated in the Python scripts as child spans to those generated by buildevents by using the HONEYCOMB_TRACE propagation data sent by buildevents cmd. buildevents is generating 16-byte span IDs, but OpenTelemetry only accepts 8-byte span IDs. I have been unable to figure out a way to set a root span in Python's OpenTelemetry implementation with the span IDs I get from buildevents.

Describe the solution you'd like
Ideally, buildevents would generate 8-byte span IDs to match the OpenTelemetry spec. I noticed the 16-byte IDs happening here in cmd_cmd.go, but it may be happening in other parts of the code.

Describe alternatives you've considered

Don't include Python spans as children of the overall CircleCI workflow traces. This is what we'll be doing for now.
Patch our own version of OpenTelemetry to allow larger Span IDs. I'd rather not incur the extra overhead of managing our own fork and keeping it updated with upstream.
Patch our own version of buildevents to generate smaller Span IDs. Similar issue with the overhead. Plus we don't currently have any go code or build systems, so we'd also need to manage that.
I'm open to other alternatives that we haven't considered yet, would love a workaround!

Additional context
I think I covered it all, but happy to answer any questions.

Make buildevents more generic by using OpenTelemetry instead of beeline lib.

Is your feature request related to a problem? Please describe.

No.

Describe the solution you'd like

The buildevents library is very handy to get insight into a CI system, but the current implementation is limited to Honeycomb.io because it builds against the Honeycomb beeline library. If we would switch to use the Opentelemetry Go SDK, output could be collected on anything supporting the OpenTelemetry Collector, still including Honeycomb.io.

Describe alternatives you've considered

Searched for alternatives, but haven't found anything which came as close to buildevents

Additional context

Occasional errors returned by buildevents

The error we see looks like this:

/usr/local/bin/buildevents: line 1: syntax error near unexpected token `newline'
/usr/local/bin/buildevents: line 1: `<!DOCTYPE html>'

For full reference we're triggering buildevents via the following command to minimise some boilerplate stuff in each project:

curl https://gitlab.example.com/pub/buildevents-boilerplate/raw/master/buildevents.sh | bash -s step

and that script looks like this:

#!/bin/sh

set -e

BUILDEVENTS_BIN_URL=https://github.com/honeycombio/buildevents/releases/latest/download/buildevents-linux-amd64
BUILDEVENTS_BIN=/usr/local/bin/buildevents

if [ -z "$BUILDEVENT_APIKEY" ]; then
    echo "error: Missing BUILDEVENT_APIKEY environment variable"
    echo "Check the CI/CD variables on Gitlab"
    exit 1
fi

if [ -z "$BUILDEVENT_DATASET" ]; then
    echo "warning: Missing BUILDEVENT_DATASET environment variable"
    echo "\`buildevents\` will be used by default"
fi

curl -L -o $BUILDEVENTS_BIN $BUILDEVENTS_BIN_URL
chmod +x $BUILDEVENTS_BIN

case "$1" in
    "init")
        echo "Initialising buildevents..."
        mkdir -p buildevents && date +%s > buildevents/buildstart
        ;;
    "start")
        echo "Logging step start..."
        mkdir -p buildevents && date +%s > buildevents/stepstart
        ;;
    "step")
        echo "Adding step span..."
        "$BUILDEVENTS_BIN" step "${CI_PIPELINE_ID}" "${CI_JOB_ID}" "$(cat buildevents/stepstart)" "${CI_JOB_NAME}"
        ;;
    "build")
        "$BUILDEVENTS_BIN" build "${CI_PIPELINE_ID}" "$(cat buildevents/buildstart)" success
        ;;
    *)
        echo "error: Unexpected <"$1"> argument for buildevents.sh"
        echo "Valid values are: init, start, step, and build"
        exit 1
        ;;
esac

Full output for that command looks like this:

$ curl https://gitlab.example.com/pub/buildevents-boilerplate/raw/master/buildevents.sh | bash -s start
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  1305  100  1305    0     0  14711      0 --:--:-- --:--:-- --:--:-- 14829
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  9269    0  9269    0     0  18919      0 --:--:-- --:--:-- --:--:-- 18955
Logging step start...
$ if [ "$SHELL" = "/bin/bash" ]; then shopt -s expand_aliases; fi && alias buildevents-cmd="/usr/local/bin/buildevents cmd ${CI_PIPELINE_ID} \"${CI_JOB_ID}\""
$ buildevents-cmd storybook-build -- yarn run storybook:build
/usr/local/bin/buildevents: line 1: syntax error near unexpected token `newline'
/usr/local/bin/buildevents: line 1: `<!DOCTYPE html>'
Running after script...
$ curl https://gitlab.example.com/pub/buildevents-boilerplate/raw/master/buildevents.sh | bash -s step
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  1305  100  1305    0     0  13885      0 --:--:-- --:--:-- --:--:-- 14032
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  9269    0  9269    0     0  48459      0 --:--:-- --:--:-- --:--:-- 48528
Adding step span...
/usr/local/bin/buildevents: line 1: syntax error near unexpected token `newline'
/usr/local/bin/buildevents: line 1: `<!DOCTYPE html>'
ERROR: Job failed: command terminated with exit code 1

I only happen to have seen this on step commands so far.

I'm not sure if this is an error being returned by the Honeycomb API and not handled correctly here? Or are we doing something wrong?

de

buildevents cmd may result in leaking sensitive information

Hey, sometimes variables are passed to CI commands that may contain sensitive information such as passwords, this makes it impossible to record those CI events with buildevents cmd since it would result in sending expanded variables with secrets to Honeycomb.

A couple of options how to try to solve it:

Record cmd attribute before parameters are expanded - not sure if this possible since buildevents would already receive an expanded bash command.
Provide an optional parameter to buildevents cmd that would allow to override or disable the cmd attribute.

add ability to use before/after invocations to wrap a `cmd`

It's lovely to pass an executable as an argument to buildevents and have it automatically time it. There are conditions during which that is not easy, and it'd be nice to have a syntax where you can run a wrap-setup command before and wrap-finish command after and still produce a cmd span.

example: in jenkinsx yml, you can override steps in a buildpack with extra things to do before and after that step. Using these you can wrap existing buildpacks with extra functionality without changing the buildpack itself. Using the current buildevents paradigm, you can use a step command to do each of those by recording the timestamp before and then using that timestamp after, but that restricts the build trace to build and step - there's no way to use cmd and get yourself 3 levels of hierarchy.

Leading whitespace around trace identifiers breaks traces

In some circumstances due to the wonders of bash quoting, it's possible to invoke buildevents with arguments that have leading or trailing spaces. The effect is that a span's parent ID may not match the parent span's ID because one will have a leading space and the other won't. This seemingly insignificant change breaks the trace waterfall view since it requires an exact match between span's parent IDs and the parent span's ID.

Closer examination of the tracing identifiers explains why it is broken

The whitespace differences break the trace.

don't require APIKEY

Is your feature request related to a problem? Please describe.
In our setup, we can send the traces directly to a local endpoint, without authentication

We can't use buildevents because it fails with:

$ buildevents build $BUILD_ID $STEP_START success
Unable to create trace URL: unable to verify API key: config.APIKey and config.WriteKey are both empty; can't verify empty key

Describe the solution you'd like

Make it possible to run buildevents without setting up an APIKEY.

Describe alternatives you've considered

Additional context

patch openSSL CVE

https://mta.openssl.org/pipermail/openssl-announce/2022-March/000219.html

build artifacts: ~~circleci/golang image is based on golang image which is based on alpine - waiting on the same update~~
- use newer circle images cimg/go:1.18.1

honeycombio / buildevents Goto Github PK

buildevents's People

Contributors

Stargazers

Watchers

Forkers

buildevents's Issues

Recommend Projects

Recommend Topics

Recommend Org