googlecloudplatform / cloud-build-notifiers

Notifier images for Cloud Build, complete with build status filtering and Google Secret Manager integration

Cloud Build Notifiers

This repo provides deployable notifier images and sources, as well as libraries for creating new notifiers.

Cloud Build notifiers are Docker containers that connect to the Cloud Build Pub/Sub topic that adapt Pub/Sub messages about Build update notifications to other services/protocols, such as SMTP for email. Cloud Build notifiers are long-lived binaries that receive notifications throughout Builds' lifecycles (e.g. from the Build starting to execute through the Build finishing).

All notifiers can be built by Cloud Build and deployed on Cloud Run. The only prerequisite is to be a Cloud Build user and to have the gcloud CLI tool installed and configured for your Cloud Build project(s).

There are currently 4 supported notifier types:

• bigquery, which writes Build updates and related data to a BigQuery table.
• http, which sends (HTTP POSTs) a JSON payload to another HTTP endpoint.
• slack, which uses a Slack webhook to post a message in a Slack channel.
• smtp, which sends emails via an SMTP server.

See the official documentation on Google Cloud for how to configure each notifier:

• Configuring BigQuery notifications
• Configuring HTTP notifications
• Configuring Slack notifications
• Configuring SMTP notifications

Setup Script

A setup script exists that should automate most of the notifier setup.

Run ./setup.sh --help for usage instructions.

Common Flags

The following are flags that belong to every notifier via inclusion of the lib/notifiers library.

--smoketest

This flag starts up the notifier image but only logs the notifier name (via type) and then exits.

--setup_check

This flag starts up the notifier, which does the following:

1. Read the notifier configuration YAML from STDIN.
2. Decode it into a configuration object.
3. Attempt to call notifier.SetUp on the given notifier using the configuration and a faked-out SecretGetter.
4. Exit successfully unless one of the previous steps failed.

This can be done using the following commands:

# First build the notifier locally.
$ sudo docker build . \
    -f=./${NOTIFIER_TYPE}/Dockerfile --tag=${NOTIFIER_TYPE}-test
# Then run the `setup_check` with your YAML.
# --interactive to allow reading from STDIN.
# --rm to clean/remove the image once it exits.
$ sudo docker run \
    --interactive \
    --rm \ 
    --name=${NOTIFIER_TYPE}-test \
    ${NOTIFIER_TYPE}-test:latest --setup_check --alsologtostderr -v=5 \
    < path/to/my/config.yaml 

License

This project uses an Apache 2.0 license.

Contributing

See here for contributing guidelines.

Support

There are several ways to get support for issues in this project:

• Cloud Build Slack channel
• Cloud Build Issue Tracker
• General Google Cloud support

Note: Issues filed in this repo are not guaranteed to be addressed. We recommend filing issues via the Issue Tracker.

cloud-build-notifiers's Issues

HTML variable SMTP sending

Hi,
I need to display the variables commitSha, branchName, tagName and repoName, in the HTML body inside main.go, can anyone help?

Thanks

Not able to use GCP Cloud Build Trigger Name in Email Subject or Body

I have tried to add to Trigger name in Subject but getting error while build the docker image.
Attached screenshot for reference.

[bigquery] notifier fails to insert messages missing step Timing information

In fixing #110, @prabenzom discovered that build notifications are often missing step.Timing information. This appears to be the case for most/all non-SUCCESSFUL statuses. The end result is that most/all non-SUCCESSful status notifications get dropped on the floor and the data in BQ is woefully inaccurate.

Example error message:

"E0527 00:54:23.418135       1 notifiers.go:449] failed to run SendNotification: Error inserting row into BQ: 1 row insertion failed (insertion of row [insertID: "YuH2meB46KYBgLxZNid82J1zGHU"; insertIndex: 0] failed with error: {Location: "steps[5].endtime"; Message: "Invalid datetime string \"0000-00-00 00:00:00\""; Reason: "invalid"})"

The relevant piece of code is here:

cloud-build-notifiers/bigquery/main.go

Lines 237 to 257 in 701ad5f

	for _, step := range build.GetSteps() {
	newStep := &buildStep{
	Name: step.Name,
	ID: step.Id,
	Status: step.GetStatus().String(),
	Args: step.Args,
	}
	if step.GetTiming().GetStartTime() != nil {
	startTime, err := parsePBTime(step.GetTiming().GetStartTime())
	if err == nil {
	newStep.StartTime = startTime
	}
	}
	if step.GetTiming().GetEndTime() != nil {
	endTime, err := parsePBTime(step.GetTiming().GetEndTime())
	if err == nil {
	newStep.EndTime = endTime
	}
	}
	buildSteps = append(buildSteps, newStep)
	}

Proposed fix

Somehow, build notifications that are missing step.Timing information should NOT be dropped on the floor. So if the BQ schema requires that non-empty timing information be present, then I think some default timing info should be set for the steps.

• Maybe you could use the overall startTime and finishTime
• Maybe you could just set them to some default golang epoch value
• Something else

[bug] aws ses send email fail ?

Hi,

I use aws ses to send cloudbuild email ,bu in the cloud run log I can see some error:
E1210 04:12:44.669479 1 notifiers.go:449] failed to run SendNotification: failed to send email: 554 Message rejected: Email address is not verified. The following identities failed the check in region US-EAST-1: XXXXXXXXXXXXXXXX(My ses key)

But I write a easy go code to test ses send, is success.

there is My test code:

import (
	"log"
	"net/smtp"
)

func main() {
	// Set up authentication information.
	auth := smtp.PlainAuth("", "ses-key", "ses-secret", "ses-server")

	// Connect to the server, authenticate, set the sender and recipient,
	// and send the email all in one step.
	to := []string{"[email protected]"}
	msg := []byte("To: [email protected]\r\n" +
		"Subject: discount Gophers!\r\n" +
		"\r\n" +
		"This is the email body.\r\n")
	err := smtp.SendMail("server:587", auth, "[email protected]", to, msg)
	if err != nil {
		log.Fatal(err)
	}
}

BQ notifier should use the new `storage` UTM medium

We added it in #54 but never assigned it.

Slack notifier fails to deploy because it doesn't listen to port 8080

I'm following steps in https://cloud.google.com/build/docs/configuring-notifications/configure-slack but on step 9, when I run the command:

 gcloud run deploy slack-notifier \
   --image=us-east1-docker.pkg.dev/gcb-release/cloud-build-notifiers/slack:latest \
   --no-allow-unauthenticated \
   --update-env-vars=CONFIG_PATH=gs://******/slack/configuration.yaml,PROJECT_ID=******

(hiding my own project id in the example above)

it fails to deploy because it says that the container is not listening in port 8080. This is the message I get:

Deploying container to Cloud Run service [slack-notifier-v1] in project [******] region [europe-southwest1]
⠧ Deploying new service... Revision 'slack-notifier-v1-00001-lr2' is not ready and cannot serve traffic. The user-provided container failed to start and listen on the port defined provided by the PORT=8080 envi
ronment variable. Logs for this revision might contain more information.                                                                                                                                          
  ⠶ Routing traffic...                                                                                                                                                                                            
Logs URL: https://console.cloud.google.com/logs/viewer?project=******&resource=cloud_run_revision/service_name/slack-notifier
-v1/revision_name/slack-notifier-v1-00001-lr2&advancedFilter=resource.type%3D%22cloud_run_revision%22%0Aresource.labels.service_name%3D%22slack-notifier-v1%22%0Aresource.labels.revision_name%3D%22slack-notifier
-v1-00001-lr2%22                                                                                                                                                                                                  
X Deploying new service... Revision 'slack-notifier-v1-00001-lr2' is not ready and cannot serve traffic. The user-provided container failed to start and listen on the port defined provided by the PORT=8080 envi
ronment variable. Logs for this revision might contain more information.                                                                                                                                          
  X Routing traffic... Revision 'slack-notifier-v1-00001-lr2' is not ready and cannot serve traffic. The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environm
Logs URL: https://console.cloud.google.com/logs/viewer?project=******&resource=cloud_run_revision/service_name/slack-notifier                                                                          
-v1/revision_name/slack-notifier-v1-00001-lr2&advancedFilter=resource.type%3D%22cloud_run_revision%22%0Aresource.labels.service_name%3D%22slack-notifier-v1%22%0Aresource.labels.revision_name%3D%22slack-notifier
-v1-00001-lr2%22                                                                                                                                                                                                  
For more troubleshooting guidance, see https://cloud.google.com/run/docs/troubleshooting#container-failed-to-startabels.service_name%3D%22slack-notifier-v1%22%0Aresource.labels.revision_name%3D%22slack-notifier
  -v1-00001-lr2%22                                                                                                                                                                                                
For more troubleshooting guidance, see https://cloud.google.com/run/docs/troubleshooting#container-failed-to-start



Deployment failed                                                                                                                                                                                                 
ERROR: (gcloud.run.deploy) Revision 'slack-notifier-v1-00001-lr2' is not ready and cannot serve traffic. The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable. Logs for this revision might contain more information.

Could it be that current image is broken?

[bigquery] notifier crashes with panic nil dereference

I'm using the BigQuery notifier, and it works sometimes but seems to crash often w/ the following stack trace:

2021/05/21 19:16:54 http: panic serving 169.254.8.129:39528: runtime error: invalid memory address or nil pointer dereference
goroutine 49 [running]:
net/http.(*conn).serve.func1(0xc0003c0e60)
	/usr/local/go/src/net/http/server.go:1824 +0x153
panic(0xd78aa0, 0x1592d30)
	/usr/local/go/src/runtime/panic.go:971 +0x499
main.(*bqNotifier).SendNotification(0xc0003d2d50, 0xff2320, 0xc0001cc240, 0xc000231500, 0x2928, 0x3000)
	/go-src/bigquery/main.go:238 +0x846
github.com/GoogleCloudPlatform/cloud-build-notifiers/lib/notifiers.newReceiver.func1(0xff09c0, 0xc000337260, 0xc000460200)
	/go-src/lib/notifiers/notifiers.go:448 +0x938
net/http.HandlerFunc.ServeHTTP(0xc0005e0ea0, 0xff09c0, 0xc000337260, 0xc000460200)
	/usr/local/go/src/net/http/server.go:2069 +0x44
net/http.(*ServeMux).ServeHTTP(0x15af920, 0xff09c0, 0xc000337260, 0xc000460200)
	/usr/local/go/src/net/http/server.go:2448 +0x1ad
net/http.serverHandler.ServeHTTP(0xc000337180, 0xff09c0, 0xc000337260, 0xc000460200)
	/usr/local/go/src/net/http/server.go:2887 +0xa3
net/http.(*conn).serve(0xc0003c0e60, 0xff23c8, 0xc0001cc180)
	/usr/local/go/src/net/http/server.go:1952 +0x8cd
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3013 +0x39b 

The config file I'm using is:

apiVersion: cloud-build-notifiers/v1
kind: BigQueryNotifier
metadata:
  name: google-cloud-cpp-bigquery-notifier
spec:
  notification:
    filter: build.substitutions["REPO_NAME"] == "google-cloud-cpp"
    delivery:
      table: projects/936212892354/datasets/cloudbuild/tables/google_cloud_cpp

My project is tracking this bug at googleapis/google-cloud-cpp#6580.

[ask] conditional on message.json

is it possible todo if condition on message.json ?

Secrets can't be read across projects

I have a bunch of projects that each run separate cloudbuild deployments. To simplify life, my Slack webhook URL is not replicated across these projects, although each project's cloudbuild service account has access to the secret.

So, I have cloud-build-notifiers running in projects 1-N, and the secret they need in project M. I pass in the full path to the secret, but apparently this is not allowed:

1 main.go:33] fatal error: failed to call SetUp on notifier: failed to get token secret: failed to get secret named "projects/some-project/secrets/cloud-build-slack-webhook-url": rpc error: code = InvalidArgument desc = Resource ID [projects/some-project/secrets/cloud-build-slack-webhook-url] is not in a valid format.

[Request] Allow using secret value for HTTP Notifier webhook

We'd like to be able to use the Http Notifier to call webhooks which have a secret token as part of the url, the same as is allowed by the the Google Chat notifier.

Update Secret Manager usage from v1beta1 to v1 API

secretmanager.googleapis.com has v1beta1 and v1 APIs. After Secret Manager went GA in March 2020, new features are only being added to the v1 version of the API.

Update `bq` CLI docs section with formatting arg

I recommend using the --format=prettyjson flag for any bq CLI commands so that the output is more readable.

Slack Notifier failed to unmarshal templating JSON

I have been following these instructions on the GCP documentation page. I have done nothing custom beyond substituting variables.

However, it is not working and I see tons of error messages in the cloud run logs saying:

1 notifiers.go:549] failed to run SendNotification: failed to write Slack message: failed to unmarshal templating JSON: unexpected end of JSON input"

There is a somewhat similar closed issue here - and the solution was to pin an older version. If thats the case, what version should it be?

Also, how would I set the retry limit invocations to the cloud run service from pubsub? I think would have racked up quite the bill if I had not been looking.

When the BQ notifier does "ensuring" work in response to an error, the logs should indicate as such

https://github.com/GoogleCloudPlatform/cloud-build-notifiers/blob/master/bigquery/main.go#L288 prints the error but then doesn't say that it's going to attempt to fix the issue, which is confusing as a user reading the logs.

[SLACK NOTIFIER] Region not contained in the url

I've set up the notifier using instructions from the official GCP page: https://cloud.google.com/build/docs/configuring-notifications/configure-slack

The messages are sent to the Slack channel, but without region, causing the GCP to throw an "Unable to find the resource that you requested" error.

Is it possible to somehow create a proper URL, with a region? Is some extra coding required or have I failed to set the hook properly?

Example:
The link on the slack channel:

https://console.cloud.google.com/cloud-build/builds/<build-id>?project=<project-id>&utm_campaign=google-cloud-build-notifiers&utm_medium=chat&utm_source=google-cloud-build

While it should be:

https://console.cloud.google.com/cloud-build/builds;region=<region>/<build-id>?project=<project-id>&utm_campaign=google-cloud-build-notifiers&utm_medium=chat&utm_source=google-cloud-build

Surfacing author / username information to Slack notifier (GH)

Is there any way to surface author information? I see some references to _COMMIT_AUTHOR_EMAIL, but, by default, neither that or anything like it is surfaced in the payload I see when using GitHub triggers:

cloud-build-notifiers/lib/notifiers/jsonpath_test.go

Line 96 in 140b8ad

"_COMMIT_AUTHOR_EMAIL": "$(build.substitutions._COMMIT_AUTHOR_EMAIL)",

Is there a way to get the author email and / or GitHub email into something I can use downstream in the notifier?

Unable to configure cloud build slack notifier

gcloud run deploy slack-ci-cd-notification \
>    --image=us-east1-docker.pkg.dev/gcb-release/cloud-build-notifiers/slack:latest \
>    --no-allow-unauthenticated \
>    --update-env-vars=CONFIG_PATH=gs://tutubi-project/slack-notifier.yaml,PROJECT_ID=tutubi-75285
Deploying container to Cloud Run service [slack-ci-cd-notification] in project [tutubi-75285] region [europe-west3]
⠛ Deploying... Cloud Run error: The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable. Logs for this revision might contain more information.
  ⠏ Creating Revision...
Logs URL: https://console.cloud.google.com/logs/viewer?pro
ject=tutubi-75285&resource=cloud_run_revision/service_name/slack-ci-cd-notification/revision_name/slack-ci-cd-notification-00004-sip&advancedFilter=resource.type%3D%22cloud_run_revision%22%0Aresource.labels.service_name%3D%22slack-ci-cd-notification%22%0Aresource.label
s.revision_name%3D%22slack-ci-cd-notification-00004-sip%22
For more troubleshooting guidance, see https://cloud.google.com/run/docs/troubleshooting#container-failed-to-start
X Deploying... Cloud Run error: The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable. Logs for this revision might contain more information.

Logs URL: https://console.cloud.google.com/logs/viewer?pro
ject=tutubi-75285&resource=cloud_run_revision/service_name/slack-ci-cd-notification/revision_name/slack-ci-cd-notification-00004-sip&advancedFilter=resource.type%3D%22cloud_run_revision%22%0Aresource.labels.service_name%3D%22slack-ci-cd-notification%22%0Aresource.label
s.revision_name%3D%22slack-ci-cd-notification-00004-sip%22
For more troubleshooting guidance, see https://cloud.google.com/run/docs/troubleshooting#container-failed-to-start
  . Routing traffic...
  ✓ Setting IAM Policy...



Deployment failed
ERROR: (gcloud.run.deploy) Cloud Run error: The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable. Logs for this revision might contain more information.

I am following this doc https://cloud.google.com/build/docs/configuring-notifications/configure-slack
Stuck on Step-9 (Deploy your notifier to Cloud Run)

should extract build id in another way when build status is unknown

In chat notifier, we extract the build id from the first 8 chars in the build id Here. But build id is empty when the build has status unknown and would throw this error:

http: panic serving 169.254.1.1:35855: runtime error: slice bounds out of range [:8] with length 0 goroutine 47

We should manually set the build id when the build status is unknown.

{{.Build.FailureInfo.Detail}} doesn't work.

{{.Build.FailureInfo.Type}} works

{{.Build.FailureInfo.Detail}} doesnt work
E1111 06:46:23.171236 1 notifiers.go:549] failed to run SendNotification: failed to write Slack message: failed to unmarshal templating JSON: invalid character 'u' after object key:value pair

full message.

  "failureInfo": {
    "type": "USER_BUILD_STEP",
    "detail": "Build step failure: build step 0 \"ubuntu\" failed: starting step container failed: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"echox\": executable file not found in $PATH: unknown"
  }

BQ notifier should store substitutions

This can just be an ARRAY of key/value pairs.

[HTTP Notifications] Empty Body

Hi! I've setup a flask http server and followed the instructions here to be able to receive notifications on our CI/ID in cloud build. Here is my configuration file:

apiVersion: cloud-build-notifiers/v1
kind: HTTPNotifier
metadata:
  name: example-http-notifier
spec:
  notification:
    filter: build.status in [Build.Status.SUCCESS, Build.Status.FAILURE, Build.Status.TIMEOUT, Build.Status.WORKING]
    delivery:
      # The `http(s)://` protocol prefix is required.
      url: https://ef02-152-32-107-144.ngrok.io/gcp/payload

I'm currently using ngrok to test locally and the filters are working and are receiving the requests send by the HTTP Notifier. However, instead of receiving a json payload, I get an empty body. Here are values of the header and body received:

Host: ef02-152-32-107-144.ngrok.io
User-Agent: GCB-Notifier/0.1 (http)
Content-Length: 0
Accept-Encoding: gzip
Content-Type: application/json
X-Forwarded-For: 2600:1900:2000:da::5
X-Forwarded-Proto: https

Body: '' (just an empty string)

Any reason for the empty body instead of the json payload?

Thank you!

secrets can't be read from different projects

I have a bunch of projects that each run separate cloudbuild deployments. To simplify life, my Slack webhook URL is not replicated across these projects, although each project's cloudbuild service account has access to the secret.

So, I have cloud-build-notifiers running in projects 1-N, and the secret they need in project M. I pass in the full path to the secret, but apparently this is not allowed:

1 main.go:33] fatal error: failed to call SetUp on notifier: failed to get token secret: failed to get secret named "projects/some-project/secrets/cloud-build-slack-webhook-url": rpc error: code = InvalidArgument desc = Resource ID [projects/some-project/secrets/cloud-build-slack-webhook-url] is not in a valid format.

Any/all BQ query docs should use the `projectID.datasetName.tableName` data source format.

I encountered an issue in the BQ UI where I was unable to query a table in my current project unless I prefixed the "data source" (the FROM noun) with the project ID.

unmarshal error thrown for pubsub notifications in slack and smtp notifiers

I recently (last week) deployed slack and smtp notifiers for some Cloud Builds from this repo using the automated setup.sh script approach as described here and here.

I deployed them in 2 projects. In one of the projects the notifications (or at least some of them) are coming through and I do see logs of valid pubsub messages received and processed amidst the errors. In the other projects I don't see any logs of valid messages received and processed and am not receiving any of the notifications.

I'm including screenshots of Cloud logging logs related to the error (which is from line 410 of notifiers.go here ) as well as the logs of the valid received/processed pubsub notifications

Lack of IGNORE_BAD_MESSAGES is logged

I'm seeing a lot of this in stderr, and it's not useful or relevant for end-users. Maybe remove it?

env var "IGNORE_BAD_MESSAGES" is empty

Huge delay between build and status

The statuses sent to Slack often happened hours/days after it actually happened on GCB.

A last example is a finished build's status was sent 30 hours after to Slack.

Failed to get reader

I followed the instructions for getting slack notifications set up but when I tried to deploy the cloudrun service, I have not been able to get past this error. I haven't been able to find a place where my configurations are different than the ones shown in the examples and my service account has access to the configuration file. (If I remove the storage permissions I get the 403 error as expected)

fatal error: failed to get config from GCS: failed to get reader for (bucket="", object="")

Received 500: Could not fetch URI /computeMetadata/v1/instance/service-accounts/default/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.full_control

I launched the GCP console and ran:
curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.full_control"

which worked successfully so I don't know where the disconnect is.

Any advice would be greatly appreciated.

regular expression matching in filter field seems to have problem

I was trying to use a regular expression match in the filter expression for an smtp-notifier but it causes a crash failure on deploy as seen in the screenshots

Template vars passed to slack.json (Block Kit template) should escape or replace double quotation marks (ie. {{.Build.FailureInfo.Detail}})

I believe this is causing issue #153.

I'm using build details in my custom slack message template:

{
        "type": "section",
        "text": {
            "type": "mrkdwn",
	    "text": "*Error Type*: {{.Build.FailureInfo.Type}}\n*Details*: {{.Build.FailureInfo.Detail}}\n<!channel>"
        }
    }

However, this throws the following error when my Cloud Build fails.

E0308 01:13:48.504548       1 notifiers.go:549] failed to run SendNotification: failed to write Slack message: failed to unmarshal templating JSON: invalid character 'g' after object key:value pair

An invalid char 'g', you say? When I check the failureInfo.detail property from this build, it has double quotes around the build step, which makes the 'g' from gcr.io/google-etcetc the first char after the quotes.

failureInfo:
  detail: 'Build step failure: build step 7 "gcr.io/google-appengine/exec-wrapper"
    failed: step exited with non-zero status: 1'
  type: USER_BUILD_STEP

[bug] Project and Region needs to be specified

After following the instructions on this page, running the setup.sh script failed for the different reasons I've specified below along with how I resolved them. Hopefully they'll help the maintainers improve the script and help others who might hit the same stumbling blocks.

Missing Documentation

The script does not set the current project for you. Before running the setup.sh script, set your current project using gcloud config set project <project-id>.

Recommended Solution
Add a step to the instructions to set the project before running the setup.sh script. Alternatively, set the project automatically within the setup.sh script.

Region Must Be Specified

The deploy_notifier() function should specify the region, otherwise it can return an error if no default region is configured. For my needs I just hard coded the region us-central1 into the deployment command, but that should be a variable passed as an argument.

deploy_notifier() {
  gcloud run deploy "${SERVICE_NAME}" \
    --image="${IMAGE_PATH}" \
    --no-allow-unauthenticated \
    --region us-central1 \
    --update-env-vars="CONFIG_PATH=${DESTINATION_CONFIG_PATH},PROJECT_ID=${PROJECT_ID}" ||
    fail "failed to deploy notifier service -- check service logs for configuration error"
}

Recommended Solution
Pass the region as another argument when running setup.sh.

Pub/Sub Invalid Push Url

The build failed a third time with the error:
ERROR: Failed to create subscription [projects/<project-id>/subscriptions/slack-subscription]: You have passed an invalid argument to the service (argument=PushConfig::push_endpoint).

The SERVICE_URL value was empty because the gcloud run services describe command was called without a region specified, so it was returning an error and pushing an empty value to the --push-url argument. My quick solution was to simply hard code the SERVICE_URL value on line 111 with the deployed Cloud Run service's URL and it worked for my needs.

Recommended Solution
Ensure the region parameter is passed to the gcloud run services describe command, like this:

  SERVICE_URL=$(gcloud run services describe "${SERVICE_NAME}" \
    --region="${REGION}" 
    --format="value(status.url)")

BQ notifier should dedupe image SHAs

When a Build creates Images that have the same SHAs (e.g. assigning multiple Docker tags to the same Image), we should dedupe so we're not storing the same data.

Please update Secret Manager name in repository description

The current repository description is:

Notifier images for Cloud Build, complete with build status filtering and Cloud Secret Manager integration

Could we please update the name of Secret Manager? It's either "Google Secret Manager" or "Secret Manager". There is no "Cloud" in the name.

Notifier images for Cloud Build, complete with build status filtering and Google Secret Manager integration.

/cc @bendory

FR: Notifier Templates and Bindings

Hello from the Cloud Build team 👋

We are aware that many Notifiers users would like to have custom templating and config bindings available to make their notifications more useful. Our product team and I are working on a requirements doc for these features that we plan on releasing here to open up the coding of this feature to the public.

I'll respond here with the doc once we have it in a good state.

Passing body and headers to HTTPNotifier

How can I use HTTPNotifier to call a web hook and pass besides url, headers and body to my request?

[enhancement]Customize Slack Notification message

I want custom notification message.

Now it is very simple message, I can't specify which trigger was pushed when using multiple CloudBuild settings.

If there are these option , It will be very useful.

• CloudBuild trigger name
• linked repository, branch, commit hash
• trigger event

allow optional configuration of log levels

Currently configuration of the logger can only be controlled from the command line which is hard-coded in the Dockerfile. For debugging notifiers, including custom notifiers, it would be nice if the notifier configuration supported optional parameters that allowed for the configuration of the logger (e.g. log level = debug).

If the configuration properties for logging are not specified, then the notifier should default to the current behavior.

[SLACK] color side bar is not working in notification messages

Currently (1.15.0 or higher), the color sidebar is not working.
The "color" is only available for an element in attachments.
But, now, "blocks" is placed outside of "attachments".

So, to get the side color bar, we should change the position of the "blocks".

-	return &slack.WebhookMessage{Attachments: []slack.Attachment{{Color: clr}}, Blocks: &blocks}, nil
+	return &slack.WebhookMessage{Attachments: []slack.Attachment{{Color: clr, Blocks: &blocks}}}, nil

But, the attachments field is not recommended.
ref: https://api.slack.com/reference/messaging/payload

I cannot find a way to add a color side bar in Block Kit.

Convert notifier codebase to Function Framework

See https://github.com/GoogleCloudPlatform/functions-framework-go.

Failure to unmarshal E2_HIGHCPU_32 machineType

Hi Team,

Similarly to what's been reported previously in #105, our notifier fails to unmarshal E2_HIGHCPU_32 machineType:

notifiers.go:440] failed to unmarshal PubSub message id="" data="<REDACTED>" publishTime="2022-08-26T19:31:45.596Z" into a Build: proto: (line 33:20): invalid value for enum type: "E2_HIGHCPU_32"

It seems that the associated proto already has it - so perhaps the proto package just need to be updated? Although the go.mod version seems to be later than the one where this machine type was added.

Thanks

The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable

When trying to deploy --image=us-east1-docker.pkg.dev/gcb-release/cloud-build-notifiers/smtp:latest in my project, I get the below error-
The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable

SlackNotifier Error "field template not found in type notifiers.Notification"

Hi guys,

I'm getting this error on deployment :

Container called exit(255).
/go-src/slack/main.go:33 +0x7b
...
line 10: field template not found in type notifiers.Notification
...
main.go: fatal error: failed to get config from GCS: failed to parse configuration from YAML at "xxx/slack/cloud-build/conf.yaml": yaml: unmarshal errors:

My config file seems reachable. When I specify a not existing config path, it throws a different error.

Any idea?

conf.yaml

apiVersion: cloud-build-notifiers/v1
kind: SlackNotifier
metadata:
  name: cloud-build-slack-notifier
spec:
  notification:
    delivery:
      webhookUrl:
        secretRef: webhook-url
    template:
      type: golang
      uri: gs://xxx/slack/cloud-build/message.json

  secrets:
    - name: webhook-url
      value: projects/xxx/secrets/xxx/versions/latest

message.json

[
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "Cloud Build {{.Build.ProjectId}} {{.Build.Id}} {{.Params.buildStatus}}"
      }
    },
    {
      "type": "divider"
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "View Build Logs"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "text": "Logs"
        },
        "value": "click_me_123",
        "url": "{{.Build.LogUrl}}",
        "action_id": "button-action"
      }
    }
  ]

 gcloud run deploy slack-notify-cloud-build \
   --image=us-east1-docker.pkg.dev/gcb-release/cloud-build-notifiers/slack:slack-1.14.0 \
   --no-allow-unauthenticated \
   --update-env-vars=CONFIG_PATH=gs://xxx/slack/cloud-build/conf.yaml,PROJECT_ID=xxx

Slack Notifier Unmarshal Failures - Cloud Build enum for machine types needs E2

Build: proto: (line 1:1900): invalid value for enum type: "E2_HIGHCPU_8"

The above error message comes from the slack notifier on image - us-east1-docker.pkg.dev/gcb-release/cloud-build-notifiers/slack:latest when using the E2_HIGHCPU_8 machineType in cloudbuild

was able to fix this by changing the machineType to use N1_HIGHCPU_8 instead

SMTP notifier should treat `sender` and `from` differently

We currently conflate these in the smtp notifier. Some users want to separate them for "accounting" and better visibility into their notifications.

See https://help.mailgun.com/hc/en-us/articles/202236494-What-is-the-difference-between-the-From-and-Sender- for more.

Mail

Hello, it shows me the following error when I configure gmail

failed to run SendNotification: failed to send email: 454 4.7.0 Too many login attempts, please try again later

Go Template `replace` function not found

I followed
https://cloud.google.com/build/docs/configuring-notifications/configure-slack tutorial, and able to get notification delivered to slack.

Build.Substitutions.REPO_NAME is coming as bitbucket_project_repo-name

Not able to find any other variable in build which can give me the repo name. my slack template is as

[
	{
		"type": "section",
		"text": {
			"type": "mrkdwn",
			"text": "Cloud Build Id: {{.Build.Id}}, Status: {{.Params.buildStatus}}"
		}
	},
	{
		"type": "divider"
	},
	{
		"type": "section",
		"text": {
			"type": "mrkdwn",
			"text": "Repo: <https://bitbucket.org/myorg/{{replace .Build.Substitutions.REPO_NAME '_' '/'}}/commits/{{.Build.Substitutions.COMMIT_SHA}}|{{replace .Build.Substitutions.REPO_NAME '_' '/'}}>"
		}
	},
	{
		"type": "divider"
	},
	{
		"type": "section",
		"text": {
			"type": "mrkdwn",
			"text": "View Build Logs"
		},
		"accessory": {
			"type": "button",
			"text": {
				"type": "plain_text",
				"text": "Logs"
			},
			"value": "click_me_123",
			"url": "{{.Build.LogUrl}}",
			"action_id": "button-action"
		}
	}
]

I tried adding replace function but container fails to start

F0630 07:52:56.913698       1 main.go:36] fatal error: failed to parse template from notiifer spec &{"golang" "gs://gcp-settings-myt/slack.json" ""}: got invalid template from path "gs://gcp-settings-my-bucket/slack.json": template: :16: function "replace" not defined

Add trigger name to slack notification

Thanks for this project, it's super useful. I'd love it if you added the name of the trigger to the slack notification, which would make it easier to see what a notification is for

setup.sh asks for /bin/sh but uses bash array

The setup.sh script doesn't work on a Debian 10.4 system when executed directly:

% ./setup.sh
./setup.sh: 86: ./setup.sh: Syntax error: "(" unexpected (expecting "}")
% which sh
/bin/sh
% ls -la /bin/sh
lrwxrwxrwx 1 root root 4 Jan 17  2019 /bin/sh -> dash
% dpkg -l dash
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  dash           0.5.10.2-5   amd64        POSIX-compliant shell

setup.sh starts with #!/bin/sh, but line 86 appears to declare a bash array:

  REQUIRED_SERVICES=('Cloud Build API' 'Cloud Run Admin API' 'Cloud Pub/Sub API')

The script works when I run it using bash ./setup.sh. You should probably change the shebang to use bash instead of sh.

New line in a substitution fails Slack notification

For the Slack notifier I added the commit message via the CloudBuild trigger substition into a new section for the Slack message:

{
    "type": "section",
    "text": {
      "type": "mrkdwn",
      "text": "Message: {{.Build.Substitutions._COMMIT_MESSAGE}}"
    }
  },

Unfortunately, this fails now since the commit message contains new lines:

notifiers.go:549] failed to run SendNotification: failed to write Slack message: failed to unmarshal templating JSON: invalid character '\n' in string literal"

As I understand, the value should be sanitized with a filter function in the template (json). Is this something I can do directly in the template or is this a needed fix for the cloud-build-notifier?

SlackNotifier message notification popup says "no preview available" in Slack desktop client for Windows

I've setup the SlackNotifier in my GCP build pipeline, everything is working pretty well but when messages come through in the Slack desktop client for Windows, the notification popup says "no preview available" even though the actual message in Slack appears fine.

This seems to be due to the fact that in the WebhookMessage post, the "Text" property is not included and Windows / Slack (on Windows) not supporting rich-text content. Slack documentation suggests including the "Text" property in the payload which will be used as fallback when rich-text is not supported.

More details are available in the StackOverfow issue I posted: https://stackoverflow.com/questions/76657027/google-cloud-build-slack-notifier-message-no-preview-available

The posted answer explains the (suspected) problem and solution and he even created a fork with suggested code changes - onix@e83dab3

Any/all BQ notifier docs/READMEs should specify the behavior of ensuring dataset + table creation.

Right now, it is unclear what happens when (potentially blank) datasets or tables exist or not. Having an opinionated approach here sounds best. Opinionated dataset/table names (but handling whatever) might be good too.