meltanolabs / tap-circle-ci

Singer Tap for CircleCI built with the Meltano SDK

tap-circle-ci

tap-circle-ci is a Singer tap for CircleCI.

Built with the Meltano Tap SDK for Singer Taps.

Streams

• Pipelines
• Workflows
• Jobs

WARNING: You must follow the projects in CircleCI to obtain their pipelines.

Installation

You can install this repository directly from the Github repo. For example, by running:

pipx install https://github.com/MeltanoLabs/tap-circle-ci.git

Configuration

Accepted Config Options

The following configuration options are available:

Field	Description	Type	Required	Default
token	Personal API Token you have generated that can be used to access the CircleCI API	string	yes
org_slug	Organization slug in the form vcs-slug/org-name. Example: org-slug=gh/CircleCI-Public	string	yes
user_agent	User-Agent to make requests with	string	no	tap-circle-ci/<version> Singer Tap for the CircleCI API
base_url	Base URL for the CircleCI API	string	no	https://circleci.com/api/v2

A full list of supported settings and capabilities for this tap is available by running:

tap-circle-ci --about

Source Authentication and Authorization

Login to your Circle CI account, go to the Personal API Tokens page, and generate a new token.

Usage

You can easily run tap-circle-ci by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-circle-ci --version
tap-circle-ci --help
tap-circle-ci --config CONFIG --discover > ./catalog.json

Developer Resources

Initialize your Development Environment

pipx install poetry
poetry install

Create and Run Tests

Create tests within the tap_circle_ci/tests subfolder and then run:

poetry run pytest

You can also test the tap-circle-ci CLI interface directly using poetry run:

poetry run tap-circle-ci --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Your project comes with a custom meltano.yml project file already created. Open the meltano.yml and follow any "TODO" items listed in the file.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-circle-ci
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-circle-ci --version
# OR run a test `elt` pipeline:
meltano elt tap-circle-ci target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.