spring-cloud / spring-cloud-skipper

A package manager that installs, upgrades, and rolls back Spring Boot applications on multiple Cloud Platforms.

⛔ Spring Cloud Dataflow has moved to a monorepo.

This repo is archived and now exists as the spring-cloud-skipper module(s) in the Dataflow monorepo.

Please file any issues/PRs in the Dataflow repo.

spring-cloud-skipper's Issues

Add Kubernetes deployer support

Add Skipper-cli

As a developer, I'd like to add the initial infrastructure for Skipper-cli, so we can use it to interact with the Skipper-server.

Depends on #6.

Acceptance:

• Decide the default to connect to Skipper-server.

Determine strategy for validationg/authenticating remote index file

Add support from undeploy operation

For a given release, the user would like to un-deploy.

Refactor Package/Release controller

Currently, all the REST operations are grouped under PackageController (package deploy/update, release rollback/undeploy). We also have Update properties explicitly for the package update operation.

We can refactor this further to move all the release related operations under ReleaseController and remove the UpdateProperties completely.

PackageIndexFile should have fields for ApiVersion and Generated Date.

Add support for nested packages

Support for packages that contain other packages . This will be a common way that Data Flow Stream definitions will be provided, e.g. ticktock stream is a collection of two 'dependent' packages, time and log.

Allow for the top level package configuration to override the dependent package configuration, e.g. version, application properties, deployment properties. The top level package should merge it's values into the dependent package configuration.

Add CI plan

As a developer, I'd like to add CI plan for Skipper.

Add complete set of metadata fields to PackageSummary entity

Rename PackageSummary to PackageMetadata

As a developer, I'd like to rename and refactor the code references of PackageSummary to PackageMetadata.

Open up logging so port is logged.

Should see a log statement such s

2017-08-02 15:37:57.414  INFO 30678 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)

Add shell command to run release rollback operation

Add check at startup/reconfiguration that the provided server URL can be contacted.

Add support for rollback operation

As a user, I'd like to rollback to the previously deployed version of my release.

Add multi-platform support

Create @ConfigurationProperties classes per platform that provide necessary connection infomation. Can model based on the approach used in config server for git/vault.

Each entry will have a name and will result in the creation of an AppDeployer instance for the relevant platform.

Add build procedure to consume Skipper UI as a library

As a developer, I'd like to add support to consume skipper-ui as a dependency similar to SCDF's Dashboard inclusion for the Local-server.

Acceptance:

• UI project is embedded in the skipper-server
• Launching http://localhost:<DEFINED_PORT> launches the skipper-ui

Update to Spring Cloud Deployer 1.3 milestone

We need to access to new features in the deployer API.

1. Support for the PCF deployer to not delete routes when deleting the applicaiton
2. Access to a new execute method, as in Spring Template class design, e.g. JdbcTemplate, to access the lower level client library.

<T> T execute(ClientCallback<T> action)

Review code to add @NotNullApi package annotation and @Nullable

Encountered a few NPE's that would have been caught if we used these annotations. For example, org.springframework.cloud.skipper.domain.Deploymnet had a null deploymentProperties value. we can set the field to an empty map. Similarly, the custom finder methods in ReleaseRepositoryImpl can return null.

See Spring 5 custom meta annotation.

Add support for update operation

Ability to update the existing release with the newly available package information.

Add shell command to run release undeploy operation

Create design for pluggable backend package storage

In CF, we can only rely on a relational database for persistent storage. A small interface, should be created such that we can also persist package.zip files to a filesystem in addition to a relational database.

Add @Version to entities for optimistic locking

Note AbstractEntity class in restbucks.

@Getter
@ToString
@EqualsAndHashCode
public class AbstractEntity implements Identifiable<Long> {

	private final @Id @GeneratedValue(strategy = GenerationType.AUTO) @JsonIgnore Long id;
	private @Version Long version;

	protected AbstractEntity() {
		this.id = null;
	}
}

Implement PackageInstallController

Add status operation that checks the platform status

For the given release, the status operation checks the status from the underlying platform.

Add check for package-info.java to checkstyle

Use undeploy/rollback properties as Path variables

Instead of explicit RequestBody objects as Undeploy/RollbackProperties, we could use releaseName and version as REST path variables.

Why is @CliOption for uri in ConfigCommands not mandatory?

Add initial support to download and resolve packages

As a developer, I'd like to add the initial support to resolve and download packages.

Design notes:

• To bootstrap the process, we will create the index archive by hand and host it in the project. The archive is similar to what is in Debian's packages.gz and also Helm's index.yaml. YAML is a good format, the file contains multiple documents main metadata that can be used to list/search packages.
• In the future, the server is expected to pull down package indexes from remote repositories.

Acceptance:

• The initial support is based on a hard coded packages.zip artifact that contains a packages.yml document.
• The Server will unpack a packages.zip and load it into memory/db.
• file:// or classpath:// can be used for test purposes at first to hose the packages.zip file.

Use int for release version

Update to Spring Shell 2.0 M2

Package search command improvements

• Rename Command from list to search
• set page size to a large value, 2000
• support wildcard expression for the name, maps on to Spring Data Repository 'like' search

Default listing is a small ascii-art table with basic information, name, version, description.

Add an --details true command option that will show full package metadata, not in a table format. A newline separator between each dump of the package metadata

Repo add/delete command

Allow repositories to be added/deleted at runtime.

Add shell command to run package update operation

Add PackageInstallController

As a developer, I'd like to create PackageInstallController, so we can create a link in the PackageMetadata Resource to install the specific package.

Notes:

@PostMapping
@RequestMapping("/install")
@ResponseStatus(HttpStatus.CREATED)
public Release install(@RequestBody InstallReleaseRequest installReleaseRequest) {
...
...
}

Acceptance:

• We can assume a single ‘default’ environment for the time being.
• Request successfully installs the package.
• A user-friendly message is available as a response.

Create package upload command

A package's .zip file should be uploaded from the client and stored on the server. When a file is uploaded, searches against package metadata are updated.

The initial implementation should store packages in the database and support the package search command. See #59

After upload, resources such as those listed below should be available, mimicking what one would expect if stored in a filesystem.

http:///repository//log/log-1.1.0.zip

Repo list command

Lists the repositories in a small ascii-art table that skipper knows about. Show the name, to be used in other commands, and the resource URL to that repository.

Name URL
test http:///<endpoint for searching repo contents/ (this is a server managed repository)
stable-packages http://github.com/markpollack/skipper-packages/stable/packages (some remote repository on a remoted hosted location)

Add repository implementation for packages

As a developer, I'd like to add repository implementation to explore packages via findAll() operation.

Acceptance:

• Repository implementation follows spring-data-jpa model
• findAll() operation searches across all the resolved packages
• Unit tests included

Add shell command to retrieve package summaries

As a developer, I'd like to add a shell command to retrieve all the available packages.

Notes:
This command will invoke the findAllPackageSummary backend operation.

Acceptance:
The new command (maybe: skipper:>package list?) retrieves all the packages.

Create Repository entity + supporting CRUD operations

A Repository entity represents a location to download package information. It refers to a location to obtain an index file of all packages hosted by that repository.

Expose via Spring Data REST for CRUD.

Do not consider keyfiles/sha hashes etc.

Add ReleaseController to expose status endpoint

As a user, I'd like to access the status of a specific release/version.

Add REST-API to search for packages

As a developer, I'd like to add the REST-APIs, so we can interact with the Skipper-server from CLI/UI.

Design notes:
A REST API living under /api is created that follows the conventions of Spring Data Rest.

@RequestMapping(“/api/packages/metadata”)

Acceptance:

• Content types are application/json and no HATEOAS resources are used yet.
• Create basic TestMvC based tests.
• Root controller for resource discovery is available for: /dashboard and /packages

Add shell command to support package deploy operation

Create only htmlsingle docs for publication, no pdf/epub

Add Travis and CodeCov integration

As a developer, I'd like to add Travis CI and CodeCov integration for Skipper.

Acceptance:

• Use sudo: required for Travis builds similar to SCDF
• Travis CI integration automatically builds each PR
• Travis + CodeCov integration automatically measure test coverage for each PR

Include package metadata in the HAL _links

As a developer, I'd like to include packageMetadata in the returned HAL _links, so it can be used to drill into all the metadata associated with a package.

Notes:
This requires creating a ResourceProcessor object for the PackageMetadata class. See [here](This requires creating a ResourceProcessor object for the PackageMetadata class. See here for more details.

Acceptance:
Drilling into packageMetadata link returns all the metadata fields associated with the selected package.

Add badges for CI and codecov to github readme

Create Spring State Machine Skipper Design

Use Spring State Machine as foundation for pipeline execution when deploying/upgrading/destroying applications.

Add package summary endpoint

As a developer, I'd like to add findAllPackageSummary operation, so it can be used to search and return a high-level summary of the package as opposed to all the metadata associated with it. This lightweight response is useful for tile-page rendering where we intend to show only the name and the icon.

Notes:
This can be done using a Spring DATA REST projection or perhaps another finder method on the PackageMetadata Spring Data repositories.

Acceptance:
The "name", "description", and "icon" of the package is returned in the response.

Add test repository data

Create structure to deploy the 'time' source. Under test/resources add

repositories/locations/testrepo.yml
repositories/sources/testrepo/log/log-1.0.0/
     - package.yml
     - values.yml
     - templates/application.yml
repositories/binaries/testrepo/
     - index.yml
     - log-1.0.0.zip

Utilities to create the index and zip files will be developed separately.

Instantiate multiple CF deployers from multiple accounts

The CF deployer needs to be instantiated from multiple accounts setup via CF Platform properties.