balena-io-modules / balena-compose Goto Github PK
View Code? Open in Web Editor NEWComplete toolkit for building docker-compose.yml files and optionally deploy them on balenaCloud
License: Apache License 2.0
Complete toolkit for building docker-compose.yml files and optionally deploy them on balenaCloud
License: Apache License 2.0
Although the library has an option to create a build stream from an archive, Builder.buildDir
is the most handy option building an image from a directory. However, if the directory has build files (e.g. node_modules
, library files, etc), these will end up in the tar, delaying the build start. Having the method read from .dockerignore
to skip files for the build, might be a way to solve the issue.
An npm install
followed by npm test
results in the type errors below during compilation. The release model create() and update()
functions expect a generic body
object, but the API post() and patch()
methods expect the body to be an AnyObject.
$ npm test
> @balena/[email protected] test /home/kbee/dev/balena-compose/test-repo
> npm run lint && ts-mocha --project ./tsconfig.test.json
> @balena/[email protected] lint /home/kbee/dev/balena-compose/test-repo
> npm run lint:lib && npm run lint:tests && tsc --noEmit
> @balena/[email protected] lint:lib /home/kbee/dev/balena-compose/test-repo
> balena-lint --typescript lib/ typings/
Warning: The 'no-floating-promises' rule requires type information.
0 errors and 0 warnings in 4 files
> @balena/[email protected] lint:tests /home/kbee/dev/balena-compose/test-repo
> balena-lint --typescript --tests test/
Warning: The 'no-floating-promises' rule requires type information.
0 errors and 0 warnings in 18 files
lib/release/models.ts:122:30 - error TS2322: Type 'U' is not assignable to type 'AnyObject | undefined'.
Type 'U' is not assignable to type 'AnyObject'.
122 return api.post({ resource, body }).catch(wrapResponseError) as Promise<T>;
~~~~
lib/release/models.ts:117:27
117 export function create<T, U>(
~
This type parameter might need an `extends AnyObject` constraint.
lib/release/models.ts:117:27
117 export function create<T, U>(
~
This type parameter might need an `extends AnyObject | undefined` constraint.
node_modules/pinejs-client-core/index.d.ts:231:5
231 body?: AnyObject;
~~~~
The expected type comes from property 'body' which is declared here on type 'Params'
lib/release/models.ts:131:35 - error TS2322: Type 'T' is not assignable to type 'AnyObject | undefined'.
Type 'T' is not assignable to type 'AnyObject'.
131 return api.patch({ resource, id, body }).catch(wrapResponseError);
~~~~
lib/release/models.ts:125:24
125 export function update<T>(
~
This type parameter might need an `extends AnyObject` constraint.
lib/release/models.ts:125:24
125 export function update<T>(
~
This type parameter might need an `extends AnyObject | undefined` constraint.
node_modules/pinejs-client-core/index.d.ts:231:5
231 body?: AnyObject;
~~~~
The expected type comes from property 'body' which is declared here on type 'Params'
Found 2 errors in the same file, starting at: lib/release/models.ts:122
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! @balena/[email protected] lint: `npm run lint:lib && npm run lint:tests && tsc --noEmit`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the @balena/[email protected] lint script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/kbee/.npm/_logs/2022-09-03T11_10_58_831Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! @balena/[email protected] test: `npm run lint && ts-mocha --project ./tsconfig.test.json`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the @balena/[email protected] test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/kbee/.npm/_logs/2022-09-03T11_10_58_846Z-debug.log
Tests fail with a timeout. Root cause is regression in dockerode v3.3.4 that hangs balena-compose use of its buildImage()
function. See Issue on that repo for details. We are awaiting response to a fix PR.
136 passing (8m)
2 pending
14 failing
1) Directory build
should build a directory image:
Error: Timeout of 60000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/kbee/dev/balena-compose/repo/test/build/all.spec.ts)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7)
...
https://github.com/opencontainers/image-spec/blob/main/image-index.md
We should be able to selectively apply the platform option when building with these images.
Presently the acceptable characters for a label on a service, network, or value are limited to ASCII letters and numbers, plus period (.) and dash (-). Supervisor is expanding the set of acceptable characters to support services like traefik and caddy-docker-proxy.
We want to support this extension of acceptable characters across all balena tools. balena-cli relies on balena-compose for label validation.
When the docker-compose file includes the image from the registry that requires authentication (e.g. ghcr.io) it always fails to get manifest with the following error (from build.ts)
debug(`${task.serviceName}: Image manifest data unavailable for ${r}`);
Reason is that inside getManifest
function, it uses docker-modem
library without passing any authentication info.
const optsf = {
path: `/distribution/${repository}/json?`,
method: 'GET',
statusCodes: {
200: true,
403: 'not found or not authorized',
500: 'server error',
},
};
optsf
shall include the object authconfig
so docker-modem
will encode it and use it as X-Registry-Auth
.
FYI: https://docs.docker.com/engine/api/v1.42/#section/Authentication
This authentication info shall be derived from registry-secrets.yml
file from Balena CLI, but that is also not the case.
As a result, it is currently impossible to use multiarch images from the major private repositories such as ghcr.io.
dockerode component pinned to v3.3.3 due to regression in v3.3.4 with buildImage()
. See Issue apocas/dockerode#696 and fix PR apocas/dockerode#695 for details.
balena-os/balena-supervisor#1935
But we are still blocking these features in the builder:
balena-compose/lib/parse/compose.ts
Lines 466 to 468 in 159cdba
This prevents applying the platform flag automatically, when it should actually be used to correctly pull multiarch base images.
eg.
[Warn] Service 'balena-supervisor':
[Warn] Multi-stage Dockerfile found with a mix of base images that require
[Warn] CPU architecture selection and base images that do not support it.
[Warn] The following base images do not support CPU architecture selection:
[Warn] - build-base
[Warn] - runtime-base
[Warn] - runtime-base
[Warn] The following base images require CPU architecture selection:
[Warn] - alpine:3.11
[Warn] - alpine:3.16
[Warn] - debian:bullseye-slim
[Warn] - alpine:3.16
[Warn] As a result, the CPU architecture of the machine where the Docker Engine
[Warn] is running will be used by default to select base images that require
[Warn] architecture selection. This may result in incorrect architecture selection
[Warn] and "exec format error" at runtime. It is usually possible to override the
[Warn] architecture in the FROM line with e.g. "FROM --platform=linux/arm/v7",
[Warn] or by adding the sha256 digest of the image for a specific architecture
While the docker-compose specification accepts both strings and numbers, using a number seems to be rejected on the engine with a JSON unmarshaling error
(HTTP code 400) unexpected - json: cannot unmarshal number into Go struct field NetworkCreateRequest.Options of type string
This after pushing this valid network configuration
networks:
default:
driver_opts:
com.docker.network.driver.mtu: 1420
Leaving this as an issue for discussion
The following composition is rejected by docker-compose with service hello declares mutually exclusive
network_modeand
networks: invalid compose project
version: '2.4'
services:
hello:
image: alpine
command: ['sleep', 'infinity']
networks: ['my-network']
network_mode: host
networks:
my-network:
While balena-compose allows it. Making the change might prevent invalid or ambiguous compositions reaching the supervisor
While this is valid with docker-compose, it doesn't make sense with balena-compose as this allows linking the container to an external network that may not exist on the final device.
Validation should probably restrict the valid values of this field to bridge
, host
or none
The parser does not currently allow volumes without a source (anonymous volumes) even though the SV supports it at runtime.
balena-compose/lib/parse/compose.ts
Lines 471 to 482 in cd8e06e
The code is here
balena-compose/lib/parse/compose.ts
Lines 42 to 65 in 0d71e1c
We should update this so we can get rid of the code referencing resin
labels on the supervisor
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.