Software update mixer and related tools
License: Apache License 2.0
This project contains tooling for using the software update mixer.
Please follow the guide at https://www.clearlinux.org/clear-linux-documentation/guides/clear/mixer.html to learn more about how to use the mixer tool.
The /usr/lib/os-release is a file with various lines KEY=VALUE that has some well-known keys (like NAME, VERSION_ID). When creating a mix, only the VERSION_ID is updated, all other information remains like in Clear Linux, so we get a confusing output.
There are some useful things to do here:
Following instructions from the official guide https://clearlinux.org/documentation/clear-linux/guides/maintenance/mixer.html?highlight=mixer when building chroots there's a yum error that looks like:
Repository 'clear': Error parsing config: Error parsing "baseurl = 'UPSTREAM_URL/releases/19420/clear/x86_64/os/'": URL must be http, ftp, file or https not ""
There are no enabled repos.
The .yum-mix.conf has the "UPSTREAM_URL" instead of a real one. The template processing seems to happen only when REPODIR (from the configuration is non-empty), but in this default case it isn't.
Add new command to support updating bundle definitions to a new version of CLR.
You have already initialized a mix (mixer init-mix -clearver n -mixver m) based on some CLR version. You have already pulled bundle definition files into your mix-bundles/ directory (mixer add-bundles -bundles somebundle). You have optionally modified any of the bundle definitions, or created your own. You may or may not have done anything else (mixer build-chroots, mixer build-update, etc.). A new version of CLR is released that adds a new package to one of the bundles you're using (among other potential changes). Now you want to base your next mix on this new CLR version to incorporate that change.
mixer get-bundlesmixer add-bundles -bundles <BUNDLE_LIST> -force with a list of every bundle in your mix-bundles/ directory to override your bundle definitions with the new ones.This is laborious and error prone.
A new command: mixer update-bundles that will take an optional -clearver flag indicating what version you want to update to. (If omitted, "latest" is assumed.)
The tool will:
-git is passed), add new git commit in mix-bundles/ directory.-prune is passed), clean up old CLR version bundle definition files.Note: There is a corner case that can result in a dangling, unneeded bundle definition file in your mix-bundles/ directory. If an old bundle definition file previously included another bundle, but that include was removed in the new version of the definition file, and that included bundle still exists in the new verison of CLR, it will be left in your mix-bundles/ direcotry, as there is no way to know if it exists there due only to an include or you manually adding it. (If it does not exist in the new version of CLR, the tool will error out saying you have a bundle that no longer exists.)
From @icarus-sparry comment #155 (comment): We ought to see exactly what wants 'root', and see if we can eliminate the need as a medium term goal.
Created this ticket to try at least get some sense of the amount of work we have to do. Note that if we have cases we do want to use root, it would be nice to "contain" them, so that other functionality can run as the regular user. The more we make "mixer" user-friendly (by creating or updating files for me), the more problematic become it creating some files as root as I create other files with my user.
Non-exhaustive list of places we currently need root permissions:
When creating the chroots we install RPMs and then expect the filesystem to look exactly like the final filesystem. I'm not sure about the status of running the dnf/yum without root, but I guess that's workable. There were previously suggested solutions for final state of the files: accept a list of files/metadata, plus the file contents. Another idea I've seen in mkosi, optionally accept a chroot tar file in place of a chroot directory: the tarfile contains the permissions, and is something one could easily create from inside a fakeroot execution.
Image creation needs privileges to mount loopback devices.
Since init is about creating the config file, what does the "--config" is used for. It seems wrong, should be removed or the documentation of it clarified.
Support building of packs for N versions back from the --to version, as an alternative to passing one or more --from options. Proposed option name: --back.
Introduce new "mixer add-bundles" command to automate the addition of bundles to the mix-bundles/ directory.
Currently adding bundles to your mix means copying them manually from the pulled clear-bundles/clear-bundles-<VER>/bundles/ dir to your mix-bundles/ dir. This is a manual process, and thus following bundle dependencies (i.e., “include (other-bundle)”) is a manual process.
This work is in progress and will be in a forthcoming PR.
At the moment the mixer-tools repository is organized that it assumes it will be set as the GOPATH to operate. This is not the conventional way to do things, which means that some of the tooling around Go either don't work or need special care to work.
The conventional way is to assume the sources will be inside an existing GOPATH, in the github.com/clearlinux/mixer-tools directory.
Changes needed:
When calling mixer init, and then calling it again, it does some of the work, but not all, but doesn't fail, which could be confusing.
I think we should fail at init if we find that the directory was already initialized. Usually this happens either by mistake or by not knowing how to bump to a new version, so could be useful to add some information on how to do that in the error message.
Section below shows examples.
$ ls
$ mixer init --clear-version 20600 --mix-version 10
Creating new builder.conf configuration file...
Adding bundle "os-core-update"
Adding bundle "kernel-native"
Adding bundle "os-core"
Adding bundle "bootloader"
Initialized empty Git repository in /home/c/src/mix3/mix-bundles/.git/
[master (root-commit) 594f30b] Initial mix version 10 from upstream version 20600
4 files changed, 76 insertions(+)
create mode 100644 bootloader
create mode 100644 kernel-native
create mode 100644 os-core
create mode 100644 os-core-update
$ mixer init --clear-version 20600 --mix-version 20
$ mixer init --clear-version 20610 --mix-version 20
$ cat .mixversion
20 # Added a fake newline here for clarity.
$ mixer init --clear-version 20610 --mix-version 30 --local-rpms
$ cat .mixversion
30 # Added a fake newline here for clarity.
$ cat .clearversion
20610 # Added a fake newline here for clarity.
$ ls
builder.conf mix-bundles
$ git -C mix-bundles/ shortlog
Caio Marcelo de Oliveira Filho (1):
Initial mix version 10 from upstream version 20600
$ grep RPM builder.conf
$
This call makes go test not point to the helper function when a failure happens, but to the caller. For example, when we have a fail in
fs.mkdir("test")
the error output in go test will point to this caller code instead of the implementation of mkdir that calls t.Fatal, because t.Helper() is called in mkdir.
Have Mixer save the upstream URL and version used when generating the new Mix.
This information is both useful historical information about the Mix as well as needed by any automation used to generate new mixes.
The primary use would be to identify the upstream's original Swupd Format when generating a new Mix. The Format of the upstream, and how many times it has bumped, are needed in order to generate the necessary mixes for upgrading the downstream client of the Mix.
Assuming a new Mix stream is created based on the Public ClearLinux distribution plus some additional bundles (i.e. for a production product), the following steps would be required.
A new production Mix, version 10 Format 1, is made based on the ClearLinux 18030 which is Format 19 . When the next Mix is needed, the public ClearLinux is now at version 19110 which is Format 21; the Upstream Format has bumped 2 times.
One suggested solution is saving the URL used for the Upstream and the version into files in the Mix when it is created.
mdhorn@d102d77
With that information, the Format of the Upstream can be found:
https://download.clearlinux.org/update/18030/format = 19
Next, the latest release of Format 19 can be determined:
https://download.clearlinux.org/update/version/format19/latest = 18500
So the first new Mix (20; still at Mix Format 1) will need to be based on Upstream's 18500.
The next Mix (30; Mix Format 2) would need to be based on the First release of Format 20:
https://download.clearlinux.org/update/version/format20/first = 18510
And then another Mix (40; Mix Format 2) based on the Last release of Format 20:
https://download.clearlinux.org/update/version/format20/latest = 18930
Next Mix (50; Mix Format 3) based on the First release of Format 21:
https://download.clearlinux.org/update/version/format21/first = 18940
Finally, we can make our last Mix (60; Mix Format 3) based on the choose 19110.
The summary would be:
| Upstream Version | Upstream Format | Mix Version | Mix Format |
|---|---|---|---|
| 18030 | 19 | 10 | 1 |
| 18500 | 19 | 20 | 1 |
| 18510 | 20 | 30 | 2 |
| 18930 | 20 | 40 | 2 |
| 18940 | 21 | 50 | 3 |
| 19110 | 21 | 60 | 3 |
Without having the URL and version the previous Mix was created from, the upgrade path could not be created.
The reason to have the Upstream URL as well as the version is that a Mix could be based on another Mix, not just the public ClearLinux release.
Mixer should be aware about the format bumps in the upstream (Clear Linux), since right now it might be producing a stream that is not usable (skipping a bump in upstream, leading to some invalid state).
"Handle" here might refer first to "be aware and fail if the input is invalid" and then to "create the intermediate mixes as needed", so might make sense implement this in steps. Just the warning would go great length to avoid issues.
See also #30 description that have a concrete example. That PR added more information with the intent of making life easy to identify this bumps.
For the record @mdhorn implemented this outside mixer for our test mix.
The current approach is to call 'yum list' and parse the results with a lot of workarounds.
See the whole story in this function
mixer-tools/builder/chroots.go
Line 357 in 64e3c26
Steps to reproduce:
VERSIONS_PATH or misspelled, i.e VERSIONS_DIR$ mixer init-mix --clear-version 20010 --mix-version 10
Mixer 3.2.1
***Error: open /.clearurl: permission denied
Works when run with sudo, but this command should not need sudo privileges.
This way a mixer user could define what gets added/removed/modified from a bundle by changing directly the chroot directory for the given bundle to mix.
A simple use case for this would be as follows:
new-bundle bundle.The flow for this today implies packaging the binary into an RPM format and create a repo so mixer can consume it and modify the bundle definition to specify that new RPM will be part of that bundle.
With this feature, user only would need to copy the proper binary/files in the chroot directory defined for new-bundle and then the mixer would detect new files in the bundle, removing entirely the RPM management from the user perspective.
So in the case new-bundle contains the following structure:
./new-bundle/
├── bin -> usr/bin
├── lib -> usr/lib
├── lib64 -> usr/lib64
├── sbin -> usr/sbin
└── usr
└── bin
└── bash
and user wants to add new file (e.g: /usr/bin/cowsay) the new chroot for new-bundle would be like:
./new-bundle/
├── bin -> usr/bin
├── lib -> usr/lib
├── lib64 -> usr/lib64
├── sbin -> usr/sbin
└── usr
└── bin
├── bash
└── cowsay
Then user could execute the mixer to create new update/mix and it would take the previously defined chroot.
Using this issue to keep of imported tests
| Original test | New test (comments) |
|---|---|
| basic | N/A (checks help output for commands) |
| contentsize-across-versions-includes | TestContentSizeAcrossVerionsIncludes |
| delete-no-version-bump | TestCreateManifestDeleteNoVerBump |
| file-name-blacklisted | TestManifestIllegalChar |
| file-name-debuginfo | TestManifestDebuginfo |
| format-bump | N/A (actions deprecated per #140) |
| format-no-decrement | TestCreateManifestFormatNoDecrement |
| fullfiles | N/A (checking command line usage of fullfiles binary) |
| full-run | TestFullRun |
| full-run-delta | TestFullRunDelta |
| ghosting | TestCreateManifestGhosted |
| includes-deduplicate | TestCreateManifestIncludesDeduplicate |
| include-version-bump | TestIncludeVersionBump |
| no-delta | TestNoDeltasForTypeChangesOrDereferencedSymlinks |
| orphaned-renames | TestCreateManifestsRenamesOrphanedDeletes and TestCreateRenamesOrphanedChain |
| pack | N/A (checking command line usage of pack binary) |
| renames | (#128) |
| renames2 | (#128) |
| state-file | TestCreateManifestsState |
| subtract-delete | TestSubtractDelete |
| update | N/A (checking command line usage of update binary) |
Cobra has built-in functionality to generate both bash and zsh autocomplete configuration for mixer, including all commands and flags. Using this functionality will significantly improve mixer's usability.
There are a number of ways to do this, so design details will likely be filled in when the issue is taken up.
Note Cobra also supports advanced, contextual auto-complete for specific functions (e.g. typing mixer bundle add edi[tab][tab] autocompletes to editors), but those will need to be implemented on a command-by-command basis.
When there was a format bump, swupd-server marked the MoM header with an "action: update".
See also the functional test: https://github.com/clearlinux/swupd-server/blob/master/test/functional/format-bump/test.bats#L55
At the moment, calling mix init with no arguments I get:
$ mixer init
Error: Required flag(s) "clear-version", "mix-version" have/has not been set
Usage:
mixer init [flags]
Flags:
--all Initialize mix with all upstream bundles automatically included
--clear-version int Supply the Clear version to compose the mix from (default 1)
--config string Supply a specific builder.conf to use for mixing
-h, --help help for init
--local-rpms Create and configure local RPMs directories
--mix-version int Supply the Mix version to build
--upstream-url string Supply an upstream URL to use for mixing (default "https://download.clearlinux.org")
Global Flags:
--new-swupd EXPERIMENTAL: Use new implementation of swupd-server when possible
Things that could be improved:
Note that one side effect is that "mixer init" would just work without arguments, so the other facilities it provides should be easy to get some other way.
This is a condensed form of a more detailed proposal on this topic. Please contact @kevincwells for information.
Currently, the way you add bundles to your mix is to copy the bundle definition file from a given version of CLR into the mix-bundles directory. That means the existence of bundle definition files in your mix-bundles directory is the specification of what bundles you want to include in your mix. Meanwhile, the mix-bundles directory is, in fact, input to other tools (bundle-chroot-builder.py).
This causes ambiguity and problems, especially when handling bundles that include other bundles. For example let’s say the developer cares about including Bundle_A in their mix, and that bundle itself includes Bundle_B. That means mix-bundles will contain both Bundle_A and Bundle_B, but there is no way to know that Bundle_B is only there as an include. So later, if Bundle_A gets modified upstream to no longer have that include, the developer is “stuck” with Bundle_B, because it is in their mix-bundles directory, and we cannot tell the difference.
Allow the user to include some bundles as-is from upstream (CLR) without maintaining a local copy.
local-bundles directory to contain user-created and edited bundles.
local-rpms directory.local-bundles directory are treated as "Upstream Bundles" and are included from upstream (CLR) as-is.mix-bundles directory (the input to bundle-chroot-builder) on the fly when chroots are built. Bundles included by other bundles are handled automatically, and local bundles always take precedence over their upstream counterparts.Add a new command ("mixer bundle check") to lint and validate a bundle definition file.
If a developer wants to write their own bundle file, or alter an existing one, it would be useful to be able to verify that it is correct.
Change mixer to automate the acquisition/editing of the ister config file (currently release-image-config.json).
curl it. Ship a local copy? Curl it ourselves in mixer init?builder.conf?
config.json file, while "format" is passed to ister as a commandline flag.builder.conf (or elsewhere), then generate the config.json file's "Bundles" list automatically?Currently the mixer tool only creates the zero packs, and a separate shell script pack-maker.sh (installed as mixer-pack-maker.sh) is used to create the packs.
In the new swupd implementation (and its test programs) we already have code to do most of what mixer-pack-maker.sh does. The interface will include the proposed "--back" flag in #12, with its new semantics of going back MoM versions.
# build all deltas from K previous versions to the current one
mixer build delta-packs --back K
# build all deltas from K previous versions to version N
mixer build delta-packs --back K --to N
# build all deltas from version M to version N
mixer build delta-packs --from M --to N
I'm also inclined to include:
# build delta pack for os-core bundle only from M to N
mixer build delta-packs --bundle os-core --from M --to N
The new command would only work with "--new-swupd", mixer-pack.sh would only be removed when we stop to use swupd-server C implementation.
When creating a pack, we may end up adding more than one delta to the same target. We should collect those and pick the smallest.
Notes:
[]Delta instead of []*Delta, when resolving this issue this might change.Note: This issue is a feature proposal and should be labeled as 'Enhancement'.
Right now, the process of creating a new mix requires the user to go through a couple manual steps that can easily be automated and packed into a mixer command. The idea behind having this "mixer init" command is to mimic the functionality of a "git init" command and have the active folder set up for a mix to be performed.
This command should automate the initial setup steps of the workflow guide found here (https://clearlinux.org/documentation/clear-linux/guides/maintenance/mixer.html), namely:
builder.conf template file into the active directorySERVER_STATE_DIR=$PWD/updatelocal/ and rpms/ directoriesRPMDIR and REPODIR to builder.conf and set them to the above directoriesFetching the Ister configuration could also be part of this command, but as this requires accessing an external server to get the config file, I believe it should not be part of it for now.
Any suggestions on extra or different behavior of this command are more than welcome.
When calling 'mixer build update' twice, the second call will try to create an update from the version to itself, leading to invalid output. This happens because after the first update, LAST_VER is also updated, and that file is used as input for the update itself.
Note that calling 'mixer build chroots' twice leads to rewriting the chroots (and reseting the environment for that version).
I don't think there are legitimate cases for LAST_VER == "version we are building right now", so we should either:
a) stop the execution of the second call ("update already exists" kind of error?), maybe pointing the user of what needs to be done to re-create the updates and/or instruct about bumping mixver.
b) work like build chroots and redo the update.
In both cases, we have a problem: we overwrite the LAST_VER in the first execution, so we don't know what is the previous version.
For case (a) we still have something even without the previous LAST_VER (we just won't be able to instruct the user easily how to redo things).
In swupd-server there was a functional test called subtract-delete (in https://github.com/clearlinux/swupd-server/blob/master/test/functional/subtract-delete/test.bats), that verified the presence of delete entries in the manifest. In swupd-server, once the file was deleted, it remained on the manifests marked as deleted.
The translated test (with the failing checks disabled) is in one of the commits of #136.
Add a command (mixer bundle delete) that removes bundle(s) from your mix.
mixer bundle add--local) also removes the bundle definition file from the local-bundles directory, if it exists.swupd dumps the following warning (from libarchive):
Warning: Linkname can't be converted from UTF-8 to current locale.
But extracting the pack with bsdtar (that also uses libarchive) doesn't complain about it. This warning is strange because we run swupd with en_US.UTF-8 locale.
See also:
Add a command (mixer bundle list) that reports what bundles are in the mix, in the local-bundles directory (see #55), or available from Upstream (CLR).
See #102 (comment).
E.g. of usage is "formatstaging"
$ mixer init-mix
ERROR: Please supply -clearver and -mixver
Ok, I'll try that
$ mixer init-mix -clearver 20000 -mixver 10
Error: unknown shorthand flag: 'c' in -clearver
Usage:
<usage text is correct>
Same with -mixver
$ mixer init-mix --clear-version 20000 -mixver 10
Error: unknown shorthand flag: 'm' in -mixver
Usage:
<usage text is correct>
Obviously this works
$ mixer init-mix --clear-version 20000 --mix-version 10
< success >
See #126 for how the output looks like. The first line (which tells the user about builder.conf) which I think is an important one gets scrolled with pages of output (even when using a lot of lines in my terminal).
Ideas:
This is a first step towards #59: first we identify the undesired situation and do not generate a potentially invalid update. This is an easier task than providing auto format bumps, and identifying the case is necessarily anyway. Optionally add a flag to allow forcing create the mix even in the invalid scenario.
This only applies to mix creation, so regular Clear Linux workflow shouldn't be affected (no mix version in that workflow).
Please consider change the major version number for the release which implement the new CLI.
4.0.0
At a minimum, bumping the minor version should be done to make this change stand out.
3.3.0
Right now the user can proceed but external tools called by mixer will complain themselves.
It would be very advantageous to allow the use of environment variables, such a $HOME, in the builder configuration file. When used in an automated build environment like Jenkins or Travis, it would be helpful to use environment variables for the various paths.
NOTE: This will require a compatible change to the bundle-chroot-builder tool as it also reads the same configuration file.
Some (if not all) information from server.ini is already available at builder.conf, so consider making our "build update" use that directly once we move to --new-swupd.
To make life easier when we ever evolve the manifest files, would be good to have a way to describe the version of the file format we are using.
I think in the past this was conflated with the "format" for the OS (as it implied what version of swupd was used and so on). But as we create mixes, this correspondence is gone.
See also the description https://github.com/clearlinux/swupd-server/wiki/Manifest-structure
Each line in the header has 2 fields; the first is a keyword, and the second is a value. Accepted keywords are:
MANIFEST: The first line of manifest (the "magic"). Second field indicates the format number used to create the manifest.
Note we also probably need to fix our manifest parsing to treat MANIFEST (or whatever we come up) like the description above.
Use goroutines. Do something similar than CreateFullfiles.
The result (if there is any) of b.SignManifestMOM() in BuildUpdate() is not checked, so failures during signing do not cause the build to abort.
For example, when the .pem file is missing or incorrectly configured:
[pid 6125] execve("/usr/bin/openssl", ["openssl", "smime", "-sign", "-binary", "-in", "/fast/build/refkit/intel-corei7-64/mixer/mix/update/www/1121509510/Manifest.MoM", "-signer", "", "-inkey", "./private.pem", "-outform", "DER", "-out", "/fast/build/refkit/intel-corei7-64/mixer/mix/update/www/1121509510/Manifest.MoM.sig"], [/* 41 vars */]) = 0
[pid 6125] +++ exited with 3 +++
[pid 6032] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=6125, si_uid=1000, si_status=3, si_utime=0, si_stime=0} ---
ERROR: Failed to sign Manifest.MoM!
Can't open for reading, No such file or directory
140317645174528:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:74:fopen('','r')
140317645174528:error:2006D080:BIO routines:BIO_new_file:no such file:../crypto/bio/bss_file.c:81:
unable to load certificate
***Error: exit status 3
Signed Manifest.MoM
strace: Process 6126 attached
[pid 6126] execve("../swupd_make_fullfiles", ["swupd_make_fullfiles", "-S", "/fast/build/refkit/intel-corei7-64/mixer/mix/update", "1121509510"], [/* 41 vars */]) = 0
strace: Process 6127 attached
...
[build continues]
```
Rewrite the mixer code that parses builder.conf using the Viper library.
builder.confViper will treat builder.conf as a TOML file. Technically, as it is read by bundle-chroot-builder.py, which relies on the Python ConfigParserr library, it is not a full TOML file, and instead limited to the subset INI format. This should not be a problem with current usage.
The full output:
$ mixer init --clear-version 20600 --mix-version 10 --all
Creating new builder.conf configuration file...
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Warning: bundle "" already exists; skipping.
Adding bundle "R-basic"
Adding bundle "R-extras"
Adding bundle "amateur-radio"
Adding bundle "application-server"
Adding bundle "big-data-basic"
Adding bundle "bootloader"
Adding bundle "bootloader-extras"
Adding bundle "c-basic"
Adding bundle "cloud-control"
Adding bundle "cloud-native-basic"
Adding bundle "clr-devops"
Adding bundle "containers-basic"
Adding bundle "containers-basic-dev"
Adding bundle "containers-virt"
Adding bundle "containers-virt-dev"
Adding bundle "cryptography"
Adding bundle "database-basic"
Adding bundle "database-basic-dev"
Adding bundle "desktop"
Adding bundle "desktop-apps"
Adding bundle "desktop-apps-extras"
Adding bundle "desktop-assets"
Adding bundle "desktop-autostart"
Adding bundle "desktop-dev"
Adding bundle "desktop-gnomelibs"
Adding bundle "desktop-locales"
Adding bundle "dev-utils"
Adding bundle "dev-utils-dev"
Adding bundle "dhcp-server"
Adding bundle "editors"
Adding bundle "editors-dev"
Adding bundle "games"
Adding bundle "games-dev"
Adding bundle "go-basic"
Adding bundle "go-basic-dev"
Adding bundle "haskell-basic"
Adding bundle "hpc-utils"
Adding bundle "java-basic"
Adding bundle "kernel-aws"
Adding bundle "kernel-container"
Adding bundle "kernel-gce"
Adding bundle "kernel-hyperv"
Adding bundle "kernel-hyperv-lts"
Adding bundle "kernel-hyperv-mini"
Adding bundle "kernel-kvm"
Adding bundle "kernel-lts"
Adding bundle "kernel-native"
Adding bundle "koji"
Adding bundle "kvm-host"
Adding bundle "letsencrypt-client"
Adding bundle "libX11client"
Adding bundle "machine-learning-basic"
Adding bundle "machine-learning-web-ui"
Adding bundle "mail-utils"
Adding bundle "mail-utils-dev"
Adding bundle "mixer"
Adding bundle "network-basic"
Adding bundle "network-basic-dev"
Adding bundle "nodejs-basic"
Adding bundle "openssh-server"
Adding bundle "os-clear-containers"
Adding bundle "os-cloudguest"
Adding bundle "os-cloudguest-aws"
Adding bundle "os-cloudguest-azure"
Adding bundle "os-cloudguest-gce"
Adding bundle "os-cloudguest-vmware"
Adding bundle "os-clr-on-clr"
Adding bundle "os-clr-on-clr-dev"
Adding bundle "os-core"
Adding bundle "os-core-dev"
Adding bundle "os-core-update"
Adding bundle "os-core-update-dev"
Adding bundle "os-installer"
Adding bundle "os-testsuite"
Adding bundle "os-testsuite-phoronix"
Adding bundle "os-utils-gui"
Adding bundle "os-utils-gui-dev"
Adding bundle "performance-tools"
Adding bundle "perl-basic"
Adding bundle "perl-basic-dev"
Adding bundle "perl-extras"
Adding bundle "php-basic"
Adding bundle "pnp-tools-basic"
Adding bundle "productivity"
Adding bundle "pxe-server"
Adding bundle "python-basic"
Adding bundle "python-basic-dev"
Adding bundle "python-extras"
Adding bundle "python3-basic"
Adding bundle "qt-basic"
Adding bundle "ruby-basic"
Adding bundle "rust-basic"
Adding bundle "shells"
Adding bundle "storage-cluster"
Adding bundle "storage-utils"
Adding bundle "storage-utils-dev"
Adding bundle "stream-server"
Adding bundle "sysadmin-basic"
Adding bundle "sysadmin-basic-dev"
Adding bundle "sysadmin-hostmgmt"
Adding bundle "sysadmin-remote"
Adding bundle "tcl-basic"
Adding bundle "telemetrics"
Adding bundle "user-basic"
Adding bundle "user-basic-dev"
Adding bundle "vnc-server"
Adding bundle "web-server-basic"
Adding bundle "xfce4-desktop"
Initialized empty Git repository in /home/c/src/mix3/mix-bundles/.git/
[master (root-commit) 67aabca] Initial mix version 10 from upstream version 20600
108 files changed, 8755 insertions(+)
create mode 100644 R-basic
create mode 100644 R-extras
create mode 100644 amateur-radio
create mode 100644 application-server
create mode 100644 big-data-basic
create mode 100644 bootloader
create mode 100644 bootloader-extras
create mode 100644 c-basic
create mode 100644 cloud-control
create mode 100644 cloud-native-basic
create mode 100644 clr-devops
create mode 100644 containers-basic
create mode 100644 containers-basic-dev
create mode 100644 containers-virt
create mode 100644 containers-virt-dev
create mode 100644 cryptography
create mode 100644 database-basic
create mode 100644 database-basic-dev
create mode 100644 desktop
create mode 100644 desktop-apps
create mode 100644 desktop-apps-extras
create mode 100644 desktop-assets
create mode 100644 desktop-autostart
create mode 100644 desktop-dev
create mode 100644 desktop-gnomelibs
create mode 100644 desktop-locales
create mode 100644 dev-utils
create mode 100644 dev-utils-dev
create mode 100644 dhcp-server
create mode 100644 editors
create mode 100644 editors-dev
create mode 100644 games
create mode 100644 games-dev
create mode 100644 go-basic
create mode 100644 go-basic-dev
create mode 100644 haskell-basic
create mode 100644 hpc-utils
create mode 100644 java-basic
create mode 100644 kernel-aws
create mode 100644 kernel-container
create mode 100644 kernel-gce
create mode 100644 kernel-hyperv
create mode 100644 kernel-hyperv-lts
create mode 100644 kernel-hyperv-mini
create mode 100644 kernel-kvm
create mode 100644 kernel-lts
create mode 100644 kernel-native
create mode 100644 koji
create mode 100644 kvm-host
create mode 100644 letsencrypt-client
create mode 100644 libX11client
create mode 100644 machine-learning-basic
create mode 100644 machine-learning-web-ui
create mode 100644 mail-utils
create mode 100644 mail-utils-dev
create mode 100644 mixer
create mode 100644 network-basic
create mode 100644 network-basic-dev
create mode 100644 nodejs-basic
create mode 100644 openssh-server
create mode 100644 os-clear-containers
create mode 100644 os-cloudguest
create mode 100644 os-cloudguest-aws
create mode 100644 os-cloudguest-azure
create mode 100644 os-cloudguest-gce
create mode 100644 os-cloudguest-vmware
create mode 100644 os-clr-on-clr
create mode 100644 os-clr-on-clr-dev
create mode 100644 os-core
create mode 100644 os-core-dev
create mode 100644 os-core-update
create mode 100644 os-core-update-dev
create mode 100644 os-installer
create mode 100644 os-testsuite
create mode 100644 os-testsuite-phoronix
create mode 100644 os-utils-gui
create mode 100644 os-utils-gui-dev
create mode 100644 performance-tools
create mode 100644 perl-basic
create mode 100644 perl-basic-dev
create mode 100644 perl-extras
create mode 100644 php-basic
create mode 100644 pnp-tools-basic
create mode 100644 productivity
create mode 100644 pxe-server
create mode 100644 python-basic
create mode 100644 python-basic-dev
create mode 100644 python-extras
create mode 100644 python3-basic
create mode 100644 qt-basic
create mode 100644 ruby-basic
create mode 100644 rust-basic
create mode 100644 shells
create mode 100644 storage-cluster
create mode 100644 storage-utils
create mode 100644 storage-utils-dev
create mode 100644 stream-server
create mode 100644 sysadmin-basic
create mode 100644 sysadmin-basic-dev
create mode 100644 sysadmin-hostmgmt
create mode 100644 sysadmin-remote
create mode 100644 tcl-basic
create mode 100644 telemetrics
create mode 100644 user-basic
create mode 100644 user-basic-dev
create mode 100644 vnc-server
create mode 100644 web-server-basic
create mode 100644 xfce4-desktop
At least "swupd verify -i" (how I was testing) seems to depend on those and not fallback to the plain versions.
<pmccarty> my idea from a few months ago was to create a "meta" directory instead
<pmccarty> and rename "image" to "chroots"
<pmccarty> so meta would live alongside chroots
<pmccarty> packages-* and *-includes would be installed in meta/
<mrsj> I think we need to have a meeting and decide what the input and output of updates should look like
<mrsj> instead of just moving things around whenever we don't like the way it looks
The outputs include:
We also have two configuration files used to communicate information to the next step (build update)
AFAIK mixer is the only user of the Python script. Both mixer and and bundle-chroot-builder read the same configuration file, which the template value provided to users come from bundle-chroot-builder but might include some mixer specific values.
The expected result is: avoid duplication of configuration reading code, making easy to maintain the default configuration for mixer, making easy to debug mixer (one less external script in the workflow).
It is always possible to provide an extra binary later if the functionality is needed by itself without mixer.
Ister supports the -S option to customize the swupd-client state directory location instead of using /var/lib/swupd. It would be nice if mixer build-image also supported a configurable client state directory, via command line option, builder.conf option, or both.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.