fuellabs / fuelup Goto Github PK
View Code? Open in Web Editor NEW⛽ The Fuel toolchain installer
Home Page: https://fuellabs.github.io/fuelup/latest
License: Apache License 2.0
⛽ The Fuel toolchain installer
Home Page: https://fuellabs.github.io/fuelup/latest
License: Apache License 2.0
3/3 of #66, (discussion)
Before this issue can be tackled we need to first decide what stable means in the context of the Fuel toolchain.
TODO:
latest
channel as stable
(#117)fuelup toolchain install stable
in the CLIWe need a version manager similar to rustup
that allows downloading, installing, and managing different versions of the compiler and forc
. I suggest calling this forcup
.
For now, the version manager should be able to:
forc-fmt
, forc-lsp
, etc.).forc
can detect when you're using the wrong version that doesn't match what's configured in the manifest file).With the latest channel now live on gh-pages thanks to #75, fuelup
can use the TOML file to source download links to the binaries instead of manually building the download links within itself.
This would mean that the parts of the code that directly queries the GitHub API to get the latest tags and builds the download URLs manually can serve as a fallback instead of being the main method of downloading the toolchain/components. The main method should instead be to download the TOML file linked above, parse the TOML and get the appropriate download link for your desired host system.
This refactor would also serve as a basis to support the other toolchains nightly
and stable
- fuelup
would not and should not contain the logic to decide what makes a nightly
or stable
toolchain, and rather would just be told by this TOML file what the correct versions are to download.
The script installs the binaries into .fuelup/bin
instead of .cargo/bin
- fuelup-init.sh
should warn users about this and recommend updating the PATH.
Following some conversations on changes in forc from 0.14.5 -> 0.15.1 causing SwaySwap to break, I think there's space for fuelup to support usage of project-specific toolchains (which is basically rustup's override mechanic, described here).
Will expand on this when I have more thoughts regarding this but my preliminary idea is that, instead of having a rust-toolchain.toml
, we can have an entry within Forc.toml
(would this create issues with parsing the toml?) stating which version of forc is built with the project. fuelup
can detect this and decide which version of forc
to run for that project.
I think this idea is also mentioned here in the original issue created by @adlerjohn.
We would need to support the toolchain pattern #63 for this.
cc: @mitchmindtree @Braqzen for comments and ideas
toml-rs
is no longer being maintainedtoml-rs
is only compliant with v0.5 of TOML, toml_edit
is up to date with v1.0.toml_edit
is format preserving and has a nicer API for editing toml (which we will need for future features here, such as supporting #65, or future enhancements to editing settings.toml
)Currently within compare-versions
, if the script fails to download the latest channel published, it automatically tries to regenerate a channel with the latest versions, which isn't exactly desirable, since they may not be compatible. (source code)
Better way to handle this:
While trying to develop new features to tackle #46 and #16, came to the realization that there were a few places where code duplication is plenty and ugly. We can follow something similar to rustup's DownloadCfg
that represents the download configuration for a component of the Fuel toolchain:
pub struct DownloadCfg {
// name of the component, eg. "forc" | "fuel-core"
name: String,
// download url, usually https://github.com/FuelLabs/<repo>/releases/download
dist_url: String,
// tarballs have different naming convention across different projects
tarball_name: String
...
}
impl DownloadCfg {
...
pub fn download() {
// same as current download_file()
...
}
...
}
fuelup-init.sh
should ask for permission to configure the user's PATH, and work to add /.fuelup/bin
to PATH if permitted. Default behaviour should still be to not alter PATH.
Perhaps, a message like this right at the beginning:
fuelup has to download binaries into /.fuelup/bin. Can fuelup modify your PATH variable? (N/y)
fuelup-init.sh
currently depends on jq
to parse the JSON response from GitHub for the latest release tag of fuelup
. It would be nice to not have to depend on jq
.
Line 28 in 69b955c
A pure-shell implementation would be fragile and so is discouraged. An alternative would be to, on release, publish a single file to the gh-pages
branch that contains the latest release tag. Then it can be downloaded with curl
(which is already an unavoidable dependency).
fuelup
should compute the hashes of the downloads against the provided hashes within channels to ensure the authenticity of the binaries.
Having a fuelup check
command that tells us if we need to update our toolchain would be useful. Although this command is technically not needed (because fuelup install
downloads the latest binaries anyway), this command would be helpful in branching out into building other commands like fuelup update
(updates all components), fuelup default [stable|nightly]
(switch between toolchains), since these commands rely on versioning checks or toolchain checks (which we do not have the concept of yet).
We can store update tags within a dedicated folder in .fuelup
, in the format [bin]-[version]
(like how our packages are released at the moment):
/.fuelup
/bin
forc
...
/binary-versions
fuel-core-0.8.0
Then we check if the tag version is the same. Simple solution for now until we have toolchain versions in place - then it's also trivial to switch over to checking for the toolchain version
Basic implementation is now at #64 - for now we are running whichever component is first available from path atm. In future there is probably a better way to do this, especially when we have multiple toolchains.
As discussed here, Apple doesn't allow binaries downloaded from the internet to be run by users unless it is signed.
We'll need to get an apple developer certificate enabled in CI to resolve that. For now we'll need to find some kind of workaround to help users requests the permissions they need to run the binary on mac. https://docs.github.com/en/actions/deployment/deploying-xcode-applications/installing-an-apple-certificate-on-macos-runners-for-xcode-development
fuelup
is largely inspired by rustup
in terms of how it should distribute packages in the Fuel toolchain in future, and this inspiration extends to the toolchain concept.
Following a discussion here, a good first step towards building the foundation of toolchain management would be to support the latest
toolchain (which is basically what we have now!)
This would mean:
~/.fuelup/bin
, this could be in the form of a settings.toml
similar to rustup, where configs (eg. the default toolchain) are set,/toolchains/latest/bin
instead, so that next time we can download other versions to, for example, /toolchains/stable/bin
, or we can even support custom toolchains i.e. /toolchains/my-toolchain/bin
https://github.com/koalaman/shellcheck
https://github.com/ludeeus/action-shellcheck
Could be useful to ensure reliability of fuelup-init.sh
Edit: this should include a formatter as well.
In the common case that the user hasn't specified a custom toolchain with specific versions, we need to decide on which version of forc
(and associated plugins) and fuel-core
to select.
I suggest that we select the version of fuel-core
based on the selected version of forc
, as forc
depends on fuel-core
, but the inverse is not true.
One way of achieving this could be to select the version of fuel-core
based on the version that was used within forc
's CI within the commit associated with the selected forc
version. We could do this by doing something like:
fuel-core-version
, which just contains the semver version and nothing more.fuel-core
to fetch in the Sway CI.forc
, determine the version of fuel-core
to use by reading the fuel-core-version
file.An alternative could be to select the version based on the timestamp of the commit (e.g. select the fuel-core
commit with the first timestamp that precedes the timestamp of the forc
release), but I think the concept of selecting commits by similar timestamps is probably much less robust than the known, CI-tested version.
When downloading binaries for a new version of forc
should also install the std and core libs, so that no further internet access is necessary. This could be done by cloning the repo for those deps exactly as forc
would, or through another mechanism.
When looking at the help section of the install
sub-command it does not tell me anything additional to what fuelup --help
tells me for that sub-command. It actually shows me less info.
All it shows me is the usage - which at this point (with the number of sub-commands and their simplicity) is useless.
Fix: Add a Description
section or something equivalent to expand on the 1 liner presented by just using fuelup --help
Note: How rustup does it.
In between publishing a release and binaries being built and uploaded by CI, a user that runs fuelup-init.sh
will be greeted with a message like
info: downloading installer
curl: (22) The requested URL returned error: 404
fuelup: installer for platform 'aarch64-apple-darwin' not found, this may be unsupported
fuelup-init.sh
should fail gracefully and warn the user that they might be in this situation, and try again in a few minutes. Maybe provide a link to the actual release page on GitHub too so they can manually confirm there are release binaries, and the time of release, etc.
curl
with rustls
is unstable, see: https://github.com/alexcrichton/curl-rust#building
rustls Enable SSL/TLS support via Rustls, a well-received alternative TLS backend written in Rust. Rustls is always statically linked. Disabled by default.
Consider using ureq
instead as a native-Rust alternative. Ref: FuelLabs/sway#651
We should have a README describing the minimum setup required to download fuelup
and start hacking right away on a Sway project.
This should be done after we have:
fuelup
should also be able to download and install plugins supported natively by Fuel Labs.
Currently the only time we warn users is within fuelup-init
, and on the first pass users might miss out on the warning. We should re-raise the warning within fuelup toolchain install <toolchain>
or fuelup update
in future.
2/3 of #66, (discussion)
After supporting the latest
toolchain, next toolchain to support would probably be nightly
.
Making this a tracking issue for:
fuelup
to support installing from nightly
channelfuelup
CI to publish nightly binaries in a TOML file, channel-fuel-nightly.toml
on the gh-pages branchnightly
Add a CONTRIBUTING.md document for raising issues and/or contributing code.
Following on from #44, fuelup
should fail gracefully if binaries for forc*
and fuel-core
can't be found.
When working on a JS project, sourcing deps like forc and fuel-core is an issue. To make this easier, we can follow the example of other non-JS projects that want to play well with the JS ecosystem and release a package to NPM called fuelup-bin
, so our toolchain can be a regular NPM dependency.
The initial plan was to release forc-bin
and fuel-core-bin
, but was closed in favor of fuelup-bin
: https://github.com/FuelLabs/pm/issues/19
Some work was already done for forc-bin
on FuelLabs/fuels-ts#287 which can be repurposed for fuelup-bin
.
Once we've got a bare-bones cargo project, we should prioritise setting up CI.
We'll likely want to publish the fuelup
binary for each platform similar to how we publish forc
for each platform too.
When running fuelup install
the output states that it first fetches forc
for the latest version followed by displaying the URL query.
Then it repeats the behaviour for fuel-core
.
Lastly, it unpacks the plugins.
If I already have a version installed that matches whatever is being fetched it should do one of the following
Note: How rustup does it. First time it updates and the second time it syncs and tells me there is nothing to change.
$ ~/.fuelup/bin/forc --help
/home/john/.fuelup/bin/forc: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
This is weird because I was expecting it to be statically linked?
fuelup
should be able to do both these things:
forc
can detect when you're using the wrong version that doesn't match what's configured in the manifest file).As mentioned here, this would probably require some design decisions on what constitutes a stable toolchain and how we reconcile forc
and fuel-core
compatibility
Update:
With FuelLabs/sway#2405 now merged, we can detect inconsistent versions by matching with the minimum forc-version
specified in the Forc.toml
.
Related to #66, followup to #75.
I wonder if there's some way we can re-use the E2E tests from the Sway repo for this? E.g. fetch the sway repo, patch the forc
and fuel-core
versions used and run those tests?
We only really need the tests that check the integration of forc
and fuel-core
, so e.g. the contract deployment ones, running scripts and checking their results, etc. Perhaps we could filter out the language tests.
Alternatively, we can start adding our own integration tests here. That said, I can't think of many forc
+fuel-core
integration tests that would be only be beneficial here that would not also be beneficial at the Sway repo. It would be a shame to have some test here that could have been added to Sway and caught some incompatibility bug earlier in the release process.
It seems as though GitHub API might be flaky with fetching the latest version of fuelup
after some reports of failure with running the fuelup-init
script with the fuelup
version somehow missing in the logs.
This issue is also somewhat a followup to #43 so that we can avoid having to curl GitHub API unnecessarily.
There is a small logic error with the way index-versions.sh
was running that causes empty hashes to appear:
curl
was not failing and exiting out of the script if the download was not successful and resulted in the script trying to calculate a hash for a missing file. As you can see from the CI workflow run below, there was a 404 but the workflow still completed successfully.
The solution is to exit out of the script should there be an attempt to curl a binary release that is not ready, and skip the indexing for that run.
https://github.com/FuelLabs/fuelup/runs/7048290707?check_suite_focus=true
Ideally the minimum viable version of fuelup
should simply download the latest stable release toolchain (forc
, fuel-core
) based on the user's OS or user input
Both as a general feature and as a workaround for #45, allow users to install specific versions of specific components of the Fuel toolchain.
Similar to how rustup
has an mdbook.
It could be worth starting this early so that we can document features as they're added.
We can take inspiration from sway
, fuels-rs
or fuel-indexer
repos on how to setup an mdbook and how to automate it's publishing with CI.
Executing the following on my MBA M1
curl --proto '=https' --tlsv1.2 -sSf https://fuellabs.github.io/fuelup/fuelup-init.sh | sh -s install
was resulting in the following:
cut: illegal option -- -
usage: cut -b list [-n] [file ...]
cut -c list [file ...]
cut -f list [-s] [-d delim] [file ...]
info: downloading installer
curl: (22) The requested URL returned error: 404
fuelup: command failed: downloader https://github.com/FuelLabs/fuelup/releases/download/v/fuelup--aarch64-apple-darwin.tar.gz /var/folders/dr/s22m32g96ksgthk_4kby3mrh0000gn/T/tmp.FAirxTJB/fuelup.tar.gz aarch64-apple-darwin
by @binggh suggestion : changing cut -c 1 --complement
to cut -c 2-
in fuelup-init.sh
seems like doing the trick.
There's a small issue with running fuelup-init.sh
to ensure that fuelup installs properly within the job when releasing fuelup
- since the script would try to source for the release that was in the process of being released, the binaries were not ready yet. This led to CI failure here.
This is not a breaking bug however, since it happens only during releases and we can rerun the workflow to have it pass once the binaries are available. We should still fix the behaviour somehow, perhaps by waiting on the binaries to be released if it is ran during a release.
Accidentally formatted with tabs - we should reformat it using 2 spaces instead.
In the case of an unavailable release, which might happen when the latest tag is published and yet the binaries are not available (happened to me with forc v0.14.1
earlier), fuelup
still outputs the installation message. This should be reworked so that only successful downloads are printed
fuelup
is a binary, so I think the lockfile should be committed. cc @mitchmindtree
1/3 of #66, (discussion)
As discussed in the issue, we will kick off selecting compatible versions of forc
and fuel-core
to be able to support the concept of a toolchain, starting with the latest
.
fuelup
needs the ability to update itself, whenever there's a new release.
Similar to how fuelup
downloads new releases for fuel-core
and forc
, it can check for new releases and download it into $HOME/.fuelup/bin
This the output I get when running fuelup self update
Fetching binary from https://github.com/FuelLabs/fuelup/releases/download/v0.2.1/fuelup-0.2.1-x86_64-unknown-linux-gnu.tar.gz
Unpacking and moving fuelup to /home/segfault_magnet/.fuelup/bin
fuelup failed to update itself: Text file busy (os error 26)
If I run it with strace
I get:
openat(AT_FDCWD, "/home/segfault_magnet/.fuelup/bin/fuelup", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0100755) = -1 ETXTBSY (Text file busy)
close(5) = 0
close(4) = 0
close(3) = 0
write(1, "fuelup failed to update itself: "..., 61fuelup failed to update itself: Text file busy (os error 26)
) = 61
This SO question might be useful
Hi,
I installed fuelup
and forc
today and managed to build and deploy a project.
After reopening a terminal, I'm now getting the following error:
/home/Dodecahedr0x/.fuelup/toolchains/latest-x86_64-unknown-linux-gnu/bin/forc: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /home/Dodecahedr0x/.fuelup/toolchains/latest-x86_64-unknown-linux-gnu/bin/forc)
I'm using the Windows Subsystem Linux (WSL) running a 18.04 Ubuntu.
I think this could be due to the default rustup toolchain being stable-x86_64-unknown-linux-gnu (default)
instead of something x86_64-linux-gnu
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.