haskell / ghcup-metadata Goto Github PK
View Code? Open in Web Editor NEWGHCup metadata repository
GHCup metadata repository
It would be good to sync updating the metadata here with each ghc release.
GHC 9.2.6 has been released:
NB. Support for new GHCs is usually swift in GHCup, I opened this issue mostly for tracking purposes.
Steps to reproduce:
https://cloud.debian.org/images/cloud/bullseye/20211011-792/debian-11-nocloud-amd64-20211011-792.qcow2
in qemucurl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh
https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-x86_64-deb9-linux.tar.xz
gets installed instead of the deb10
variant, which is more appropriate (IMO)Probable cause:
Linux_Debian:
'( >= 9 && < 10 )': &ghc-8107-64-deb9
dlUri: https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-x86_64-deb9-linux.tar.xz
dlSubdir: ghc-8.10.7
dlHash: ced9870ea351af64fb48274b81a664cdb6a9266775f1598a79cbb6fdd5770a23
'( >= 10 && < 11 )': &ghc-8107-64-deb10
dlUri: https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-x86_64-deb10-linux.tar.xz
dlSubdir: ghc-8.10.7
dlHash: a13719bca87a0d3ac0c7d4157a4e60887009a7f1a8dbe95c4759ec413e086d30
unknown_versioning: *ghc-8107-64-deb9
The above falls back to deb9
for Debian 11.
Note that the fallback is also used with Debian testing or unstable, as these don't have any version in /etc/os-release.
(Same thing happens with GHC 9.2.1, it's not specific to old GHC versions.)
Discussions:
Main concern that we have here is that alpine bindists seem to be busted according to @mpilgrem:
I'll probably discuss this with @mpickering tomorrow.
@bgamari @wz1000 sorry for the drastic measures, but I revoked push access until we figure out what's going on here. I did not have enough time to verify that the cross signing has succeeded and which keys exactly we need to use.
The gpg feature is not about convenience. It's a security feature and it's important that we get it right, even if it's off by default. Also see haskell/ghcup-hs#858
I sent an email to both your WT addresses with your pubkeys cross signed. The idea was that you will then import them and upload them (uploading for other people is discouraged).
I still don't know if you received that mail. If you did not, why? Did you check your spam filter? These emails need to be valid and available for end-users to contact you in private for key verification.
I followed these instructions: https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84
So let's clear up some confusion:
Error: [GHCup-05841] Download failed: Process "curl" with arguments ["-fL", "-o",
"/home/phadej/.ghcup/tmp/ghcup-5089968bf31a9814/ghc-9.8.0.20230727-x86_64-linux-ubuntu18_04.tar.xz.tmp",
"https://downloads.haskell.org/~ghc/9.8.0.20230727/ghc-9.8.0.20230727-x86_64-ubuntu18_04-linux.tar.xz"] failed with exit code 22.
URL is incorrect.
As far as I can see there is https://downloads.haskell.org/ghc/9.8.0.20230727/ghc-9.8.0.20230727-x86_64-deb9-linux.tar.xz
i.e. deb9-linux
, not ubuntu18_04
.
stack upstream doesn't provide them yet, so we should roll our own for now: https://gitlab.haskell.org/maerwald/stack/-/pipelines/44275
Issue opened for tracking
@Kleidukos once you added 3.10.2.1
here, please close this issue.
But the test bindist is busted: https://gitlab.haskell.org/ghc/ghc/-/issues/22723
Currently ghcup
metadata maintenance is one of the more manual (and consequently error-prone) aspects of cutting a GHC release. Specifically, it involves manually adding a snippet to the 1000+ LoC metadata file and then carefully editing the YAML anchors of said snippet to ensure that they are globally unique.
It seems to me that this process could be streamlined by splitting the metadata into individual files which can be combined into the final monolithic metadata file by CI (e.g. when merging to master
). For instance, one might imagine this repository consisting of a directory structure like:
metadata
+ ghc
| + 9.4.5.yaml
| + 9.4.5.yaml.asc
| + 9.6.3.yaml
| + 9.6.3.yaml.asc
+ cabal
| + 3.2.0.0.yaml
| + 3.2.0.0.yaml.asc
...
This, of course, poses the problem of ensuring that the final metadata is signed. I can see at least three approaches that might be used here:
ghcup
itself to distribute the individual metadata files (e.g. via a cabal-style tar
archive) and validate each individually.It seems that ghcup-0.0.7.yaml
is more up-to-date than ghcup-0.0.8.yaml
.
Why there is different YAML version for each channel? Because when new version of Haskell tools release, the maintainer will append that version to YAML file instead of overwrite it.
Can anyone give me some explanation?
It's been several months. Several regressions exist but nothing egregious. So, maybe we can get the badge for 3.7 3.8?
The URL in metadata seems to be wrong: metadata says https://downloads.haskell.org/~ghc/9.8.0.20230727/
but the correct URL is https://downloads.haskell.org/~ghc/9.8.1-alpha1/
.
Log:
$ ghcup install ghc 9.8.1-alpha1
[ Info ] downloading: https://downloads.haskell.org/~ghc/9.8.0.20230727/ghc-9.8.0.20230727-x86_64-apple-darwin.tar.xz as file $HOME/.ghcup/tmp/ghcup-d515ee862ba6ad4e/ghc-9.8.0.20230727-x86_64-darwin.tar.xz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404
[ Error ] [GHCup-05841] Download failed: Process "curl" with arguments ["-fL", "-o",
[ ... ] "$HOME/.ghcup/tmp/ghcup-d515ee862ba6ad4e/ghc-9.8.0.20230727-x86_64-darwin.tar.xz.tmp",
[ ... ] "https://downloads.haskell.org/~ghc/9.8.0.20230727/ghc-9.8.0.20230727-x86_64-apple-darwin.tar.xz"] failed with exit code 22.
[ Error ] Also check the logs in $HOME/.ghcup/logs
Input:
https://downloads.haskell.org/~ghc/9.0.1
ghcup-0.0.6.yaml
)Output:
Things to consider:
viPostRemove
, viPostInstall
etc rightrecommended
and latest
), some are not (e.g. old
))ghcup list
gives now this:
✗ ghc 9.4.0.20220501 prerelease,base-4.17.0.0
✗ ghc 9.4.0.20220523 prerelease,base-4.17.0.0
✗ ghc 9.4.0.20220623 prerelease,base-4.17.0.0
✗ ghc 9.4.0.20220721 prerelease,base-4.17.0.0
✗ ghc 9.4.1 base-4.17.0.0
...
✗ ghc 9.4.6 base-4.17.2.0
✗ ghc 9.6.0.20230111 prerelease,base-4.18.0.0
✗ ghc 9.6.0.20230128 prerelease,base-4.18.0.0
✗ ghc 9.6.0.20230210 prerelease,base-4.18.0.0
✗ ghc 9.6.0.20230302 latest-prerelease,base-4.18.0.0
✗ ghc 9.6.1 base-4.18.0.0
✗ ghc 9.6.2 latest,base-4.18.0.0 hls-powered
✗ ghc 9.8.1-alpha1 prerelease,base-4.19.0.0 2023-07-28
The last is an outlier, please add it as 9.8.0.20230727
as well.
Follow-up of:
ATTN: @bgamari
ghcup-metadata/ghcup-0.0.7.yaml
Lines 9 to 33 in 3e07ac0
From ghcup-0.0.7.yaml:
...
Darwin:
unknown_versioning:
dlUri: https://downloads.haskell.org/~ghcup/0.1.17.7/x86_64-apple-darwin-ghcup-0.1.17.7
dlHash: d3d0644dc5d9b51ed1c345fc006e936e9284b3181e5a9cccf4cf70a7184398fe
FreeBSD:
'( >= 12 && < 13 )':
dlUri: https://downloads.haskell.org/~ghcup/0.1.17.7/x86_64-freebsd12-ghcup-0.1.17.7
dlHash: d3d0644dc5d9b51ed1c345fc006e936e9284b3181e5a9cccf4cf70a7184398fe
'( >= 13 )':
dlUri: https://downloads.haskell.org/~ghcup/0.1.17.7/x86_64-freebsd13-ghcup-0.1.17.7
dlHash: d3d0644dc5d9b51ed1c345fc006e936e9284b3181e5a9cccf4cf70a7184398fe
The hash for Darwin binary is the same as FreeBSD binary. Is this really correct? I could not do a ghcup upgrade
on my mac without first changing this to 9702f30c9374a122d79f7ef11170b34deb248a0f3cd92d671c0aab747be4add7
(the hash for the darwin binary pointed to by the url).
$ ghcup list
✗ ghcup 0.1.19.4 latest,recommended
✔✔ ghcup 0.1.19.5 latest,recommended
Recently the gpg key used to sign the the ghcup ghcup-0.0.7.yaml
file changed.
When following the updated instructions at https://www.haskell.org/ghcup/guide/#gpg-verification I get the following message:
$ gpg --batch --keyserver keys.openpgp.org --recv-keys 7D1E8AFD1D4A16D71FADA2F2CCC85C0E40C06A8C
gpg: key CCC85C0E40C06A8C: new key but contains no user ID - skipped
gpg: Total number processed: 1
gpg: w/o user IDs: 1
So, while the key is found, it does not get imported as it does not have any user ID associated with it.
The impact of this is that ghcup verification when doing ghcup tui
fails predictably as [ Error ] [GHCup-00210] GPG verify failed: GPG verify failed
Manually trying to verify the yaml results in the following
$ gpg --verify ghcup-0.0.7.yaml.sig ghcup-0.0.7.yaml
gpg: Signature made Thu Mar 30 21:04:04 2023 IST
gpg: using RSA key 7D1E8AFD1D4A16D71FADA2F2CCC85C0E40C06A8C
gpg: issuer "[email protected]"
gpg: Can't check signature: No public key
Interestingly here there is some sort of user id attached to the signature.
E.g. https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS to check that files exist, or actually GET
but not reading the body (not so nice for the server).
https://github.com/haskell/ghcup-metadata/actions/runs/4180841985/jobs/7242206970
Fails silently.
We install wrong GHC:
Need to mimic the test from hls CI:
From the perspective of the Stack project, there is no reason for GHCup not to recommend the latest Stack release, at any point - Stack aims to have a high degree of backwards compatibility and it would become clear within a few weeks of any release if there were any undetected regressions introduced into a release.
The hls-powered
tags produced by ghcup list
do not match what is written at: https://haskell-language-server.readthedocs.io/en/latest/support/ghc-version-support.html
A new version of stack was released. No idea how one would update stack metadata, though. Would be cool, if added to README instructions.
https://github.com/commercialhaskell/stack/releases/tag/v2.9.1
I added vanilla release channel to be able to install GHC-9.6.1, and now my builds started failing on GHC-9.0.2
Is GHC-9.0.2 bindists in the main release channel different? Is there away to add a release-channel so it doesn't override anything in main channel?
<no location info>: error:
<command line>: /github/home/.cabal/store/ghc-9.0.2/regex-base-0.94.0.2-74ad7a23e807bdf2[42](https://github.com/haskell-CI/haskell-ci/actions/runs/4397158283/jobs/7701773044#step:21:43)4625a40f382dedae9acc7d83e88b587217df1216929b61/lib/libHSregex-base-0.94.0.2-74ad7a23e807bdf2424625a40f382dedae9acc7d83e88b587217df1216929b61-ghc9.0.2.so: undefined symbol: base_GHCziBase_zpzpzuzdszpzp_info
@amesgen please? We would have needed this for HLS...
https://github.com/haskell/ghcup-metadata/blob/develop/.github/workflows/bindists.yaml
Check
While looking through ghcup-prereleases-0.0.7.yaml
I realized that I have no idea what the intended semantics of the old
tag are. We should probably have a canonical reference document describing the set of valid tags (and check consistency as described in #135).
Error: [GHCup-05841] Download failed: Process "curl" with arguments ["-fL", "-o",
"/home/phadej/.ghcup/tmp/ghcup-0e8c66d3020a1a24/ghc-9.6.2.20230523-x86_64-linux-alpine3_12-cross_wasm32-wasi-release+fully_static.tar.xz.tmp",
"https://gitlab.haskell.org/api/v4/projects/3223/jobs/1530829/artifacts/ghc-x86_64-linux-alpine3_12-cross_wasm32-wasi-release+fully_static.tar.xz"] failed with exit code 22.
ormolu
is a popular Haskell formatter for developers. It would be nice to provide a static compiled ormolu
in ghcup toolchain.
stack install ormolu
is difficult to clean and do version management. This tool is pretty feature-completed.
haskell/haskell-language-server#2424
tinfo failures
As can be seen from #12 I wasn't able to attend to required maintenance work for a couple of weeks, due to a hardware repair.
As all yaml files in this repo are gpg signed, we need a 1-2 contributors who can take over this task in case I'm unavailable. Their gpg keys should be added to: https://www.haskell.org/ghcup/guide/#gpg-verification and https://www.haskell.org/ghcup/install/#manual-install
@tomjaguarpaw has had concerns about GHCup devs changing binaries. It's true, we may.
Users may want to opt out of that. We could trivially add a metadata file that only supplies vanilla upstream bindists. For GHC that will mean that 30% of them will be broken due to this bug: https://gitlab.haskell.org/ghc/ghc/-/issues/19646
But vanilla is vanilla, so users will have to deal with it.
This problem is tracked at haskell/cabal#9334 and was first encountered in #127
This bindist is statically linked and seems to cause issues for HLS compilation: haskell/haskell-language-server#2615 (comment)
Following https://gitlab.haskell.org/ghc/ghc/-/issues/23043 I have decided that GHCup will not distribute static alpine bindists anymore in the default release channel.
CI will be enhanced to catch that.
Instead, static alpine will only be provided through a separate release channel: #91
So that we don't have to change the source manually, and can always get the newest tools by just checking ghcup list
.
Currently it's very easy to overlook or otherwise mess up the tags when adding a new tool version. CI should ideally perform some consistency checks to catch this failure mode:
Latest
and one LatestPrerelease
, and one Recommended
version for each toolghc
tool should have a base-...
tag@amesgen following the suggestion from https://discourse.haskell.org/t/hf-board-meeting-minutes-2022-10-06/5143/12
Any ideas how to make a nightly job that installs:
recommended
as well as latest
tag?Apparently, this has caused issues with HLS, e.g. Bodigrim/mod#13
TODO:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.