It would be good to sync updating the metadata here with each ghc release.

GHC 9.2.6 has been released:

• https://www.haskell.org/ghc/download_ghc_9_2_6.html

NB. Support for new GHCs is usually swift in GHCup, I opened this issue mostly for tracking purposes.

Steps to reproduce:

1. boot https://cloud.debian.org/images/cloud/bullseye/20211011-792/debian-11-nocloud-amd64-20211011-792.qcow2 in qemu
2. curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh
3. observe that https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-x86_64-deb9-linux.tar.xz gets installed instead of the deb10 variant, which is more appropriate (IMO)

Probable cause:

          Linux_Debian:
            '( >= 9 && < 10 )': &ghc-8107-64-deb9
              dlUri: https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-x86_64-deb9-linux.tar.xz
              dlSubdir: ghc-8.10.7
              dlHash: ced9870ea351af64fb48274b81a664cdb6a9266775f1598a79cbb6fdd5770a23
            '( >= 10 && < 11 )': &ghc-8107-64-deb10
              dlUri: https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-x86_64-deb10-linux.tar.xz
              dlSubdir: ghc-8.10.7
              dlHash: a13719bca87a0d3ac0c7d4157a4e60887009a7f1a8dbe95c4759ec413e086d30
            unknown_versioning: *ghc-8107-64-deb9

The above falls back to deb9 for Debian 11.
Note that the fallback is also used with Debian testing or unstable, as these don't have any version in /etc/os-release.

(Same thing happens with GHC 9.2.1, it's not specific to old GHC versions.)

Discussions:

• https://discourse.haskell.org/t/rfc-bumping-recommended-ghc-to-9-4-5-in-ghcup/6866
• https://www.reddit.com/r/haskell/comments/14t5elx/rfc_bumping_recommended_ghc_to_945_in_ghcup/?utm_source=share&utm_medium=web2x&context=3
• https://kbin.social/m/haskell/t/150476/RFC-Bumping-recommended-GHC-to-9-4-5-in-GHCup

Main concern that we have here is that alpine bindists seem to be busted according to @mpilgrem:

• https://discourse.haskell.org/t/rfc-bumping-recommended-ghc-to-9-4-5-in-ghcup/6866/3?u=hasufell
• https://gitlab.haskell.org/ghc/ghc/-/issues/23043#note_506034

I'll probably discuss this with @mpickering tomorrow.

@mpickering

@bgamari @wz1000 sorry for the drastic measures, but I revoked push access until we figure out what's going on here. I did not have enough time to verify that the cross signing has succeeded and which keys exactly we need to use.

The gpg feature is not about convenience. It's a security feature and it's important that we get it right, even if it's off by default. Also see haskell/ghcup-hs#858

I sent an email to both your WT addresses with your pubkeys cross signed. The idea was that you will then import them and upload them (uploading for other people is discouraged).

I still don't know if you received that mail. If you did not, why? Did you check your spam filter? These emails need to be valid and available for end-users to contact you in private for key verification.

I followed these instructions: https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84

So let's clear up some confusion:

1. did you received my email?
2. which are your keys exactly?
   • from Ben I now have two keys and I don't know which one to use:
     • 0x73EDE9E8CFBAEF01 was expired last time I checked and is only cross-signed by me, but not by Zubin
     • 0x2de04d4e97db64ad is cross signed by Zubin, but not by me (because apparently Ben did not receive my email with the signed pubkey)
   • from Zubin, I'm guessing it's 0x588764fbe22d19c4 but it's neither cross-signed by Ben, nor by me

Error: [GHCup-05841] Download failed: Process "curl" with arguments ["-fL", "-o",
                                                "/home/phadej/.ghcup/tmp/ghcup-5089968bf31a9814/ghc-9.8.0.20230727-x86_64-linux-ubuntu18_04.tar.xz.tmp",
                                                "https://downloads.haskell.org/~ghc/9.8.0.20230727/ghc-9.8.0.20230727-x86_64-ubuntu18_04-linux.tar.xz"] failed with exit code 22.

URL is incorrect.

As far as I can see there is https://downloads.haskell.org/ghc/9.8.0.20230727/ghc-9.8.0.20230727-x86_64-deb9-linux.tar.xz

i.e. deb9-linux, not ubuntu18_04.

stack upstream doesn't provide them yet, so we should roll our own for now: https://gitlab.haskell.org/maerwald/stack/-/pipelines/44275

#62

Issue opened for tracking

• haskell-actions/setup#53

@Kleidukos once you added 3.10.2.1 here, please close this issue.

https://github.com/haskell/ghcup-metadata/actions/runs/4262127875/jobs/7417258308

But the test bindist is busted: https://gitlab.haskell.org/ghc/ghc/-/issues/22723

Currently ghcup metadata maintenance is one of the more manual (and consequently error-prone) aspects of cutting a GHC release. Specifically, it involves manually adding a snippet to the 1000+ LoC metadata file and then carefully editing the YAML anchors of said snippet to ensure that they are globally unique.

It seems to me that this process could be streamlined by splitting the metadata into individual files which can be combined into the final monolithic metadata file by CI (e.g. when merging to master). For instance, one might imagine this repository consisting of a directory structure like:

metadata
  + ghc
  |    + 9.4.5.yaml
  |    + 9.4.5.yaml.asc
  |    + 9.6.3.yaml
  |    + 9.6.3.yaml.asc
  + cabal
  |    + 3.2.0.0.yaml
  |    + 3.2.0.0.yaml.asc
...

This, of course, poses the problem of ensuring that the final metadata is signed. I can see at least three approaches that might be used here:

• Approach A: Teach CI to verify each of the signatures of the individual per-version metadata files and apply its own signature (using its own key) to the final metadata.
• Approach B: Teach ghcup itself to distribute the individual metadata files (e.g. via a cabal-style tar archive) and validate each individually.
• Approach C: Rework the signature scheme to instead sign a canonicalized representation of the per-version metadata (e.g. teach the verification scheme to render the metadata as Canonical JSON and verify the signature with respect to that representation)

It seems that ghcup-0.0.7.yaml is more up-to-date than ghcup-0.0.8.yaml.

Why there is different YAML version for each channel? Because when new version of Haskell tools release, the maintainer will append that version to YAML file instead of overwrite it.

Can anyone give me some explanation?

It's been several months. Several regressions exist but nothing egregious. So, maybe we can get the badge for 3.7 3.8?

The URL in metadata seems to be wrong: metadata says https://downloads.haskell.org/~ghc/9.8.0.20230727/ but the correct URL is https://downloads.haskell.org/~ghc/9.8.1-alpha1/.

Log:

$ ghcup install ghc 9.8.1-alpha1
[ Info  ] downloading: https://downloads.haskell.org/~ghc/9.8.0.20230727/ghc-9.8.0.20230727-x86_64-apple-darwin.tar.xz as file $HOME/.ghcup/tmp/ghcup-d515ee862ba6ad4e/ghc-9.8.0.20230727-x86_64-darwin.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0   146    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
[ Error ] [GHCup-05841] Download failed: Process "curl" with arguments ["-fL", "-o",
[ ...   ]                                                 "$HOME/.ghcup/tmp/ghcup-d515ee862ba6ad4e/ghc-9.8.0.20230727-x86_64-darwin.tar.xz.tmp",
[ ...   ]                                                 "https://downloads.haskell.org/~ghc/9.8.0.20230727/ghc-9.8.0.20230727-x86_64-apple-darwin.tar.xz"] failed with exit code 22.
[ Error ] Also check the logs in $HOME/.ghcup/logs

Input:

1. base url like https://downloads.haskell.org/~ghc/9.0.1
2. SHA256SUM file
3. metadata file (e.g. ghcup-0.0.6.yaml)
4. changelog URL

Output:

• updated YAML

Things to consider:

1. getting the yaml anchors right
2. getting viPostRemove, viPostInstall etc right
3. not missing the source tarball
4. getting the tags right (some tags are unique (e.g. recommended and latest), some are not (e.g. old))
5. getting the subdirectories right

ghcup list gives now this:

✗  ghc   9.4.0.20220501 prerelease,base-4.17.0.0                   
✗  ghc   9.4.0.20220523 prerelease,base-4.17.0.0                   
✗  ghc   9.4.0.20220623 prerelease,base-4.17.0.0                   
✗  ghc   9.4.0.20220721 prerelease,base-4.17.0.0                   
✗  ghc   9.4.1          base-4.17.0.0                              
...
✗  ghc   9.4.6          base-4.17.2.0                              
✗  ghc   9.6.0.20230111 prerelease,base-4.18.0.0                   
✗  ghc   9.6.0.20230128 prerelease,base-4.18.0.0                   
✗  ghc   9.6.0.20230210 prerelease,base-4.18.0.0                   
✗  ghc   9.6.0.20230302 latest-prerelease,base-4.18.0.0            
✗  ghc   9.6.1          base-4.18.0.0                              
✗  ghc   9.6.2          latest,base-4.18.0.0            hls-powered
✗  ghc   9.8.1-alpha1   prerelease,base-4.19.0.0        2023-07-28 

The last is an outlier, please add it as 9.8.0.20230727 as well.

Follow-up of:

• #101
• #102

ATTN: @bgamari

• Maybe using ghc as example

From ghcup-0.0.7.yaml:

         ...
          Darwin:
            unknown_versioning:
              dlUri: https://downloads.haskell.org/~ghcup/0.1.17.7/x86_64-apple-darwin-ghcup-0.1.17.7
              dlHash: d3d0644dc5d9b51ed1c345fc006e936e9284b3181e5a9cccf4cf70a7184398fe
          FreeBSD:
            '( >= 12 && < 13 )':
              dlUri: https://downloads.haskell.org/~ghcup/0.1.17.7/x86_64-freebsd12-ghcup-0.1.17.7
              dlHash: d3d0644dc5d9b51ed1c345fc006e936e9284b3181e5a9cccf4cf70a7184398fe
            '( >= 13 )':
              dlUri: https://downloads.haskell.org/~ghcup/0.1.17.7/x86_64-freebsd13-ghcup-0.1.17.7
              dlHash: d3d0644dc5d9b51ed1c345fc006e936e9284b3181e5a9cccf4cf70a7184398fe

The hash for Darwin binary is the same as FreeBSD binary. Is this really correct? I could not do a ghcup upgrade on my mac without first changing this to 9702f30c9374a122d79f7ef11170b34deb248a0f3cd92d671c0aab747be4add7 (the hash for the darwin binary pointed to by the url).

Recently the gpg key used to sign the the ghcup ghcup-0.0.7.yaml file changed.

When following the updated instructions at https://www.haskell.org/ghcup/guide/#gpg-verification I get the following message:

$ gpg --batch --keyserver keys.openpgp.org     --recv-keys 7D1E8AFD1D4A16D71FADA2F2CCC85C0E40C06A8C
gpg: key CCC85C0E40C06A8C: new key but contains no user ID - skipped
gpg: Total number processed: 1
gpg:           w/o user IDs: 1

So, while the key is found, it does not get imported as it does not have any user ID associated with it.

The impact of this is that ghcup verification when doing ghcup tui fails predictably as [ Error ] [GHCup-00210] GPG verify failed: GPG verify failed

Manually trying to verify the yaml results in the following

$ gpg --verify ghcup-0.0.7.yaml.sig ghcup-0.0.7.yaml
gpg: Signature made Thu Mar 30 21:04:04 2023 IST
gpg:                using RSA key 7D1E8AFD1D4A16D71FADA2F2CCC85C0E40C06A8C
gpg:                issuer "[email protected]"
gpg: Can't check signature: No public key

Interestingly here there is some sort of user id attached to the signature.

E.g. https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS to check that files exist, or actually GET but not reading the body (not so nice for the server).

Issues like #104 or #102 are relatively easily preventable.

https://github.com/haskell/ghcup-metadata/actions/runs/4180841985/jobs/7242206970

Fails silently.

We install wrong GHC:

Need to mimic the test from hls CI:

https://github.com/haskell/haskell-language-server/blob/2598fcec399835a3ac83e76e8b3451f1dd9a86a1/.github/scripts/test.sh#L27-L48

From the perspective of the Stack project, there is no reason for GHCup not to recommend the latest Stack release, at any point - Stack aims to have a high degree of backwards compatibility and it would become clear within a few weeks of any release if there were any undetected regressions introduced into a release.

The hls-powered tags produced by ghcup list do not match what is written at: https://haskell-language-server.readthedocs.io/en/latest/support/ghc-version-support.html

A new version of stack was released. No idea how one would update stack metadata, though. Would be cool, if added to README instructions.

https://github.com/commercialhaskell/stack/releases/tag/v2.9.1

I added vanilla release channel to be able to install GHC-9.6.1, and now my builds started failing on GHC-9.0.2

Is GHC-9.0.2 bindists in the main release channel different? Is there away to add a release-channel so it doesn't override anything in main channel?

<no location info>: error:
    <command line>: /github/home/.cabal/store/ghc-9.0.2/regex-base-0.94.0.2-74ad7a23e807bdf2[42](https://github.com/haskell-CI/haskell-ci/actions/runs/4397158283/jobs/7701773044#step:21:43)4625a40f382dedae9acc7d83e88b587217df1216929b61/lib/libHSregex-base-0.94.0.2-74ad7a23e807bdf2424625a40f382dedae9acc7d83e88b587217df1216929b61-ghc9.0.2.so: undefined symbol: base_GHCziBase_zpzpzuzdszpzp_info

@amesgen please? We would have needed this for HLS...

https://github.com/haskell/ghcup-metadata/blob/develop/.github/workflows/bindists.yaml

Check

• that the recommended GHC works with the recommended HLS
• that the latest GHC works with the latest HLS
• that the recommended GHC works with the latest HLS

While looking through ghcup-prereleases-0.0.7.yaml I realized that I have no idea what the intended semantics of the old tag are. We should probably have a canonical reference document describing the set of valid tags (and check consistency as described in #135).

ormolu is a popular Haskell formatter for developers. It would be nice to provide a static compiled ormolu in ghcup toolchain.

stack install ormolu is difficult to clean and do version management. This tool is pretty feature-completed.

haskell/haskell-language-server#2424

tinfo failures

As can be seen from #12 I wasn't able to attend to required maintenance work for a couple of weeks, due to a hardware repair.

As all yaml files in this repo are gpg signed, we need a 1-2 contributors who can take over this task in case I'm unavailable. Their gpg keys should be added to: https://www.haskell.org/ghcup/guide/#gpg-verification and https://www.haskell.org/ghcup/install/#manual-install

@bgamari @gbaz ...volunteers?

@tomjaguarpaw has had concerns about GHCup devs changing binaries. It's true, we may.

Users may want to opt out of that. We could trivially add a metadata file that only supplies vanilla upstream bindists. For GHC that will mean that 30% of them will be broken due to this bug: https://gitlab.haskell.org/ghc/ghc/-/issues/19646

But vanilla is vanilla, so users will have to deal with it.

• FreeBSD x86_64
• Alpine i386

This problem is tracked at haskell/cabal#9334 and was first encountered in #127

This bindist is statically linked and seems to cause issues for HLS compilation: haskell/haskell-language-server#2615 (comment)

Following https://gitlab.haskell.org/ghc/ghc/-/issues/23043 I have decided that GHCup will not distribute static alpine bindists anymore in the default release channel.

CI will be enhanced to catch that.

Instead, static alpine will only be provided through a separate release channel: #91

@mpickering @bgamari

Why

So that we don't have to change the source manually, and can always get the newest tools by just checking ghcup list.

Currently it's very easy to overlook or otherwise mess up the tags when adding a new tool version. CI should ideally perform some consistency checks to catch this failure mode:

• tags should be valid (matching a regular expression capturing known tags)
• there should be one Latest and one LatestPrerelease, and one Recommended version for each tool
• each ghc tool should have a base-... tag

@amesgen following the suggestion from https://discourse.haskell.org/t/hf-board-meeting-minutes-2022-10-06/5143/12

Any ideas how to make a nightly job that installs:

1. for all tools
2. for both recommended as well as latest tag?

• It has been released in github and hackage
• Some builds has failed in gitlab: https://gitlab.haskell.org/haskell/haskell-language-server/-/pipelines/44095

Apparently, this has caused issues with HLS, e.g. Bodigrim/mod#13

TODO:

• ghc-8.10.7
• ghc-8.10.6
• ghc-9.0.1
• PR