Make package URLs match Hackage about cardano-haskell-packages

[michaelpj]

Can you elaborate on your use case? It works fine for Cabal.

[arrowd]

Many downstream packagers (including FreeBSD) are prefetching all required sources before the build to allow caching and to be able to disable network access during build. I implemented all the necessary machinery for multi-repository Cabal projects, but it turned out that URL patterns are different. I just wanted to make it easy for me, if it is cheap to implement on your side.

[michaelpj]

The only robust way to do that is to get cabal to do it for you, I think. I don't think we want to promise that we will always match the layout of Hackage if that's not a requirement on Cabal package repositories.

[michaelpj]

@andreabedini

[arrowd]

Interestingly, Stackage URLs also do not have that extra /foo-1.2.3/ part. I wonder if it is because both CHaP and Stackage do not support package revisions, only Hackage does. In this case, I think, nothing should be done on the CHaP side.

[andreabedini]

Interesting, thanks for bringing this up @arrowd. I originally based my code on what hackage-repo-tool does and never really dug into the details once I saw the code working. Looking at the code It turns out both schemes are kind of correct: the long one is a legacy one, the short one is the new. I don't know if there's a fallback between the two.

-- | Generate the URI of the tarball for a given package.
--
packageURI :: RemoteRepo -> PackageId -> URI
packageURI repo pkgid | isOldHackageURI (remoteRepoURI repo) =
  (remoteRepoURI repo) {
    uriPath = FilePath.Posix.joinPath
      [uriPath (remoteRepoURI repo)
      ,prettyShow (packageName    pkgid)
      ,prettyShow (packageVersion pkgid)
      ,prettyShow pkgid <.> "tar.gz"]
  }
packageURI repo pkgid =
  (remoteRepoURI repo) {
    uriPath = FilePath.Posix.joinPath
      [uriPath (remoteRepoURI repo)
      ,"package"
      ,prettyShow pkgid <.> "tar.gz"]
  }

[arrowd]

I wonder how revision URLs look like in the new scheme.

[andreabedini]

My understanding is that revisions are not fetched by url but are read off the index file (which contains all the cabal files of all the packages).

[arrowd]

Fetching by URL works for Hackage. I suppose the index file contains only the last revision?

[andreabedini]

I think I didn't explain myself well.

Hackage exposes a large api hackage.haskell.org/api but to build a project cabal only uses two bits:

1. the index file https://hackage.haskell.org/01-index.tar.gz
2. the package tarballs https://hackage.haskell.org/package/package-id.tar.gz (as above)

The index file is append-only and contains all the cabal files of all packages and their revisions (plus some other metadata like preferred versions and the hashes of the package tarballs).

The index file alone is enough for cabal to orchestrate a build, but it needs to download the package tarball to actually execute them. In particular cabal never downloads the revisions from the url since they are all already included in the index file (even older revisions).

[michaelpj]

I'm closing this with "working as reasonably intended".