acme4j example is creating zero- length crt files about acme4j

[shred]

The master branch is currently WIP, as I am making major changes to the code in preparation for v3.0.0. However, integration tests against the Pebble test server are still working, so I don't think that there is a bug in acme4j.

If an empty certificate file is written, it is usually caused by problems with the used JDK/JRE or Java Security Provider. Please make sure that you are using acme4j v2.13 or higher. In v2.13 a bug was fixed that caused empty certificates on Android/Conscrypt.

Also, please tell me what JDK you are using (Oracle, Corretto, Temurin, etc)? Maybe I can reproduce it here.

You can enable debug logging and see if a certificate was actually downloaded. As soon as the certificate is downloaded, you should get a log line like:

[main] DEBUG org.shredzone.acme4j.Certificate - download

It is followed by the response headers of the server. They should give a content-type application/pem-certificate-chain and a reasonable content-length.

[shred]

Debugging the application, I am seeing that when I call the order.getCertificate() method, the cert I am getting looks like it has a lot of file URL information, but its "alternate" and "certChain" properties are null. I am unsure if this means anything but it is a little strange.

The alternate and certChain properties are loaded lazily, so it is normal that the initial value is null. As soon as you access the Certificate (via one of the getter methods, writeCertificate(), or download()), the certificate is downloaded from the CA, and the alternate and certChain properties are set accordingly.

For debugging, the simplest way would be to invoke download(), and then check the state of the instance.

[RobertBrown3]

Greetings:

Thanks for the swift responses.

Some things to tell you: (1) the JDK I am using is OpenJDK v17. (2) the version of acme4j I am using (utils and client) is 2.16.
(3) After reading your suggestions, I did call the download() method after successfully ordering the cert, then I used getCertificate() after doing the download. The alternates and certChain properties are still null.

[shred]

OpenJDK 17 should definitely work, I'm using it here as well.

alternates and certChain properties can only be null after the invocation of download() if it throws an exception. If there is also no exception, then something is definitely going very wrong there.

Can you enable debug logging and check the response headers of the download?

[RobertBrown3]

Greetings:

I have attached a text file showing the download and the headers that came in response. There are two downloads that occur, possibly due to me calling downlad() then calling getCertificate().

headers.txt

[shred]

The headers look allright so far. A certificate and alternate certificate links are provided by the CA.

However, download should not occur twice. If you invoke download(), it downloads the cert and keeps it in your local Certificate instance. If you invoke getCertificate() after that, it should use that local copy instead of downloading it again.

Your Certificate instance is unable to keep its internal state, for whatever reason. We already noticed that above, when you said that after invoking download(), the alternates and certChain properties are still null. This is just impossible if no exception is thrown:

    public void download() throws AcmeException {
        if (certChain == null) {
            LOG.debug("download");
            try (Connection conn = getSession().connect()) {
                conn.sendCertificateRequest(getLocation(), getLogin());
                alternates = new ArrayList<>(conn.getLinks("alternate"));
                certChain = new ArrayList<>(conn.readCertificates());
                // <-- here
            }
        }
    }

As you can see, in the happy path alternates and certChain will point to freshly created ArrayList instances. They cannot be null at the end of the try block. Can you set a breakpoint at the try block above, and confirm that alternates and certChain are still null at the point marked "here"?

How do you create your Certificate instances? Is it just via order.getCertificates(), or do you use serialization?

[RobertBrown3]

I found and solved the problem.

It wasn't with the download() method. I stepped into it and once it was done I saw the collection of X509 stuff that was placed in CertChain.

The reason why my getCertificate() method was returning certs with null CertChain even after the download() was called is that apparently calling another getCertificate() returns another cert that needs to download the CertChain and alternate again. Not that this matters, because the download apparently gets called as needed when I try to write the cert.

My problem was that when writing the cert, the way I was, I called the cert.writeCertificate() method in a traditional try- catch block that did not automatically close the FileWriter. The cert chain was being downloaded without problems, but it simply wasn't being saved. I had to add a flush() and close() call in order to actually get the contents of CertChain written to the file.

Embarrassing...

Anyway, I do have a (hopefully quick) question: I would like to separate the generated certs generated into different files (in the same way that the CertBot application does). When debugging acme4j, I see that despite the fact that the example puts everything into a single .crt file, the writeCertificate() code actually looks like it is saving everything in a PEM format.

My question is: is there a particular order that the different keys are stored in the CertChain object? I'd like to put the public key in a public.pem file, and the private key into a private.pem file. Is the collection of keys put in a consistent order? Or is there any way to separate out the different keys in order to write them into separate files?

[shred]

The reason why my getCertificate() method was returning certs with null CertChain even after the download() was called is that apparently calling another getCertificate() returns another cert that needs to download the CertChain and alternate again. Not that this matters, because the download apparently gets called as needed when I try to write the cert.

It's still not optimal. I should change getCertificate() to return the same instance again.

My problem was that when writing the cert, the way I was, I called the cert.writeCertificate() method in a traditional try- catch block that did not automatically close the FileWriter. The cert chain was being downloaded without problems, but it simply wasn't being saved. I had to add a flush() and close() call in order to actually get the contents of CertChain written to the file.

Hehe... We all have made that mistake once I guess. 😉 A flush() should not be necessary though. Just make sure the writer is properly closed.

My question is: is there a particular order that the different keys are stored in the CertChain object? I'd like to put the public key in a public.pem file, and the private key into a private.pem file. Is the collection of keys put in a consistent order? Or is there any way to separate out the different keys in order to write them into separate files?

writeCertificate() actually just invokes getCertificateChain(), and writes each part of the certificate as PEM block into the crt file. getCertificateChain() returns the certificates in a strict order. The first object is always "your" certificate, i.e. the certificate that belongs to your domain. The next in the list is the certificate of the CA that signed your certificate, then the certificate of the place that signed your CA, and so on.

However, this is only the certificate for your domain. It does not contain your private key.

In the example, the key pair of your domain is created and written in the loadOrCreateDomainKeyPair() method. Actually it only writes the private key, because the public key can be derived from it.

If you want to write your domain's public key only, you can use BouncyCastle:

KeyPair domainKeyPair = //... your domain's KeyPair
try (JcaPEMWriter jw = new JcaPEMWriter(new FileWriter("public-key.pem")) {
    jw.writeObject(domainKeyPair.getPublic());
}

But I will also add a utility method for that in the next version.

[RobertBrown3]

Based on what you have told me, it is clear that my understanding of certificates and keys is less than I originally thought it was.

One last favor I must ask: can you point me to somewhere where I can learn more about the different keys and certs that I would use when getting them using this library? For example: how does one get the public key from the private key? Or what the difference between the private cert (in the CRT file generated by the example) and the keypair created/found by the loadOrCreateDomainKeyPair() method? I would need to know which of these I would use for a site using HTTPS and how to do it.
I don't want to take up your time getting a lesson on these keys and certs. I would, however, be grateful if you could point me to any information you used to learn about them so that I can use them properly.

Thanks in advance. I will close this issue since effectively it has been resolved.

[shred]

Thank you for your feedback. I'm glad this issue could be resolved. 😄

how does one get the public key from the private key?

Just read the private key, e.g. with KeyPairUtils.readKeyPair(). It will return a KeyPair instance, which provides the public and private key part with getPublic() and getPrivate(), respectively.

Or what the difference between the private cert (in the CRT file generated by the example)

The CRT file contains a description of your domain order. It contains the domain names that you'd like to order a certificate for, and also your domain's public key. It is signed with your domain's private key. This is why you need it.

and the keypair created/found by the loadOrCreateDomainKeyPair() method?

This is the keypair that is later used by your web server for these domains. But the keypair is also used for signing the CRT, and the certificate will later contain the public key part of it.

I would need to know which of these I would use for a site using HTTPS and how to do it.

It depends on your web server. For nginx, all you need is the private key and the certificate chain that is written by writeCertificate(). The CRT file is only needed for the order, and maybe for certificate renewal (but can just be recreated if lost). The web servers have no use for it.

Maybe a good starting point would be the documentation of your web server. It explains what files it needs for setting up the encryption. Some web servers might need a separate private and public key PEM file, then you need to write the public key part like mentioned above. Some web servers might need the certificate and the issuer chain in different files, then you can use Certificate.getCertificateChain(), and write the resulting X509Certificate instances into different files. Some web servers might need another format than PEM, this is where things then start to become complicated.

So unfortunately there is no simple, general answer here.

Closing this issue because it has been resolved.