Out of memory - any way to optimize? about okhttp-eventsource

[jkodumal]

@rexxars Our main use case for this library does something very similar-- we use heartbeats sent every 3 minutes. We have not encountered this problem-- but we'd be happy to help investigate. Can you share the version of the library you're using, as well as a heap dump if you can produce one?

[rexxars]

I can't seem to reproduce the problem on the Android emulator, which is annoying. It seems to be garbage collecting memory just fine.

I'm using:

compile 'com.squareup.okhttp3:okhttp:3.8.1'
compile 'com.launchdarkly:okhttp-eventsource:1.4.1'

I'll try updating to okhttp-eventsource 1.5.0, though it shouldn't make any difference judging by the commits.

[jkodumal]

@rexxars keep us posted if you discover more. We have still not encountered this issue in our own usage.

[rexxars]

Out of interest, are you using this over HTTP or HTTPS? HTTP2?

I'm encountering this issue over HTTPS with HTTP2, not sure if that makes any difference.
Not too familiar with Java, but seeing a lot of these, which isn't stopping it from connecting:
09-01 02:58:58.783 3050-23749/codes.espen.digit D/NativeCrypto: returned from sslSelect() with result 1, error code 2

When it crashes, I'm seeing two different OutOfMemoryErrors. One being the one I mentioned in the original post, and the other is:

09-01 02:58:58.342 3050-3070/codes.espen.digit E/AndroidRuntime: FATAL EXCEPTION: Thread-11
 java.lang.OutOfMemoryError: (Heap Size=32775KB, Allocated=29814KB, Bitmap Size=0KB)
     at java.io.ByteArrayOutputStream.expand(ByteArrayOutputStream.java:91)
     at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:216)
     at org.bouncycastle.asn1.DEROutputStream.write(DEROutputStream.java:104)
     at org.bouncycastle.asn1.DEROutputStream.writeEncoded(DEROutputStream.java:50)
     at org.bouncycastle.asn1.DERSequence.encode(DERSequence.java:78)
     at org.bouncycastle.asn1.DEROutputStream.writeObject(DEROutputStream.java:127)
     at org.bouncycastle.asn1.ASN1Encodable.getEncoded(ASN1Encodable.java:48)
     at org.bouncycastle.jce.provider.X509CertificateObject.getEncoded(X509CertificateObject.java:531)
     at org.bouncycastle.jce.provider.X509CertificateObject.calculateHashCode(X509CertificateObject.java:585)
     at org.bouncycastle.jce.provider.X509CertificateObject.hashCode(X509CertificateObject.java:574)
     at java.util.HashMap.put(HashMap.java:394)
     at java.util.HashSet.add(HashSet.java:95)
     at okhttp3.internal.tls.TrustRootIndex$BasicTrustRootIndex.<init>(TrustRootIndex.java:115)
     at okhttp3.internal.tls.TrustRootIndex.get(TrustRootIndex.java:48)
     at okhttp3.internal.tls.TrustRootIndex.get(TrustRootIndex.java:43)
     at okhttp3.internal.platform.Platform.buildCertificateChainCleaner(Platform.java:167)
     at okhttp3.internal.platform.AndroidPlatform.buildCertificateChainCleaner(AndroidPlatform.java:173)
     at okhttp3.internal.tls.CertificateChainCleaner.get(CertificateChainCleaner.java:41)
     at okhttp3.OkHttpClient.<init>(OkHttpClient.java:251)
     at okhttp3.OkHttpClient.<init>(OkHttpClient.java:223)
     at com.launchdarkly.eventsource.EventSource$Builder.<init>(EventSource.java:307)
     at codes.espen.digit.MainActivity$SseRunnable.buildAndStart(MainActivity.java:309)
     at codes.espen.digit.MainActivity$SseRunnable.updateStateIfNecessary(MainActivity.java:332)
     at codes.espen.digit.MainActivity$SseRunnable.run(MainActivity.java:342)
     at java.lang.Thread.run(Thread.java:1027)

After the crashing, I'm seeing one of these:

09-01 02:59:03.067 3050-23989/codes.espen.digit W/OkHttp: A connection to https://<url>/ was leaked. Did you forget to close a response body?

It might be interesting to note that this doesn't happen for quite a while, we're talking hours of just receiving ping-replies and reconnecting every 30 minutes (server closes connection, client reconnects). I'm also curious if this could be related to the way the load balancer forcefully terminates the connection. The transfer encoding used is chunked, and it expects the stream to end with a zero-byte chunk. However, the load balancer is not aware of this fact and just forcefully closes after 30 minutes. I'm pondering if somehow this could cause connections to remain open or leak memory.

I'm going to try reproducing the issue with a real device against a local HTTP endpoint, and might also take the time to see if I can create a server replicating the load balancer cut-off behaviour.

[rexxars]

Sorry, this was all on my end. When the eventsource closed, I had written some faulty code that triggered a new EventSource to be instantiated for every disconnect. With a 30 minute timeout, this took quite a while to build up to a out of memory error, but locally with a 15 second timeout the error was much easier to spot.

Sorry about this, thanks for a great library!