airlift / airlift Goto Github PK

View Code? Open in Web Editor NEW

571.0 571.0 326.0 28.44 MB

Airlift framework for building REST services

License: Apache License 2.0

Java 98.18% HTML 0.95% Shell 0.36% Python 0.51%

airlift's People

Stargazers

Watchers

Forkers

martint dain electrum johngmyers andrewcox mattstep twilliamson hgschmie groys jaxlaw 11xor6 erichwang hexiaofeng ifedorenko davidhagar-zz rikima bonifaido imace rishidwivedi arjenroodselaar zoechalfant knowledgehacker nileema dhanji nagaseyasuhito liaochy kuguobing res-life sasakitoa zhenyuy-fb dparda akwirick gpreda haozhun krasnyanskiy simpleofac chakra-coder miniway wdong87 drudenko yujingwang kokosing jongyoul kyleburke danielting sangkyunyoon phamthaithinh hero1122 iloveyou416068 jlentz han-wang losipiuk billonahill gnomix arhimondr sderosiaux nezihyigitbasi raghavsethi kaschaeffer geraint0923 harunurhan future310 amrutagokhale kintondo hibernatexj y130 teradata mradamlacey poiu72 maniacs-sfa hiramj shimaomao kekenalog quantopian rmerizalde jxiang tonek shenguoquan rongrong coderlxl leesong-stack wubiaoi elonazoulay akyekth flakytestdetection matrixy hellium01 highker rash67 leusonmario spasam rynemccall unikrau cltforever yingsu00 hexopensource mway08 weijiezheng mbasmanova jthompson6

airlift's Issues

Http stats handler can throw exception

Stats measurement code can throw an IllegalArgumentException which kills the response. Exceptions from this code should be caught and suppressed. Also, the measurement code may be broken.

java.lang.IllegalArgumentException: value is negative
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) ~[guava-12.0.jar:na]
    at io.airlift.units.Duration.<init>(Duration.java:38) ~[units-0.67-SNAPSHOT.jar:na]
    at io.airlift.http.server.StatsRecordingHandler.log(StatsRecordingHandler.java:38) ~[http-server-0.67-SNAPSHOT.jar:na]
    at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:68) ~[jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) ~[jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) ~[jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:121) ~[jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47) ~[jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) ~[jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.Server.handle(Server.java:351) ~[jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) [jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890) [jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944) [jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) [jetty-http-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) [jetty-http-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76) [jetty-server-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609) [jetty-io-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45) [jetty-io-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) [jetty-util-8.1.1.v20120215.jar:8.1.1.v20120215]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) [jetty-util-8.1.1.v20120215.jar:8.1.1.v20120215]
    at java.lang.Thread.run(Thread.java:722) [na:1.7.0_07]

HttpServer does not decode gzip "Content-Encoding" requests

HttpServer properly encodes responses to gzip with the "Accept-Encoding" header, but does not decode requests with the gzip "Content-Encoding" header.

curl -v -X POST -T [local_file_name.gz] -H 'Content-Encoding: gzip'

Add jackson-datatype-jdk7 to JSON module

Does not compile

Just forked the project, ran mvn clean install and got the following

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
objc[85520]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
    at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
Caused by: java.lang.RuntimeException: Class java/util/UUID could not be instrumented.
    at org.jacoco.agent.rt.internal_5d10cad.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:138)
    at org.jacoco.agent.rt.internal_5d10cad.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:99)
    at org.jacoco.agent.rt.internal_5d10cad.PreMain.createRuntime(PreMain.java:51)
    at org.jacoco.agent.rt.internal_5d10cad.PreMain.premain(PreMain.java:43)
    ... 6 more
Caused by: java.lang.NoSuchFieldException: $jacocoAccess
    at java.lang.Class.getField(Class.java:1703)
    at org.jacoco.agent.rt.internal_5d10cad.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:136)
    ... 9 more
FATAL ERROR in native method: processing of -javaagent failed
/bin/sh: line 1: 85520 Abort trap: 6           /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/bin/java -javaagent:/Users/frenaud/.m2/repository/org/jacoco/org.jacoco.agent/0.6.2.201302030002/org.jacoco.agent-0.6.2.201302030002-runtime.jar=destfile=/Users/frenaud/projects/me/airline/target/jacoco.exec -jar /Users/frenaud/projects/me/airline/target/surefire/surefirebooter5947238246567954039.jar /Users/frenaud/projects/me/airline/target/surefire/surefire8523588895070006246tmp /Users/frenaud/projects/me/airline/target/surefire/surefire_04622422368411571876tmp

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.676 s
[INFO] Finished at: 2015-10-07T05:05:23-07:00
[INFO] Final Memory: 22M/383M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.14:test (default-test) on project airline: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.14:test failed: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ?
[ERROR] Command was/bin/sh -c cd /Users/frenaud/projects/me/airline && /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/bin/java -javaagent:/Users/frenaud/.m2/repository/org/jacoco/org.jacoco.agent/0.6.2.201302030002/org.jacoco.agent-0.6.2.201302030002-runtime.jar=destfile=/Users/frenaud/projects/me/airline/target/jacoco.exec -jar /Users/frenaud/projects/me/airline/target/surefire/surefirebooter5947238246567954039.jar /Users/frenaud/projects/me/airline/target/surefire/surefire8523588895070006246tmp /Users/frenaud/projects/me/airline/target/surefire/surefire_04622422368411571876tmp
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

My java:

java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Config validation errors for prefixed configs have an extra dot

For example, after making HttpClientConfig.connectTimeout default to null:

1) Error: Invalid configuration property discovery..http-client.connect-timeout: may not be null (for class io.airlift.http.client.HttpClientConfig.connectTimeout)

v0.99 io.airlift.configuration.testing.ConfigAssertions crash in Java 8

The exception is:
java.lang.IllegalArgumentException: null
at net.sf.cglib.asm.ClassReader.(Unknown Source)
at net.sf.cglib.asm.ClassReader.(Unknown Source)
at net.sf.cglib.asm.ClassReader.(Unknown Source)
at net.sf.cglib.proxy.BridgeMethodResolver.resolveAll(BridgeMethodResolver.java:61)
at net.sf.cglib.proxy.Enhancer.emitMethods(Enhancer.java:911)
at net.sf.cglib.proxy.Enhancer.generateClass(Enhancer.java:498)
at net.sf.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:216)
at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:377)
at net.sf.cglib.proxy.Enhancer.create(Enhancer.java:285)
at net.sf.cglib.proxy.Enhancer.create(Enhancer.java:679)
at io.airlift.configuration.testing.ConfigAssertions.recordDefaults(ConfigAssertions.java:276)
at com.facebook.di.smc.TestSmcUnionDataSourceConfig.testDefaults(TestSmcUnionDataSourceConfig.java:16)
which should related with an old version of cglib. However, cglib doesn't have an official version to support Java 8.
Someone provided a mitigation: https://jira.codehaus.org/browse/EASYMOCK-140
but not work for airlift as if I exclude the current version of cglib, there will be runtime error:
java.lang.NoClassDefFoundError: net/sf/cglib/proxy/Callback
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at bla.testDefaults(TestBlaConfig.java:15)

Presto UI broken as a result of replacing GzipFilter with GzipHandler

We've been using a SNAPSHOT version of Airlift internally because we need a Kerberos fix (932d490) that isn't part of any release. With this SNAPSHOT version we noticed that the Presto UI was "broken". See the attached screenshot for a concrete example on what I mean by "broken". Running queries doesn't change anything, it remains this way. We tried a couple of different browsers and a couple of different installations.

The Airlift 0.124 release does not break the UI. I tried three different versions of Airlift, each excluding a different commit since the 0.124 release and narrowed the culprit down to the last commit (218bbfe). I don't understand how that commit can break the UI, which is why I'm not putting a PR along with this. If it's a relatively simple fix, let me know and I'd be happy to provide a patch.

TestMoreFutures.testTimeout() failures due to race

The assertion

assertTrue(rootFuture.isDone());

intermittently fails because TimeoutFutureTask.run() completes timeoutFuture before canceling rootFuture. The assertion can run and fail before rootFuture is canceled.

@dain should it be a requirement that the rootFuture be canceled before the timeoutFuture is completed?

Incorrect dependency on "jackson-datatype-jdk7" version "2.7.4"

Looks like "io.airlift-json" and "io.airlift-http-client" artifacts looks for "jackson-datatype-jdk7" library when you give <dep.jackson.version>2.7.4</dep.jackson.version>.

jackson's 2.7.4 version does not contain "jackson-datatype-jdk7" library. I tried excluding this from the dependencies but it broke one of my unit test with this exception -

Caused by: java.lang.NoClassDefFoundError: com/fasterxml/jackson/datatype/jdk7/Jdk7Module
[ERROR] at io.airlift.json.ObjectMapperProvider.(ObjectMapperProvider.java:56)
[ERROR] at io.airlift.json.JsonCodec$1.get(JsonCodec.java:39)
[ERROR] at io.airlift.json.JsonCodec$1.get(JsonCodec.java:36)
[ERROR] at com.google.common.base.Suppliers$MemoizingSupplier.get(Suppliers.java:125)
[ERROR] at io.airlift.json.JsonCodec.listJsonCodec(JsonCodec.java:65)
[ERROR] at com.facebook.puma.router.TestServer.(TestServer.java:82)
[ERROR] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[ERROR] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[ERROR] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[ERROR] at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
[ERROR] at org.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:29)
[ERROR] at org.testng.internal.ClassHelper.createInstance1(ClassHelper.java:387)
[ERROR] ... 23 more
[ERROR] Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.datatype.jdk7.Jdk7Module
[ERROR] at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
[ERROR] at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
[ERROR] at java.security.AccessController.doPrivileged(Native Method)
[ERROR] at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
[ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
[ERROR] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
[ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

HttpUriBuilder does not support IPv6

The host is appended as-is, but for IPv6 it should be surrounded by brackets.

logging no longer compiles (or works) with logback > 1.0.11

[ERROR] /Users/hgschmie/scratch/airlift/log-manager/src/main/java/io/airlift/log/Logging.java:[27,34] cannot find symbol
symbol: class StatusChecker
location: package ch.qos.logback.core.status
[ERROR] /Users/hgschmie/scratch/airlift/log-manager/src/main/java/io/airlift/log/Logging.java:[267,9] cannot find symbol
symbol: class StatusChecker
location: class io.airlift.log.Logging
[ERROR] /Users/hgschmie/scratch/airlift/log-manager/src/main/java/io/airlift/log/Logging.java:[267,37] cannot find symbol
symbol: class StatusChecker
location: class io.airlift.log.Logging

Logger does not print Java 7 suppressed exceptions

TestBoundedExecutor.testExecutorCorruptionDetection() race

TestBoundedExecutor.testExecutorCorruptionDetection() fails at the first

            boundedExecutor.execute(() -> fail("Should not be run"));

due to a race condition.

The call to BoundedExecutor.execute() can increment BoundedExecutor.queueSize before the previous call's BoundedExecutor.drainQueue() invocation tests it in its do-while loop.

HTML form of /v1/jmx/mbean broken

The browser form of /v1/jmx/mbean is busted as of 4149c4e. The Firefox web console shows the error

[15:26:06.794] GET http://localhost:8080/v1/jmx/mbean?pretty&jsonp=jQuery152036713108270773465_1371939966720&_=1371939966739 [HTTP/1.1 500 Incompatible types: declared root type ([collection type; class java.util.List, contains [simple type, class io.airlift.jmx.MBeanRepresentation]]) vs com.fasterxml.jackson.databind.util.JSONPObject 170ms]

MinDuration/MaxDuration have unhelpful default message

The default message for MinDuration is {io.airlift.units.MinDuration.message}, which is not very informative. This appears to be a reference to a resource, but no such resource is defined anywhere.

airlift jmx-http requires using discovery

JmxHttpModule, line 74 reads

// todo move this to the jmx announcement when this module is promoted from experimental
DiscoveryBinder.discoveryBinder(binder).bindHttpAnnouncement("jmx-http");

but that did not happen. As a result, unless a service uses Discovery, jmx-http can not be used.

SOCKS unit test failing on Linux

On Linux, getting harness failures in the Socks code with Airlift trunk.

JDK 1.7u51, iptables not running, Both CentOS 5.10 and CentOS 6.5

Tests run: 180, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 26.49 sec <<< FAILURE!
testPostMethod(io.airlift.http.client.jetty.TestJettyHttpClientSocksProxy)  Time elapsed: 0.019 sec  <<< FAILURE
!
java.io.IOException: Invalid response from SOCKS4 proxy
        at org.eclipse.jetty.client.Socks4Proxy$Socks4ProxyConnection.onFillable(Socks4Proxy.java:158)
        at org.eclipse.jetty.io.AbstractConnection$1.run(AbstractConnection.java:505)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
        at java.lang.Thread.run(Thread.java:744)

TestAsyncJettyHttpClient failures in Travis CI

testConnectTimeout(io.airlift.http.client.jetty.TestAsyncJettyHttpClient)  Time elapsed: 2.073 sec  <<< FAILURE!
java.lang.AssertionError: unexpected exception: [java.util.concurrent.TimeoutException: Idle timeout 2000 ms
    at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onIdleExpired(HttpConnectionOverHTTP.java:104)
    at org.eclipse.jetty.io.AbstractEndPoint.onIdleExpired(AbstractEndPoint.java:162)
    at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
    at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
]
    at org.testng.Assert.fail(Assert.java:94)
    at io.airlift.http.client.AbstractHttpClientTest.testConnectTimeout(AbstractHttpClientTest.java:225)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
    at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54)
    at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)


testConnectionRefused(io.airlift.http.client.jetty.TestAsyncJettyHttpClient)  Time elapsed: 0.464 sec  <<< FAILURE!
java.net.SocketTimeoutException: Connect Timeout
    at org.eclipse.jetty.io.ManagedSelector$ConnectTimeout.run(ManagedSelector.java:673)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)


testConnectTimeout(io.airlift.http.client.jetty.TestJettyHttpsClient)  Time elapsed: 2.023 sec  <<< FAILURE!
java.lang.AssertionError: unexpected exception: [java.util.concurrent.TimeoutException
    at org.eclipse.jetty.client.util.InputStreamResponseListener.get(InputStreamResponseListener.java:220)
    at io.airlift.http.client.jetty.JettyHttpClient.execute(JettyHttpClient.java:345)
    at io.airlift.http.client.jetty.TestJettyHttpsClient.executeRequest(TestJettyHttpsClient.java:79)
    at io.airlift.http.client.AbstractHttpClientTest.testConnectTimeout(AbstractHttpClientTest.java:219)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
    at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54)
    at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
]
    at org.testng.Assert.fail(Assert.java:94)
    at io.airlift.http.client.AbstractHttpClientTest.testConnectTimeout(AbstractHttpClientTest.java:225)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
    at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54)
    at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Update dense HLL to guarantee a min cardinality when derived from sparse HLL

In the current HLL implementation, it is possible to report a "drop" in cardinality even though more values were added. This occurs in boundary when a sparse HLL gets converted to a dense HLL. This is very unintuitive for reporting use cases.

To correct for this oddity, in the conversion between sparse to dense, we should record the sparse cardinality and make sure the dense form never reports a cardinality lower than the sparse cardinality.

Servlet Filters not applied to static resources

The comment in the HttpServer constructor states that resource handlers appear after all built-in filters, user provided filters, and the servlet. The code places the resource handlers first, before all those. Thus, the various filters do not get an opportunity to modify the behavior of static resources.

What was the intent?

Empty pid file makes launcher think process is running

# ls -l data/var/run/launcher.pid 
-rw-rw-rw- 1 root root 0 Mar  6 16:37 data/var/run/launcher.pid
# installation/bin/launcher status
Running as 0

JmxAgent not properly handling alternate JVMs

java.lang.NoSuchFieldException: jmxServer
        at java.lang.Class.getDeclaredField(Unknown Source) ~[na:1.7.0]
        at io.airlift.jmx.JmxAgent.getField(JmxAgent.java:146) [jmx-0.84.jar:0.84]
        at io.airlift.jmx.JmxAgent.getRunningAgentAddress(JmxAgent.java:108) [jmx-0.84.jar:0.84]
        at io.airlift.jmx.JmxAgent.<init>(JmxAgent.java:49) [jmx-0.84.jar:0.84]
        at io.airlift.jmx.JmxAgent$$FastClassByGuice$$553f0c8a.newInstance(<generated>) [guice-3.0.jar:0.84]

Duration does not handle locales correctly

Duration will not round trip in locals that use a comma for the decimal separator

Airlift HyperLogLog cardinality estimates over the same data can have large variance

Given an input data set of about 5 million unique numbers, if you randomly shuffle the values and add them in chunks of 24k to different hyperloglogs and then merge them together, the cardinality estimate can vary by as much as 0.1% to 0.3%. This variability gets much worse the more buckets you use in the hyperloglogs.

Two observations:

Forcing all hyperloglogs to start off as dense does not have this problem.
Using a single hyperloglog over this whole data set does not have this problem (no merging).

Test program:
http://pastebin.com/zm5ZHswK

Contribute overridden Jetty classes back to jetty project

I've just noticed the airlift project overrides several jetty classes - like schedulers and threadpool, to do such stuff like support thread groups.

The Jetty project would be more than pleased to update our classes to support things like thread groups. So when you need stuff like that, please ask on https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Jetty and we shall probably deliver. Better yet, just give us a pull request against https://github.com/eclipse/jetty.project

cheers

Update TestingHttpClient to use Function from JDK

Netty 3.6 is a direct memory hog

See bug report on Netty here: netty/netty#478

When using the AsyncHttpClient to send large messages, the above issue (or something related to it) results in a gargantuan bloat of direct memory (I've seen numbers as high as 1.7GB).

Query about wiki and contribution guide

Where I can learn more about this framework and how to use this?
Any contribution guide if I want to make any contribution to this?
May be if someone can help me understand this, I can work on creating wiki for this.

RequestStats in JettyHttpClient are very expensive

RequestStats.record() accounts for a very large amount (likely even the majority) of the CPU time when using a JettyHttpClient. This appears to be due to the TimeStat being very expensive to update.

Disable more features in ObjectMapperProvider

New features to disable:

INFER_PROPERTY_MUTATORS
ALLOW_FINAL_FIELDS_AS_MUTATORS

Support registering a custom LogHandlers

In Presto we need to register a customer LogHandler that sends our logs to our log collection system. For example, in the config maybe we could do this:

handlers=io.airlift.log.RollingFileHandler,my.other.custom.LogForwardingHandler

The handlers are currently hardcoded to only RollingFileHandler and the handlers and the Logging class are buried below Bootstrap, which makes adding a custom handler not possible.

If the config option is not desired, another way to do this in the code would also work. For example if Logger exposed a protected method to add a handler, then one might be able to subclass it and override logToFile to log additional handlers. That would require telling Bootstrap that initializeLogging=false and taking on initialization of the subclass into our own hands. This starts to get messy though so it's not ideal. There might be another option using Guice that could work.

I could take on this ticket if we reach agreement around what approach is ideal.

TestingHttpClient's processor should be able to throw checked exceptions

...such as SocketTimeoutException and the like.

jetty-9.3.13.M0 not stable ?

Hello,

Using Facebook PrestoDB, which uses airlift, I see in log:

2017-03-24T20:21:31.146+0100	INFO	main	org.eclipse.jetty.server.Server	jetty-9.3.13.M0
2017-03-24T20:21:31.146+0100	WARN	main	org.eclipse.jetty.server.Server	THIS IS NOT A STABLE RELEASE! DO NOT USE IN PRODUCTION!
2017-03-24T20:21:31.146+0100	WARN	main	org.eclipse.jetty.server.Server	Download a stable release from http://download.eclipse.org/jetty/

Do you think this could be a problem?

NettyIoPools are not closed

NettyIoPoolProvider needs to track and destroy the pools using @PreDestroy, just like HttpClientProvider does.

BodyGeneratorContentProvider adding to full BlockingQueue

Seeing the following, which suggests that the BodyGeneratorContentProvider exception handling isn't quite right.

2014-12-11T18:42:21.485+0000     WARN   http-client-shared-228  org.eclipse.jetty.util.thread.QueuedThreadPool          
java.lang.IllegalStateException: Queue full
        at java.util.AbstractQueue.add(AbstractQueue.java:98) ~[na:1.7.0_55]
        at java.util.concurrent.ArrayBlockingQueue.add(ArrayBlockingQueue.java:283) ~[na:1.7.0_55]
        at com.proofpoint.http.client.jetty.JettyHttpClient$BodyGeneratorContentProvider$1.run(JettyHttpClient.java:580) ~[http-client-1.02.jar:na]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) ~[jetty-util-9.1.4.v20140401.jar:9.1.4.v20140401]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) ~[jetty-util-9.1.4.v20140401.jar:9.1.4.v20140401]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]

Config system should not allow multiple instances of same property

This should not be allowed in a config file:

foo=abc
foo=xyz

Rename update method in CounterStat

The method update() would be better named add() both to match its usage and to match the other stats objects. It should be deprecated and a new add() method added.

HTTP client test failures

io.airlift.http.client.jetty.TestJettyHttpClient
testDeleteMethod
java.io.UncheckedIOException: java.io.EOFException: HttpConnectionOverHTTP@1c830ad8(l:/127.0.0.1:54358 <-> r:/127.0.0.1:54356,closed=false)[HttpChannelOverHTTP@18bc9d90(exchange=HttpExchange@5dbe00e3 req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@68781e42(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator@6c291c5f{s=START}],recv=HttpReceiverOverHTTP@10055c57(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of 0}]]]
	at io.airlift.http.client.ResponseHandlerUtils.propagate(ResponseHandlerUtils.java:21)
	at io.airlift.http.client.StatusResponseHandler.handleException(StatusResponseHandler.java:44)
	at io.airlift.http.client.StatusResponseHandler.handleException(StatusResponseHandler.java:28)
	at io.airlift.http.client.jetty.JettyHttpClient.execute(JettyHttpClient.java:365)
	at io.airlift.http.client.jetty.TestJettyHttpClient.executeRequest(TestJettyHttpClient.java:47)
	at io.airlift.http.client.AbstractHttpClientTest.testDeleteMethod(AbstractHttpClientTest.java:327)
Caused by: java.io.EOFException: HttpConnectionOverHTTP@1c830ad8(l:/127.0.0.1:54358 <-> r:/127.0.0.1:54356,closed=false)[HttpChannelOverHTTP@18bc9d90(exchange=HttpExchange@5dbe00e3 req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@68781e42(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator@6c291c5f{s=START}],recv=HttpReceiverOverHTTP@10055c57(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of 0}]]]
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:293)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1377)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:185)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:132)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:70)
	at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:90)
	at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:115)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
	at java.lang.Thread.run(Thread.java:745)
... Removed 28 stack frames

io.airlift.http.client.jetty.TestAsyncJettyHttpClientHttp2
testHandlesUndeclaredThrowable
java.lang.UnsupportedOperationException: not yet implemented
	at io.airlift.http.client.AbstractHttpClientTest$ThrowErrorResponseHandler.handleException(AbstractHttpClientTest.java:976)
	at io.airlift.http.client.AbstractHttpClientTest$ThrowErrorResponseHandler.handleException(AbstractHttpClientTest.java:970)
	at io.airlift.http.client.jetty.JettyHttpClient$JettyResponseFuture.failed(JettyHttpClient.java:875)
	at io.airlift.http.client.jetty.JettyHttpClient$BufferingResponseListener.onComplete(JettyHttpClient.java:1113)
	at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
	at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:185)
	at org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:457)
	at org.eclipse.jetty.client.HttpReceiver.abort(HttpReceiver.java:528)
	at org.eclipse.jetty.client.HttpChannel.abortResponse(HttpChannel.java:129)
	at org.eclipse.jetty.client.HttpChannel.abort(HttpChannel.java:122)
	at org.eclipse.jetty.http2.client.http.HttpChannelOverHTTP2.abort(HttpChannelOverHTTP2.java:99)
	at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:257)
	at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:141)
	at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:708)
	at org.eclipse.jetty.http2.client.http.HttpConnectionOverHTTP2.abort(HttpConnectionOverHTTP2.java:117)
	at org.eclipse.jetty.http2.client.http.HttpConnectionOverHTTP2.close(HttpConnectionOverHTTP2.java:99)
	at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2$SessionListenerPromise.onFailure(HttpClientTransportOverHTTP2.java:230)
	at org.eclipse.jetty.http2.HTTP2Session.notifyFailure(HTTP2Session.java:1086)
	at org.eclipse.jetty.http2.HTTP2Session.abort(HTTP2Session.java:994)
	at org.eclipse.jetty.http2.HTTP2Session.onShutdown(HTTP2Session.java:867)
	at org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:224)
	at org.eclipse.jetty.util.thread.strategy.ProduceConsume.execute(ProduceConsume.java:69)
	at org.eclipse.jetty.http2.HTTP2Connection.onOpen(HTTP2Connection.java:94)
	at org.eclipse.jetty.http2.client.HTTP2ClientConnectionFactory$HTTP2ClientConnection.onOpen(HTTP2ClientConnectionFactory.java:164)
	at org.eclipse.jetty.io.SelectorManager.connectionOpened(SelectorManager.java:324)
	at org.eclipse.jetty.io.ManagedSelector.createEndPoint(ManagedSelector.java:425)
	at org.eclipse.jetty.io.ManagedSelector.access$1600(ManagedSelector.java:56)
	at org.eclipse.jetty.io.ManagedSelector$CreateEndPoint.run(ManagedSelector.java:605)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedChannelException
	... 15 more

Improve heuristic for detecting if JMX agent is already initialized

2015-05-10T19:04:59.006-0700    WARN    main    io.airlift.jmx.JmxAgent Cannot determine if JMX agent is already running (not an Oracle JVM?). Will try to start it manually.

Archetype doesn't use correct packages.

The archetypes use the specified project package for airlift imports instead of io.airlfit

As a very small example if my project package was 'foo.bar' then the imports from the generated archetype would look like this:
import foo.bar.bootstrap.Bootstrap;
instead of:
import io.airlift.bootstrap.Bootstrap;

Insufficient threads in TestingHttpServerModule

TestingHttpServerModule does not work on 32 core machines:

java.lang.IllegalStateException: Insufficient max threads in ThreadPool: max=32 < needed=49

airlift / airlift Goto Github PK

airlift's People

Stargazers

Watchers

Forkers

airlift's Issues

Recommend Projects

Recommend Topics

Recommend Org