Coder Social home page Coder Social logo

playframework / play-ws Goto Github PK

View Code? Open in Web Editor NEW
220.0 15.0 87.0 2.06 MB

Standalone Play WS, an async HTTP client with fluent API

Home Page: https://www.playframework.com/documentation/latest/JavaWS

License: Apache License 2.0

Scala 69.93% Java 30.07%
asynchttpclient scala http-client java asynchronous reactive reactive-streams playframework

play-ws's Introduction

Play WS Standalone

Twitter Follow Discord GitHub Discussions StackOverflow YouTube Twitch Status OpenCollective

Build Status Maven Javadocs Repository size Scala Steward badge Mergify Status

Play WS is a powerful HTTP Client library, originally developed by the Play team for use with Play Framework. It uses AsyncHttpClient for HTTP client functionality and has no Play dependencies.

We've provided some documentation here on how to use Play WS in your app (without Play). For more information on how to use Play WS in Play, please refer to the Play documentation.

Getting Started

To get started, you can add play-ahc-ws-standalone as a dependency in SBT:

libraryDependencies += "org.playframework" %% "play-ahc-ws-standalone" % "LATEST_VERSION"

// Before version 3.0.0:
libraryDependencies += "com.typesafe.play" %% "play-ahc-ws-standalone" % "LATEST_VERSION"

Where you replace LATEST_VERSION with the version shown in this image: Latest released version.

This adds the standalone version of Play WS, backed by AsyncHttpClient. This library contains both the Scala and Java APIs, under play.api.libs.ws and play.libs.ws.

To add XML and JSON support using Play-JSON or Scala XML, add the following:

libraryDependencies += "org.playframework" %% "play-ws-standalone-xml" % playWsStandaloneVersion
libraryDependencies += "org.playframework" %% "play-ws-standalone-json" % playWsStandaloneVersion

// Before version 3.0.0:
libraryDependencies += "com.typesafe.play" %% "play-ws-standalone-xml" % playWsStandaloneVersion
libraryDependencies += "com.typesafe.play" %% "play-ws-standalone-json" % playWsStandaloneVersion

Shading

Play WS uses shaded versions of AsyncHttpClient and OAuth Signpost, repackaged under the play.shaded.ahc and play.shaded.oauth package names, respectively. Shading AsyncHttpClient means that the version of Netty used behind AsyncHttpClient is completely independent of the application and Play as a whole.

Specifically, shading AsyncHttpClient means that there are no version conflicts introduced between Netty 4.0 and Netty 4.1 using Play WS.

NOTE: If you are developing play-ws and publishing shaded-asynchttpclient and shaded-oauth using sbt publishLocal, you need to be aware that updating ~/.ivy2/local does not overwrite ~/.ivy2/cache and so you will not see your updated shaded code until you remove it from cache. See http://eed3si9n.com/field-test for more details. This bug has been filed as sbt/sbt#2687.

Shaded AHC Defaults

Because Play WS shades AsyncHttpClient, the default settings are also shaded and so do not adhere to the AHC documentation. This means that the settings in ahc-default.properties and the AsyncHttpClient system properties are prepended with play.shaded.ahc, for example the usePooledMemory setting in the shaded version of AsyncHttpClient is defined like this:

play.shaded.ahc.org.asynchttpclient.usePooledMemory=true

Typed Bodies

The type system in Play-WS has changed so that the request body and the response body can use richer types.

You can define your own BodyWritable or BodyReadable, but if you want to use the default out of the box settings, you can import the type mappings with the DefaultBodyReadables / DefaultBodyWritables.

Scala

import play.api.libs.ws.DefaultBodyReadables._
import play.api.libs.ws.DefaultBodyWritables._

More likely you will want the XML and JSON support:

import play.api.libs.ws.XMLBodyReadables._
import play.api.libs.ws.XMLBodyWritables._

or

import play.api.libs.ws.JsonBodyReadables._
import play.api.libs.ws.JsonBodyWritables._

To use a BodyReadable in a response, you must type the response explicitly:

import scala.concurrent.{ ExecutionContext, Future }

import play.api.libs.ws.StandaloneWSClient
import play.api.libs.ws.XMLBodyReadables._ // required

def handleXml(ws: StandaloneWSClient)(
  implicit ec: ExecutionContext): Future[scala.xml.Elem] =
  ws.url("...").get().map { response =>
    response.body[scala.xml.Elem]
  }

or using Play-JSON:

import scala.concurrent.{ ExecutionContext, Future }

import play.api.libs.json.JsValue
import play.api.libs.ws.StandaloneWSClient

import play.api.libs.ws.JsonBodyReadables._ // required

def handleJsonResp(ws: StandaloneWSClient)(
  implicit ec: ExecutionContext): Future[JsValue] =
  ws.url("...").get().map { response =>
    response.body[JsValue]
  }

Note that there is a special case: when you are streaming the response, then you should get the body as a Source:

import scala.concurrent.ExecutionContext
import org.apache.pekko.util.ByteString
import org.apache.pekko.stream.scaladsl.Source
import play.api.libs.ws.StandaloneWSClient

def useWSStream(ws: StandaloneWSClient)(implicit ec: ExecutionContext) =
  ws.url("...").stream().map { response =>
     val source: Source[ByteString, _] = response.bodyAsSource
     val _ = source // do something with source
  }

To POST, you should pass in a type which has an implicit class mapping of BodyWritable:

import scala.concurrent.ExecutionContext
import play.api.libs.ws.DefaultBodyWritables._ // required

def postExampleString(ws: play.api.libs.ws.StandaloneWSClient)(
  implicit ec: ExecutionContext) = {
  val stringData = "Hello world"
  ws.url("...").post(stringData).map { response => /* do something */ }
}

You can also define your own custom BodyReadable:

import play.api.libs.ws.BodyReadable
import play.api.libs.ws.ahc.StandaloneAhcWSResponse

case class Foo(body: String)

implicit val fooBodyReadable = BodyReadable[Foo] { response =>
  import play.shaded.ahc.org.asynchttpclient.{ Response => AHCResponse }
  val ahcResponse = response.asInstanceOf[StandaloneAhcWSResponse].underlying[AHCResponse]
  Foo(ahcResponse.getResponseBody)
}

or custom BodyWritable:

import org.apache.pekko.util.ByteString
import play.api.libs.ws.{ BodyWritable, InMemoryBody }

implicit val writeableOf_Foo: BodyWritable[Foo] = {
  // https://tools.ietf.org/html/rfc6838#section-3.2
  BodyWritable(foo => InMemoryBody(ByteString.fromString(foo.toString)), "application/vnd.company.category+foo")
}

Java

To use the default type mappings in Java, you should use the following:

import play.libs.ws.DefaultBodyReadables;
import play.libs.ws.DefaultBodyWritables;

followed by:

public class MyClient implements DefaultBodyWritables, DefaultBodyReadables {    
    public CompletionStage<String> doStuff() {
      return client.url("http://example.com").post(body("hello world")).thenApply(response ->
        response.body(string())
      );
    }
}

Note that there is a special case: when you are using a stream, then you should get the body as a Source:

class MyClass {
    public CompletionStage<Source<ByteString, NotUsed>> readResponseAsStream() {
        return ws.url(url).stream().thenApply(response ->
            response.bodyAsSource()
        );
    }
}

You can also post a Source:

class MyClass {
    public CompletionStage<String> doStuff() {
        Source<ByteString, NotUsed> source = fromSource();
        return ws.url(url).post(body(source)).thenApply(response ->
            response.body()
        );
    }
}

You can define a custom BodyReadable:

import play.libs.ws.ahc.*;
import play.shaded.ahc.org.asynchttpclient.Response;

class FooReadable implements BodyReadable<StandaloneWSResponse, Foo> {
    public Foo apply(StandaloneWSResponse response) {
        Response ahcResponse = (Response) response.getUnderlying();
        return Foo.serialize(ahcResponse.getResponseBody(StandardCharsets.UTF_8));
    }
}

You can also define your own custom BodyWritable:

public class MyClient {
    private BodyWritable<String> someOtherMethod(String string) {
        org.apache.pekko.util.ByteString byteString = org.apache.pekko.util.ByteString.fromString(string);
      return new DefaultBodyWritables.InMemoryBodyWritable(byteString, "text/plain");
    }
}

Instantiating a standalone client

The standalone client needs Pekko to handle streaming data internally:

Scala

In Scala, the way to call out to a web service and close down the client:

package playwsclient

import org.apache.pekko.actor.ActorSystem
import org.apache.pekko.stream._
import play.api.libs.ws._
import play.api.libs.ws.ahc._

import scala.concurrent.Future

object ScalaClient {
  import DefaultBodyReadables._
  import scala.concurrent.ExecutionContext.Implicits._

  def main(args: Array[String]): Unit = {
    // Create Pekko system for thread and streaming management
    implicit val system = ActorSystem()
    system.registerOnTermination {
      System.exit(0)
    }

    implicit val materializer = SystemMaterializer(system).materializer

    // Create the standalone WS client
    // no argument defaults to a AhcWSClientConfig created from
    // "AhcWSClientConfigFactory.forConfig(ConfigFactory.load, this.getClass.getClassLoader)"
    val wsClient = StandaloneAhcWSClient()

    call(wsClient)
      .andThen { case _ => wsClient.close() }
      .andThen { case _ => system.terminate() }
  }

  def call(wsClient: StandaloneWSClient): Future[Unit] = {
    wsClient.url("http://www.google.com").get().map { response =>
      val statusText: String = response.statusText
      val body = response.body[String]
      println(s"Got a response $statusText: $body")
    }
  }
}

You can also create the standalone client directly from an AsyncHttpClient instance:

object ScalaClient {
  def main(args: Array[String]): Unit = {
    // Use 
    import play.shaded.ahc.org.asynchttpclient._
    val asyncHttpClientConfig = new DefaultAsyncHttpClientConfig.Builder()
      .setMaxRequestRetry(0)
      .setShutdownQuietPeriod(0)
      .setShutdownTimeout(0).build
    val asyncHttpClient = new DefaultAsyncHttpClient(asyncHttpClientConfig)
    val wsClient = new StandaloneAhcWSClient(asyncHttpClient)
    /// ...
  }
}

This is useful when there is an AsyncHttpClient configuration option that is not available in the WS config layer.

Java

In Java the API is much the same:

package playwsclient;

import org.apache.pekko.actor.ActorSystem;
import org.apache.pekko.stream.*;
import com.typesafe.config.ConfigFactory;

import play.libs.ws.*;
import play.libs.ws.ahc.*;

public class JavaClient implements DefaultBodyReadables {
    private final StandaloneAhcWSClient client;
    private final ActorSystem system;

    public static void main(String[] args) {
        // Set up Pekko materializer to handle streaming
        final String name = "wsclient";
        ActorSystem system = ActorSystem.create(name);
        system.registerOnTermination(() -> System.exit(0));
        Materializer materializer = SystemMaterializer.get(system).materializer();

        // Create the WS client from the `application.conf` file, the current classloader and materializer.
        StandaloneAhcWSClient ws = StandaloneAhcWSClient.create(
                AhcWSClientConfigFactory.forConfig(ConfigFactory.load(), system.getClass().getClassLoader()),
                materializer
        );

        JavaClient javaClient = new JavaClient(system, ws);
        javaClient.run();
    }

    JavaClient(ActorSystem system, StandaloneAhcWSClient client) {
        this.system = system;
        this.client = client;
    }

    public void run() {
        client.url("http://www.google.com").get()
                .whenComplete((response, throwable) -> {
                    String statusText = response.getStatusText();
                    String body = response.getBody(string());
                    System.out.println("Got a response " + statusText);
                })
                .thenRun(() -> {
                    try {
                        client.close();
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                })
                .thenRun(system::terminate);
    }
}

Likewise, you can provide the AsyncHttpClient client explicitly from configuration:

public class JavaClient implements DefaultBodyReadables {
     public static void main(String[] args) { 
        // ...
        // Set up AsyncHttpClient directly from config
        AsyncHttpClientConfig asyncHttpClientConfig =
            new DefaultAsyncHttpClientConfig.Builder()
                .setMaxRequestRetry(0)
                .setShutdownQuietPeriod(0)
                .setShutdownTimeout(0)
                .build();
        AsyncHttpClient asyncHttpClient = new DefaultAsyncHttpClient(asyncHttpClientConfig);
    
        // Set up WSClient instance directly from asynchttpclient.
        WSClient client = new AhcWSClient(asyncHttpClient, materializer);
        // ...
    }
}

Caching

Play WS implements HTTP Caching through CachingAsyncHttpClient, AhcHTTPCache and CacheControl, a minimal HTTP cache management library in Scala.

To create a standalone AHC client that uses caching, pass in an instance of AhcHttpCache with a cache adapter to the underlying implementation. For example, to use Caffeine as the underlying cache, you could use the following:

import scala.concurrent.Future
import java.util.concurrent.TimeUnit
import com.github.benmanes.caffeine.cache.{ Caffeine, Ticker }

import play.api.libs.ws.ahc.StandaloneAhcWSClient
import play.api.libs.ws.ahc.cache.{
  AhcHttpCache, Cache, EffectiveURIKey, ResponseEntry
}

class CaffeineHttpCache extends Cache {
  val underlying = Caffeine.newBuilder()
    .ticker(Ticker.systemTicker())
    .expireAfterWrite(365, TimeUnit.DAYS)
    .build[EffectiveURIKey, ResponseEntry]()

  def remove(key: EffectiveURIKey) =
    Future.successful(Option(underlying.invalidate(key)))

  def put(key: EffectiveURIKey, entry: ResponseEntry) =
    Future.successful(underlying.put(key, entry))

  def get(key: EffectiveURIKey) =
    Future.successful(Option(underlying getIfPresent key ))

  def close(): Unit = underlying.cleanUp()
}

def withCache(implicit m: org.apache.pekko.stream.Materializer): StandaloneAhcWSClient = {
  implicit def ec = m.executionContext

  val cache = new CaffeineHttpCache()
  StandaloneAhcWSClient(httpCache = Some(new AhcHttpCache(cache)))
}

There are a number of guides that help with putting together Cache-Control headers:

Releasing a new version

See https://github.com/playframework/.github/blob/main/RELEASING.md

License

Play WS is licensed under the Apache license, version 2. See the LICENSE file for more information.

play-ws's People

Contributors

benoit-ponsero avatar billyautrey avatar bomgar avatar cchantep avatar dependabot[bot] avatar dwijnand avatar eed3si9n avatar ennru avatar gmethvin avatar gurkankaymak avatar htmldoug avatar ignasi35 avatar ihostage avatar marcospereira avatar maxcellent avatar mergify[bot] avatar mkurz avatar octonato avatar ophirr33 avatar raboof avatar richdougherty avatar scala-steward avatar schmitch avatar sethtisue avatar shasts avatar sovaalexandr avatar sullis avatar suya55 avatar wsargent avatar xuwei-k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

play-ws's Issues

Add typesafe config parser

The WSConfigParser from Play WS takes a configuration and environment. It should take a com.typesafe.config.Config here and have a version that reads from ConfigFactory / application.conf

Provide API for adding cookies

Currently, the WSRequest API has

def addCookies(cookies: WSCookie*):

This means that you can carry cookies through from a response to another request, but there is no way to instaniate a cookie, since AHCWSCookie is a private class and WSCookie is a pure trait with no implementation. So add a case class implementation, and tests to ensure that this is available. There should also be a clear cookie builder for Java and Scala APIs.

Persistent caching of HTTP responses fails due to presence of non-serializable classes

Play WS Version (2.5.x / etc)

1.0.0-M10

API (Scala / Java / Neither / Both)

Scala

Operating System (Ubuntu 15.10 / MacOS 10.10 / Windows 10)

Arch Linux (4.10.11-1-ARCH)

JDK (Oracle 1.8.0_72, OpenJDK 1.8.x, Azul Zing)

java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

Library Dependencies

"org.ehcache" % "ehcache" % "3.3.1"

Expected Behavior

Please describe the expected behavior of the issue, starting from the first action.

  1. send an HTTP request
  2. get an HTTP response
  3. cache the response to disk

Actual Behavior

  1. works fine
  2. works fine
  3. java.io.NotSerializableException is thrown:
[] INFO   - Ehcache key GET http://localhost:8181/greeting?name=Foo recovered from (o.e.c.i.r.LoggingRobustResilienceStrategy)
java.io.NotSerializableException: play.api.libs.ws.ahc.cache.CacheableHttpResponseBodyPart
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
        at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
        at scala.collection.immutable.List$SerializationProxy.writeObject(List.scala:476)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
        at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
        at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
        at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
        at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
        at org.ehcache.impl.serialization.CompactJavaSerializer.serialize(CompactJavaSerializer.java:94)
        ... 54 common frames omitted
Wrapped by: org.ehcache.spi.serialization.SerializerException: java.io.NotSerializableException: play.api.libs.ws.ahc.cache.CacheableHttpResponseBodyPart
        at org.ehcache.impl.serialization.CompactJavaSerializer.serialize(CompactJavaSerializer.java:100)
        at org.ehcache.impl.internal.store.offheap.portability.OffHeapValueHolderPortability.encode(OffHeapValueHolderPortability.java:52)
        at org.ehcache.impl.internal.store.offheap.portability.OffHeapValueHolderPortability.encode(OffHeapValueHolderPortability.java:31)
        at org.terracotta.offheapstore.storage.PortabilityBasedStorageEngine.writeMapping(PortabilityBasedStorageEngine.java:64)
        at org.terracotta.offheapstore.OffHeapHashMap.tryWriteEntry(OffHeapHashMap.java:703)
        at org.terracotta.offheapstore.OffHeapHashMap.writeEntry(OffHeapHashMap.java:687)
        at org.terracotta.offheapstore.OffHeapHashMap.computeWithMetadata(OffHeapHashMap.java:1947)
        at org.terracotta.offheapstore.AbstractLockedOffHeapHashMap.computeWithMetadata(AbstractLockedOffHeapHashMap.java:582)
        at org.terracotta.offheapstore.concurrent.AbstractConcurrentOffHeapMap.computeWithMetadata(AbstractConcurrentOffHeapMap.java:729)
        at org.ehcache.impl.internal.store.offheap.EhcacheConcurrentOffHeapClockCache.compute(EhcacheConcurrentOffHeapClockCache.java:152)
        at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.computeWithRetry(AbstractOffHeapStore.java:1094)
        ... 44 common frames omitted
Wrapped by: org.ehcache.core.spi.store.StoreAccessException: org.ehcache.spi.serialization.SerializerException: java.io.NotSerializableException: play.api.libs.ws.ahc.cache.CacheableHttpResponseBodyPart
        at org.ehcache.core.exceptions.StorePassThroughException.handleRuntimeException(StorePassThroughException.java:70)
        at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.computeWithRetry(AbstractOffHeapStore.java:1109)
        at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.put(AbstractOffHeapStore.java:316)
        at org.ehcache.impl.internal.store.tiering.TieredStore.put(TieredStore.java:135)
        at org.ehcache.core.Ehcache.put(Ehcache.java:198)
        at api.ProxyService$EhcacheHttpCache$1.put(HttpService.scala:69)
        at play.api.libs.ws.ahc.cache.AhcHttpCache.put(AhcHttpCache.scala:60)
        at play.api.libs.ws.ahc.cache.AhcHttpCache.cacheResponse(AhcHttpCache.scala:281)
        at play.api.libs.ws.ahc.cache.AsyncCachingHandler.processFullResponse(AsyncCachingHandler.scala:201)
        at play.api.libs.ws.ahc.cache.AsyncCachingHandler.onCompleted(AsyncCachingHandler.scala:142)
        at play.shaded.ahc.org.asynchttpclient.netty.NettyResponseFuture.getContent(NettyResponseFuture.java:193)
        at play.shaded.ahc.org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:227)
        at play.shaded.ahc.org.asynchttpclient.netty.handler.HttpHandler.finishUpdate(HttpHandler.java:58)
        at play.shaded.ahc.org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:159)
        at play.shaded.ahc.org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:187)
        at play.shaded.ahc.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:76)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
        at play.shaded.ahc.io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
        at play.shaded.ahc.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
        at play.shaded.ahc.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
        at play.shaded.ahc.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
        at play.shaded.ahc.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
        at play.shaded.ahc.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
        at play.shaded.ahc.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
        at play.shaded.ahc.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
        at play.shaded.ahc.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
        at play.shaded.ahc.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
        at play.shaded.ahc.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:625)
        at play.shaded.ahc.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:560)
        at play.shaded.ahc.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:477)
        at play.shaded.ahc.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:439)
        at play.shaded.ahc.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
        at play.shaded.ahc.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:745)

Reproducible Test Case

https://github.com/grigoriy/play-ws-persistent-cache-bug

Notes

The problem is with CacheableHttpResponseStatus and CacheableHttpResponseBodyPart which are not serializable.

Make Cache return Future[T]

The cache interface is synchronous:

package play.api.libs.ws.ahc.cache

trait Cache {
  def get(key: EffectiveURIKey): ResponseEntry
  def put(key: EffectiveURIKey, entry: ResponseEntry): Unit
  def remove(key: EffectiveURIKey): Unit
  def close(): Unit
}

Should be made to return Future[ResponseEntry] so that IO blocking / disk based cache system will work with it.

Create Java WSClientConfig builder API

It's a bit strange to have the case classes be exposed through the Java API. There should be an actual Java builder that can be passed around, and can eventually return an AsyncHttpClientConfig object back out.

Make the client mockable with Java mocking frameworks

Play WS Version (2.5.x / etc)

1.0.0-M8

API (Scala / Java / Neither / Both)

Scala

Expected Behavior

It should be able to mock the client with Java mocking libraries like mockito, EasyMock or JMock.

Actual Behavior

The following code doesn't compile:

val wsRequest = mock[WSRequest]
val wsResponse = mock[WSResponse]

wsResponse.json throws new RuntimeException("Unexpected character ('<' (code 60))")
wsResponse.body returns "<html></html>"
wsRequest.withHeaders(any) returns wsRequest
wsRequest.post[Map[String, Seq[String]]](any)(any) returns Future.successful(wsResponse)
/home/travis/build/mohiva/play-silhouette/silhouette/test/com/mohiva/play/silhouette/impl/providers/OAuth2ProviderSpec.scala:274: type mismatch;
[error]  found   : this.wsRequest.type (with underlying type play.api.libs.ws.WSRequest)
[error]  required: this.wsRequest.Self
[error]       wsRequest.withHeaders(any) returns wsRequest

/home/travis/build/mohiva/play-silhouette/silhouette/test/com/mohiva/play/silhouette/impl/providers/OAuth2ProviderSpec.scala:275: type mismatch;
[error]  found   : this.wsResponse.type (with underlying type play.api.libs.ws.WSResponse)
[error]  required: this.wsRequest.Response
[error]       wsRequest.post[Map[String, Seq[String]]](any)(any) returns Future.successful(wsResponse)

Reproducible Test Case

mohiva/play-silhouette#509

Links

https://groups.google.com/forum/#!topic/play-framework/mYPRPbfa8Uk

Notes

Both Scala testing libraries support mocking with Java mocking frameworks. ScalaTest supports one Scala and three Java mocking frameworks. Specs2 provides syntactic sugar for mockito. This shows that using Java mocking frameworks in the Scala world is a preferred method.

Switching to play-mockws as suggested by @wsargent , isn't a good solution for a play related library, because then the library is bound to the release cycle of that play related dependency.

[2.5.3 Java] Not possible to set infinite WS request timeout with stream()

Migrated from playframework/playframework#6225

Play Version (2.5.x / etc)

2.5.3

API (Scala / Java / Neither / Both)

Java

Operating System (Ubuntu 15.10 / MacOS 10.10 / Windows 10)

Windows 8.1

JDK (Oracle 1.8.0_72, OpenJDK 1.8.x, Azul Zing)

Oracle JDK 1.8.0_92

Library Dependencies

None

Expected Behavior

  1. Setting setRequestTimeout(-1)for a WSClient request and calling stream() should respect the timeout being infinite.
  2. In the case of a timeout it should result in a Throwable being emitted to whenComplete()

Actual Behavior

The following code reaches .whenComplete() without an exception after 120 seconds.

    CompletionStage<StreamedResponse> events = ws.url("https://example.com/events")
            .setMethod("GET")
            .setHeader("Accept", "text/event-stream")
            .setRequestTimeout(-1)
            .stream();
    events.thenCompose(res -> {
        Source<ByteString, ?> responseBody = res.getBody();
        return responseBody.via(Framing.delimiter(ByteString.fromString("\n"), Integer.MAX_VALUE, FramingTruncation.ALLOW)).map(ByteString::utf8String).runForeach(data -> logger.info(data), materializer);
    }).whenComplete((done, throwable) -> {
        if (throwable != null) {
            logger.error(throwable.getMessage(), throwable);
        }
        logger.info("Done");
    });

When setting the timeout to Integer.MAX_VALUE the call doesn't timeout after 120 seconds.
When changing to get() instead of stream() the infinite setting seems to be respected. However this doesn't help with Server-Sent Events and Akka Streams. Might be related to #4846.

Is there a workaround to get an infinite event-stream processing with Play 2.5?

Exceptions when using together with AsyncHttpClient

Sorry for cross-posting. I've written down the details of this issue at:
playframework/playframework#7056

It looks like (at least in my Play 2.6.0-M1 app) that the shaded AsyncHttpClientDefaults class reads the ahc-default.properties that are pulled in from the non-shaded version on my classpath.

The issue probably comes from
https://github.com/AsyncHttpClient/async-http-client/blob/async-http-client-project-2.0.27/client/src/main/java/org/asynchttpclient/config/AsyncHttpClientConfigHelper.java#L47

Where the current ThreadContext Classloader is used to lookup the properties instead a classloader based on the current class (which would then find the ahc-default.properties with the play.shaded prefix.

json responses with 'application/json' and no charset are parsed in ISO-8859-1 charset

JSON in HTTP are always encoded in UTF-8. Responses are parsed correctly when server writes content type header like application/json; charset=utf-8.

However, many servers (like Play framework itself) uses application/json without charset. In Play 2.6, that responses are parsed in ISO-8859-1 charset.

I think that problem is in JsonBodyReadables implementation. Json.parse(response.body) should be replaced with Json.parse(response.bodyAsBytes.utf8string). Charset for response.body is always ISO-8859-1 in async http client when no charset info is provided in content type header.

Add withCookies to WSRequest

The WsCookie support could be improved.

def obtainCookie = ws.url(myurl).post(credentials).map {
      response => response.cookies
 }

returns a Seq[WsCookie]'s but there is no WsCookie support in the builder:

ws.url("request").withCookies(obtainCookies)

Fix thread leak in CachingAsyncHttpClient

CachingAsyncHttpClient has

class CachingAsyncHttpClient(underlying: AsyncHttpClient, cache: Cache[EffectiveURIKey, ResponseEntry])
    extends AsyncHttpClient
    with TimeoutResponse
    with Debug {

  private val cacheThreadPool = Executors.newFixedThreadPool(2)

}

But does not close down the thread pool on exit. Also, wouldn't it be better to handle this with Akka?

Add BodyWritable / BodyReadable for form data

The Java API does not have a body writable for java.util.Map[String, Array[String]] -- so you can't write the theoretically clear:

Map<String, List<String>> formData = parseFormData("key1=value1&key2=value2");
CompletionStage<WSResponse> responseStage = ws.url().post(formData);

Likewise, you'd like to be able to read data back out:

Map<String, List<String>> formData = response.body(formData());

Replace the set* methods in Java API

Probably everything should use withFoo rather than setFoo to keep it inline with the Scala API.

Should deprecate rather than remove old API to prevent breakage.

How to add standalone version to SBT project ?

Hi, help me please. Maybe i am doing something wrong,
but i can't even install this lib.

I am adding this to build.sbt
libraryDependencies += "com.typesafe.play" %% "play-ahc-ws-standalone" % "2.6.0-M1"

and getting:

[info] Resolving jline#jline;2.12.1 ...
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	::          UNRESOLVED DEPENDENCIES         ::
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	:: com.typesafe.play#play-ahc-ws_2.11;2.6.0-M1: not found 

Also i tried to clone this repo and open sbt console in templeate/standalone project,
but i am still getting the same error.

Make nullable getters use Optional<T> or Option[T]

In many places, the Java WSRequest and WSResponse still use methods which return null:

public interface StandaloneWSRequest {
    String getUsername();

    String getPassword();

    WSAuthScheme getScheme();

    WSSignatureCalculator getCalculator();

    Duration getRequestTimeoutDuration();

    boolean getFollowRedirects();

    String getContentType();
}

And they should return Optional<String> etc where appropriate.

Should preserve existing headers when using withBody

Play WS Version (2.6.x / etc)

1.0.0-M8

API (Scala / Java / Neither / Both)

StandaloneAhcWSRequest.withBodyAndContentType

Expected Behavior

request.addHttpHeaders(
"hl-app-id" -> appId,
"hl-timestamp" -> timestamp,
"hl-echo-str" -> echoStr,
"hl-access-token" -> accessToken
)

Send the HTTP Headers I have set the custom header.

Actual Behavior

Send the HTTP Headers and I did not set the value.

I think here: withBody(wsBody).withHttpHeaders(HttpHeaders.Names.CONTENT_TYPE, contentType)
Should be amended as follows: withBody(wsBody).addHttpHeaders(HttpHeaders.Names.CONTENT_TYPE, contentType)

Turn WSClientConfig into an interface

There should be only one config object, defined by the implementation. WSClientConfig should be an interface or trait that the config object implements.

Fix sbt-release for sonatype publishing

Currently the publish to Sonatype fails, and so the artifacts are sent to staging but have to be published from the staging repository by hand (by clicking "close" and then "publish") rather than directly.

Not sure what is causing this behavior, and it's hard to fake publishing to Sonatype.

Can't configure Content-Type for Multipart Form Uploading

Play Version (2.5.x / etc)

2.5.10

API (Scala / Java / Neither / Both)

Java

Operating System (Ubuntu 15.10 / MacOS 10.10 / Windows 10)

Ubuntu 16.04.1 LTS

JDK (Oracle 1.8.0_72, OpenJDK 1.8.x, Azul Zing)

1.8.0_121

Expected Behavior

Following the spec set by RFC 2388 a POST with Content-Type=multipart/form-data should be able to support multiple content-types and character encoding in all of its parts.

Actual Behavior

https://www.playframework.com/documentation/2.5.x/JavaWS#Submitting-multipart/form-data

The play.mvc.Http.MultipartFormData.DataPart doesn't allow any sort of content-type configuration and it defaults to text/plain with no charset, which our back-end (JAX-RS) interprets as the default for text/plain: US-ASCII. RFC 1341
The play.mvc.MultipartFormatter.boundaryToContentType() beeing called uppon the format of a multipart Post also forces a content-type in the header without a charset configuration and therefore when using play.libs.ws.WSRequest.setContentType("multipart/form-data; charset=UTF-8") will make the request carry 2 Content-type entity headers.

#6928 seems like it had a similar problem.

Example Request sent by

POST /multipartform HTTP/1.1
Cache-Control: no-cache
Accept-Language: en
Content-Type: multipart/form-data; boundary=LK9jYpDyGz-nbNxWCA
Accept-Encoding: gzip,deflate
Transfer-Encoding: chunked
Host: 10.112.98.101:8010
Accept: /
User-Agent: AHC/2.0

161
--LK9jYpDyGz-nbNxWCA
Content-Disposition: form-data; name="request"
Content-Type: text/plain

{"stuff":null,"dummyDate":1487772284795,"action":null,"stuff":null}
6c

--LK9jYpDyGz-nbNxWCA
Content-Disposition: form-data; name="filename"
Content-Type: text/plain

SUM.csv
8d

--LK9jYpDyGz-nbNxWCA
Content-Disposition: form-data; name="data"; filename="SUM.csv"
Content-Type: multipart/form-data; charset=UTF-8

13
1;1;1;1;1;1;1;1;1;1
18

--LK9jYpDyGz-nbNxWCA--
0

Unshaded versions of asynchttpclient showing up in ivy repo?

Expected Behavior

play-ws shades asynchttpclient and oauth.signpost -- the unshaded versions should not show up as POM dependencies of the project.

Actual Behavior

If you create a sample project including only Play-WS, then you can see it depends on the unshaded code still in an Intellij IDEA project:

libraryDependencies += ("com.typesafe.play" %% "play-ahc-ws-standalone" % "1.0.0-M2")

Although it does not show up in mvnrepository for some reason:

https://mvnrepository.com/artifact/com.typesafe.play/play-ahc-ws-standalone_2.12/1.0.0-M2

Add back request filters

May not be able to have request filters in the general case, but should at least see if they fit into the AhcWSRequest use case.

Since JDK 1.8 is out and we have 2.12, this may be a good point to reexamine the API and see if something more functional can be used i.e. java.util.function.

Refine WSRequestFilter

On the Scala API side, we could define WSRequestFilter as

class WSRequestFilter extends (StandaloneWSRequest => Future[StandaloneWSResponse])

along with some Singleton apply methods that make wrappers work.

On the Java side, we can define it as a functional interface so we can use lambdas and such.

Add Source WSBody handling to Scala API

It's a bit strange, but the WSRequest on the Java API can handle Source and Inputstream, and I don't think the Scala API handles that specifically.

Should not add InputStream as that is deprecated in the Java API.

Application does not terminate correctly

I am using the standalone WS in a script-like program. Sadly, it does not terminate correctly, but hangs. The same happens on a colleagues computer, as well as on the build server, so I doubt it's a local problem for me. I also created a minimal example from the minimal scala seed with sbt new sbt/scala-seed.g8.

When terminating with Ctrl-C, it will terminate the actor system only after some time.

I used the example code from the README.

Play WS Version (2.5.x / etc)

libraryDependencies += "com.typesafe.play" %% "play-ahc-ws-standalone" % "1.0.0-M10"
but it also happens with RC2.

API (Scala / Java / Neither / Both)

Scala

Operating System (Ubuntu 15.10 / MacOS 10.10 / Windows 10)

Linux fresenius 4.8.0-53-generic #56~16.04.1-Ubuntu SMP Tue May 16 01:18:56 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

JDK (Oracle 1.8.0_72, OpenJDK 1.8.x, Azul Zing)

java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Library Dependencies

none

Expected Behavior

The application terminates correctly when the main method terminates. An sbt run terminates after we receive the "Got a response OK" (see minimal example)

Actual Behavior

The application does not terminate (within a time frame of <20s). See here:

$ sbt run
[info] Loading project definition from /home/brueder/tmp/minimal-ws-error/project
[info] Set current project to Hello (in build file:/home/brueder/tmp/minimal-ws-error/)
[info] Running playwsclient.ScalaClient 
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Got a response OK
^C[WARN] [06/06/2017 11:03:50.477] [Thread-1] [CoordinatedShutdown(akka://default)] 
CoordinatedShutdown from JVM shutdown failed: Futures timed out after [10000 milliseconds]

Reproducible Test Case

see linked minimal example: http://wikisend.com/download/941182/minimal-ws-error.zip

[2.5.x] Add API to get a single header

Currently, in 2.5.x there is no way to get a single header from a WSRequest : The API only has getHeaders, which returns a Map<String, Collection>

We can easy get a simple header from the map, but having a bit more clean API would be nice.

WSRequest should have a setCookies() method

Play WS Version (2.5.x / etc)

Using Play WS 2.5

API (Scala / Java / Neither / Both)

Java (don't know about Scala)

Operating System (Ubuntu 15.10 / MacOS 10.10 / Windows 10)

All I guess

Improvement

It would be very convenient if there was a .setCookies(List<WSCookie> cookies) method on WSRequest to easily share cookies between different requests. WSResponse have a .getCookies() so it is easy getting the cookies out.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.