pepegar / hammock Goto Github PK

View Code? Open in Web Editor NEW

175.0 7.0 20.0 1.5 MB

Purely functional HTTP client

Home Page: https://hammock.pepegar.com/

License: MIT License

Scala 96.17% Shell 0.13% Nix 3.70%

scala http fp cats typelevel

hammock's Introduction

Hammock

Hammock is yet another HTTP client for Scala. It tries to be typeful, purely functional, and work along other technologies that you're already using such as akka-http, circe, or cats.

Installation

Add the following to your build.sbt.

// For Scala 2.10, 2.11, or 2.12
libraryDependencies ++= Seq(
  "com.pepegar" %% "hammock-core" % "0.10.0",
  
  // Hammock for standard Scala doesn't ship with a standard implementation
  "com.pepegar" %% "hammock-apache-http" % "0.10.0"
)

// For ScalaJS
libraryDependencies += "com.pepegar" %%% "hammock-core" % "0.10.0"

Rationale

It's easy to use, has a high level API
It's typeful, tries to represent effects at type level.
It does not force a specific target context. You can run your computations in any type F[_] that has an instance of cats-effect's Sync[F].
It has good documentation.
It's modular

Modules

Module name	Description	Version
`hammock-core`	the core functionality of hammock, using XHR in JS	`0.10.0`
`hammock-circe`	encode and decode HTTP entities with Circe	`0.10.0`
`hammock-apache-http`	run your HTTP requests with Apache HTTP commons	`0.10.0`
`hammock-akka-http`	run your HTTP requests with akka-http	`0.10.0`
`hammock-asynchttpclient`	run your HTTP requests with AsyncHttpClient	`0.10.0`

How does Hammock look in action?

import cats.effect.IO
import hammock._
import hammock.marshalling._
import hammock.apache.ApacheInterpreter
import hammock.circe.implicits._

object HttpClient {
  // Using the Apache HTTP commons interpreter
  implicit val interpreter = ApacheInterpreter.instance[IO]

  val response = Hammock
    .request(Method.GET, uri"https://api.fidesmo.com/apps", Map()) // In the `request` method, you describe your HTTP request
    .as[List[String]]
    .exec[IO]
}

Code of conduct

People are expected to follow the Typelevel Code of Conduct when discussing Hammock on the Github page, Gitter channel, or other venues.

hammock's People

Contributors

Stargazers

Watchers

hammock's Issues

2.12 version is not published

https://jcenter.bintray.com/com/pepegar/ there is no 2.12 artifacts

Make interptrans not require `F` to be a `Sync`

Better documentation

Create a new structure for the microsite, grouping concepts in a clearer way. It will have the following categories:

Interpreters (Apache hc, AHC(#73), Akka HTTP, and JS XHR)
Algebras (Marshalling (#74),and HttpRequestF)
Marshalling/unmarshalling (Codec typeclass and marshalling algebra)
High level DSL

Should address #65

FIX: We're depending on an SNAPSHOT

Right now we depend on a Snapshot version of Atto. Fix that once released.

publish 0.7.0

In the readme you mention 0.7.0 version while only 0.6.1 is that latest published one

add documentation on Post + Codecs

It will be useful to have some documentation on Posts and Codecs valuable for the body part.
For instance, it is not clear to me how to send zipped file via Method.POST with hammock and how to send multipart body, where some fields are strings and some are zip files

Create AWS4 authentication

FWIW https://github.com/bos/wreq/blob/master/Network/Wreq/Internal/AWS.hs#L43-L117

ScalaJS support

Make hammock work with ScalaJS

Separate API and implementation

Currently, Apache HTTP Client is always included. Even when using alternative implementations. I scanned through the code and could not find a specific reason for this. Since this is a rather heavy dependency, I'm reluctant to use the library as it is right now.

Special characters are interpreted as `?` when using Hammock with Circe

Hammock's standard ContentType is text/plain, which Apache's StringEntity and ContentType maps to the encoding ISO-8859-1.

To support UTF-8, this ContentType needs to be application/json.
As Circe is used for encoding and decoding json, I have proposed a solution where HammockEncoderForCirce explicitly passes application/json; charset=utf-8 as the ContentType: #94

Does this look like a viable solution?

Submit to Typelevel incubator

Hammock could benefit a lot from the Typelevel community, and maybe the community could benefit from Hammock as well?

Runtime exceptions with Apache HTTP backend

Not sure what your policy on runtime exception is, so I just leave this issue for your consideration - whether you want to make it typesafe or not.

First error - `ClientProtocolException`

Reproduction:

import cats.effect.IO
import hammock._
import hammock.jvm.Interpreter

implicit val interpreter = Interpreter[IO]

Hammock.
  .request(Method.GET, uri"github.com", Map())
  .exec[IO]
  .unsafeRunSync

Result:

org.apache.http.client.ClientProtocolException: org.apache.http.ProtocolException: Target host is not specified
  at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
  at hammock.jvm.Interpreter.$anonfun$doReq$3(Interpreter.scala:41)
  at cats.effect.internals.IORunLoop$.step(IORunLoop.scala:167)
  at cats.effect.IO.unsafeRunTimed(IO.scala:307)
  at cats.effect.IO.unsafeRunSync(IO.scala:242)
  ... 36 elided
Caused by: org.apache.http.ProtocolException: Target host is not specified
  at org.apache.http.impl.conn.DefaultRoutePlanner.determineRoute(DefaultRoutePlanner.java:71)
  at org.apache.http.impl.client.InternalHttpClient.determineRoute(InternalHttpClient.java:125)
  at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
  ... 43 more

Second error - `UnknownHostException`

Easiest way to reproduce this is to unplug your Internet cable / turn off wifi and run:

import cats.effect.IO
import hammock._
import hammock.jvm.Interpreter

implicit val interpreter = Interpreter[IO]

Hammock.
  .request(Method.GET, uri"http://github.com", Map())
  .exec[IO]
  .unsafeRunSync

Result:

java.net.UnknownHostException: github.com: Temporary failure in name resolution
  at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
  at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
  at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
  at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
  at java.net.InetAddress.getAllByName(InetAddress.java:1192)
  at java.net.InetAddress.getAllByName(InetAddress.java:1126)
  at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
  at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
  at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
  at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
  at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
  at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
  at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
  at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
  at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
  at hammock.jvm.Interpreter.$anonfun$doReq$3(Interpreter.scala:41)
  at cats.effect.internals.IORunLoop$.step(IORunLoop.scala:167)
  at cats.effect.IO.unsafeRunTimed(IO.scala:307)
  at cats.effect.IO.unsafeRunSync(IO.scala:242)
  ... 36 elided

publish API documentation

Auth

Add support for auth

BasicAuth renders wrong header name

Dear Hammock-crew,

Auth.BasicAuth("user", "password") produces the correct rendition of "Basic <Base64 "$user:$pass">"

But it renders the wrong header name->value-pair. It produces:

Authentication: Basic <...>

It should produce:

Authorization: Basic <...>

I have verified this using my own tests.

hammock scalajs does not work

I updated my cromwell-client lib to the latest hammock version and discovered that ScalaJS version of cromwell does not work.
For example this line ( https://github.com/antonkulaga/cromwell-client/blob/master/client/jvm/src/test/scala/group/research/aging/cromwell/Hello.scala#L23 )that works perferctly fine from JVM , leads to match-ing errors on ScalaJS:

[Ljava.lang.String;@d (of class [Ljava.lang.String;)
"Error
    at $c_s_MatchError.$c_jl_Throwable.fillInStackTrace__jl_Throwable (http://localhost:8080/public/cromwell-web-fastopt.js:27930:14)
    at $c_s_MatchError.$c_jl_Throwable.init___T__jl_Throwable (http://localhost:8080/public/cromwell-web-fastopt.js:28039:8)
    at $c_s_MatchError.init___O (http://localhost:8080/public/cromwell-web-fastopt.js:62153:52)
    at $c_Lhammock_js_Interpreter.parseHeaders__p1__T__sci_Map (http://localhost:8080/public/cromwell-web-fastopt.js:24169:35)
    at http://localhost:8080/public/cromwell-web-fastopt.js:24135:35
    at $c_sjsr_AnonFunction0.apply__O (http://localhost:8080/public/cromwell-web-fastopt.js:44602:23)
    at $c_Lcats_effect_internals_IORunLoop$.step__Lcats_effect_IO__Lcats_effect_IO (http://localhost:8080/public/cromwell-web-fastopt.js:4741:24)
    at $c_Lcats_effect_IO$Bind.$c_Lcats_effect_IO.unsafeRunTimed__s_concurrent_duration_Duration__s_Option (http://localhost:8080/public/cromwell-web-fastopt.js:4395:51)
    at $c_Lcats_effect_IO$Bind.$c_Lcats_effect_IO.unsafeRunSync__O (http://localhost:8080/public/cromwell-web-fastopt.js:4368:15)
    at $c_Lgroup_research_aging_cromwell_web_AppCircuit$$anon$1$$anonfun$handle$1.applyOrElse__O__F1__O (http://localhost:8080/public/cromwell-web-fastopt.js:68093:172)"

Upgrade to cats 1.0.0

🎉

Open the interpretation to more backends

Currently Hammock is just a wrapper over either Apache HTTP Client or XmlHTTPRequest. We can do better by adding more clients, or at least specifying in the docs how an interpreter is implemented for any client.

Check laws of typeclasses we're creating instances for.

From the top of my head:

optics in the high level DSL
circe Codec

Cookies

Add support for cookies

content type is wrong

When I try to get a result that generates plain text (like https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM1698570&targ=self&form=text&view=full ) I get json/application content type instead and all "\n" symbols are deleted from there

automatic deploy of microsite

Tests...

Only circe-codec is tested right now... We need to test:

Use cats-effect bracket

We should use it for interpreters, where the class instantiating InterpTrans uses the lower level interpreter as a resource that's aquired, used and released afterwards.

Documentation Request: How to consume response and headers in one request.

The docs make it pretty clear how to make a request and decode the entity of the HttpResponse:

  val response = Hammock
    .request(Method.GET, uri"https://api.fidesmo.com/apps", Map())
    .as[List[String]]
    .exec[IO]

And you can get the headers like:

  val response = Hammock
    .request(Method.GET, uri"https://api.fidesmo.com/apps", Map())
    .map(_.headers)
    .as[List[String]]
    .exec[IO]

But how could one decode the response and consume the headers and body into a case class like:

case class BodyAndHeaders[T](body: T, responseHeaders: Map[String, String])

Entity is not fully consumed and the connection is not released in some cases

I tried to use hammock in a simple project that uses Gitlab API, but it turned out that simple GET requests started hanging after few connections.

Reproducing this (against Gitlab 9 v4 API) is pretty easy if you have more than 2 open merge requests.

  def openMergeRequests: Try[List[MergeRequest]] = Hammock
    .request(GET, GitlabUris.mergeRequests(projectId, "opened"), Map.empty)
    .exec[Try]
    .as[List[MergeRequest]]

  def mergeRequestComments(mergeRequest: MergeRequest): Try[List[Comment]] = Hammock
    .request(GET, GitlabUris.mergeRequestComment(projectId, mergeRequest.iid), Map.empty)
    .exec[Try]
    .as[List[Comment]]

  for {
    openMergeRequests ← GitlabClient.openMergeRequests
    comments ← openMergeRequests.flatTraverse(GitlabClient.mergeRequestComments)
  } yield comments.size

Make Uri parser more restrictive

Currently it allows for some invalid URIs, which lead to runtime errors when passed to Apache HTTP (thrown by java.net.URI) or AsyncHttpClient.

Examples of invalid URIs which pass hammock.Uri.fromString parser:

::
^&%$#@!
http://^&%$#@!/foo
empty string and/or whitespace

A couple of ideas for how to solve this:

Wrap and use java.net.URI constructor as a parser, in type-safe way (probably the easiest)
Use a library like: https://github.com/lemonlabsuk/scala-uri
Improve existing parser/write/port a parser from scratch (https://tools.ietf.org/html/rfc3986)

I'd be happy to contribute once you decide on direction.

Support multipart requests

Very slick library, but can't send multipart requests out-of-the-box. Other libraries I've tried do support it but have other caveats:

RosHttp but not as nice as this one (and a bit buggy).
sttp but doesn't support Node.js ATM.

Create an example of use with ScalaJS

Model headers

add standard's headers to the model
create DSL for manipulating them

tut-check README

Replace java.util.Date in Cookie with java.time.ZonedDateTime

use https://github.com/scala-js/scala-js-java-time for JS.

convert interpreters to methods of objects

Currently we're using classes that extend InterpTrans. We don't need to create those classes, we could just create objects that expose a single method (potentially implicit) that returns an InterpTrans instance.

New Release with cats 1.0.0-MF

It seems migration was done some weeks ago. Is there any plan of new release? Thanks.

compile with scalaz

Investigate doobie/yax and aiyou/yax

Reduce allocation overhead by removing Kleisli layer

Hammock Interpreter have HttpClient as constructor parameter. So why use Kleisli[F, HttpClient, ?] in the internal machinery?

We could just use Async effect without any other layer.

Documentation: How does hammock compare to other frameworks.

Convert `InterpTrans` into a typeclass

NullPointerException on NoContent (204) response

I have code similar to this:

val response = Hammock
.request(Method.POST,
uri,
headers,
Some(body))
.exec[IO]
.unsafeRunSync

where the HTTP response is 204 (No Content). The following code in hammock.Interpreter gets a null pointer exception:

private def doReq(reqF: HttpF[HttpResponse])(implicit F: Sync[F]): Kleisli[F, HttpClient, HttpResponse] =
Kleisli { client =>
for {
req <- getApacheRequest(reqF)
resp <- F.delay(client.execute(req))
body <- F.delay(responseContentToString(resp.getEntity.getContent))
status <- Status.get(resp.getStatusLine.getStatusCode).pure[F]
responseHeaders <- resp.getAllHeaders.map(h => h.getName -> h.getValue).toMap.pure[F]
_ <- F.delay(EntityUtils.consume(resp.getEntity))
} yield HttpResponse(status, responseHeaders, new Entity.StringEntity(body))
}

The issue appears to be that on a NoContent HTTP response, resp.getEntity returns null. The above code references through the null reference with resp.getEntity.getContent.

Any way to work around this?

use lenses for user-facing high-level API

Instead of using the lenses under the hood, It'd be cool to experiment how the API would look like if lenses (not lenses actually, all kind of optics) were exposed to modify the Options type.

URI class

We need an URI model

example is broken

Example subproject is broken and does not compile. I also cannot use .as[T] method now

Add functionality for checking status codes of a response

It would be very helpful to check a status codes of a response with methods :

@Lenses case class Status(code: Int, text: String, description: String) {
  def isInformational: Boolean = this.code / 100 == 1
  def isSuccess: Boolean = this.code / 100 == 2
  def isRedirection: Boolean = this.code / 100 == 3
  def isClientError: Boolean = this.code / 100 == 4
  def isServerError: Boolean = this.code / 100 == 5
}

URI params method

Hi there!
Could you make a param(s) method of Uri class like in STTP library? It would be very useful.
Here is an implicit class I'm using in my production code:

  implicit class UriOps(val self: Uri) extends AnyVal {
    def param(key: String, value: String): Uri = self.copy(query = self.query + (key → value))
    def params(ps: (String, String)*): Uri = ps match {
      case Seq() ⇒ self
      case _     ⇒ ps.foldLeft(self) { case (uri, (k, v)) ⇒ uri.copy(query = uri.query + (k → v)) }
    }
  }

I could extend Uri class or provide this implicit class by myself and open a pull request

Add syntax to the `Codec` typeclass

Expose the methods on Codec in an implicit class

[Deprecation] Synchronous XMLHttpRequest on the main thread is deprecated

Here is what I get in ScalaJS when I do .unsafeToFuture()

Interpreter.scala:34 [Deprecation] Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.

I think .unsafeToFuture() should send async requests by default in ScalaJS

  implicit object HttpFMonad extends Monad[HttpF] {
    override def pure[A](x: A): HttpF[A] = Get(HttpRequest(Uri(), Map(), None)).asInstanceOf[HttpF[A]]
    override def flatMap[A, B](fa: HttpF[A])(f: A ⇒ HttpF[B]): HttpF[B] = Get(fa.req).asInstanceOf[HttpF[B]]
    override def tailRecM[A, B](a: A)(f: A ⇒ HttpF[Either[A, B]]): HttpF[B] = f(a).asInstanceOf[HttpF[B]]
  }

It's obvious that such monad is incorrect but free.runTailRec.req produces correct output:

[error] java.lang.Exception: Failed auth, 
request: HttpRequest(Uri(Some(https),Some(Authority(None,Other(portal.admixer.net),None)),/Token,Map(),None),Map(),None) 
response: HttpResponse(Status(400,Bad Request,The request contains bad syntax or cannot be fulfilled.),Map(Access-Control-Allow-Headers -> Content-Type, x-xsrf-token, X-Requested-With, Authorization, Server -> Microsoft-IIS/10.0, Access-Control-Allow-Methods -> POST, GET, OPTIONS, Expires -> -1, Content-Length -> 34, Cache-Control -> no-cache, Content-Type -> application/json;charset=UTF-8, Pragma -> no-cache, Date -> Tue, 20 Mar 2018 11:22:07 GMT, X-Powered-By -> ASP.NET, X-ApiVersion -> 1.0.6638.23204),StringEntity({"error":"unsupported_grant_type"},hammock.ContentType$$anon$2@1e498b00))

So the question is:

Are there a simple way to get original request
If not, should I provide such method with the code above? (hiding this peace of... bad monad!)

Apache HTTP module duplicates headers, Akka HTTP ignores them all alltogether

The base, Apache HTTP intepreter adds headers twice (here and here). Duplicate headers are ignored by many APIs.

Then, the Akka HTTP interpreter does not add them at all.