Coder Social home page Coder Social logo

zio / zio-http Goto Github PK

View Code? Open in Web Editor NEW
748.0 28.0 379.0 9.33 MB

A next-generation Scala framework for building scalable, correct, and efficient HTTP clients and servers

Home Page: https://zio.dev/zio-http

License: Apache License 2.0

Scala 99.83% Dockerfile 0.06% Shell 0.12%
zio http http-server http-client websocket scala

zio-http's Introduction

ZIO Http

ZIO HTTP is a scala library for building http apps. It is powered by ZIO and Netty and aims at being the defacto solution for writing, highly scalable and performant web applications using idiomatic Scala.

ZIO HTTP is designed in terms of HTTP as function, where both server and client are a function from a request to a response, with a focus on type safety, composability, and testability.

Development CI Badge Sonatype Snapshots ZIO Http

Some of the key features of ZIO HTTP are:

ZIO Native: ZIO HTTP is built atop ZIO, a type-safe, composable, and asynchronous effect system for Scala. It inherits all the benefits of ZIO, including testability, composability, and type safety.

Cloud-Native: ZIO HTTP is designed for cloud-native environments and supports building highly scalable and performant web applications. Built atop ZIO, it features built-in support for concurrency, parallelism, resource management, error handling, structured logging, configuration management, and metrics instrumentation.

Imperative and Declarative Endpoints: ZIO HTTP provides a declarative API for defining HTTP endpoints besides the imperative API. With imperative endpoints, both the shape of the endpoint and the logic are defined together, while with declarative endpoints, the description of the endpoint is separated from its logic. Developers can choose the style that best fit their needs.

Type-Driven API Design: Beside the fact that ZIO HTTP supports declarative endpoint descriptions, it also provides a type-driven API design that leverages Scala's type system to ensure correctness and safety at compile time. So the implementation of the endpoint is type-checked against the description of the endpoint.

Middleware Support: ZIO HTTP offers middleware support for incorporating cross-cutting concerns such as logging, metrics, authentication, and more into your services.

Error Handling: Built-in support exists for handling errors at the HTTP layer, distinguishing between handled and unhandled errors.

WebSockets: Built-in support for WebSockets allows for the creation of real-time applications using ZIO HTTP.

Testkit: ZIO HTTP provides first-class testing utilities that facilitate test writing without requiring a live server instance.

Interoperability: Interoperability with existing Scala/Java libraries is provided, enabling seamless integration with functionality from the Scala/Java ecosystem through the importation of blocking and non-blocking operations.

JSON and Binary Codecs: Built-in support for ZIO Schema enables encoding and decoding of request/response bodies, supporting various data types including JSON, Protobuf, Avro, and Thrift.

Template System: A built-in DSL facilitates writing HTML templates using Scala code.

OpenAPI Support: Built-in support is available for generating OpenAPI documentation for HTTP applications, and conversely, for generating HTTP endpoints from OpenAPI documentation.

ZIO HTTP CLI: Command-line applications can be built to interact with HTTP APIs by leveraging the power of ZIO CLI and ZIO HTTP.

Installation

Setup via build.sbt:

libraryDependencies += "dev.zio" %% "zio-http" % "<version>"

NOTES ON VERSIONING:

  • Older library versions 1.x or 2.x with organization io.d11 of ZIO HTTP are derived from Dream11, the organization that donated ZIO HTTP to the ZIO organization in 2022.
  • Newer library versions, starting in 2023 and resulting from the ZIO organization started with 0.0.x, reaching 1.0.0 release candidates in April of 2023

Getting Started

ZIO HTTP provides a simple and expressive API for building HTTP applications. It supports both server and client-side APIs. Let's see how it is simple to build a greeting server and call it using the client API.

Greeting Server

The following example demonstrates how to build a simple greeting server. It contains 2 routes: one on the root path, it responds with a fixed string, and one route on the path /greet that responds with a greeting message based on the query parameter name.

import zio._
import zio.http._

object GreetingServer extends ZIOAppDefault {
  val routes =
    Routes(
      Method.GET / Root -> handler(Response.text("Greetings at your service")),
      Method.GET / "greet" -> handler { (req: Request) =>
        val name = req.queryParamToOrElse("name", "World")
        Response.text(s"Hello $name!")
      }
    )

  def run = Server.serve(routes).provide(Server.default)
}

Greeting Client

The following example demonstrates how to call the greeting server using the ZIO HTTP client:

import zio._
import zio.http._

object GreetingClient extends ZIOAppDefault {

  val app =
    for {
      client   <- ZIO.serviceWith[Client](_.host("localhost").port(8080))
      request  =  Request.get("greet").addQueryParam("name", "John")
      response <- client.request(request)
      _        <- response.body.asString.debug("Response")
    } yield ()

  def run = app.provide(Client.default, Scope.default)
}

Documentation

Learn more on the ZIO Http homepage!

Contributing

For the general guidelines, see ZIO contributor's guide.

Code of Conduct

See the Code of Conduct

Support

Come chat with us on Badge-Discord.

License

License

zio-http's People

Contributors

987nabil avatar adamgfraser avatar afsalthaj avatar amitksingh1490 avatar d11-amitsingh avatar d11kaushik avatar dependabot[bot] avatar devsprint avatar dinojohn avatar ex0ns avatar frekw avatar gciuloaica avatar girdharshubham avatar github-actions[bot] avatar jdegoes avatar khajavi avatar kyri-petrou avatar rajcspsg avatar renovate[bot] avatar runtologist avatar scala-steward avatar scottweaver avatar shrutiverma97 avatar smehta91d11 avatar swoogles avatar tomtriple avatar tusharmath avatar vigoo avatar wosin avatar wpoosanguansit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zio-http's Issues

QueryParams multimap

Tapir requires to extract from queryParams: Seq[(String, Seq[String])]

Inspiration could be extract http4s, akka-http

ApacheBench times out

Here's cURL working.

$ curl -v  http://localhost:8090/text
*   Trying ::1:8090...
* TCP_NODELAY set
* Connected to localhost (::1) port 8090 (#0)
> GET /text HTTP/1.1
> Host: localhost:8090
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 13
< server: ZIO-Http
< date: Wed, 17 Mar 2021 01:55:56 GMT
< 
Hello World!
* Connection #0 to host localhost left intact

Here's ApacheBench timing out.

$ ab http://localhost:8090/text
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)...apr_pollset_poll: The timeout specified has expired (70007)

Here's ApacheBench and pressing ^C immediately .

$ ab http://localhost:8090/text
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)...^C

Server Software:        ZIO-Http
Server Hostname:        localhost
Server Port:            8090

Document Path:          /text
Document Length:        0 bytes

Concurrency Level:      1
Time taken for tests:   0.891 seconds
Complete requests:      0
Failed requests:        0
Total transferred:      107 bytes
HTML transferred:       13 bytes

support for 2.12

in order tapir support for 2.12 zio-http needs a 2.12 release

Support configurable request/response serialization

Please add a way to configure binary serialization for requests and responses.

We have a protobuf http application that does its own Array[Byte] decoding/encoding and speed is very important. I would love to try using zio-http in it.

`ServerRequestHandler.executeAsync` should close connection on failure without cause

Current

case Exit.Failure(cause) =>
    cause.failureOption match {
      case Some(Some(e)) => cb(SilentResponse[Throwable].silent(e))
      case Some(None)    => cb(Response.fromHttpError(HttpError.NotFound(Path(jReq.uri()))))
      case None          => () /// Callback not being called can keep the connection hanging. We should close the connection in this case.
    }

Create an Authentication middleware facility

It is for sure possible to create manually an authentication middleware on top of the current API, but it would be beneficial for newcomers to have an API similar to Http4s to collect routes requiring authentication.

Uploading large body logs exception in stderr, mangles input

Using the code:

import zhttp.http._
import zhttp.service.Server
import zio._

object Main extends App {
  val echo = Http.collect[Request] { case req @ Method.POST -> Root / "echo" =>
    Response.http(content = req.data.content)
  }

  override def run(args: List[String]): URIO[ZEnv, ExitCode] =
    Server.start(8090, echo).exitCode
}

Edit: made some adjustments after finding out #35 was an incorrect report.

# 10 million bytes
> cat /dev/random | head -c 10000000 > in.zip

> curl localhost:8090/echo --data-binary @in.zip -v > out.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8090 (#0)
> POST /echo HTTP/1.1
> Host: localhost:8090
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 10000000
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
} [65536 bytes data]
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< content-length: 9469184
<
{ [102400 bytes data]
100 18.5M  100 9247k  100 9765k  32.7M  34.5M --:--:-- --:--:-- --:--:-- 67.2M
* Connection #0 to host localhost left intact
* Closing connection 0

# output is smaller
> wc -c in.zip                                                                                                                                                                                                            
 10000000 in.zip
> wc -c out.zip
 9469184 out.zip

shasum --algorithm 256 *.zip
8e27ef091cef676413a1e39f002a1fe5fafaa7ac4f80a2bb41a3b8092837bf52  in.zip
86807e45c1a2c61d4545954e62d6da7b1504691454ea593cfee79af61eba33b1  out.zip
[E] Mar 16, 2021 2:22:47 AM io.netty.channel.DefaultChannelPipeline onUnhandledInboundException
[E] WARNING: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
[E] io.netty.channel.StacklessClosedChannelException
[E] 	at io.netty.channel.AbstractChannel.close(ChannelPromise)(Unknown Source)
[E]

Sending a 960MB file, however, hangs around the end (client-side) after outputting this from curl:

 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8090 (#0)
> POST /echo HTTP/1.1
> Host: localhost:8090
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 1000000000
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
} [65536 bytes data]
 99  953M    0     0   99  950M      0  25.5M  0:00:37  0:00:37 --:--:-- 13.5M* We are completely uploaded and fine
100  953M    0     0  100  953M      0  21.1M  0:00:45  0:00:45 --:--:--     0^C #I cancelled here
After a couple seconds from disconnecting I got this stack trace
[E] Mar 16, 2021 2:55:54 AM io.netty.channel.DefaultChannelPipeline onUnhandledInboundException
[E] WARNING: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
[E] java.lang.IndexOutOfBoundsException: writerIndex(0) + minWritableBytes(-1454240449) exceeds maxCapacity(2147483647): UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf(ridx: 0, widx: 0, cap: 1810942847)
[E] 	at io.netty.buffer.AbstractByteBuf.ensureWritable0(AbstractByteBuf.java:296)
[E] 	at io.netty.buffer.ByteBufUtil.reserveAndWriteUtf8Seq(ByteBufUtil.java:681)
[E] 	at io.netty.buffer.ByteBufUtil.writeUtf8(ByteBufUtil.java:638)
[E] 	at io.netty.buffer.Unpooled.copiedBufferUtf8(Unpooled.java:599)
[E] 	at io.netty.buffer.Unpooled.copiedBuffer(Unpooled.java:582)
[E] 	at zhttp.http.Response$HttpResponse.toJFullHttpResponse(Response.scala:37)
[E] 	at zhttp.service.server.ServerRequestHandler.writeAndFlush(ServerRequestHandler.scala:60)
[E] 	at zhttp.service.server.ServerRequestHandler.channelRead0(ServerRequestHandler.scala:76)
[E] 	at zhttp.service.server.ServerRequestHandler.channelRead0(ServerRequestHandler.scala:13)
[E] 	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[E] 	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[E] 	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
[E] 	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
[E] 	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
[E] 	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[E] 	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[E] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[E] 	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
[E] 	at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:544)
[E] 	at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:382)
[E] 	at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:211)
[E] 	at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:289)
[E] 	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
[E] 	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
[E] 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
[E] 	at java.base/java.lang.Thread.run(Thread.java:834)
[E]

For the whole time of the upload, out.zip's size never changed (0B) and the application kept consuming memory (I can now see the JVM use 7GB according to macOS's activity monitor, 24GB according to htop and bottom). Reminder: the file is <1GB.

jconsole:

image

Note that this took around 37 seconds to send the payload.

For comparison, http4s 0.21.18:

import scala.concurrent.ExecutionContext

import cats.effect.ExitCode
import cats.effect.IO
import cats.effect.IOApp
import org.http4s._
import org.http4s.dsl.io._
import org.http4s.implicits._
import org.http4s.server.blaze.BlazeServerBuilder

object Main extends IOApp {
  def run(args: List[String]): IO[ExitCode] =
    BlazeServerBuilder[IO](ExecutionContext.global)
      .bindHttp(8090, "0.0.0.0")
      .withHttpApp(
        HttpRoutes
          .of[IO] { case req @ POST -> Root / "echo" =>
            Ok(req.body)
          }
          .orNotFound
      )
      .resource
      .use(_ => IO.never)
}
> curl localhost:8090/echo --data-binary @in.zip -v > out.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8090 (#0)
> POST /echo HTTP/1.1
> Host: localhost:8090
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 1000000000
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 200 OK
< Date: Tue, 16 Mar 2021 02:03:21 GMT
< Transfer-Encoding: chunked
<
* Done waiting for 100-continue
  0  953M    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0} [65536 bytes data]
 63 1401M    0  447M   46  448M   152M   152M  0:00:06  0:00:02  0:00:04  305M* We are completely uploaded and fine
{ [65552 bytes data]
100 1907M    0  953M  100  953M   248M   248M  0:00:03  0:00:03 --:--:--  497M
* Connection #0 to host localhost left intact
* Closing connection 0
*

(3 seconds)

> shasum --algorithm 256 *.zip
b021c1864684a4133e918330979f406fd372da0a2aeb94cb86d1cf9802d181aa  in.zip
b021c1864684a4133e918330979f406fd372da0a2aeb94cb86d1cf9802d181aa  out.zip

Let the server to be terminated programmatically

Hey guys!

I've been playing around with the project and was going to try to build some integration tests, basically starting http server for a spec and terminating it afterwards. Though I found myself unable to shut it down programmatically.

I suppose that to be a useful feature for a server.

Best regards.

Support for Http error handler

Currently, if there is an exception on the channel, there is no way for the HTTP app to handle it.
Typically someone might want to log it and probably close the connection after an exception is thrown.

Example

Server.port(PORT) ++       
  Server.app(app) ++
  Server.error(cause => log.throwable(cause).as(true))

Scalafix error when running HelloWorld example

Hi,
When trying to run the example HelloWorld from an otherwise empty project, sbt will initially crash with error:
ThisBuild / scalafixScalaBinaryVersion from Global / scalafixInterfaceProvider ((scalafix.sbt.ScalafixPlugin.globalSettings) ScalafixPlugin.scala:213)

this can be resolved by creating a project/plugins.sbt with the following line:
addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.9.26")

It would be great if this could be added to the documentation or maybe this implicit dependency can somehow be removed.

Thanks!
Willem

Release a zio-http-test module with test helpers

It would be nice if the existing zhttp.service.HttpRunnableSpec (currenlty in zio-http test directory) was available for users, for example in a zio-http-test module.

So it would possible for users to write tests like :

testM("test response") {
    val path = Root / "api" / "hello"
    for {
     a1 <- assertM(status(path))(equalTo(Status.OK))
     a2 <- assertM (request(path).map(r => responseAsString(r)))(equalTo("Hello world"))
    } yield a1 && a2
}

With responseAsString looking to something like :

def responseAsString(reponse: UHttpResponse): String = {
    reponse.content match {
      case CompleteData(data) => new String(data.toArray)
      case StreamData(data) => data.run(Sink.foldLeft("")(???)) 
      case Empty => ""
    }
  }

Incorrect Content-Length header with multibyte characters with HttpContent.Complete

Hello,

When using HttpContent.Complete with a multibyte character, the calculation for the Content-Length header does not take into account that this is a multibyte character. This is because of the length is calculated with String.length() instead of with String.getBytes.length . Illustration:

"λ".length() // 1
"λ".getBytes.length // 2

With a minimal example:

object Main extends zio.App {
	val app = Http.collect{ case Method.GET -> Root => Response.text("λλ") }
	override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] = Server.start(8080, app).exitCode
}

we can use curl -iv --raw localhost:8080 to see that we get content-length: 2, an "Excess found" of size 2, and only one "λ"

* Excess found in a non pipelined read: excess = 2, size = 2, maxdownload = 2, bytecount = 0
* Connection #0 to host localhost left intact
λ⏎                

This is in the latest release 1.0.0.0-RC15 and on main. Let me know if you want me to put together an MR.

Support for socket life cycle events

Currently, WebSocket API only provides access to the frames.
A new spec needs to be designed to support channel events to detect socket initiation, failure & completion.

SSL Support

Can you please provide SSL support? I would like an option to provide an SSL context when creating the server please.

Make the fields in SocketConfig optional

Current

case class SocketConfig[-R, +E](
  onTimeout: ZIO[R, Nothing, Unit] = ZIO.unit,
  onOpen: Connection => ZStream[R, E, WebSocketFrame] = (_: Connection) => ZStream.empty,
  onMessage: WebSocketFrame => ZStream[R, E, WebSocketFrame] = (_: WebSocketFrame) => ZStream.empty,
  onError: Throwable => ZIO[R, Nothing, Unit] = (_: Throwable) => ZIO.unit,
  onClose: Connection => ZIO[R, Nothing, Unit] = (_: Connection) => ZIO.unit,
  protocolConfig: JWebSocketServerProtocolConfig = SocketConfig.protocolConfigBuilder.build(),
)

Proposed

case class SocketConfig[-R, +E](
  onTimeout: Option[ZIO[R, Nothing, Unit]] = None,
  onOpen: Option[Connection => ZStream[R, E, WebSocketFrame]] = None,
  onMessage: Option[WebSocketFrame => ZStream[R, E, WebSocketFrame]] = None,
  onError: Option[Throwable => ZIO[R, Nothing, Unit]] = None,
  onClose: Option[Connection => ZIO[R, Nothing, Unit]] = None,
  protocolConfig: JWebSocketServerProtocolConfig = SocketConfig.protocolConfigBuilder.build(),
)

Why?

  1. Especially for onError if someone doesn't configure it, all the errors get escaped.
  2. For others, the control unnecessary goes to the ZIO thread pool, does a no-op, and comes back to netty. This is a performance hog.

Make Http response more composable

package zhttp.http

import io.netty.handler.codec.http.{
  DefaultHttpResponse => JDefaultHttpResponse,
  HttpResponse => JHttpResponse,
  HttpVersion => JHttpVersion,
}
import zhttp.http.Status.OK

import scala.annotation.{implicitAmbiguous, implicitNotFound, unused}
import scala.language.implicitConversions

sealed trait HttpResponseBuilder[+S, +A] { self =>
  import HttpResponseBuilder._

  def ++[S1 >: S, S2, S3, A1 >: A, A2, A3](other: HttpResponseBuilder[S2, A2])(implicit
    @unused @implicitNotFound("Status is already set once")
    s: CanCombine[S1, S2, S3],
    @unused @implicitNotFound("Content is already set once")
    a: CanCombine[A1, A2, A3],
  ): HttpResponseBuilder[S3, A3] =
    HttpResponseBuilder.Combine(
      self.asInstanceOf[HttpResponseBuilder[S3, A3]],
      other.asInstanceOf[HttpResponseBuilder[S3, A3]],
    )

  def widen[S1, A1](implicit evS: S <:< S1, evA: A <:< A1): HttpResponseBuilder[S1, A1] =
    self.asInstanceOf[HttpResponseBuilder[S1, A1]]

  private[zhttp] def jHttpResponse[S1 >: S](implicit ev: S1 <:< Status): JHttpResponse = {
    val jResponse: JHttpResponse = new JDefaultHttpResponse(JHttpVersion.HTTP_1_1, OK.toJHttpStatus)

    def loop(response: HttpResponseBuilder[Status, A]): Unit = {
      response match {
        case HttpResponseStatus(status) => jResponse.setStatus(status.toJHttpStatus)
        case HttpResponseHeader(header) => jResponse.headers().set(header.name, header.value)
        case Combine(a, b)              => loop(a); loop(b)
        case _                          => ()
      }
      ()
    }

    loop(self.widen)
    jResponse
  }

  private[zhttp] def content[A1 >: A](implicit @unused ev: HasContent[A1]): A1 = {
    val nullA = null.asInstanceOf[A1]
    def loop(response: HttpResponseBuilder[S, A1]): A1 = {
      response match {
        case Combine(a, b)                =>
          val a0 = loop(a)
          if (a0 == null) loop(b) else a0
        case HttpResponseContent(content) => content
        case _                            => nullA
      }
    }
    loop(self)
  }
}

object HttpResponseBuilder {
  private final case class HttpResponseStatus[S](status: S)   extends HttpResponseBuilder[S, Nothing]
  private final case class HttpResponseHeader(header: Header) extends HttpResponseBuilder[Nothing, Nothing]
  private final case class HttpResponseContent[A](data: A)    extends HttpResponseBuilder[Nothing, A]
  private final case class Combine[S, A](a: HttpResponseBuilder[S, A], b: HttpResponseBuilder[S, A])
      extends HttpResponseBuilder[S, A]

  sealed trait CanCombine[X, Y, A]

  object CanCombine {
    implicit def combineL[A]: CanCombine[A, Nothing, A]                = null
    implicit def combineR[A]: CanCombine[Nothing, A, A]                = null
    implicit def combineNothing: CanCombine[Nothing, Nothing, Nothing] = null
  }

  @implicitNotFound("Response doesn't have status set")
  sealed trait HasStatus[S]
  implicit object HasStatus extends HasStatus[Status]

  @implicitAmbiguous("Response doesn't have status set")
  implicit object HasNoStatus0 extends HasStatus[Nothing]
  implicit object HasNoStatus1 extends HasStatus[Nothing]

  trait HasContent[A]
  object HasContent {
    implicit object NoContent0 extends HasContent[Nothing]
    implicit def hasContent[A]: HasContent[A] = null.asInstanceOf[HasContent[A]]
  }

  implicit def status(status: Status): HttpResponseBuilder[Status, Nothing] =
    HttpResponseStatus(status)

  implicit def header(header: Header): HttpResponseBuilder[Nothing, Nothing] =
    HttpResponseHeader(header)

  implicit def content[R, E](data: HttpData[R, E]): HttpResponseBuilder[Nothing, HttpData[R, E]] =
    HttpResponseContent(data)

  implicit def asResponse[S, A, R, E](self: HttpResponseBuilder[S, A])(implicit
    evS: S =:= Status,
    @unused evStatus: HasStatus[S],
    evA: HasContent[A],
    evD: A =:= HttpData[R, E],
  ): Response[R, E] = Response.FromResponseBuilder(self.widen)
}

Enhance support for cookies

Current
Cookies are supported only as header values which is a string.

Proposed

  • Have first-class support of features such as secure and domain and httpOnly etc.
  • Use an expressive DSL to build and query cookies

Streaming request body for server

It would be really nice if we can get the server incoming request body as a stream of bytes instead of a string. If done properly, this would allow to (among other things):

  • extract all kinds of content from the request instead of only string content-types in HttpContent.Complete
  • provide back pressure to the socket instead of loading all content into memory at once.

In doing so, I would propose merging HttpContent.Complete and HttpContent.Chunked into something like HttpContent.Raw. By adding some utilities on HttpContent.Raw (or on the Request itself) like asText and possible asJSON you have nearly the same ergonomics as with HttpContent.Complete while also providing access to the raw stream of bytes. More content types can then be added such as HttpContent.MultipartFormData in #114 .

On the Netty level, the challenge would be to provide the request body in a ZStream[R, E, Byte]. In a first iteration this can be done like here but it would be nicer if we can let Netty provide backpressure to the socket instead of loading everything in memory first. This can be done by removing the Netty HttpObjectAggregator and writing or own handler in which we process individual HttpContent objects and take control of the channel read config ourselves. Roughly that could look like

// for every bit of `HttpContent` that we receive in the handler do
ctx.channel().config().setAutoRead(false)
// offer content bytes to the ZStream (via a bounded ZQueue of size 1?)
ctx.channel().config().setAutoRead(true)

Let me know what you think. I can create a small prototype in you think it's worth trying.

Using `ByteBuf` instead of `Chunk[Byte]` for exposing bytes of data.

Using ByteBuf is going to be more performant when compared to Chunk[Byte].

And an immutable version of ByteBuf can be implemented as follows —

HBuf.scala

package zhttp.core

import scala.annotation.unused
import zio.Chunk

object HBufExmp {
  trait Nat
  trait Zero                extends Nat
  trait Successor[N <: Nat] extends Nat

  type One = Successor[Zero]
  type Two = Successor[One]

  trait >[A <: Nat, B <: Nat]
  object > {
    implicit def base[A <: Nat]: >[Successor[A], Zero]                                                = new >[Successor[A], Zero] {}
    implicit def other[A <: Nat, B <: Nat](implicit @unused ev: A > B): >[Successor[A], Successor[B]] =
      new >[Successor[A], Successor[B]] {}
  }

  sealed trait Direction
  object Direction {
    case class In()  extends Direction
    case class Out() extends Direction
  }
  import Direction._

  trait Flip[A <: Direction, B <: Direction]
  object Flip {
    implicit object in  extends Flip[In, Out]
    implicit object out extends Flip[Out, In]
  }

  trait HBuf[Count <: Nat, D <: Direction] { self =>
    def retain(implicit ev: Count > Zero): HBuf[Successor[Count], D]                          = ???
    def release[A <: Nat](implicit ev: Count > Zero, ev0: Successor[A] =:= Count): HBuf[A, D] = ???
    def bytes(implicit ev: Successor[Count] =:= Two, ev0: Count > Zero): Chunk[Byte]          = self.retain.read
    def read(implicit ev: Count =:= Two): Chunk[Byte]                                         = ???
    def flip[D0 <: Direction](implicit ev: Flip[D, D0]): HBuf[Count, D0]                      = ???
  }

  object HBuf {
    def fromString[D <: Direction](str: String): HBuf[One, D]       = ???
    def fromChunk[D <: Direction](bytes: Chunk[Byte]): HBuf[One, D] = ???
  }

  trait Request  {
    val content: HBuf[One, In]
  }
  trait Response {
    val content: HBuf[One, Out]
  }

  // Goal
  // - One read should not affect others
  // - Final ref count of the buffer should be 1

  def requestHandler(req: Request): Response = {
    // Async

    val a = req.content.bytes // 1 ~ 2 ~ 1

    val b = req.content.bytes // 1 ~ 3 ~ 3

    val c = req.content.bytes // 1 ~ 4 ~ 2

    println(s"$a $b $c")

    new Response {
      val content: HBuf[One, Out] = req.content.flip
    }
  }

  // writeAndFlush( requestHandler(req))
  // req.release() /// FAIL

}

Getting started
TypeLevel programming and Nat implementation

https://www.youtube.com/watch?v=qwUYqv6lKtQ

Forwarding body from request seems to drop newlines

Edit: forgot to paste the code 😂

With this server code:

import zio._
import zhttp.http._
import zhttp.service.Server

object Main extends App {
  val echo = Http.collect[Request] { case req @ Method.POST -> Root / "echo" =>
    Response.http(content = req.data.content)
  }

  override def run(args: List[String]): URIO[ZEnv, ExitCode] =
    Server.start(8090, echo).exitCode
}

Calling the endpoint with a file that contains newlines seems to drop them from the output:

> cat input.txt
hello
world, and
more world

> curl localhost:8090/echo --data @input.txt -v
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8090 (#0)
> POST /echo HTTP/1.1
> Host: localhost:8090
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 25
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 25 out of 25 bytes
< HTTP/1.1 200 OK
< content-length: 25
<
* Connection #0 to host localhost left intact
helloworld, andmore world* Closing connection 0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.