marblejs / marble Goto Github PK

Marble.js v3 is right around the corner. Before its official release I would like to introduce the incoming new features and potential API breaking changes. This is a place for general discussion about all the changes, their relevance and potential impact to your codebase. Feel free to ask questions and propose improvements. 😊

General overview of new features:

• official support for TypeScript v3.7
• official support for RxJS v6.5
• official support for fp-ts v2.x
• simplified dependency injection via new useContext hook
• introducing new module @marblejs/messaging for building Microservices
  • for MVP version with support for AMQP (RabbitMQ) and Redis transport layers
  • I expect more transport layers to be introduced after release
  • I'll post another issue when needed (or expect a surprise 😎🤪)

Incoming breaking changes::

Context API

[email protected] brought a major breaking change in it's API (see changelog). It introduced changes that have a major impact to Context API (eg. Reader monad). What's new?

More explicit dependency binding. Previous API wasn't so precise, which could result to confusion, eg. when the dependency is lazily/eagerly evaluated.

Old way:

// eager
bindTo(WsServerToken)(websocketsServer.run),

// lazy
bindTo(WsServerToken)(websocketsServer),

New way:

// eager
bindEagerlyTo(WsServerToken)(websocketsServer),

// lazy
bindTo(WsServerToken)(websocketsServer),
bindLazilyTo(WsServerToken)(websocketsServer),

Reader creation:

Old way:

import { reader } from '@marblejs/core';

const someService = reader.map(ctx => {
  // ...
});

New way:

import { reader } from '@marblejs/core';
import { pipe } from 'fp-ts/lib/pipeable';
import { map } from 'fp-ts/lib/Reader';

const someService = pipe(reader, map(ctx => {
  // ...
}));

The release of fp-ts also had an impact to HTTP and WebSocket server creators. Since the run() method on Reader, etc. has been replaced with a thunk, server creation also applied to this change. Bootstrap thunks are promisified, which means that they will return an instance only when started listening, if not then will throw an error.

Old way:

const server = createServer({
  // ...
});

server.run();

New way:

const server = createServer({
  // ...
});

await server();

Effect interface changes:

Currently Effect interface defines three arguments where the second one is used for accessing contextual client, eg. HttpResponse, WebSocketClient, etc. Typicaly the second argument was not used very often. That's why in the next major version client parameter will be moved to context object which will result to reduced available number of parameters from 3 to 2:

Old way:

const foo$: WsEffect = (event$, client, meta) =>
  event$.pipe(
    matchEvent('FOO'),
    // meta.ask       ---    context reader
  );

New way:

const foo$: WsEffect = (event$, ctx) =>
  event$.pipe(
    matchEvent('FOO'),
    // ctx.client    ---    contextual client
    // ctx.ask       ---    context reader
  );

This change also implies a much cleaner base Effect interface:

interface Effect<I, O, Client> {
  (input$: Observable<I>, ctx: EffectContext<Client>): Observable<O>;
}

interface EffectContext<T, U extends SchedulerLike = SchedulerLike> {
  ask: ContextProvider;
  scheduler: U;
  client: T;
}

With that change the last argument of Effect interface is no more called as EffectMetadata but rather as EffectContext

When dealing with error or output Effect, the developer had to use the attribute placed in the third effect argument, eg. const effect = (req$, client, { initiator, error }) => ... In the case of ErrorEffect the thrown error is passed to stream directly:

const error$: HttpErrorEffect<HttpError> = req$ =>
  req$.pipe(
    map(({ req, error }) => {
      // ...
    }),
  );

In the case of OutputEffect the message initiator (eg. initial request) is passed to stream directly:

const output$: HttpOutputEffect = out$ =>
  res$.pipe(
    map(({ req, res }) => {
      // ...
    }),
  );

Nightly builds

If you would like to take part in beta testing feel free to try out canary releases.
I'll try to inform here about new builds and changes that they introduce.

CC @tstelzer

Describe the bug
When using validators with module @marblejs/middleware-io and then call the REST service I have recieved server internal error. The error message is below.

{
"error":{
"status": 500,
"message": "testing_1.isTestingMetadataOn is not a function"
}

To Reproduce

Error when use validator

import { HttpEffect, use, HttpError, HttpStatus } from '@marblejs/core';
import { map,  mergeMap, catchError } from 'rxjs/operators';
import { requestValidator$, t } from '@marblejs/middleware-io';
import { of, throwError } from 'rxjs';
import { getCustomRepository } from 'typeorm';
import { UserRepository } from '@api/user/dao/user.repository';

const emailValidator$ = requestValidator$({
  body: t.type({
    email: t.string
  })
});

export const findUserByEmailEffect$: HttpEffect = req$ =>
  req$.pipe(  
    use(emailValidator$),
    mergeMap(req => of(req.body.email).pipe(
      mergeMap(getCustomRepository(UserRepository).findUserByEmail),
      map(user => ({body: user}),
      catchError(() => throwError(
        new HttpError("Could not found user", HttpStatus.NOT_FOUND)
      ))
      )
    ))
  )

Without validator everything work fine but the body is uknown which is not also useful.

import { HttpEffect, HttpError, HttpStatus } from '@marblejs/core';
import { map,  mergeMap, catchError } from 'rxjs/operators';
// import { requestValidator$, t } from '@marblejs/middleware-io';
import { of, throwError } from 'rxjs';
// import { getCustomRepository } from 'typeorm';
// import { UserRepository } from '@api/user/dao/user.repository';

// const emailValidator$ = requestValidator$({
//   body: t.type({
//     email: t.string
//   })
// });

export const findUserByEmailEffect$: HttpEffect = req$ =>
  req$.pipe(  
    // use(emailValidator$),
    mergeMap(req => of(req.body).pipe(
      // mergeMap(getCustomRepository(UserRepository).findUserByEmail),
      map(user => ({body: user}),
      catchError(() => throwError(
        new HttpError("Could not found user", HttpStatus.NOT_FOUND)
      ))
      )
    ))
  )

Expected behavior
I would expect to be able get value from the database when the validator successfully validate the request body.

Desktop (please complete the following information):

• OS - Windows 10
• Package + Version: @marblejs/middleware-io + latest
• Node version: v10.15.3

I'm trying to serve static files with Marble, but it does not seem possible because of a JSON.stringify() applied to every outgoing response:
https://github.com/marblejs/marble/blob/master/packages/core/src/response/response.handler.ts

It would make sense (simplest solution I could think) to stringify objects only, not strings.

Also, taking into account headers provided in the EffectResponse object could be of value.

Hey guys 👋

There are few typos in the docs mathPath vs matchPath:
https://marblejs.gitbook.io/marble/?q=mathPath

Everyone loves math though.

Problem: When using use(multipart$()), I can't seem to expand the Observable with additional values beyond the multipart upload. For example:

const a$ = r.pipe(
    r.matchPath('a'),
    r.matchType('POST'),
    r.useEffect(Rx.pipe(
        use(multipart$()),
        // Expanding the Observables with additional values.
        Ro.combineLatest(Rx.from([1, 2, 3])),
        // Seems to only log once, apparently the last value in above array.
        Ro.tap(([upload, n]) => console.log(n)),
        Ro.count(),
        // Responds with 1. Should be 3.
        Ro.map(body => ({status: 200, body})),
    )),
)

I created a repository to reproduce the issue: https://github.com/tstelzer/marble-playground

node: v13.2.0
"@marblejs/core": "^3.0.0-next.78",
"@marblejs/middleware-body": "^3.0.0-next.78",
"@marblejs/middleware-cors": "^3.0.0-next.78",
"@marblejs/middleware-io": "^3.0.0-next.78",
"@marblejs/middleware-logger": "^3.0.0-next.78",
"@marblejs/middleware-multipart": "^3.0.0-next.78",
"rxjs": "^6.5.3"

Overview

The current official package/modules ecosystem consist of:

• @marblejs/core
• @marblejs/messaging
• @marblejs/websockets
• @marblejs/testing
• @marblejs/middleware-logger
• @marblejs/middleware-io
• @marblejs/middleware-joi
• @marblejs/middleware-jwt
• @marblejs/middleware-body
• @marblejs/middleware-cors
• @marblejs/middleware-multipart

Problem statement

1. Main framework monorepo contains deprecated packages (eg. @marblejs/middleware-joi) that are no maintained anymore - their versions are automatically bumped-up from release to release. The Joi validation middleware was the first community driven package that was used for HTTP request validation. Due to the difficulties in proper type inference, it was deprecated with the introduction of version v2.0, when io-ts-based middleware was released.
2. Community doesn't have a dedicated environment/space where they can propose new non-core modules and easily contribute to the ecosystem, eg. like fp-ts-contrib does.
3. @marblejs/core API layer is too big. It contains a core/common abstractions that are used across other modules, like: messaging, testing, websockets, middlewares mixing core functionalities with HTTP protocol context. With an introduction of messaging module HTTP protocol is not the only one available transport layer that Marble.js apps can be built on.

Proposed solution

1. New, separate monorepo marblejs/contrib (?) for community driven packages that are out of scope core modules. First candidates for migration: @marblejs/middleware-joi and maybe @marblejs/middleware-jwt.
2. @marblejs/core package should be split into two parts: reduced API layer for @marblejs/core and brand new @marblejs/http package. All existing HTTP related API would be moved to the new scope, where the core package will include all basic and common interfaces and API's that are used across all dependent modules: testing, websockets, messaging and new http.
3. The new package layer reorganization can be introduced with the release of version v4.0

Things to consider

• Some middlewares can be baked into main modules, eg. cors, multipart, body or logger can be scoped and baked into the future @marblejs/http module. The only problem that I foresee is the maintenance experience/cost - it will be hard to maintain so many things in one place. Maybe separation into dedicated packages is still valid and good path. 🤔

Migration path

• It is not so drastic braking change
  • joi-based middleware due to it's deprecation should not be used so widely,
  • the change mostly affects imports:

Instead of:

import { r, HttpStatus, useContext, combineRoutes } from '@marblejs/core';

we will have:

import { useContext } from '@marblejs/core';
import { r, HttpStatus, combineRoutes } from '@marblejs/http';

Hello guys,
thank you so much for the hard work creating a gorgeous framework like marblejs.
I would like to know if exists a potentially release date and if the test package will come with the new release.
Also will the examples repository be updated?

Thank you again

Problem statement

Marble.js defines an Effect as follows:

export interface Effect<I, O, Client> {
  (input$: Observable<I>, ctx: EffectContext<Client>): Observable<O>;
}

const foo:$ HttpEffect = (req$, ctx) =>
  req$.pipe(
    mergeMap(compute),
    map(body => ({ status: HttpStatus.OK, body }),
  );

The result of effect function returns some computation on Observable<HttpRequest> stream. The current implementation of HTTP request processing is not continuous. #181

Internally the HTTP request processing stream is broken into something similar to this (for demo purpose it is presented in a very simplified form):

// 1. get Tuple [req, res]
// 2. parse request
// 3. find the route in routing table
// 4. put the request to the stream -> apply middlewares -> apply effect
// 5. send the output to the client


// request stream processing...

req$.pipe(
  mergeMap(req => effect(of(req), ctx).pipe(
    tap(output => /* corelate output with `req` */),
    catchError(error => /* catch the error and tie it to `req` */),
  ),
);

Every time the req is passing through the stream the effect function is evaluated.

Target solution

Ideally eachEffect should be bootstrapped eagerly, which means that the route should be initialized with a subject (common or dedicated per route - doesn't matter right now). When the effect stream throws an error it should be caught underneath in order to tie it to the initial request.

req$.pipe(
  publish(req$ => effect(req$, ctx).pipe(
    // correlate output with request
    // catch the error and correlate it with request
  ),
);

And the effect implementation:

const foo$: HttpEffect = (req$, ctx) => {
  // ...
  // eager computation, context injection, etc.

  return req$.pipe(
    mergeMap(...),
    map(...),
    map(body => ({ status: HttpStatus.OK, body })),
  );
}

The problem is that the effect which produces requests doesn't guarantee the order, so we cannot zip streams together.

req$   ---a1---b1-------
👇
effect
👇
out$   -----b2---a2----

Requirements

• avoid an explicit error catching via catchError operator in effect stream since it is very repetitive. Each route should be protected with this kind of error handling underneath.
• preserve the continuity of the stream
• each effect should be bootstrapped eagerly - it shouldn't evaluate every time

The API that I would like to avoid:

const foo$: HttpEffect = (req$, ctx) =>
  req$.pipe(
    mergeMap(req => of(req).pipe(
      // request processing
      map(output => /* correlate `req` with produced output */),
      catchError(error => /* catch the error and tie it to `req` */),
    )),
  );

In my development environment, I'm adding CORS headers to my API responses. But my API endpoints are combined :
combineRoutes('/api', effects)

I would like a way to use my CORS middleware on all API effects, without having to call use() on each of them. I was thinking of something similar to httpListener's signature :
combineRoutes('/api', { middlewares, effects })

Hello
I really love marblejs!! I want to use it on my project and also contribute it if I can.
BTW is there a talk channel for marblejs? like spectrum, gitter, discord

I wanna compress response(gzip) like express-compression.
Middleware looks like only update req but not res

// effects.combiner.ts
export const combineMiddlewares = <T, U>
  (...effects: Effect<T, T, U>[]) =>
  (input$: Observable<T>, client: U, meta: EffectMetadata): Observable<T> =>
    effects.reduce((i$, effect) => effect(i$, client, meta!), input$);

So user can not add compress feature by add a compression middleware.

I’m a big fan of marblejs - I’ve been following it closely since the early releases and I love the simplicity of its design.

Has anyone proposed that marblejs expand to include messaging capabilities (i.e. NATS, grpc, SQS, etc.). The fact that it is lightweight and simple makes it a great candidate for microservices that rely on messaging. I think NATS in particular could be a good fit.

I have used fp-ts in my project and that has a version of ^2.0.1 and I have used @marblejs/middleware-logger to log my requests. when I request my server that exited with the error below.

TypeError: Option_1.fromNullable(...).map is not a function
in the file, /node_modules/@marblejs/middleware-logger/dist/logger.util.js:8:10)

Then, I inspected your code, that package also using fp-ts in the old style (.map, .getOrElse).
I don't know, how to debug this.

And, I hesitated to use other middlewares. Because of other package also use that lib in the same manner.

Visit https://marblejs.gitbook.io/marble/available-middlewares/logger#usage

Missing $ from the const assignments in both examples

Examples

Expected

export const logger$ = loggerWithOpts$(); 

Actual

export const logger$ = loggerWithOpts(); // add missing $ from loggerWithOpts

Describe the bug
I send post request from client:

  const sendDataToServer = () =>
    fetch('/long-answer', {
      method: 'post',
      headers: {
        'Content-Type': 'text/plain;charset=utf-8',
      },
      body: `is body exist?`
    })
      .then(response => response.text())
      .then(body => console.log(`body:`, body))
      .catch(e => console.error(`fetch`, e))

and on the server side I get body as undefined:

export const api_long_answer$ = r.pipe(
  r.matchPath('/long-answer'),
  r.matchType('POST'),
  r.useEffect(req$ => req$.pipe(
    tap(({method, url, body}) => console.log(`post req`, {method, url, body})),
    delay(3_000),
    map(({body}) => ({
      headers: {
        'Content-Type': 'text/plain;charset=utf-8'
      },
      body
    })),
  ))
);

To Reproduce

1. https://github.com/MyTempGitForCommunity/marble-body-undefined
2. run script "server:run"
3. open in browser http://127.0.0.1:1337/
4. push the button 'Send Data To Server'
5. view the log in the server console. In my case, I see the following:
   post req { method: 'POST', url: '/long-answer', body: undefined }

Expected behavior
In theory if I understood everything correctly the body should not be equal to undefined.

Desktop (please complete the following information):

• OS: Win10, Mac
• Package + Version: @marblejs v.3
• Node version: v12.14.0

For example, is there a way to implement a proxy via middleware?

Hi 👋
first of all, thanks for providing this awesome package - I am having great time using it. Unfortunately I ran into an issue, that I believe is not on my side (however I am happy to be proven wrong, or pointed to a valid solution).

Describe the bug
When closing connections of two clients that are connected and notified of each other state via a common channel (e.g. a Subject), an Error: WebSocket is not open: readyState 2 (CLOSING) is thrown. Happens only when clients are closed immediately after another (e.g. via a script).

Full stack trace:

Error: WebSocket is not open: readyState 2 (CLOSING)
    at WebSocket.send (/*path to the project*//node_modules/@marblejs/websockets/node_modules/ws/lib/websocket.js:322:19)
    at WebSocket.sendResponse (/*path to the project*//node_modules/@marblejs/websockets/dist/response/websocket.response.handler.js:7:12)
    at SafeSubscriber.input$.pipe.subscribe.type [as _next] (/*path to the project*//node_modules/@marblejs/websockets/dist/server/websocket.server.listener.js:64:42)
    at SafeSubscriber.__tryOrUnsub (/*path to the project*//node_modules/rxjs/src/internal/Subscriber.ts:265:10)
    at SafeSubscriber.next (/*path to the project*//node_modules/rxjs/src/internal/Subscriber.ts:207:14)
    at Subscriber._next (/*path to the project*//node_modules/rxjs/src/internal/Subscriber.ts:139:22)
    at Subscriber.next (/*path to the project*//node_modules/rxjs/src/internal/Subscriber.ts:99:12)
    at TakeUntilSubscriber.Subscriber._next (/*path to the project*//node_modules/rxjs/src/internal/Subscriber.ts:139:22)
    at TakeUntilSubscriber.Subscriber.next (/*path to the project*//node_modules/rxjs/src/internal/Subscriber.ts:99:12)
    at CatchSubscriber.Subscriber._next (/*path to the project*//node_modules/rxjs/src/internal/Subscriber.ts:139:22)

To Reproduce

1. Link two clients using a common channel.
2. Using the finalize operator notify each user about another user being disconnected
3. Open the connections
4. Close the connections directly one after another
   (5.) First client that happens to be disconnected causes a message to be pushed to the second client, that also disconnected in the meantime.
   (6.) Message is being pushed, even though the websocket is in CLOSING state

Minimal epic causing the error:

class UserPoolPubSub {
    private readonly pubSub = new Subject<Message>();
    public join(id: string): Observable<Message> {
        this.publish(id, 'Hello there!');
        return this.pubSub.pipe(filter(({ publisher }) => publisher !== id));
    }
    public publish(publisher: string, message: string): void {
        this.pubSub.next({ publisher, message });
    }
}

const pool = new UserPoolPubSub();

const join$: WsEffect = (event$, { client: { id } }) => {
    const leave$ = event$.pipe(matchEvent('LEAVE'));

    return event$.pipe(
        matchEvent('JOIN'),
        exhaustMap(() =>
            pool.join(id).pipe(
                map(({ message }) => ({ type: 'RECEIVED', payload: message })),
                takeUntil(leave$),
            ),
        ),
        finalize(() => pool.publish(id, 'I was disconnected, sorry')),
    );
};

Minimal client interaction:

function run() {
// after opening the ws connections and exchanging messages
  clientA.close();
  clientB.close();
}

I published a runnable reproduction here: https://github.com/GrzegorzKazana/marblejs-ws-issue-repro.
It can be run by

npm i
npm run dev
# in separate terminal
npm run client

Expected behavior
Server should not try to push the event to user that is already closing.

Desktop (please complete the following information):

OS: MacOS 10.14.6
@marblejs/core: 3.4.8
@marblejs/websockets: 3.4.8
Node version: 12.18.0, also happens on 14.15.1

Additional context
I run into this issue when doing integration tests for my app, that uses @marblejs/websocket for its backend. When the test runner was halting on a failed test, it caused closing all connections (2 in my case) at the same time - this in turned was causing crash of my backend service.

I hope I did not miss any obvious error on my side, but I will be happy to be corrected or pointed to a mistake.

Thanks in advance!

Describe the bug
Following the Marble docs for creating an amqp publisher I receive the following error on app start-up
(node:90349) UnhandledPromiseRejectionWarning: TypeError: rabbitMqClient.send is not a function
because the resolved rabbitMQ client is still a Promise even though it was eagerly bounded to the context

To Reproduce
Amqp publisher

export const AmqpClientToken = createContextToken<MessagingClient>('MessagingClient');
export const amqpClient = messagingClient({
    transport: Transport.AMQP,
    options: {
        host: 'amqp://localhost:5672',
        queue: 'hello_queue'
    },
});

Http effect

export const getRoot$ = r.pipe(
    r.matchPath('/'),
    r.matchType('GET'),
    r.useEffect((req$, ctx) => {
        const rabbitMqClient = useContext(AmqpClientToken)(ctx.ask);

        return req$.pipe(
            mergeMapTo(rabbitMqClient.send({ type: 'HELLO', payload: 'John' })),
            mapTo({ status: HttpStatus.ACCEPTED }),
        );
    }),
);

App

const httpServerListener = httpListener({
    middlewares: [
        logger$({ silent: isTestEnv() }),
        bodyParser$(),
    ],
    effects: [
        getRoot$
    ],
});

export const server = createServer({
    port: getPortEnv(),
    listener: httpServerListener,
    dependencies: [
        bindEagerlyTo(AmqpClientToken)(amqpClient),
    ],
});

export const main: IO.IO<void> = async () =>
    await (await server)();

Expected behavior
App starts up.

Desktop (please complete the following information):

• OS: Ubuntu
• Package + Version: Marble ^3.4.9
• Node version: 14.15.4

Additional context

Problem statement:
Consider an example to build utility to print all the API effects/routes at the time of starting a server. This problem cannot be solved by middlewares. Also, Marble.js provides no hooks/events to let application developer know about the state of the server.

Solution
Similar to how we have ErrorEffect, provide one more type of effect for common events. These server effects would be streams like other effects. They will be used by an application developer to handle the required scenarios.

Current alternative
Since there is no abstraction on top of createServer, it leaves the code organization to the application developer. The developer will manually add callbacks or listen for various server events.

Notes:
If we consider Marble.js as a framework, then it makes sense to provide developers with these hooks as framework essentially implies Inversion of Control (IOC). If the scope of Marble.js is more confined an express-like or a library-like feature-set, then it probably doesn't make sense.

Hi, guys ✋
I want to implement json-schema-validator for marblejs.

Why

Have written json-schema validation middleware for my pet project, and I wish to contribute to marblejs and thing that the solution can be useful for the marblejs community.

Why Joi middleware is not enough?
Joi looks pretty good but I need to generate swagger specs for my API. So use the same schema for validation and documentation is a good idea in my opinion.

Internal realization
In my project I use typescript-json-schema for generation schema from typescript interfaces. So I have custom logic for schema loading and not sure that such solution will be good for marble.

Implementation proposal

I propose one of three ways implementation json-schema validator for marble.

Use typescript-json-schema API for generating schema from typescript types or interfaces

The way is fully based on typescript interfaces or types. So we have a simple described type and generate JSON schema in runtime (when application start). Then we keep generated schemas in memory and use for validation or/and serving swagger specs (if we need).

Prebuild schemas for validation

This way is similar to previous, but instead compiling JSON schema on the fly we will build schemas before and just read the schemas from files.

Pass JSON schema manually

This way is simplest from this list. Just pass JSON schema as a simple object (and marble users should implement source yourself). It will like as Joi middleware, but instead passing Joi builder we will pass JSON schema object.

I think last way it preferred for marble, such as not all users need the functionality for generation schemas and serving specs.

Questions

1. What you guys think about implementing such middleware?
2. If you approve implementing the middleware which way you prefer?
3. Should I clarify something from the description above?

Is your feature request related to a problem? Please describe.

Hello,

I noticed that Marble is inspired by the Angular ecosystem. In my everyday developer experience I really appreciate what the CLI tools bring to me. It helps to quickly scaffold main building blocks and enforce consistency.

It could be really nice to have CLI tools for Marble.

Describe the solution you'd like

The @angular/dev-kit and schematics can be a good fit to provide CLI functionality.

Describe alternatives you've considered

After digging in other CLI tools it appeared to me that there is no strong alternative to the @angular/dev-kit package.

Problem statement:

I intend to build cookie parser middleware similar to what Hapi or Express has. Additionally, provide helper functions to manage cookie lifecycle.

Solution:

Originally, I intended to build cookie parser as part of marble-mix. But after some thought, I believe cookie-parsing should be part of official marblejs. I also doubt if monorepo is the right thing to do with cookies middleware as it doesn't really form the crux of marble. Additionally, this middleware will not be changing once stabilized. Seems no point in bumping version each time with Lerna.

Implementation:

1. Build on top of proven cookie and cookie-signature or cookies.
2. Decorate request object with state which will basically be the Record<string, string>. Or it could be the ES Proxy object for some enhanced behavior.
3. Provide additional helpers to deal with setting cookies as part of the response. - The idea is not fully clear right now. But it will be similar to Hapi.js way of handling things.

Questions:

1. What are your thought about this middleware?
2. If it should be part of an official marble ecosystem, should it be part of this monorepo or separate repo?
3. Will you accept PR for this middleware?
4. Also, does the concept of response middleware exists with marblejs?
5. If I am going to decorate incoming request object with state, how should I do it? Just add additional key? Same question as stackoverflow question.

First, thanks much for this neat stuff! Being big fan of reactiveness I found myself at home writing these effects and middlewares.

Can I please ask for adding 422 error code to HttpStatus enum? REST how-tos advise using it when server validation failed. Not suffering much, but would be great to have!

When I have two effects, matching:
/user/search
/user/:id

Both effects get applied, resulting in:
Error: Can't set headers after they are sent.

It would make sense to have only one effect applied, intuitively that would be the first one in effects array.

Describe the bug
Im triying to create some test to a project with jest and im using marblejs for building the server.
The point is that there is no problem when im running the code, but when I try to execute any file with jest which require the @marblejs/core module or any file that internally imports it, its returns the following error:

Cannot find module 'rxjs/internal/scheduler/AsyncScheduler' from 'node_modules/@marblejs/core/dist/effects/effectsContext.factory.js'

    Require stack:
      node_modules/@marblejs/core/dist/effects/effectsContext.factory.js
      node_modules/@marblejs/core/dist/http/server/http.server.js
      node_modules/@marblejs/core/dist/index.js
      utils.helper.js
      utiles.test.js

Im not sure if this is a marblejs bug or maybe a jest bug, but i will appreciate any help to discover it.

To Reproduce
I have create a stackblitx for it, just run jest command
https://stackblitz.com/edit/node-sbtfbd?file=utils.test.js

Expected behavior
The test fails because my own mistake :)

Desktop (please complete the following information):

• Windows 10
• MarbleJS: 3.5.0
• Jest: 27.0.6
• Node: 14.17.1

Can I use regex in the marble routes?

I know that are some especial chars that I can use like:

EffectFactory
  .matchPath('/:dir*')

But what about more complex regex?

EffectFactory
  .matchPath(/^\/(api|rest)\/.+$/)

Describe the bug
It seems that rxjs version 7 breaks something inside marble.

To Reproduce

yarn add @marblejs/core fp-ts rxjs

Run the example as shown on the website

Desktop (please complete the following information):

• MacOS 11.3
• Node 16.0

The following error is thrown:

Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './internal/scheduler/AsyncScheduler' is not defined by "exports" in 'node_modules/rxjs/package.json'

hi @JozefFlakus, I am suggesting to have GitBook website for marble documentation.

Is your feature request related to a problem? Please describe.
Right now HttpRequest is can be assigned any property. Thanks to that you can pass custom properties down the stream as middleware-jwt does.

However, custom properties are of type any. I can't see a way to make them type-safe. Am I missing something?

Describe the solution you'd like
I'd like to pass custom properties in a type-safe manner as it is done with params, query, and body.

Describe alternatives you've considered
I tried to extend the HttpRequest type and make my middleware return the extended version, but with no success.

Additional context

Problem statement

createEvent function is a handy event constructor which unfortunately comes with its own limitations and drawbacks.

1. constructor is a way too verbose - you have to define a list of arguments first and then map to the corresponding attributes

import { createEvent } from '@marblejs/core';

const foo = createEvent(
  'FOO',
  (bar: string, baz: boolean) => ({ bar, baz }),
);

2. By passing an variadic number of arguments the implementation allows to create non-unary event creators

3. While calling the constructor, parameters names are not preserved.

// foo(arg1: string, arg2: boolean)
   foo(...
   foo('bar', false);

4. Inferred type definition is a bit messy

typeof foo === {
  type: "FOO"
} & ((arg1: string, arg2: boolean) => {
    type: "FOO";
    payload: {
        bar: string;
        baz: boolean;
    };
    metadata?: EventMetadata | undefined;
})

5. In order to parse (validate) the I/O event you have to create a separate schema (eg. io-ts schema type) which has to be in sync with an event creator.

6. The implementation is dirty 🤮

// No args
export function createEvent<T extends string, Payload>(
  type: T, creator?: () => Payload,
): { type: T } & (() => { type: T; payload: Payload; metadata?: EventMetadata });

// 1 arg, 1 maybe
export function createEvent<T extends string, Payload, Arg1>(
  type: T, creator: (arg1?: Arg1) => Payload,
): { type: T } & ((arg1?: Arg1) => { type: T; payload: Payload; metadata?: EventMetadata });
// 1 arg
export function createEvent<T extends string, Payload, Arg1>(
  type: T, creator: (arg1: Arg1) => Payload,
): { type: T } & ((arg1: Arg1) => { type: T; payload: Payload; metadata?: EventMetadata });

...

Proposed solution

• Proposed implementation is much more simpler and can be found here: #272
• Much simpler and cleaner API
• event function creates an io-ts compatible codec for decoding and creating I/O events
• Expose an additional event.create function for building the event
• One place for defining both event schema and creator
• Event type can also be inferred from matchEvent operator

Usage:

import { event } from '@marblejs/core';

const FooEvent = event('FOO')(t.type({
  bar: t.string,
  baz: t.boolean,
}));

const fooEvent = FooEvent.create({ 
  bar: 'bar',
  baz: false,
});

typeof FooEvent === EventSchemaWithPayload<'FOO', t.TypeC<{
    bar: t.StringC;
    baz: t.BooleanC;
}>> & {
    create: EventCreatorWithPayload<'FOO', t.TypeC<...>>;
}

typeof fooEvent === EventWithPayload<{
 bar: string;
 baz: boolean;
}, OfferCommandType>

export const foo$: MsgEffect = event$ =>
  event$.pipe(
    matchEvent(FooEvent),
    // {
    //   type: 'FOO';
    //   payload: {
    //     bar: string;
    //     baz: boolean;
    //   };
    // } & {
    //   metadata: EventMetadata;
    // }
    act(event => ...),
  );

Things to consider

• The different function naming should be taken into consideration.

Migration steps

1. event I/O decoder won't take place of createEvent function - their purposes are different
2. matchEvent operator has to be adapted to support new event creator interface - done in #272

Based on the tests and documentation it seems like this should work, but I'm getting a 404.

What am I missing?

const findTag$: Effect = request$ =>
    request$
      .pipe(
        matchPath('/:id'),
        matchType('GET'),
        map(req => req.params.id),
        switchMap(FakeService.findTag),
        map(response => ({ body: response }))
      );

Great library by the way - nice and lightweight, composable and readable.

Describe the bug
I am using marblejs in one of my applications and encountered a weird behaviour if I send a query looking like this GET 127.0.0.1:[email protected].
Turns out that it breaks because of the .com at the end of the query.
I would like to consume a JSON file there, but somehow it is always interpreted with an application/x-msdownload header and results my application to return a 500 HTTP error.
I used your "hello world" example to reproduce it.

To Reproduce
Setup the "hello world" example.
Do a request like this GET 127.0.0.1:[email protected].
Receive a download file called download.exe

Expected behavior
Do not interpret the request as a download request.

Desktop (please complete the following information):

• OS: macOS Mojave Version 10.14.4
• [email protected]
• Node version 8.15.10

I think it will be good if user can receive request using '*' symbol.
As I noticed you use "path-to-regexp" library, If it's good feature and you want to have this I can help you, just I need to know should I use "path-to-regexp" or something other?

Thanks

Is your feature request related to a problem? Please describe.
I have read the description to this project and I like the idea of functional programming related to Web development. The point I didn't get is that you call HTTP single-event based. HTTP has a set of declared endpoints with a declared processing logic.
I researched a bit about RxJS and found out that creating observable every time you want to process an event is WAY MORE expensive than using the same observable and pushing new events to it.
I have also used a debugger a bit with Marble.js and saw that it really creates a new observable for each request. The nature of EffectFactory about that.

Describe the solution you'd like
I think if marble used effectFactory callback upon application startup to create a processing pipe Once instead of calling it on each request, it would be much more performant.
Maybe you'll need some extra RxJS operators to perform routing, but I can't tell for sure (didn't dive deep to marble.js sources).

Additional context
Maybe there are there fundamental problems about it?
If not, I'd like to make some help with this subject (maybe make a version for this).

Describe the bug
I'm using fp-ts-rxjs bindings https://github.com/gcanti/fp-ts-rxjs inside a marble effect using the fp-ts pipe.

export const hasuraActions$ = r.pipe(
    r.matchPath('/keycloak'),
    r.matchType('POST'),
    r.useEffect((req$) => pipe(
        req$,
        requestValidator$({body: HasuraKeycloakAction}),
        OB.map(() => ({
            body: {
                name: 'Tester McTesterson',
                email: '[email protected]'
            }
        })),
        )
    ));

When I call the endpoint with an invalid body I get a validation error as expected but when I call it with a valid body I get the marble error bellow and my whole server crashes:

setTimeout(function () { throw err; }, 0);
^

[
  @marblejs/core error:

  🚨  An effect returned a response: "{"body":{"name":"Tester McTesterson","email":"[email protected]"}}" without bound request

  ] {
  name: 'CoreError'
}

If i change up the code to match the docs and use the standard rxjs pipe operator it does whats expected

export const hasuraActions$ = r.pipe(
    r.matchPath('/keycloak'),
    r.matchType('POST'),
    r.useEffect((req$) => req$.pipe(
        requestValidator$({body: HasuraKeycloakAction}),
        map(_ => ({
            body: {
                name: 'Tester McTesterson',
                email: '[email protected]'
            }
        }))
        )
    ));

There clearly somthing I'm not understanding about what's happing in the requestValidator$ as all the types seem to line up with my original example but still cause the error.

Any insight anyone can offer would be great?

I don't know if this is the correct place to ask a question, is there a better place for discussion on marblejs e.g. slack?

Expected behavior
To match the behaviour when used within the rxjs pipe.

Hello 👋 a big thanks for this library, RxJS as http server is really awesome 😍

Describe the bug

I got a typing issue with @marblejs/core on my typescript project,
I created a listener (httpListener()) and created a http server (http.createServer(app)) but got a typing error :

Argument of type '(req: HttpRequest, res: HttpResponse) => void' is not assignable to parameter of type '((request: IncomingMessage, response: ServerResponse) => void) | undefined'.
  Type '(req: HttpRequest, res: HttpResponse) => void' is not assignable to type '(request: IncomingMessage, response: ServerResponse) => void'.
    Types of parameters 'req' and 'request' are incompatible.
      Type 'IncomingMessage' is not assignable to type 'HttpRequest'.
        Property 'matchPath' is missing in type 'IncomingMessage'.

createServer(app).listen(

This error is thrown only when strictFunctionTypes is activated on my compiler config

{
  "compilerOptions": {
    "strictFunctionTypes": true
  }
}

To Reproduce

I create a branch on my project to reproduce the issue (https://github.com/zap4you/server/tree/marblejs-issue)

$ git clone [email protected]:zap4you/server.git marblejs-issue
$ cd marblejs-issue
$ git checkout marbejs-issue
$ npm i
$ npm run build

Expected behavior
I'm not sure but the function that httpListener returns should be more strictly typed to match with http.createServer expectation ?!.

Desktop (please complete the following information):

 - macOs 10.13.4
 - [email protected] / [email protected]
 - [email protected]
 - @marblejs/[email protected]
 - @types/[email protected]

Hi, i've discovered another minor issue.

Describe the bug
define content-type: text/html as response header results in duplicate content type header:

To Reproduce
create a route that returns a html response like

  return ({
      headers: {
          ...filterProxyRequestHeaders(response.headers),
          "content-type": "text/html"
      },
      body: bodyStream
  });

Expected behavior
response contains only "content-type": "text/html" as content type header.

Desktop (please complete the following information):

• OS => MacOS BigSur
• Package + Version

   "@marblejs/core": "^3.4.8",
   "@marblejs/middleware-body": "^3.4.8",
   "@marblejs/middleware-logger": "^3.4.8",

• Node version => v14.12.0

Additional context
i found this one as a short explainer that the headers in http protocol are case-insensitive: https://stackoverflow.com/a/5259004

My use case for a bit more context: I use undici a nodejs http client: https://github.com/nodejs/undici
All response headers are in lower case letters and my plan is to forward them, as my marble server acts like a proxy in this case.

Workaround
For now i can map the content-type header to the upper case one. But this can be tedious process if more headers are affected.

Let me know if i can help with that if you want to improve this.

Default import of uuid was deprecated in uuid@3, it's now been removed in latest uuid@7.

Example:

https://github.com/marblejs/marble/blob/master/packages/core/src/context/context.token.factory.ts

import * as uuid from 'uuid';

should be replaced with:

import { v4 as uuid } from 'uuid';

(This is backwards compatible and will work with both uuid@3 and uuid@7)

For more details: https://github.com/uuidjs/uuid#upgrading-from-uuid3

First of all, want to say that you doing an awesome job by creating marble.

I have a proposal by better request flow typing.

Let's consider an example:
We have an effect middleware for body validation

const foo$ = EffectFactory
  .matchPath('/foo')
  .matchType('GET')
  .use(req$ => req$.pipe(
    use(validator$({
      body: Joi.object({
        id: Joi.number().min(1).max(10),
      })
    }));
  ))
     .map(req => ({ body: req.body.id }));

After validation we stiil have body as any. But we would say that body have type

type FooBodyT = { id: number }

All the same with any other middlewares that modify request property with use. When we will modify some Observable with map we will get new type after map operator, but use not affect the type anyway.

I found useful FP helpers in marble

• marble/packages/core/src/+internal/fp/compose.ts

I think we can use middleware composition this way with correct typing. But now it not applicable to HttpRequest.

What do you think about this guys?

I'm trying to rewrite part of my express application to marble.js. And I'm not able to implement something like https://expressjs.com/en/api.html#res.sendFile without modification of httpListener itself.

The problem is that I want to use this library for file streaming https://github.com/pillarjs/send. And I need to change default behaviour of requestSubject$.

I think that it can be a useful feature to have option specify custom res.send method instead of default one. And with that it should be possible also specify custom error handler.

If you think that would be useful, I can make some proposal. I have my own httpListener currently.

Is there any way to use the filter of the marblejs/middleware-logger to filter specific routes?
I have some frequently called routes which I do not want to see in the log.
I know I can just use some grep to filter them, but it would be nice to filter it with the filter option of the marblejs/middleware-logger.

I really like the principles behind this framework I would like to know how can we use graphql here, I did not see anything in the docs related to graphql

Hi there,

Is there any way to known when a websocket client's connection is broken so that I can do a follow-up action, e.g. update the database about the client's status.

Forgive me if I've missed this somewhere, but is it possible to send, rather than receive, a http request with marble?

Hi, thanks for the great work on this project. It looks very promising and i have choosen it to build a framework for microfrontends on top of it. While developing it i faced the following problem:

Describe the bug
Server with a named route parameter crashes if an invalid parameter is defined. Before the server is terminated it shows a URIError: URI malformed Exception.

/.../src/server/node_modules/@marblejs/core/dist/http/router/http.router.resolver.v2.js:73
        const resolvedRoute = find(urlPath, req.method);
                              ^
URIError: URI malformed
    at decodeURIComponent (<anonymous>)
    at /.../src/server/node_modules/@marblejs/core/dist/http/router/http.router.matcher.js:18:55
    at resolve (/.../src/server/node_modules/@marblejs/core/dist/http/router/http.router.resolver.v2.js:73:31)
    at Server.handle (/.../src/server/node_modules/@marblejs/core/dist/http/server/http.server.listener.js:22:9)
    at Server.emit (events.js:326:22)
    at Server.EventEmitter.emit (domain.js:486:12)
    at parserOnIncoming (_http_server.js:863:12)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:126:17)
error Command failed with exit code 1.

To Reproduce

• define a route with a named parameter

const getMicrofrontendHtml$ = r.pipe( 
    r.matchPath('/mf/:name'), 
    r.matchType('GET'),
    r.useEffect((req$, ctx) => {
       // .... 
    })
);

I pass a string like @some-name/another-name but before i use encodeURIComponent("@some-name/another-name") to use it in a url. So it results in a Request like: GET /mf/%40some-name%2Fanother-name. The parameter is passed decoded and everything works as expected!

But if the URL contains invalid characters, e.g. GET /mf/%mf-test%2Ffoobar, the decoding fails with an exception and the whole server crashes.

Expected behavior
Instead of crashing the whole server it should keep running stable and return a Bad-Request Error.

Desktop (please complete the following information):

• OS => MacOS BigSur
• Package + Version

    "@marblejs/core": "^3.4.7",
    "@marblejs/middleware-body": "^3.4.7",
    "@marblejs/middleware-logger": "^3.4.7",

• Node version => v14.12.0

If you can point me in the right direction, i'm happy to implement a change and submit a pull request.

Hi there!
I believe there is a bug createServer factory:

event$(serverEventSubject.pipe(takeWhile(e => !isCloseEvent(e))), server, metadata).subscribe();

takeWhile takes a second argument, inclusive - setting it to true should allow server close callback to get called.

event$(serverEventSubject.pipe(takeWhile(e => !isCloseEvent(e), true)), server, metadata).subscribe();

thanks!

Describe the bug
Sometimes, I need to validate some data encoded as a sting with help of io-ts. For example, I have a query parameter page (?page=123) that must be a number. So, I use custom type:

const numberFromStringCodec = new t.Type<number, string, unknown>(
  'NumberFromStringCodec',
  t.number.is,
  (u: unknown, c: t.Context) => {
    const validation = t.string.validate(u, c);
    if (validation.isLeft() || u === '') {
      return t.failure(u, c);
    }
    const s = validation.value;
    const n = Number(s);
    return isNaN(n) ? t.failure(s, c) : t.success(n);
  },
  (n: number) => n.toString()
);
const data = { page: '1' };
const schema = t.interface({ page: numberFromStringCodec });
schema.decode(data) // right({ page: 1 }) <-- string changed to number

But middleware-io doesn't update validated values. So, after (successful) validation with requestValidator$ I have new static type but old unchanged value.

To Reproduce

...
r.useEffect((req$, _, { ask }) =>
  req$.pipe(
    use(
      requestValidator$({
        query: t.interface({ page: numberFromStringCodec })
      })
    ),
    mergeMap(async ({ req, state }) => {
      // `req.query.page`:
      //   static type is `number`
      //   runtime type is `string`
      //   value is '1' (should be 1)
    })
...

and send request with ?page=1

Expected behavior
Same values as with manual run of codec.decode().

Desktop (please complete the following information):

• OS Linux
• Package + Version @marblejs/core 2.1.1 + @marblejs/middleware-io 2.1.1
• Node version 10.15.3

Hey there!

Is there a way in the current version to pass arguments to middlewares?

If not, is this metadata field in the Effect type useful for it?

Using the latest version of core and middleware-io, defining a codec directly via io-ts and using it in requestValidator$ leads to a failed build due to seemingly incompatible type guards. Bumping the io-ts dependency seems to resolve the issue. A PR has been created.

To Reproduce
Check out https://github.com/tstelzer/marble-playground with minimal code to reproduce the issue.

Describe the bug
The @marblejs/core package utilizes chalk in the error/error.factory.js file. This is not listed as a required dependency, which causes the installed dependencies to be incorrect when this is installed in production mode (npm i --production). This causes any marblejs application to crash since the package cannot be imported.

To Reproduce
Create a boilerplate server with @marblejs/core and install dependencies via npm i --production with no pre-existing node_modules.

Expected behavior
I expect the chalk dependency to be added to the @marblejs/core package.

Desktop (please complete the following information):

• OS: Mac OS X High Sierra
• Package + Version: @marblejs/core v1.2.0
• Node version: v10.13.0

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
About @marblejs/middleware-io,
I want validate with regex

Now, I validate which using requestValidator (type),
But I have to validate with other pipe function (regexp)

Describe the solution you'd like
A clear and concise description of what you want to happen.
Add option to validate body parameter with regex

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

import { requestValidator$, t } from '@marblejs/middleware-io'

const registerVaildater$ = requestValidator$({
  body: t.type({
    email: t.regex(new Regexp(something)),
    name:  t.regex(new Regexp(something)),
    password:  t.regex(new Regexp(something)),
    rPassword:  t.regex(new Regexp(something))
  })
});

Additional context
Add any other context or screenshots about the feature request here.