maskingtechnology / jitar Goto Github PK

We need to release a new version with the new dependency

With TypeScript 4.8 installed, not all examples compile and run. We need to fix this

When requesting a module the client id is directly passed by the controller without checking its type and content. These checks need to be added.

According to the security scanner this issue is a serious threat because the id is injected into the path. Because the id is removed from the path before its used, it's not as bad as it looks. The id already got checked by the repository to validate that the id has the correct format, and it a registered client. So we are only missing the checks in the controller.

Tracking issue for:

• https://github.com/MaskingTechnology/jitar/security/code-scanning/2

When calling the RPC api, there are only three possible return codes;

• 200 when the call is succesful;
• 404 when the procedure is not found;
• 500 all other errors.

When calling a procedure we want to send back a Bad Request when the parameters sent do not match the expected parameters of the procedure.

There are two scenarios where the server fails on the parameter validation. The #extractParameterValues in the implementation.ts throws two errors; UnknownParameter and MissingParameterValue.

The RPC controller needs to be extended to recognize the errors and send a 400 response.

#createErrorResponse(errorData: unknown, response: Response, serialize: boolean): Response
{
    const content = this.#createResponseContent(errorData, serialize);
    const contentType = this.#createResponseContentType(content);
    const statusCode = this.#createResponseStatusCode(errorData);

    response.setHeader('Content-Type', contentType);

    return response.status(statusCode).send(content);
}

#createResponseStatusCode(errorData: unknown): number
{
    // implement me
    // if unknown error return 500
}

Both the introduction and the features sections contain a spelling error that needs to be fixed.

Introduction: "It lets you to build monolithic applications" (remove "to")
Features: "is placed on of the servers" (add "one")

After #59, we killed the dependabot configuration. The only thing is, it runs only once a month, so we only found out recently it did not work.

We need a new configuration for the dependabot to scan the new packages.

We want to scan

• jitar
• jitar-nodejs-server
• jitar-vite-plugin

We leave the examples out of scanning, as these are examples and not meant for actual development.

Describe the solution you'd like
We want to automate the process of publishing and managing the versions of our packages.

Currently the docs work well on a desktop, but lacks support for mobile devices. Therefore we need to add responsiveness to the layout.

Also there are some issues with the scrolling and overflow on an iPad that need to be fixed in this process.

We want to use dependabot to detect outdated dependencies.

We'll start with a monthly check of our npm dependencies.

Currently we only have a React example that loads the libraries using a CDN. This is not how real-world developers work. At this point in time Vite is a popular tool used by many frameworks. It would be great if we can create a Jitar plugin.

Currently the docs contain spelling errors that need to be fixed. A few error are already found but won't be listed here because the docs need a total review still.

Vita has released version four. To keep up with the latest version, we need to update and test our plugin.

Release all packages to npm.

For optimizing full-stack apps we need to add cashing options. We can do this on both ends:

1. Browser caching by setting 'cache-control' header;
2. Server caching by storing RPC responses in the gateway (and/or node).

The caching must be configured per procedure:

• Type (browser of server)
• Maximum age (before refresh)

By default the caching is disabled.

When scaling an application onto multiple servers we want to be fail safe. Currently we have a health check system that is used for monitoring the availability of nodes. In case one of the nodes is not healthy or doesn't respond, the gateway will stop using it. But when the gateway already forwarded a request to such a node, the request will fail.

As an improvement we need to add extends the gateway with a retry strategy. When a node has become unavailable, the gateway should look for an alternative node for the requested procedure and try it again on that node before it fails.

The getting stated page needs two updates in order to match the current implementation:

1. Jitar configuration still uses the "old" configuration style
2. The tsconfig.json misses the skipLibCheck setting (causing unnecessary errors)

We need to publish our package to npm manually and then update all the examples.

Current situation
When running a procedure dynamically (using the RPC API) or the runProcedure hook we currently only check for missing parameters. This means that additional parameters that are not part of the procedure are accepted, although they are not used.

Example
The example below shows the current situation using the runProcedure hook.

// sayHello.ts
export async function sayHello(name: string): Promise<string>
{
    return `Hello ${name}`;
}

// Run the procedure without the name parameter
runProcedure('sayHello', '0.0.0', {}); // Error -> Missing value for parameter 'name'

// Run the procedure with additional parameters
runProcedure('sayHello', '0.0.0', { name: 'John Doe', additional: 'some value' }); // No error

This situation can lead to confusion, so we want to be more strict.

Desired situation
Running procedures with additional parameters should result into the following error: Unknown parameter ${parameterName}

Implementation details
The procedures are a part of the core module of Jitar. A procedure has one or more implementation. Each implementation represents a version of the procedure. This means that running a procedure actually means running an implementation.

The following steps define what needs to be done for the realization:

1. Add a new error src/core/errors/UnknownParamater.ts
2. Import the error in src/core/Implementation.ts
3. Add the validation logic to the Implementation.ts file
4. Create tests

Our previous attempt to make things works broke the publish functionality to npm. The package.json on the root causes npm publish to skip each package.json for each of our packages. This resulted in an empty 0.3.1 publication.

Lerna seems to be suitable for the things we need to do.

Currently the repository does not protect the access to any of the source files. This means that files containing sensitive information are accessible for any client.

For example, the following API call will result into the full content of the requested file.

GET http://repository.example.com:3000/path/to/secretAlgorithm.js

While the desired result would be a 404 - Not found.

Adding protection
By default we should make everything protected (this is the safest option), and whitelist files or folders that are publicly accessible. For the whitelist we need to add an additional configuration option to the repository. Something like:

{
    "url": "http://repository.example.com:3000",
    "repository":
    {
        "source": "./src",
        "cache": "./cache",
        "index": "index.html",
        "assets": [ "*.html", "assets/**/*" ]
    }
}

In this option we simply specify what the assets are. The given paths are relative to the source folder.

Note that components from protected files always need to be placed into a segment in order to make them internally accessible.

Context
The support for authentication as a middleware is application specific.

Solution
To help developers setting up authentication, we need to write a few examples.

We have a dependency on cpx for development purposes. This package has been last updated 6 years ago and contains a severe issue.

We need to remove the cpx dependency, which we seem to be replace with cpx2.

Jitar needs to have an option to validate incoming data through its apis. As with the rest of the product, we want to have a configuration only option to define the validation schemas.

What needs to be done for this story;

• update the segment file configuration with a validation model
• add the configuration to generated cache files
• validate the incoming data on the server with the provided configuration
• translate validation errors to http 400

Split the jitar and plugin implementations into packages

In the documentation of the startClient hook we use a wrong segment reference in the example. This is only wrong in the documentation. The actual example uses the correct reference.

Current situation
We reference a segment.js file, but doesn't work anymore (old style).

const client = await startClient('client.segment.js');

Desired situation
Only reference the name of the segment.

const client = await startClient('client');

Review the dependabot pull requests and merge them together.

To release the next version, we need to;

• update documentation
• update release notes
• tag release
• npm publish
• remove package-lock.json for examples

Using the RPC API from external clients requires controlling the CORS headers. We need to implement middleware that sets the following headers:

• Access-Control-Allow-Origin
• Access-Control-Allow-Methods

The values can be set with the constructor of the middleware.

new CorsMiddleware(origin, methods)

To make this work we need to extends the middleware implementation. Currently we only support request headers, so we can't set response headers yet. For the implementation the we need to do two things:

1. clear the headers map after the execution of the procedure (so that middleware can set response headers after executing the next() function)
2. set the returned headers in the response (in the Node.js server)

This issue also requires an update of the documentation.

The current CORS implementation only supports setting the allowed origin. To get more control over the CORS rules, setting the allowed headers is required.

We have a nodejs implementation to execute jitar as runtime. Now it's time to add the next javascript server; deno

The deserializers currently assume that the given object contains the fields with the expected type. We need to add more validation to make sure the object meets all expectations.

In the current implementation the request body is passed as deserialization data directly. We need to do this because we don't (and shouldn't) know if the data is a simple string or a complex object. This has to be figured out in the deserialization process. Therefore we need to implement the validation in the deserializers.

Tracking issue for:

• https://github.com/MaskingTechnology/jitar/security/code-scanning/1

We've received some useful feedback on our website that we want to process quickly:

1. Improve the readability of the articles (set the font-weight to light instead of extra-light)
2. Open a new tab for all external links (including the docs)
3. Improve the framework agnostic sentence
4. Fix the icon of the 'source code' button

The feedback contains more items, but need more thought and can't be done quickly.

Glob-promise updated to 5.0.1 in the last couple of days. This introduced a dependency on an old peer dependency installer, containing a lot of vulnerabilities.

The glob promise is now updated to 6.0.0, so we need to release a new version of the nodejs server implementation to get rid of these warnings.

When sharing (parameter) data between nodes, we currently support the following data types:

• Primary types (string, number, boolean, null, undefined)
• Collection types (array, set, map)
• Object types (plain objects, class objects)
• Binary data (TypedArray)

To support more common cases we also need to add the (de)serialization of dates.

When an error is passed from a node to the gateway, the gateway isn't able to de-serialize the error.

Reproduction
This issue can be reproduced by running the segmentation example (only node1 is required). The following request works as expected.

GET http://localhost:3000/rpc/greetings/sayHi?firstName=John

When removing the firstName parameter from the request, it results into the following error.

Module '../..//jitar/core/errors/MissingParameterValue.js' could not be loaded | Cannot find module '/path/to/jitar/examples/1-basic//jitar/core/errors/MissingParameterValue.js' imported from /path/to/jitar/examples/1-basic/2-segmentation/dist/start.js

The expected result is as follows.

Missing value for parameter 'firstName'

Context
The packges for jitar are currently unscoped and all start with jitar. We want to change the package structure and have the following package structure for npm.

• jitar - a single package that developers can include and enjoy the full jitar experience
• @jitar/runtime - #158
• @jitar/reflection - #115
• @jitar/serialization - #154
• @jitar/nodejs - #156
• @jitar/caching - #155
• @jitar/plugin-vite - #157

Solution
This story is about integration all the newly created packages and bundle them together in a jitar package. The old packages that are currently on npm need to be deprecated and the new packages will be released.

This is a breaking change and will need to be communicated properly.

To enable #153, we need to parse javascript files. We checked other existing libraries, but they do not provide the functionality we need.

We will write our own reflection library.

We have mappings for not found exceptions and bad request exceptions. We need to extend the number of exceptions to support a wider range of status codes.

The most used exceptions that will be added are:

• Unauthorized (401)
• Payment required (402)
• Forbidden (403)
• I'm a teapot (418)
• Not implemented (501)

Context
To make it easier to start jitar using the frameworks developers are known with, we need to create integrations and examples for the most popular frameworks.

Solution
To support developers that use vue we need a plugin and an example how to use it with vue.

We need to add middleware support

Context
To make it easier to start jitar using the frameworks developers are known with, we need to create integrations and examples for the most popular frameworks.

Solution
To support developers that use nextjs we need a plugin and an example how to use it with nextjs.

With the new deno implementation it would be nice to add a working example to the repository

When sharing (parameter) data between nodes, we currently support the following data types:

• Primary types (string, number, boolean, null, undefined)
• Collection types (array, set, map)
• Object types (plain objects, class objects)

To support more complex cases we need to add (de)serialization of binary data.

Implementation
For the implementation we use the ArrayBuffer object that is supported by all major browsers and Node.js. We need to implement a ArrayBufferSerializer in the src/runtime/serialization module.

Describe the bug
When trying to call a function that runs on another node and has private access results into the following error: "TypeError: ... is not a function".

To Reproduce
Steps to reproduce the behavior:

1. Clone the react-jitar example;
2. Set the access to private in the server.segment.json file;
3. Build and run the application;
4. Open the application in a browser with dev-tools open: http://localhost:3000
5. See error in the console

Expected behavior
Get a descriptive error telling that the remote function is private and therefore can not be accessed.

Additional context
Requesting a remote private function results into an empty module file that doesn't export anything, while the module file contains the remote implementation for a public function.

Context
To make it easier to start jitar using the frameworks developers are known with, we need to create integrations and examples for the most popular frameworks.

Solution
To support developers that use remix we need a plugin and an example how to use it with remix.