cubos / sdkgen Goto Github PK

Nowadays there's no way to consume the API in a node application, we could fix that in a number of ways:

• Make the current typescript_web target compatible with node;
• Create a new target for node applications that works with packages;
• Create a new standalone target for node;

The problem with the first approach is the bundle size increment, in order to make the web target compatible we would need to polifyll some functions and that would increase the bundle size in about 30kb (that's a lot for the web and we're trying to reduce it, not increment it)

With that in mind I think we should stick with the last two options, but the sdkgen itself doesn't generate a npm package, It only generates the api.ts file (for the web target) that will be bundled into a npm package by another tool.

So, for now, I'm attacking the last option and creating a new node target that will output a standalone client for the api.

Besides that I'm also opening a new issue suggesting the sdkgen to return a npm package as output instead of a file to be bundled for a third tool.

Once #24 is done and we can use npm for versioning, breaking changes will become less scary. With that out of the way, I propose we implement an opt-in (maybe as a plugin?) object mode for endpoints' arguments, as follows:

// object mode disabled
function logIn(username: string, password: string) { /* stuff */ }

// object mode enabled
function logIn(args: { username: string, password: string }) { /* stuff */ }

By doing so, breaking changes can be more easily detected and fixed: imagine the logIn function changes, so that it starts to receive a username and a token, both strings. Without object mode, the change wouldn't be detected by type systems (as password and token have the same type), introducing subtle bugs.

Another kind of bug easily introduced without object mode but avoided with it, is based on the fact that arguments order matter: switching arguments of the same type (e.g. logIn(password, username)) is easily done and hardly detectable.

Any thoughts?

When a try to run a sdkgen with the code below this produce a nil exception fired by the crystal.

function funcName(param1: string):

Error Logging

Unhandled exception: Nil assertion failed (Exception)
  from /usr/lib/crystal/class.cr:148:0 in 'not_nil!'
  from /home/icaro/cubos/sdkgen/src/syntax/token.cr:0:3 in 'filename'
  from /home/icaro/cubos/sdkgen/src/syntax/token.cr:11:5 in 'location'
  from /home/icaro/cubos/sdkgen/src/syntax/parser.cr:289:35 in 'parse_type'
  from /home/icaro/cubos/sdkgen/src/syntax/parser.cr:211:7 in 'parse_operation'
  from /home/icaro/cubos/sdkgen/src/syntax/parser.cr:49:9 in 'parse'
  from /home/icaro/cubos/sdkgen/src/main.cr:38:9 in '__crystal_main'
  from /usr/lib/crystal/crystal/main.cr:97:5 in 'main_user_code'
  from /usr/lib/crystal/crystal/main.cr:86:7 in 'main'
  from /usr/lib/crystal/crystal/main.cr:106:3 in 'main'
  from __libc_start_main
  from _start
  from ???

We need to start our documentation in the repository, I thought about:

• Contribution Guidelines
• Code of Conduct
• Read me, containing how to run the project, software dependencies and use cases.

You guys think that we need more topics? If yes, which ones?

Problem: currently the output for the typescript_web target is a single ts file with multiple unresolved dependencies.

Proposal: Add a --package option to the generator, with this flag the user would be capable to generate a npm package (with a package.json describing the dependencies) or get a standalone file with the dependencies already resolved.

Documentation has been regarded as a must for a while now, but little has been done to plan how that would be implemented. I propose we work in two main functionalities:

• A specification for documentation comments in the sdkgen language (prior art: JSDoc, JavaDoc), compatible with its equivalent in the target languages (namely, Javascript, Java, Kotlin and Swift) + include that documentation on the generated code when transpiling
• A documentation target for sdkgen, that could use some ready-to-use documentation solution (such as Docusaurus an GitBook) for generating publishable, human-friendly documentation

Any thoughts?

With simple changes to the build_and_publish.sh script, we could bundle both a CommonJS and a ESModule versions of the API, so that we allow bundle size optimizations such as tree shaking without sacrificing backwards compatibility.

I should be shipping a proposal as a PR soon.

Just for branding and to improve editor extensions:

Sometimes we need to point the generated target to a specific URL other than the one set by the .sdkgen file.

In order to do that I propose that we implement a method called .setUrl, .setEndpoint or .setEndpointUrl, which one do you guys prefer most?

This proposal affects the targets: typescript_web and typescript_nodeclient.

Motivation

Currently, sdkgen-based API projects use a docker container for compiling the sdkgen files on CI, and compile them by hand locally. There are two main issues with that:

• It's hard to attach a project to a specific version of sdkgen (specially, it's hard to work with projects using different versions of it)
• It's hard to integrate with tools like TypeScript and Webpack and therefore hard to automate (current workaround is to include a bash script that assumes the sdkgen source code is in a specific location and up to date)

Proposed solution

Libraries like node-sass compile their binaries from languages like C on post-install. A package for sdkgen would solve versioning and allow better tooling:

• A sdkgen-loader for webpack would prevent developers from working on out-of-date APIs
• So would a typescript plug-in that allowed to directly import sdkgen files and have niceties such as auto completion on modern editors

Possible issues

• I couldn't found any example of npm packages compiling binaries with crystal, so it's feasibility is yet to be evaluated

Fate of api_calls need to be decided. Maybe we can have database plugins.

Currently each connected client is device, that has an ID and is saved in the database and is exposed as ctx.device. The client holds a device token. Session is done by saving some data on the devices table.

Problems:

1. There is no way to get an history previous sessions of a device.
2. If the device token is leaked, then a future authentication of that device will be leaked as well. This breaks "Backward Secrecy".
3. It is hard to attach actions to a particular session of a specific user on a particular device, for auditing.

Also, the device ID is not always persisted when the target app is removed and then reinstalled, which creates two devices for the same physical device. Currently this is not done because uninstalling and reinstalling an app (or cleaning data) should also logout the user. As the user is attached to our device, we end up removing the whole device by losing the device token on device. It also makes it hard to black list a device.

##Solution:

1. Do a best effort to store the device token on the device using permanent storage. A cryptographic secure vault if possible. (This is doable on iOS and Android. TODO: Investigate about what is the best effort for web @dygufa @vhfmag)
2. Create the concept of device stored data:

Device-stored Data:

Any api call can send to the device data to be stored, this can be session tokens, authentication info, list of permissions, anything relevant. Before sending, this chunk of data will be JSON encoded, then encrypted and then signed by the server with a device specific key (this is pretty much JWT). The receiver device won't be able to read or modify the information, the only thing it can possibly do is either store it or destroy it. It also can't be transferred to another device even if leaked, it is linked to the device token, that is stored in some safer way.

The device should store it in a way that if the user want to clear all data, it will be removed as well (localStorage, SharedPreferences, ...). This should be sent on every request to the sever.

Upon receiving, the server will decrypt the information and verify the signature. If it doesn't match, simply discard and act as if nothing was sent.

For the implementation of an api function, the data will be available as ctx.data. It can be modified directly and after the function returns, the modified version will be encrypted, signed and sent back alongside the usual response. It is by default an empty object {}.

This enforces that device has no piece of information that sdkgen is not aware about.

For backward compatibility with clients that are in production and aren't aware of this data, we can store it in the database and load/store ourselves.

With these two features it is possible to implement the concept of sessions without special support from sdkgen. An implementation can have a table of sessions attaching an user to a device. The id of the session can be saved on ctx.data.sessionId (it doesn't need to be a cryptographic-secure token or anything special).

This also doesn't touch on authentication itself. The implementation can still validate who is who by any means. Any function can read and modify a device's data.

What do you think? @dygufa @davidcpires @vhfmag @tironiigor

As the title says, sdkgen doesn't work with the latest version of crystal(0.32.0).
It seems that this version limited the maximum size of tuples, which makes the sdkgen scripts fail with errors like:

tuple size cannot be greather than 300 (size is 639)

The erros points to crystal's default library at:

/src/string.cr:4536:27

Steps to reproduce:
Just run any sdkgen project with the latest crystal version.

Moment is way too big and we are only using it to format dates during codegen for DatePrimitiveType.

There could be an officially supported way to mark endpoints that require authentication.

Maybe there could be decorators akin to the @deprecated, such as @requiresAuth. However, how to tell the library to attach or detach a device and a user, so that access to an endpoint that requires authentication can be denied, isn't well defined. If sdkgen is going to stick to Typescript/Javascript, maybe it could export a module providing runtime functionality. If so, there could be functions for signaling login and logout.

It is kind of opinionated, so maybe it could be a better idea to provide an API for extensibility, providing access to intercepting requests, for example, so that there could be an auth plugin.

There is room for at least two improvements there:

• Function and parameters signatures are available, so sdkgen could provide that information to the API consumer targets (JSDoc for JS/TS, Javadoc for Java/Kotlin, etc)
• There could be support for documentation directly on the sdkgen file, that would be passed down to the target files.

It would be a feature so we could have an enum like

type Gender enum {
    female
    male
}

and then we could have

translator pt-br, default {
    Gender: {
        female: Feminino
        male: Masculino
    }
}

and it would generate a function to get the value for that translation given the phone Locale, and using the default value if the Locale trasnlation is not avaiable

Problem: the need to mark methods as deprecated in order to enforce warning on the clients.

Implementation:

@deprecated($message)
get plansWithPrice()

get fullPlans()

If message is set the client will show it otherwise the message will be: "The method getPlansWithPrice is deprecated"

When in sdkgen is created a function with param called "version" this produce a compile errors in node-server target.

Following below the error text produced by the behavior above.

ERROR in [at-loader] ./src/user/api.ts:2268:48 
        TS7022: 'version' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly in its own initializer.
    
    ERROR in [at-loader] ./src/user/api.ts:2268:109 
        TS2448: Block-scoped variable 'version' used before its declaration.
    
    ERROR in [at-loader] ./src/user/api.ts:2301:48 
        TS7022: 'version' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly in its own initializer.
    
    ERROR in [at-loader] ./src/user/api.ts:2301:118 
        TS2448: Block-scoped variable 'version' used before its declaration.
    
    ERROR in [at-loader] ./src/user/api.ts:2334:48 
        TS7022: 'version' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly in its own initializer.
    
    ERROR in [at-loader] ./src/user/api.ts:2334:111 
        TS2448: Block-scoped variable 'version' used before its declaration.
    
    ERROR in [at-loader] ./src/user/api.ts:2367:48 
        TS7022: 'version' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly in its own initializer
        
 ERROR in [at-loader] ./src/user/api.ts:2367:116 
        TS2448: Block-scoped variable 'version' used before its declaration.`

This bug is introduced by 0cbc20c commit. in the following line
@io << ident ident ident ident "const {key, expirationSeconds, version} = await cacheConfig.#{op.pretty_name} (#{(["ctx"] + op.args.map(&.name)).join(", ")});\n"

I'm think the same occurs with functions with parameters called "key" and "expirationSeconds" .

Rationale: Currently, the web client target generates a single file. If it generated a common module and a module per endpoint, bundle size could be hugely optimized by clients using modern bundlers, like webpack, parcel & rollup.

Implementation details: It would still be necessary to generate a single file module for compatibility with CommonJS, and it could be done by bundling the ESModule version. The CommonJS version could be exposed on package.json's main field, while the ESModule version could be expose in its module field (it's still a proposed field, but is recognized by most bundlers).

Objections? Thoughts?

This code below compile without problems. updateIntermediateReport is declared twice in .sdkgen file.

get AnswersIntermediateReport(): string?
function submitIntermediateReport(Answers: Answer[])
function updateIntermediateReport(Answers: Answer[])

get AnswersFinalReport(): string?
function submitFinalReport(Answers: Answer[])
function updateIntermediateReport(Answers: Answer[])

When is declared a duplicate error types in skdgen no advice or error are presented by the compiler and this will produce a invalid typescript code.

iex.

error NotLoggedIn
error WrongLogin
error InvalidCall
error InvalidToken

error NotLoggedIn
error WrongLogin
error InvalidCall
error InvalidToken