Hi, Iโm Vincent! Iโm a software engineer who loves:
- Simplifying things
- Building open source libraries
- Building products (full stack)
- Building developer tooling (including programming language design)
๐ Find me on the web:
Elegant validation library for type-safe input data (for TypeScript and Flow)
Home Page: https://decoders.cc
License: MIT License
Can I create a Decoder
such as this one
const withKey = <K extends string, T>(k: K, d: Decoder<T>) => object({[k]: d})
so that the resulting type is exact? For example Decoder<{a: number}>
for withKey("a", number)
?
Currently this results in something like Decoder<{[x: string]: number}>
(simplified).
Perhaps related, is it possible to take a Decoder<{...}>
and "merge" a property into it / "extend" it?
zod
equivalent: https://github.com/colinhacks/zod#extending-objects
This case throws a horrendously looking error (all useful information is invisible):
const g = guard(
either(
object({ foo: string }),
object({ bar: number }),
)
);
g({
// Number + string deliberately flipped, so both branches throw a field error
foo: 123,
bar: 'not a number',
})
This outputs:
Decoding error:
{
"foo": 123,
"bar": "not a number",
}
^
Either:
-
-
Note the missing error info in the Either: section.
See also https://github.com/nvie/decoders/tree/improve-either-errs for a demonstration.
I have a set of 30 possible objects, all of which have different type
properties. Their additional properties also differ based off of which type
they have. For example,
const certificateSignerExample = { type: 1, signer: 'BillyGates' }
// ...
const antimalwarePathExample = { type: 30, path: 'lol/not/a/path' }
To decode these objects, I needed to use dispatch
, since (quickly does some math) 30 is more than 9, which means I cannot use one of the friendly either
composite decoders.
However, in my case, I could not use the type
field, since the object of decoders that we supply to dispatch is... well, a javascript object :), i.e., all of the keys are inevitably strings.
const certSignerMatcher = object({ id: constant(1 as 1), signer: string })
// ...
const antimalwarePathMatcher = object({ id: constant(30 as 30), path: string })
const matcherDecoder: Decoder<PolicyMatcherDetails> = dispatch(
'type',
{
1: certSignerMatcher,
// ...
30: antimalwarePathMatcher
}
)
I was able to work around this, of course, thanks to the very friendly map
and compose
higher-order decoders (thanks for those, by the way).
My solution was to:
map
on the input objects to add an additional __type
field (which is just the stringified version of the object's type
)compose
said map
decoder with a Dispatch
decoder that makes use of the __type
field insteadconst addStringifiedTypeDecoder = map(
jsonObject,
o => ({ ...o, __type: o?.type?.toString() })
)
const mapHackyStringTypeToDecoder = dispatch(
'__type',
{
1: certSignerMatcher,
// ...
30: antimalwarePathMatcher
}
)
const matcherDecoder: Decoder<PolicyMatcherDetails> = compose(
addStringifiedTypeDecoder,
mapHackyStringTypeToDecoder
)
This solution works fine for me, but it made me think that the dispatch
composite decoder could possibly be rewritten to be a tiiiiiny bit more flexible.
One thought I had is that there could be an optional transformation of the value located at field
(in my case, type
), which would convert it to a stringified key:
const matcherDecoder: Decoder<PolicyMatcherDetails> = dispatch(
'type',
{
1: certSignerMatcher,
// ...
30: antimalwarePathMatcher
},
(valueOfTypeField: number): string => `${valueOfTypeField}`
)
At any rate, I'm curious to hear what others think!
Hello ๐
I'd like to check if the documentation for dispatch
is correct because I'm running into a problem with it at the moment!
import {object, string, constant, dispatch, guard } from 'decoders'
type SandboxEnviroment = {
status: "sandbox";
name: string;
}
type LiveEnvironment = {
status: "live";
name: string;
}
type Environment = SandboxEnviroment | LiveEnvironment
const decodeSandbox = object({
name: string,
status: constant('sandbox')
})
const decodeLive = object({
name: string,
status: constant('live')
})
const decoder = dispatch('status', {
live: decodeLive,
sandbox: decodeSandbox
})
const run = (): Environment =>
guard(decoder)({
status: 'live',
name: 'my-app'
})
The error I get (in run
) is the following:
TS2322: Type '$Values<{ live: { status: string; name: string; }; sandbox: { status: string; name: string; }; }>' is not assignable to type 'Environment'.
Type '{ status: string; name: string; }' is not assignable to type 'Environment'.
Type '{ status: string; name: string; }' is not assignable to type 'LiveEnvironment'.
Types of property 'status' are incompatible.
Type 'string' is not assignable to type '"live"'
However the documentation for dispatch
indicates that the inner type of the decoder will be a union! Is this a bug or an error in the documentation?
Of course, I can leave off the annotations in my run
function to get this compiling so it's not a blocker. Thought I'd give you a heads up about it though! ๐
Description of the bug
The global URL was introduced in NodeJS in version 10, and is referred to in the url decoder.
Trying to use this package on an earlier version of NodeJS (e.g. 8.17.0) results in an error when importing the module.
Expected behavior
I expect NPM to prevent me from installing (version 2?) of this package, due to having {engines: { node: '>=10'}}
in package.json
To Reproduce
index.js
require('decoders');
Command line:
node index.js
Additional context
Decoders version: 2.0.1
TypeScript version: 4.5.5
Node version: 8.17.0
I know Node10 is a relatively ancient version, so its reasonable to expect its features to exist, but it would be really appreciated if the minimum expected engine version was specified, ideally in both the README/documentation and in the package.json .
One pain point is the decoder is more-or-less just the flowtype rewritten. The advantage of something like https://github.com/codemix/flow-runtime is that this is done automatically, but without the flexibility of making more complex runtime decoders and/or deciding where to apply the runtime check.
What are your thoughts about a tool that would generate decoders for all exported types? For example if you have a file ./src/myModule.js
and you run $ decoder-codegen ./src/
, it would create a new file ./src/__generated__/myModule.decoders.js
. Then we can import the decoders into the source file.
myModule.js
import { guard } from 'decoders';
import { pointDecoder } from './__generated__/myModule.decoders';
const checkPointType = guard(pointDecoder);
export type Point = {
x: number,
y : number
}
export function moveUp(point: Point) {
checkPointType(point);
point.y = point.y - 1;
}
Inspired by https://github.com/apollographql/apollo-codegen#generate
Hi there,
I have only started with TypeScript and front-end development recently and I discovered the decoders
package just now.
Quick question: What is the recommend way to import the basic decoders such as object
, number
, string
, etc.? All these are built-in TypeScript keywords. Should I just rename the imports or is there any better technique?
Maybe this question could also become a feature request for export the basic decoders with some wrapper object called JsonDecoders
which can then be imported and used like JsonDecoders.object
, JsonDecoders.number
, JsonDecoders.string
, etc.
Just for convenience when using IDEs with auto-import, e.g. VSCode.
Description of the bug
When I use the .decode
method, I end up with errors of type Annotation, but none of the utility functions to do anything with that type are exported. My error handling library only accepts instances of type Error. The non-exported format
function handles converting and formatting an Annotation into an Error for verify
, but it is not exported/accessible.
I must use .decode
for other behaviors, but would still like to be able to use the error as an Error, not an Annotation in places.
Expected behavior
Be able to import formatInline
and other custom formatting functions/utils to process the annotation before handling/logging it.
To Reproduce
Try to import formatInline
from package.
Additional context
Decoders version: 2.0
When trying to decode a string to one of 3 constants, am receiving an error TypeError: (0 , _Result.Ok) is not a function
Codepen example (open browser console to see error).
Snippet:
import { guard, either3, constant, Decoder } from "decoders";
const statusDecoder: Decoder<'complete' | 'closed' | 'failed'>
= either3(constant<'complete'>('complete'), constant<'closed'>('closed'), constant<'failed'>('failed'));
const statusString = 'closed';
const status = guard(statusDecoder)(statusString);
console.log('Decoded:', status);
Please confirm:
What is the problem?
Description
I had a map
which was converting a string to a luxon DateTime.
import { Decoder, map, string } from 'decoders'
import { DateTime } from 'luxon'
export const transformDateTime: Decoder<DateTime> = map(string, (isoDate) => {
const dt = DateTime.fromISO(isoDate)
if (dt.invalidExplanation) {
throw `Unable to parse date ${isoDate}: ${dt.invalidExplanation}`
}
return dt
}
)
I use this whenever I use DateTime - e.g.:
const decodeFoo = object({
id: string,
expires: transformDateTime,
})
I migrated it like this (simple!):
import { Decoder, string } from 'decoders'
import { DateTime } from 'luxon'
export const transformDateTime: Decoder<DateTime> = string.transform(
(isoDate) => {
const dt = DateTime.fromISO(isoDate)
if (dt.invalidExplanation) {
throw `Unable to parse date ${isoDate}: ${dt.invalidExplanation}`
}
return dt
}
)
Now my page fails to load with an error:
Uncaught TypeError: string.transform is not a function
I've looked at the generated code of the private package and it still looks correct to me. I get this (the .d.ts):
import { Decoder } from 'decoders';
import { DateTime } from 'luxon';
export declare const transformDateTime: Decoder<DateTime>;
and this (the .js):
import { string } from 'decoders';
import { DateTime } from 'luxon';
export const transformDateTime = string.transform((isoDate) => {
const dt = DateTime.fromISO(isoDate);
if (dt.invalidExplanation) {
throw `Unable to parse date ${isoDate}: ${dt.invalidExplanation}`;
}
return dt;
});
Possibly useful extra info:
transformDateTime
is only used internally in the private package (none of the decoders are exported from the package)Any suggestions? Thank you!
I see the export in the code on the repo but not in the package I pulled from npm. I believe all that's needed is a new release.
I am currently using your library for type-safe decoding of API responses, which sometimes contain arrays of items. In its current implementation, the D.array
decoder will throw an error if any of the items in the array fail to decode against the provided schema, like so:
export const item: D.Decoder<Item> = D.object({
id: D.uuidv4,
title: D.string,
});
export const paginatedItems: D.Decoder<PaginatedResponse<Item>> = D.object({
results: D.array(item),
next_page: D.optional(D.nullable(D.string)),
});
I'd like to propose a feature where the array decoder has a "resilient" or "filter" mode that continues decoding despite individual errors, collects these errors separately, and omits the failing items from the resulting array. This could provide a more user-friendly, error-resilient approach while still being within the bounds of the library's design philosophy.
const decoder = object({
id: number,
name: string,
address: optional(string),
});
decoder({ id: 1, name: "test" }).unwrap();
// returns this
{ id: 1, name: "test", address: undefined };
const decoder = object({
id: number,
name: string,
address: optional(string),
});
decoder({ id: 1, name: "test" }).unwrap();
// returns this
{ id: 1, name: "test" };
type Params = {|
host?: string,
port?: number,
|};
const decoder = exact({
host: optional(string),
port: optional(number),
});
const validate = guard(decoder);
const params = validate({});
const options = {
host: "localhost",
port: 3000,
...params,
};
console.log(options);
/*
{
host: undefined,
port: undefined
}
*/
const decoder = exact({
host: optional(string),
port: optional(number),
});
const validate = guard(decoder);
const params = JSON.parse(JSON.stringify(validate({})));
console.log(params);
/*
{}
*/
2.0.0-beta9
Decoder<O, I>
where I
is a type other than unknown
Decoder<O, I>
to any of the builtin composite decodersE.g.
import { Decoder, DecodeResult, optional } from 'decoders';
import { unknown } from 'decoders/annotate';
import { err, ok } from 'decoders/result';
import * as uuid from 'uuid';
const binaryUUIDFromStringUUID: Decoder<Uint8Array, string> = (uuidString: string): DecodeResult<Uint8Array> => {
try {
return ok(Uint8Array.from(uuid.parse(uuidString)));
} catch (error) {
return err(unknown(error));
}
};
const optionalBinaryUUIDFromStringUUID = optional(binaryUUIDFromStringUUID);
No type errors.
TS2345: Argument of type 'Decoder<Uint8Array, string>' is not assignable to parameter of type 'Decoder<Uint8Array, unknown>'.
ย ย Type 'unknown' is not assignable to type 'string'.
guard
compose
predicate
transform
array
nonEmptyArray
set
describe
dict
exact
inexact
mapping
object
either
lazy
maybe
nullable
optional
taggedUnion
tuple
const bigIntFromString: Decoder<bigint, string> = ...;
const bigIntArrayFromStringArray: Decoder<bigint[], string[]> = array(bigIntFromString);
const booleanFromString: Decoder<boolean, string> = ...;
const bigIntFromString: Decoder<bigint, string> = ...;
const recordFromStringifiedValueRecord: Decoder<{ foo: boolean, bar: bigint }, { foo: string, bar: string }> =
object({ foo: booleanFromString, bar: bigIntFromString });
const bigIntFromString: Decoder<bigint, string> = ...;
const bigIntFromJSON: Decoder<bigint> = compose(compose(json, string), bigIntFromString);
Having used Elm decoders, decoding a particular field or a combination of them into another field is a common use case for me.
An example could look something like this:
const decoder: Decoder<T> = object({ targetField: compose(field('sourceField'), string) })
So essentially it would be a 'mixed' decoder that takes its value from another field than its target field.
Is there currently a way to do this? If not, would this be possible to implement?
Description of the bug
Using a combination of lazy and either, decoders can become very slow.
Expected behavior
The ~13 second runtime mentioned below would be a few milliseconds.
This may not be an issue. While writing the first iteration of this code I imagined it would be slow--but this does feel needlessly slow.
In the below example each AST type has a constant type field. I decreased the runtime of the below code by using the commented out "fast ast decoder." The fast version of this decoder first checks for the constant type field, then will choose the appropriate decoder to use (formerly returned in lazy calls) within a .transform
. Given these type fields are constant I think decoders could implement an optimization to handle this specific case.
To Reproduce
// test.js
const value = {
"type": "label",
"tok": "eq",
"body": {
"type": "lambda",
"params": {
"type": "ident",
"tok": "x",
"sibling": {
"type": "ident",
"tok": "y"
}
},
"body": {
"type": "eq",
"left": {
"type": "ident",
"tok": "x"
},
"right": {
"type": "ident",
"tok": "y"
}
}
},
"sibling": {
"type": "label",
"tok": "not",
"body": {
"type": "lambda",
"params": {
"type": "ident",
"tok": "status"
},
"body": {
"type": "cond",
"body": {
"type": "condset",
"left": {
"type": "ident",
"tok": "status"
},
"right": {
"type": "false"
},
"sibling": {
"type": "condset",
"left": {
"type": "true"
},
"right": {
"type": "true"
}
}
}
}
},
"sibling": {
"type": "list",
"body": {
"type": "ident",
"tok": "not",
"sibling": {
"type": "list",
"body": {
"type": "ident",
"tok": "eq",
"sibling": {
"type": "strlit",
"tok": "this was a",
"sibling": {
"type": "strlit",
"tok": "triumph"
}
}
}
}
}
}
}
}
const decoders = require('decoders');
const typeDecoder = decoders.either(
decoders.constant("cdr"),
decoders.constant("cons"),
decoders.constant("cond"),
decoders.constant("condset"),
decoders.constant("lambda"),
decoders.constant("ident"),
decoders.constant("label"),
decoders.constant("list"),
decoders.constant("atom"),
decoders.constant("quote"),
decoders.constant("eq"),
decoders.constant("strlit"),
decoders.constant("numlit"),
decoders.constant("true"),
decoders.constant("false"),
decoders.constant("car"),
);
// fast ast decoder
// const astDecoder = decoders.inexact({type: typeDecoder}).transform(value => {
// switch (value.type) {
// case "cdr": return cdrDecoder.verify(value);
// case "cons": return consDecoder.verify(value);
// case "cond": return condDecoder.verify(value);
// case "condset": return condSetDecoder.verify(value);
// case "lambda": return lambdaDecoder.verify(value);
// case "ident": return identDecoder.verify(value);
// case "label": return labelDecoder.verify(value);
// case "list": return listDecoder.verify(value);
// case "atom": return atomDecoder.verify(value);
// case "quote": return quoteDecoder.verify(value);
// case "eq": return eqDecoder.verify(value);
// case "strlit": return StrlitDecoder.verify(value);
// case "numlit": return NumlitDecoder.verify(value);
// case "true": return TDecoder.verify(value);
// case "false": return NilDecoder.verify(value);
// case "car": return carDecoder.verify(value);
// }
// })
// slow ast decoder
const astDecoder = decoders.either(
decoders.lazy(() => TDecoder),
decoders.lazy(() => NilDecoder),
decoders.lazy(() => listDecoder),
decoders.lazy(() => identDecoder),
decoders.lazy(() => atomDecoder),
decoders.lazy(() => quoteDecoder),
decoders.lazy(() => eqDecoder),
decoders.lazy(() => StrlitDecoder),
decoders.lazy(() => NumlitDecoder),
decoders.lazy(() => carDecoder),
decoders.lazy(() => cdrDecoder),
decoders.lazy(() => consDecoder),
decoders.lazy(() => condDecoder),
decoders.lazy(() => lambdaDecoder),
decoders.lazy(() => labelDecoder),
)
const listDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('list'),
body: astDecoder,
});
const identDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('ident'),
tok: decoders.string,
})
const atomDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('atom'),
body: astDecoder,
})
const quoteDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('quote'),
body: astDecoder,
})
const eqDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('eq'),
left: astDecoder,
right: astDecoder,
})
const StrlitDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('strlit'),
tok: decoders.string,
})
const NumlitDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('numlit'),
tok: decoders.string,
})
const TDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('true'),
})
const NilDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('false'),
})
const carDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('car'),
body: astDecoder,
})
const cdrDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('cdr'),
body: astDecoder,
})
const consDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('cons'),
left: astDecoder,
right: astDecoder,
})
const condSetDecoder = decoders.object({
sibling: decoders.optional(decoders.lazy(() => condSetDecoder)),
type: decoders.constant('condset'),
left: astDecoder,
right: astDecoder,
})
const condDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('cond'),
body: condSetDecoder,
})
const lambdaArgDecoder = decoders.object({
sibling: decoders.optional(decoders.lazy(() => lambdaArgDecoder)),
type: decoders.constant('ident'),
tok: decoders.string,
})
const lambdaDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('lambda'),
params: decoders.optional(lambdaArgDecoder),
body: astDecoder,
})
const labelDecoder = decoders.object({
sibling: decoders.optional(astDecoder),
type: decoders.constant('label'),
tok: decoders.string,
body: lambdaDecoder,
})
const start = new Date().getTime();
astDecoder.verify(value);
const end = new Date().getTime();
console.log(`run time: ${(end - start) / 1000} seconds`)
$ node test.js
The fast version of this decoder commonly runs in 0.002 seconds while the slow verson runs in ~13 seconds on my machine.
Additional context
Decoders version: 2.0.0
TypeScript version: N/A
Flow version: N/A
Node version: v16.14.0
Here's my problem. The data comes from the backend with the compound names of the properties with underline, but in the frontend I want them to be camelCase.
How the data comes from backend:
{ id: 'some string', status: 'some string', token: 'some string', due_date: 'some string', bank_slip_url: 'some string', invoice_url: 'some string', created: 'some string' }
Here's my frontend implementation:
export class LicensePaymentModel { id?: string; status?: PaymentStatusEnum; token?: string; dueDate?: Date; bankSlipUrl?: string; invoiceUrl?: string; created?: Date; };
export const licenseDecoder: Decoder<LicensePaymentModel> = object({ id: string, status: string, token: string, due_date: date, bank_slip_url: string, invoice_url: string, created: date });
I realized that since I'm using Decoder in my decoder, the object must have the properties in camelCase, as in my LicensePaymentModel model. I was wondering if there is any way for me to associate the properties that comes from JSON underline with the model's camelCase.
When adding decoders to an already functioning production environment, many times it is helpful to use a weakGuard
temporarily. Rather than throwing an exception it would console.error
the message (or maybe an option for a custom action?). This way the code can continue to function like it has, but the type can be refined as more variances come in. Eventually the weakGuard can be replaced with guard
once one is confident the type is correct.
How could one write the following Elm decoder?
intFromStringDecoder : Decoder Int
intFromStringDecoder =
Decode.string
|> Decode.map
(\str ->
String.toInt str
|> Maybe.map Decode.succeed
|> Maybe.withDefault (Decode.fail "Not a number")
)
If an object is passed in that refers to itself (like the express req
object), an infinite loop occurs.
Smallest reproducible example:
const { string } = require('decoders');
// Take any decoder and pass in some self-referential object
const req = {};
req.myself = req;
console.log(string(req)); // <--- RangeError: Maximum call stack size exceeded
Either decoders signature does not seem to infer types correctly from the input decoders:
import { object, string, either } from "decoders";
const UnionDecoder = either(
object({
/* dependent fields */
dependency: string,
dependant: string,
/* common fields */
common: string
}),
object({
common: string
})
)
type UnionDecoderType = DecoderType<typeof UnionDecoder>
/**
* { common: string; }
*/
/* this is not assignable */
const decoderTypeValue: UnionDecoderType = {
dependency: "this does not compile",
dependant: "value",
common: "hello"
}
the use case here is that some of the fields have to be defined together, so they cannot be simply optional
I'm not sure what exactly is wrong with the type signature, but I've had a similar pattern in a personal project and found that the following signature yields correct type:
import { Decoder, DecoderType, object, string } from "decoders";
const either = <A extends Array<Decoder<DecoderType<A[number]>>>>(
...arr: A
): Decoder<DecoderType<A[number]>> => {
throw Error("implementation")
}
const UnionDecoder = either(
object({
/* dependent fields */
dependency: string,
dependant: string,
/* common fields */
common: string
}),
object({
common: string
})
)
type UnionDecoderType = DecoderType<typeof UnionDecoder>
/*
{
dependency: string;
dependant: string;
common: string;
} | {
common: string;
}
*/
/* now this is assignable */
const decoderTypeValue: UnionDecoderType = {
dependency: "this does compile",
dependant: "value",
common: "hello"
}
/* and this is not assignable */
const decoderTypeValue2: UnionDecoderType = {
dependency: "this does not compile",
dependant: "value"
}
the signature can of course be cleaned up and extracted to utility types for readability, but that's the gist of it.
Tested with versions:
"decoders": "^2.0.1",
"typescript": "^4.9.4"
Imagine a view where you verify input, like a user setting a new password:
const bodyGuard = guard(object({
oldPassword: string,
newPassword: string,
}));
const body = bodyGuard(req.body);
And then a user does not send in a new password, so the guard throws an error which contains the following error message:
{
oldPassword: 'ohNo3s-thisSldntBeXposed!',
newPassword: undefined,
^^^^^^^^^ Must be string
}
Normally, these values provide great debugging context, but in some cases you may want to omit sensitive data from those places. Some ideas:
password
, passwd
, secretKey
, etc. This is pretty rigid.password
instead of string
. Or sensitive(string)
for a slightly more composable touch.guard(decoder, { echo: false })
or something like that.Currently, debrief only collects values and then annotates a string message to it. As of right now, it does not have a way to transform the values it serializes, but adding support for that in should be possible.
Description of the bug
Looks like the documentation of inexact
decoder is missing field for y
.
It is now
const decoder = inexact({
x: number,
});
While should be
const decoder = inexact({
x: number,
y: number,
});
as in exact
decoder above.
Or am I missing something?
First off, great library!
To guard against a non-empty string (in TypeScript) I have to do this now:
regex(/./, 'Field is required')
Q: Is there a cleaner way to do this?
I don't really like having to use a regexp for this - yes it is quite simple, but maybe there is (/can be invented) an even simpler/idiomatic way to do this. Empty strings are almost never useful in real-life JSON.
Note that the message is mandatory in the type definition of the regex guard, which is kinda annoying for my use, as I'm putting "Field is required" there (all my string fields use this regex guard, I never want empty strings in my JSON).
Maybe I'm missing something, interested to hear what you think.
In Flow, if you have a type User
:
type User = {|
name: string,
age: number,
address: {
street: string,
city: string,
state: string,
zip: number,
}
|}
You can grab the address type by using $PropertyType utility type:
type Address = $PropertyType<User, 'address'>
I have found this to be extremely useful in the case of using flow types that are generated from graphql queries, because it allows for better consistency in types throughout the code. My proposal is to add some similar functionality for decoders. If you had a similar decoder for that same user, looking like this:
import { object, number, string } from 'decoders';
const userDecoder = object({
name: string,
age: number,
address: object({
street: string,
city: string,
state: string,
zip: number,
})
});
That you would be able to access the address in this way:
import { propertyType } from 'decoders';
...
const addressDecoder = propertyType(userDecoder, 'address');
It would also be ideal to be able to access these even when nested within an array, similar to $ElementType.
I would find this extremely useful with generated decoders, for the same reason that it is useful with flow types. I would love to get your thoughts on this (or maybe it is already possible!)
Hi,
We wanted to publish a similar library at OuiCar based on our old blog post, but your library is much more mature. I especially like the way you handle the typing of the object
primitive. Good to see, thanks!
One remark: the parameter of a decoder is of type any
:
export type anything = any;
export type Guard<T> = anything => T;
export type Predicate<T> = T => boolean;
export type DecodeResult<T> = Result<Annotation, T>;
export type Decoder<T, F = anything> = F => DecodeResult<T>;
Could you use a type mixed
instead? This would force Flow to check that the casting are correct.
Is there an "intersection" composition available, such that
const a = object({a: number})
const b = object({b: boolean})
const c = ???(a, b)
would result in c
being a Decoder<{a: number, b: boolean}>
? Can this be achieved with some of the existing combinators?
io-ts
equivalent: https://gcanti.github.io/io-ts/modules/index.ts.html#intersection
It seems like AllowImplicit
is confusing Typescript into thinking that required properties are optional.
interface A {
a: string;
}
const aDecoder: Decoder<A> = object({
a: string
});
Output:
Type 'Decoder<{ a?: string; }, unknown>' is not assignable to type 'Decoder<A, unknown>'.
Type '{ a?: string; }' is not assignable to type 'A'.
Property 'a' is optional in type '{ a?: string; }' but required in type 'A'.ts(2322)
Live example: https://codesandbox.io/s/nostalgic-sinoussi-lkrq1?file=/src/index.ts
Hi there,
how would I decode recursive data structures, such as
type Folder = {
name: string,
subfolders: Folder[]
}
Elm has the lazy
decoder for this:
https://package.elm-lang.org/packages/elm/json/latest/Json-Decode#lazy
How would I solve this using this decoders
package?
These are missing from the docs:
Thanks for your work here! I love Elm Decoders/Encoders and I was missing it for Flow. I added this library in my project and flow is complaining about constants that are reassigned.
Error โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ node_modules/debrief/lib/serialize.js.flow:15:5
Cannot reassign constant s [1].
[1] 6โ function serializeString(s: string, width: number = 80) {
:
12โ }
13โ
14โ // Cut off a bit
15โ s = s.substring(0, width - 15) + '...';
16โ ser = JSON.stringify(s) + ' [truncated]';
17โ return ser;
18โ }
Error โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ node_modules/decoders/lib/either.js.flow:13:5
Cannot reassign constant s [1].
10โ */
11โ // istanbul ignore next
[1] 12โ function itemize(s: string = ''): string {
13โ s = indent(s);
14โ return '-' + s.substring(1);
15โ }
16โ
Found 2 errors
I am using Flow 0.72.0 in strict local mode.
The error is:
Cannot call object with object literal bound to mapping because:
โข property x is missing in object type [1] but exists in object literal [2].
โข property y is missing in object type [1] but exists in object literal [2].
[2] 18โ const pointDecoder = object({
19โ x: number,
20โ y: number,
21โ });
node_modules/decoders/object.js.flow
[1] 78โ export function object<O: { +[field: string]: AnyDecoder }>(mapping: O): Decoder<$ObjMap<O, $DecoderType>> {
For example:
export interface AnObject extends Record<string, string> {
id: number;
name: string;
}
I would need to find a way to combine
object({id: number, name: string})
//And
dict(string)
The following code works fine on TS 3.4.5:
type Example = {
eg: string;
};
const exampleDecoder: Decoder<Example> = object({
eg: string,
});
In TS 4.0.3 however, I got this message:
Type 'Decoder<{ eg?: string; }, unknown>' is not assignable to type 'Decoder<Example, unknown>'.
Type '{ eg?: string; }' is not assignable to type 'Example'.
Property 'eg' is optional in type '{ eg?: string; }' but required in type 'Example'.ts(2322)
Any ideas?
Firstly, apologies if this isn't an actual issue and is instead related to my naivety as to how to properly use the library.
Secondly, this is a great library.
My issue is relating to determining that a type is required, and not ever undefined, as far as the Typescript compiler is concerned.
If I have some code:
const testDecoder = object({ success: boolean });
TS determines the type to be:
Decoder<{
success?: boolean;
}, unknown>
My question is, how can I have typescript infer that the key success
is not optional?
If we create a Decoder
called dFoo
:
import * as d from 'decoders';
const dFoo = d.object({
id: d.string,
name: d.optional(d.string)
});
Then infer the type from it via:
type Foo = d.$DecoderType<typeof dFoo>;
The inferred type is:
type Foo = {
id: string;
name: string | undefined;
}
That means when we use Foo
we need to set a name
key equal to undefined
, instead of just omitting it:
const someFoo: Foo = {
id: '42',
name: undefined
};
Is it possible to have the inferred type use ?
so we can just omit the optional key:
type Foo = {
id: string;
name?: string;
}
Thanks
I would like to make a decoder similar to dispatch. Except instead of having a type
field, I have a series of objects each with a single key:
type Foo =
| { circle: { x: number; y: number; radius: number } }
| { square: { x: number; y: number; size: number } }
| { rectangle: { x: number; y: number; length: number; width: number } };
// Not sure how to get this line to correctly infer the type.
const fooDecoder: Decoder<Foo> = dispatch({
circle: exact({ x: number, y: number, radius: number }),
square: exact({ x: number, y: number, size: number }),
rectangle: exact({ x: number, y: number, length: number, width: number }),
});
const verifyFoo = guard(fooDecoder);
verifyFoo({ circle: { radius: 5 } });
// ^^ Decoder error: Missing fields x and y
Composing these either9
get cumbersome and I would like easily digestible error messages. I am not sure how to formulate the types (typescript) so that inference works as expected.
Any Ideas or pointers? Much appreciated.
Let's say there is an enum like this:
enum Alphabet {
a = "a",
b = "b"
}
A possible decoder would be:
const decoder = either2(constant("a"), constant("b"))
This doesn't work for enums with more than 9 cases atm. What do you think of introducing another method that checks input agains an array of values with the same type?
Something like oneOf(values: [T])
Description of the bug
As subject says, D.url
will reject URLs that have forward slashes in the query part.
Expected behavior
Those URLs are valid and should be accepted. See https://datatracker.ietf.org/doc/html/rfc3986#section-3.4 where / is explicitly called out as a valid character in query.
To Reproduce
const D = require('decoders')
D.url.verify("https://example.com/foo?q=foo/bar")
Additional context
Decoders version: 2.0.3
TypeScript version: 5.1.3
Node version: 18.14.2
As per here: https://nvie.com/posts/introducing-decoders/#error-messages
Thanks for the library btw, it's great!
Please confirm:
What is the problem?
Description
We use something like WeakGuard
as the default. Is there some built-in mechanism that doesn't involve wrapping verify in a try/catch ?
// Paste your v1.x code here
Running into flow errors, even when running the same version of flow as project.
Stevens-MacBook-Pro:labeltron stevefar$ flow --version
Flow, a static type checker for JavaScript, version 0.92.1
Stevens-MacBook-Pro:labeltron stevefar$ flow
Error โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ node_modules/decoders/lib/string.js.flow:61:5
Missing type annotation for V. V is a type parameter declared in function [1] and was implicitly instantiated at call of
compose [2].
node_modules/decoders/lib/string.js.flow
58โ * - url(['https', 'git+ssh']) accepts both https:// and git+ssh:// URLs
59โ */
60โ export const url = (schemes: Array<string> = DEFAULT_SCHEMES) =>
[2] 61โ compose(
62โ string,
63โ (value: string) => {
64โ const matches = value.match(url_re);
65โ if (!matches) {
66โ return Err(annotate(value, 'Must be URL'));
67โ } else {
68โ const scheme = matches[1];
69โ if (schemes.length === 0 || schemes.includes(scheme.toLowerCase())) {
70โ return Ok(value);
71โ } else {
72โ return Err(annotate(value, `URL scheme must be any of: ${schemes.join(', ')}`));
73โ }
74โ }
75โ }
76โ );
77โ
node_modules/decoders/lib/utils.js.flow
[1] 41โ export function compose<T, V>(decoder: Decoder<T>, next: Decoder<V, T>): Decoder<V> {
Found 1 error
Stevens-MacBook-Pro:labeltron stevefar$
Hi, can you help me fix the below?
THANKS
This:
object({
name: string,
address: optional(string),
})
Should decode to type:
{
name: string;
address?: string;
}
But actually decodes to type:
{
name: string;
address: string | undefined;
}
Which according to my TypeScript (3.7.5) makes address a mandatory key that may have value undefined.
I extract the type like this, maybe not right?
export const propGuard = guard(
object({
name: string,
address: optional(string),
})
);
export type MyProps = ReturnType<typeof propGuard>;
// This doesn't compile:
const props: MyProps = {
name: '123',
}
Runtime is fine and does allow for missing key.
I don't want to use a guard because I don't want to deal with a bunch of try catch
blocks but I do want to know when it has failed so I can log the issue and respond with an HTTP message. This write-up got me down the path of calling the decoder directly, rather than always wrapping a decoder in a guard.
I'm trying to do decoder(VAL).map(() => {}).mapError((e) => {})
and execute logic when I know VAL
has been decoded and typed or do something with e
in mapError
. I've examined the code and the docs in lemons.js and it's not clear to me how to get the nice err.message
that using a guard gives you. I see the inbound argument to mapError
is an Annotation
; how can I extract the message from an Annotation
?
I've tried e.message
and e.errValue()
and neither of them worked. I am using version 1.25 of decoders.
Using rest spread with a javascript Map does not throw any runtime errors or flow errors, but it un-intuitively returns an empty object.
const someMapping = guard(mapping(string))({ foo: 'bar' });
console.log({ ...someMapping });
// -> logs {}
Can we consider change the output of a mapping to a object or add a new decoder that acts like a mapping decoder, but returns an object?
A bit cryptic description, I know, but maybe the following example clarifies the issue:
import { array, Decoder, object } from "decoders"
type GenericArrayContainer<F> = {
value: F[]
}
// inferred type matches declared type
const genericArrayContainer = <F>(decoder: Decoder<F>): Decoder<GenericArrayContainer<F>> => object({
value: array(decoder),
})
type GenericValueContainer<F> = {
value: F
}
// inferred type does not match declared type
const genericValueContainer = <F>(decoder: Decoder<F>): Decoder<GenericValueContainer<F>> => object({
value: decoder,
})
as commented, in the first case the type is inferred correctly when the generic value is wrapped in another decoder, but if the field is declared with the dynamically passed decoder directly, type inference no longer works.
This was tried with:
"decoders": "^2.0.1",
"typescript": "^4.9.4"
Let me know if there is another approach to achieving what I'm trying to do here or if any further info is required.
I have a situation where I'm receiving a JSON event where one field contains a string containing serialized JSON. I would like to build a decoder that decodes the outer event as well as the nested serialized JSON string. Some pseudo-code to show what I mean:
const history = JSON.stringify({ employment: ["Captain of Costa Concordia", "Designer of the Vasa"] })
const event = {
name: 'John Smith',
age: 34,
history
}
const decoder = object({
name: nonEmptyString,
age: positiveInteger,
history: serializedJson(object({
employment: array(string)
}))
})
const parsed = decoder.verify(event)
console.log(parsed.history.employment[0]) // "Captain of Costa Concordia"
I went through the documentation on how to construct your own decoders, and I got it to work at runtime, but the resulting Typescript type is not quite right:
function serializedJson<T>(decoder: Decoder<T>): Decoder<T> {
return nonEmptyString.transform(value => JSON.parse(value)).then(decoder.decode)
}
The result of this is that the event I defined above would complain about my history
field being a string instead of an object, which makes sense since serializedJson<T>
has a return type of Decoder<T>
, but then I'm clearly missing how I can express that I expect the input type to be of type string
but the return type from verify
should be T
.
Any guidance would be appreciated. ๐
Hi again ๐ ,
in my front-end TypeScript code I have the following discriminated union (DU) type:
type FooA = {
duKind: "FooA",
myPropA: string
}
type FooB = {
duKind: "FooB",
myPropB: number
}
type Foo = FooA | FooB
My back-end REST API returns either
{
"foo" : {
"fooA": {
"myPropA": "Hello World"
}
}
}
or
{
"foo" : {
"fooB": {
"myPropB": 42
}
}
}
depending on the DU case.
How can I map this to my DU types using decoders?
const fooADecoder: Decoder<FooA> = ???;
const fooBDecoder: Decoder<FooB> = ???;
const fooDecoder: Decoder<Foo> = either2<Foo>(fooADecoder, fooBDecoder);
I guess for the duKind
I can use the constant
decoder, but the main problem is that I have to reduce the "object nesting" from { "foo": { "fooA": { "myPropA": ...} } }
in JSON to { myPropA: ... }
in TypeScript.
This problem probably is not limited to DUs, but a general question of how to reduce/compress/squeeze JSON object nesting levels within a decoder.
Any help would be much appreciated, since this is a show stopper for me right now.
You likely already know this, but I felt pretty chuffed when I got it working, and figured others might benefit from it!
https://gist.github.com/girvo/b4207d4fc92f6b336813d1404309baab
This demonstrates how one can extract the T
from a given Decoder<T>
into a real type definition, for use elsewhere in a code-base. Prior to this, I was duplicating it: writing a type Example = ...
then writing a const example: Guard<Example> = guard(object(...
with the run-time versions duplicated.
This lets one define them once for run-time, and re-use them at type-time!
About the only "gripe" I have, is that I'd love to be able to make the type exact.
// @flow
type Thing = {|
name: string
|}
But I don't know whether this is possible through inference yet?
Perhaps by using the $Exact<T>
utility type this can be achieved: https://flow.org/en/docs/types/utilities/#toc-exact
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.