Bi-directional encoding/decoding about valibot

[fabian-hiller]

Thank you for your feedback! We can leave this open for now so developers can use this issue to discuss it. I will probably not investigate this further before v1, but it may be a feature we think about again for v2.

With the two schema approach, would we be able to differentiate at the type level the encoder and the decoder?

I am not quite sure what you mean. It is possible to infer both types. You can also determine the input and output type of the encoder based on the decoder. Here is an example:

import * as v from 'valibot';

const DateDecodeSchema = v.pipe(
  v.string(),
  v.isoDate(),
  v.transform((input) => new Date(input))
);

type DateDecodeInput = v.InferInput<typeof DateDecodeSchema>;
type DateDecodeOutput = v.InferOutput<typeof DateDecodeSchema>;

const DateEncodeSchema = v.pipe(
  v.date(),
  v.transform((date) => date.toISOString())
) satisfies v.GenericSchema<DateDecodeOutput, DateDecodeInput>; // <--

function doStuff<
  TDecoder extends v.GenericSchema,
  TEncoder extends v.GenericSchema<
    v.InferOutput<TDecoder>, // <--
    v.InferInput<TDecoder> // <--
  >,
>(decoder: TDecoder, encoder: TEncoder) {
  // More code here
}

const result = doStuff(DateDecodeSchema, DateEncodeSchema);

[fabian-hiller]

To be honest, I never thought about this use case. That is why Valibot's dataflow is optimized for a single direction.

Technically we could e.g. add an isomorphicTransform action that accepts a decode and an encode function. The encode method would then recursively search for 'isomorphic_transform' actions in the pipelines of the schema and execute its encode function. This seems a bit hacky at first and I am not sure if this is the right approach as bugs can easily occur when mixing transform or other transformation actions with isomorphicTransform.

const DateSchema = v.pipe(
    v.string(),
    v.isoDate('The date is badly formatted.'),
    v.isomorphicTransform(i => new Date(i), (date) => date.toISOString())
);

const encoded = v.encode(DateSchema, data);

Another and perhaps less buggy approach would be to add an encoder method that adds an encoding function to a schema. This way, encode could take only the type SchemaWithEncoder<...> as its first argument to directly execute its encoding function.

const DateSchema = v.encoder(
  v.pipe(
    v.string(),
    v.isoDate('The date is badly formatted.'),
    v.transform((i) => new Date(i))
  ),
  (date) => date.toISOString()
);

const encoded = v.encode(DateSchema, data);

Neither approach will work for complex nested data without writing a very complex and non-modular encode method. How do io-ts and effect/schema handle complex nested data? I think to natively support encoding a single entry within a complex and nested schema, we need to rethink the entire implementation.

A workaround for now might be to write two schemas. One that decodes and one that encodes.

import * as v from 'valibot';

const DateDecodeSchema = v.pipe(
  v.string(),
  v.isoDate(),
  v.transform((input) => new Date(input))
);

const DateEncodeSchema = v.pipe(
  v.date(),
  v.transform((date) => date.toISOString())
);

const decoded = v.parse(DateDecodeSchema, '2024-01-01');
const encoded = v.parse(DateEncodeSchema, decoded);

[Magellol]

How do io-ts and effect/schema handle complex nested data

I'd have to dig into the details of their source code, I'm not entirely sure at the moment. I would imagine there is some way of recursively calling the encode/decode function as it goes through each schema/properties.

Neither approach will work for complex nested data without writing a very complex and non-modular encode method

I think the second approach (v.encoder) will break co-location of the encode/decode functions. It would be totally possible to write the transform in some modules, import that schema, compose it with others and only later on write out the encoding function on a larger type, which can lead to potentially complex (and possibly easier to get out of sync) encoding functions. I think keeping encoding function co-located to the schema might lead to smaller, more composable encoders.

Perhaps the first approach would be a bit simpler 🤔 — I'm trying to think. I wrote a more complex example using io-ts here. It might be a bit far-fetched but this could represent a realistic scenario or a good test bed.

We have a few levels of codecs (DateFromString, Value/ValueFromString, Values/ValuesFromString). Using the first approach we could possibly imagine something like this

const DateFromString = v.pipe(
    v.string(),
    v.isoDate('The date is badly formatted.'),
    v.isomorphicTransform(i => new Date(i), (date) => date.toISOString())
);

const Value = v.tuple([v.string(), DateFromString]);
const ValueFromString = v.pipe(
  v.string(),
  // How do we handle potential failures?
  v.isomorphicTransform(x => JSON.parse(i), JSON.stringify),
  Value
);

const Values = v.record(v.string(), Value);
const ValuesFromString = v.pipe(
  v.string(),
  v.isomorphicTransform(() => ..., () => ...)
)

v.parse(ValuesFromString, 'name:["key","2020-01-01"];name2:["key2","2021-01-01"]') // Should parse
v.encode(ValuesFromString, {
  name: ["key", new Date("2020-01-01")]
}) // Should encode back to string

I don't know the details of valibot yet but during encode, would there be a way to walk up the chain of schema and call their isomorphic transform action as you encounter one? In a pseudo stack

- `DateFromString#isomorphicTransform`
- `ValueFromString#isomorphicTransform`
- `ValuesFromString#isomorphicTransform`

Let's keep brainstorming if that's something you'd like to see in the library. That said, the two schema approach works too and might be simple enough to not introduce additional complexity. With the two schema approach, would we be able to differentiate at the type level the encoder and the decoder? If so that might be fine.

const doStuff = (decoder: Schema<...>, encoder: Schema<...>) => {}

[fabian-hiller]

Valibot is currently implemented in such a way that each schema has it's own ._run method. This method usually contains only a few lines with the complete validation logic of the schema. This makes Valibot modular and small in terms of bundle size. When calling parse or safeParse, the internal ._run method of the schema is called. For nested schemas, the parent schema simply calls the ._run method of its children. To support encode without putting a lot of code into the encode function, each schema and action must support an internal ._decode method, but this would increase the bundle size for all schemas. Even if this functionality is not used.

[Magellol]

but this would increase the bundle size for all schemas

I see. It's definitely a tradeoff. I understand valibot is optimized for bundle size so if you feel that adding such functionality would undo some of the effort done to reduce its footprint as much as possible then perhaps this capability isn't worth it and we can close this issue. We can still use this issue as future reference should users of valibot feel they would benefit from isomorphic schemas.

Perhaps my last question is still relevant though?

With the two schema approach, would we be able to differentiate at the type level the encoder and the decoder?