red-blox / zap

A lightning fast networking solution for roblox.

Zap

Zap is a blazingly fast networking solution for Roblox.

Features

• Zap packs data into buffers with no overhead. The same data can be sent using a fraction of the bandwidth.
• Zap doesn't compromise on performance. Zap's packing and unpacking is typically faster than Roblox's generic encoding.
• Extended support of data types sendable through the network.
• For both the IDL and API, Zap is a joy to use. It's easy to learn, easy to use, and easy to debug. It's the best DX you'll find.
• Zap is fully secure. Buffers make reverse engineering your game's networking much harder and Zap validates all data received.

Find Zap at...

Zap is currently undergoing a rewrite, which can be found at the rewrite branch.

Otherwise versions 0.6.x are being maintained by @sasial-dev, on the 0.6.x branch.

Documentation

Documentation can be found here. Please note these docs are for 0.6.x.

Contributing

Contributions are welcome! Please open an issue or discuss in the Roblox Open Source Software Community Discord before you start working on a feature or bug fix so we can discuss it.

Logo

Zap's Logo is sourced from Twitter and is under the CC BY 4.0 license.

zap's Issues

[FEAT] Vector2 class support

Describe your feature

Support for the Vector2 class; this is more of a personal need so I apologize in advance. I use Zap for voxel replication and I only need to save 2 values instead of 3.

[FEAT] Channels

Describe your feature

Currently zap batches all reliable events together, and doesn't batch unreliables at all. This behavior can be undesirable in both cases, as batched reliables aren't ordered with the rest of Roblox's replication, and unbatched unreliables can use more bandwidth than expected.

Adding channels would enable users to specify what events are batched together, if they're batched at all. We can also enable users to decide exactly how many events are batched into one call, and how often zap should empty the queue.

This increased control of what events are batched enables zap to staticly verify that a batch of the maximum size can be sent with unreliables and won't be dropped.

channel A = {
	type: Reliable,
	batch: {
		size: 10, -- maximum of 10 events per batch
		time: 1, -- maxiumum of 1 second in between each send
	},
	-- if queue isn't specified the queue type is `event(64)`
}

channel B = {
	type: Unreliable,
	-- if batch isn't specified the events are unbatched
	queue: frames(10), -- 10 frames before remote is connected to
}

event Uses_A = {
	channel: A,
	from: Client,
	call: SingleAsync,
}

event Uses_B = {
	channel: B,
	from: Client,
	call: SingleAsync,
}

event Uses_B_Again = {
	channel: B,
	from: Client,
	call: SingleAsync,
}

The syntax allows for channels to specify their reliability type and batching. Events specify their channel with the new channel field. As channels are now specifying the type, events no longer specify their type field. Please note that all of these events are dataless events, but that is not a requirement with channels.

As queues are now specified by channel, the queue-related options will be removed.

Due to events no longer specifying their type, two channels should be included by default: Reliable and Unreliable. These channels use their respective reliability types, have an event(64) queue type, and have no batching. For almost all users these channels will be sufficient.

In the future when namespaces are added, channel declarations will behave the same as type declarations with namespaces.

Alternatives

We could make multiple different types to go in the type field for events and add a batch field, but this feels very weak from a user standpoint, and doesn't offer much from a performance standpoint.

Implementation Details

Each channel should create it's own remote, and it's own connections. If a channel has batching enabled it will have a stored outgoing queue. Channels should have separate event Ids, this way we can reuse Ids and potentially stay under the 256 limit.

[FEAT] Remove shadowing variable redefinitions

Describe your feature

Selene goes wild due to lots of 'shadowing variable'

Implementation Details

You could silence selene, however keeping track of what is already defined shouldn't be too hard in Codegen

[FEAT] Outgoing queue flushing

Describe your feature

If the server uses Zap to replicate reliable events to the client including reference to an Instance, then deletes that Instance, there's no guarantee the client will receive that Instance as non-nil, because deletion will replicate before Zap replicates its event triggers. If developers could flush a Zap queue, triggering immediate RemoteEvent:FireClients, they could ensure that the Instance reference exists when received by the client because property writes replicate on the same ordering channel as reliable RemoteEvents.

Developers would be able to flush a queue immediately before deleting important Instances.

Implementation Details

Generate a casing variant of flush_queue(channel) where channel has type of a union of string singleton channel names.

Additional context

While developers can type Instances as optional, this is an awkward solution. The server may use Zap to inform clients that a round has ended, then delete the map. Until the Zap event trigger replicates, other client code won't expect the map to be deleted, even if Zap network callbacks see it's nil.

Channels would complement this. Instead of flushing one queue for all reliable events, only the relevant queues need to be flushed. While channels will allow disabling batching, in this case batching is still desirable most of the time.

I think this could be introduced before channels and still be useful.

[FEAT] Add support for buffer as event data

Describe your feature

Add support for sending buffers as event data. This facilitates the ability to use custom compression algorithms.

Alternatives

N/A

[BUG] Zap incorrectly shadows variables which causes deserialization to fail

Describe the bug

Zap de-serialization will shadow variables and try to use the value of the variable before it was shadowed

Reproduction

https://zap.redblox.dev/playground.html?code=ZXZlbnQgZXhhbXBsZSA9IHsNCiAgICBmcm9tOiBTZXJ2ZXIsDQogICAgdHlwZTogUmVsaWFibGUsDQogICAgY2FsbDogU2luZ2xlU3luYywNCiAgICBkYXRhOiBtYXAgeyBbdThdOiBtYXAgeyBbdThdOiBib29sZWFuIH0gfQ0KfQ0K

If you look at the client you will see:

local key
local val
key = buffer.readu8(incoming_buff, read(1))
val = {}
for _ = 1, buffer.readu16(incoming_buff, read(2)) do
	local key
	local val
	key = buffer.readu8(incoming_buff, read(1))
	val = buffer.readu8(incoming_buff, read(1)) == 1 -- shadows the `val` table
	val[key] = val -- errors because `val` is now a boolean instead of the table
end

[FEAT] Add "OrderedUnreliable" Type

Describe your feature

An ordered, unreliable event type would help developers handle data that immediately expires. It would reduce boiler-plate code in which expired data packets are thrown out and is a natural addition to a networking generated-library. Consider the following example demonstrating the problem such an addition would solve:

• Some client sends three unordered packets modeling an object's physics data at a given moment in time via an unreliable remote in the following order: packet 1, packet 2, packet 3. Note that data in packet 1 is thus older than data in packet 2, and likewise to packets 2 and 3.
• The server receives the packets out of order (say 2, 1, 3), however, nonetheless processes the data (in the order received) and replicates it to other clients.
• Other clients apply the physics data in the order received (say 2, 1, 3 again). Because packet 2's data is older than packet 1's, some "rubberbanding" occurs when applying the physics data to the game world.

Alternatives

1. The control alternative would be to not have this type and simply leave packet ordering to developers. This is fine, but as said before such a type would be a natural addition to a networking library and would reduce boiler-plate code.
2. A new field ("order") could be added to the event constructor. I probably wouldn't do this because such a field would only be relevant for events with type set to Unreliable.

Implementation Details

Approach 1

A single, u32 "packetCount" variable for each event, incremented by the sender with each fire and used by the receiver to determine what order packets were sent, throwing out packets with a packetCount less than what has already been read.

Approach 2 (recommended)

Same functionality as before, but packetCount would be u16 and reset to 0 after the 65,535 limit is reached. When the receiver is at or near the limit, it would accept the first packet with a substantially small packetCount value (0, 1, 2, etc., not just 0 because the packet with packetCount 0 could fail to deliver). The receiver would not accept packets with huge packetCount differences (in case packet A has a packetCount of 65,535 and packet B's is 0, and packet B is delivered before packet A). At max sending speeds (every heartbeat), this result would result in a reset every ~18 min. I'm recommending this approach because it seems to be the sweet spot for timing resets and avoiding additional overhead.

Approach 3

Same as before, except a u8 packetCount is used. This would mean packetCount would reset every ~4 seconds at max speeds.

Additional context

Feel free to message me on discord: I'm @ReyDelCodigo.

[FEAT] Addition of an `OR` operator for types

Describe your feature

The addition of an OR operator for types will create better data definitions with less overhead, compared to tagged enum alternatives. The operator can function similar to Luau i.e. type bool_or_u8 = boolean | u8

Alternatives

Using tagged enums will in essence function as an OR, however there is more overhead.
i.e.

enum "Type" {
    Number {
        Value: f64,
    },

    String {
        Value: string,
    },
}

WILL CREATE -->

({
	Type: "Number",
	Value: (number),
} | {
	Type: "String",
	Value: (string),
})

meaning that there is now an overhead of both a string and the value's type.

[FEAT] Option for putting remote instances in a folder

We have some wally packages such as these, that we use internally:

• Character animation (replicated headtracking, hand/arm ik, etc)
• Interactables (proximity prompts with extras like better seat handling, also has custom replication)
• Physics package (better network ownership, with actions and low-cost replication to clients)
  and some others, but these are the important ones.

I'd like to migrate each individual package to use Zap for optimized networking, but I dont want to delegate all the remote definitions and handling to the consumers of these packages (games).
Right now each package can use the remote_scope option, and migrating to zap will work, but it means we'll have a lot of remotes directly in ReplicatedStorage. That indirectly means:

• Finding instances whenever ReplicatedStorage is opened is more difficult
• IDE autocomplete for instances under ReplicatedStorage gets cluttered

It would be nice to have a remote_folder setting, or something similar, where as long as all packages use the same option value, all remotes will be under this one folder. It doesn't need to support nesting or anything fancy, a single folder is fine.

[FEAT] Support for CFrames

Describe your feature

Zap should support the CFrame datatype.

Alternatives

The alternative is not supporting CFrames.

Implementation Details

The slowest way we can impliment it is by sending i32[12] from CFrame:GetComponents. This is expensive and wastes network bandwidth.

The challenge with CFrames is that Roblox uses a lot of compression methods for sending CFrames. We also should do this - we just need to research how it is done to avoid sending 12 i32s.

[FEAT] Roblox-ts Type generation

Add a flag that can be added to the compile such as --roblox-ts or opt you could add to enable it.
This would not require to rewrite the already existing code generation and would generate a .d.ts in the output location as well.

If you aren't apposed to the feature but don't feel like developing it i am more than happy to fork it and create a pull request.

Here is an example

event Test = {
    from: Server,
    type: Reliable,
    call: ManyAsync,
    data: struct {
        foo: u32,
        bar: string,
    },
}

Would generate something like this on the client

type Test = {
    foo: number;
    bar: string;
};

type Callback<T extends {}> = (value: T) => void;

interface ClientEvent<T extends {}> {
    On: (cb: Callback<T>) => void;
}

interface Generated {
    Test: ClientEvent<Test>;
}

declare const Exports: Generated;

export = Exports;

[FEAT] Seperate Output File Locations

Describe your feature

Currently, zap outputs in a folder with the client and shared code together. When using rojo, it can become very painful to have server and client code in the same directory. Zap should instead specifiy in the config file where the outputs should be.

Alternatives

• Stick with the extisting network folder for output.
• Define the outputs through the CLI instead of the config file

[FEAT] Support RemoteFunctions

Describe your feature

Events should be able to return a value

Alternatives

Creating two events (one for each way)

Implementation Details

Careful decision will have to be made as to if we use RemoteEvents under-the-hood and then create a mock remotefunction-like interface in zap (so we don't yield the thread!).

[FEAT] Provide support for Packet Profiler

Hi!

After a discussion in the packet profiler thread in the Roblox OSS server, I'd like to request zap adding support for the packet profiler plugin. As zap uses the one-remote schema, reading RemoteObject.Name becomes moot for the profiler.
In consideration of this, packet profiler provides support for custom naming of remotes. You can find the description detailed on the DevForum thread:
https://devforum.roblox.com/t/version-200-packet-profiler-easily-measure-all-of-your-remotes/1890924#support-for-remote-compression-libraries-8

[BUG] Luau reports typechecking error on strict

Describe the bug

Luau currently reports a type error in strict mode for code generated for the client.
This is due to when using :WaitForChild, the return value is Instance while we know that it's a RemoteEvent.
The function FireServer does not exist on type Instance thus correctly reporting an error.

Reproduction

Just generate any code for the client. It should cause a type error in strict mode.

Expected behavior

The generated code should not give off any type error.

Potential Fix

Typecast the resulting return value to be RemoteEvent / UnreliableRemoteEvent

[BUG] Zap fails to get the proper max size of enums

Describe the bug

Zap fails to get the max size of an enum type and defaults to 255 as its max value.

This does not cause an error, but a warning is generated saying "potentially oversize" due to this behavior in certain situations.

Reproduction

playground link

Expected behavior

Zap should infer the proper max size

Additional context

https://github.com/red-blox/zap/blob/0.6.x/zap/src/config.rs#L258

[FEAT] Support for roblox-ts

Describe your feature

Zap would create .d.ts files alongside the luau code to allow for roblox-ts support.

Alternatives

Outputting a TypeScript AST/file.

Additional context

N/A

[FEAT] Support for Color3s

Describe your feature

Zap should support the Color3 datatype.

Alternatives

Sending 3 doubles is less efficient than sending a single Color3.

Implementation Details

I think most people only need 8 bits of accuracy per color channel, but for HDR colors I guess you need to bump it to the next largest size which is 16 bits of accuracy per color channel. I would assume a Color3 channel range of [0, 1].

[FEAT] Set type

Describe your feature

A set type would allow for sending dictionaries where all the values are true. This can be optimized over a map since only the keys have to be sent.

Alternatives

Literal types like true or false could be added. Then a map { [string]: true } could be used instead assuming zap would be smart enough to optimize for this.

[FEAT] Support for Namespaces

Describe your feature

In a large complex game, the remotes used can generally be grouped into categories. Zap should support a namespace under the event struct, and when a namespace is specified, you should be able to access it through remotes.namespace.remote_name.

Alternatives

• Do not organise the events
• Use mutiple zap config files

Implementation Details

This should be implimented under the event struct with an optional namespace field.

[FEAT] Trim dead code in codegen

Describe your feature

Codegen currently has a lot of dead code if network definition files don't define enough variations of events. This makes linters upset because they flag unused functions and variables.

Alternatives

A simple workaround for this issue would be to include comments at the top of the generated code to silence Selene and the Luau linter. However, this isn't ideal and the better solution is to actually trim dead code.

alloc can reassign `outgoing_buff` which causes the write call to write to the old buffer

title explains it

[FEAT] Add option for exporting zap types into separate file.

Describe your feature

Currently types you write inside your zap file get exported in both the server and client files, this is usually fine. However, I'm pretty nitpicky about separation, and I am not a fan of having my shared files access client files just to access types.

Implementation Details

opt types_output = "src/shared/types/zap.types.luau
This would create a new file in the location specified, and then your server & client would gather the types necessary from there.

fire except doesn't generate closing end after the firing for loop

title explains itself

[BUG] TypeScript type generation should use semicolons

Describe the bug

Given an input like

opt typescript = true

event MyEvent = {
	from: Server,
	type: Reliable,
	call: ManyAsync,
	data: u8,
}

Zap will generate

// Server generated by Zap v0.6.1 (https://github.com/red-blox/zap)
export const MyEvent: {
	Fire: (Player: Player, Value: number) => void
	FireAll: (Value: number) => void
	FireExcept: (Except: Player, Value: number) => void
	FireList: (List: Player[], Value: number) => void
};

But TypeScript style usually dictates that object types should use semicolons like:

export const MyEvent: {
	Fire: (Player: Player, Value: number) => void;
	FireAll: (Value: number) => void;
	FireExcept: (Except: Player, Value: number) => void;
	FireList: (List: Player[], Value: number) => void;
};

Technically, this is a stylistic thing and not required.. but it will trigger common lint warnings.

Reproduction

playground link

Expected behavior

TypeScript type generation should properly use semicolons.

[FEAT] Add a fireSet function to events

Describe your feature

Events would get a fireSet(players: { [Player]: true }, ...) function. It's common to construct a set of players over a list of players. This API will be convenient in those scenarios since you don't have to convert to a list.

[FEAT] Support multiple instances of zap within one place

Describe your feature

I'm suggesting there is a new opt parameter to allow segmenting multiple zap net codes:

opt remote_scope = "name"

where this option will modify the remote emits from:

ZAP_UNRELIABLE
ZAP_RELIABLE

to:

name_UNRELIABLE
name_RELIABLE

or some other reasonable naming convention.

That way any conflicts between packages and place net code can be resolved.

Alternatives

There is no known alternatives as of writing that would allow this functionality.

Implementation Details

Implementation should change the emit of the remote name based on whether or not this option is set. The implementation is relatively simple as it involves just adding a condition to check if an option is set and changing the remote name in emits.

Additional context

Right now, zap can only be used once within a given place file. While this is useful enough for many people, this implementation, especially as a package writer, this is not enough. Zap should be able to segment net code in packages away from other net code in other places.

[BUG] des ranges aren't max inclusive

Describe the bug

Ranges aren't max inclusive, the value is 1, but it's checking if it's < 1

Reproduction

Link to playground with accurate reproduction
https://zap.redblox.dev/playground.html?code=ZXZlbnQgcm91bmQgPSB7Cglmcm9tOiBTZXJ2ZXIsCgl0eXBlOiBSZWxpYWJsZSwKCWNhbGw6IFNpbmdsZVN5bmMsCglkYXRhOiBTdHJpbmdbNF0KfQoKZXZlbnQgY29tbWFuZCA9IHsKCWZyb206IENsaWVudCwKCXR5cGU6IFVucmVsaWFibGUsCgljYWxsOiBTaW5nbGVTeW5jLAoJZGF0YTogewoJCWRlbHRhVGltZTogZjMyLAoJCVg6IGk4ICgtMS4uMSksCgkJWjogaTggKC0xLi4xKQoJfSwKfQo=

Expected behavior

That ranges are max inclusive, so that this bug doesn't happen

[BUG] TypeScript type generation should use `declare` keyword

Describe the bug

Given an input like

opt typescript = true

event MyEvent = {
	from: Server,
	type: Reliable,
	call: ManyAsync,
	data: u8,
}

Zap will generate

// Server generated by Zap v0.6.1 (https://github.com/red-blox/zap)
export const MyEvent: {
	Fire: (Player: Player, Value: number) => void
	FireAll: (Value: number) => void
	FireExcept: (Except: Player, Value: number) => void
	FireList: (List: Player[], Value: number) => void
};

This is missing the declare keyword. Not technically a requirement, but good practice.
TypeScript docs

Reproduction

playground link

Expected behavior

// Server generated by Zap v0.6.1 (https://github.com/red-blox/zap)
export declare const MyEvent: {
	Fire: (Player: Player, Value: number) => void
	FireAll: (Value: number) => void
	FireExcept: (Except: Player, Value: number) => void
	FireList: (List: Player[], Value: number) => void
};

[BUG] String arrays in nested structs does not generate valid Luau

Describe the bug

String arrays in nested structs causes resulting Luau code to attempt to create invalid variable names.

event Event = {
	from: Server,
	type: Unreliable,
	call: ManyAsync,
	data: struct {
		nested: struct {
			tags: string[]
		}
	}
}

will generate

local Value.nested_tags_v = Value.nested.tags[i]

Reproduction

Playground

Expected behavior

It should properly access nested values and generate valid code.

[BUG] Maps with numeric keys drop non-sequential keys

Reproduction

event test = {
    from: Server,
    type: Reliable,
    call: ManyAsync,
    data: map {[i32]: string}
}

Send this table:

{
	[1] = "1",
	[2] = "2",
	[50] = "50",
	[100] = "100",
}

Keys 50 and 100 will be dropped.

Expected behavior

Current behavior is consistent with Roblox's convention that an array is a table with a continuous sequence of integer keys beginning at 1. My Zap type is not string[] though, it's a map from i32. Expected behavior is for Zap to correctly serialize and deserialize what Roblox doesn't.

[FEAT] make range max inclusive by default

Describe your feature

Rust uses the range syntax of 1..=5 because they have arrays/vecs which start indexing at 0, and the highest index of an array is one less than the length. In luau indexing starts at 1, and the highest index of an array is the length, so it would make more sense and be more intuitive to make the max part of a range inclusive.

This is a breaking change. It would change the behavior of any currently non-inclusive ranges, and cause parse errors for any inclusive ranges. As we are in early 0.x, this is acceptable for the benefit it will bring over time.

Alternatives

• Using mathematical interval notation: [0, 10) = inclusive 0, exclusive 10
  This would likely create more confusion, and it becomes harder to provide good syntax highlighting and other editor features.

• Using a function-like syntax: Limit(u8, 0, 9) = inclusive 0, inclusive 9
  This has the change of making max inclusive, but creates large syntax that only works for numeric types.

• Using commas instead of ..: 1, 9
  This would not allow for one sided ranges or exact ranges.

Implementation Details

This makes the parsing and implementation for Ranges simpler, and should have minimal effect on Range usage.

Additional context

N/A

syntax error when generating event listener

When zap generates the reliable/unreliable event listener it uses if for an event with id 1, and uses elseif for all others. This is a syntax error if the event with id 1 isn't being listened to in that single listener.

[FEAT] Make data field in events optional

Describe your feature

Making data optional would enable sending remotes without any data.

Alternatives

Adding a void type. This adds another type that isn't necessary and it's more difficult to maintain because you need to make sure it's being used correctly.

[FEAT] Zap LSP

Zap should have an LSP with a companion VSCode extension. This opens the door for other editor's support down the line.

[FEAT] Remove Instance Overhead

Describe your feature

Currently instances are placed into a single large array, and then the index of the instance in that array is written to the buffer. This wastes the two bytes written to the buffer. As Zap serializes and deserializes in the same order, instances will be written to the array in the same order.

A type like this: Instance(BasePart) is currently outputted as the following:

function types.write_basePart(value: basePart)
	local pos = alloc(2)
	buffer.writeu16(outgoing_buff, pos, alloc_inst(value))
end
function types.read_basePart()
	local value;
	value = incoming_inst[buffer.readu16(incoming_buff, read(2))]
	assert(value ~= nil)
	return value
end

With this change implemented this will be changed to this:

function types.write_basePart(value: basePart)
	table.insert(outgoing_inst, value)
end
function types.read_basePart()
	local value;
	value = table.remove(incoming_inst)
	assert(value ~= nil)
	return value
end

Again, this is possible because we serialize and deserialize in the same order - so instances will be written and read from the inst buffer in the same order.

Alternatives

I have not considered any alternatives at this moment.

Implementation Details

This could cause issues if we ever have non-deterministic serialization or deserialization order in the future.

Additional context

N/A

[BUG] `unknown` type can't be used in `map` values

Describe the bug

unknown can't be used everywhere. Specifically, I found that it can't be used as map keys or values.

Reproduction

https://zap.redblox.dev/playground.html?code=ZXZlbnQgdmFsdWUgPSB7DQogICAgZnJvbTogU2VydmVyLA0KICAgIHR5cGU6IFJlbGlhYmxlLA0KICAgIGNhbGw6IFNpbmdsZVN5bmMsDQogICAgZGF0YTogbWFwIHsgW3N0cmluZ106IHVua25vd24gfSwNCn0NCg0KZXZlbnQga2V5ID0gew0KICAgIGZyb206IFNlcnZlciwNCiAgICB0eXBlOiBSZWxpYWJsZSwNCiAgICBjYWxsOiBTaW5nbGVTeW5jLA0KICAgIGRhdGE6IG1hcCB7IFt1bmtub3duXTogc3RyaW5nIH0sDQp9

[BUG] TypeScript type generation should only use tuples for small arrays

Describe the bug

Given an input like this

opt typescript = true

event MyEvent = {
	from: Server,
	type: Reliable,
	call: ManyAsync,
	data: u8[5],
}

Zap will generate TypeScript types like:

Fire: (Player: Player, Value: [number, number, number, number, number]) => void

However.. if you use something like data: u8[128][128][128], this very quickly gets out of hand with 2 million number types.

Instead, if an array type exceeds a certain size, Zap should just emit number[].

Reproduction

playground link

Expected behavior

Zap should emit TypeScript types which don't make the IDE explode.

constrain max size for unreliable events to 900 bytes

It's entirely possible for zap to calculate the maximum size (potentially infinity) for any given type. Zap should refuse to generate code for unreliable events that could possible fire with more than 900 bytes. This will enable developers to know ahead of time that they're unreliable events will always work.

[BUG] `unknown` typed as `unknown?`

Describe the bug

Using unknown results in unknown?:

Reproduction

https://zap.redblox.dev/playground.html?code=ZXZlbnQgdmFsdWUgPSB7DQogICAgZnJvbTogU2VydmVyLA0KICAgIHR5cGU6IFJlbGlhYmxlLA0KICAgIGNhbGw6IFNpbmdsZVN5bmMsDQogICAgZGF0YTogdW5rbm93biwNCn0NCg==

[FEAT] Structs, Enums, and Tagged Unions

Describe your feature

Recently enums were modified to be wrapped in () instead of {} due to ambiguity issues with structs. This change would remove ambiguity issues by using keywords, and would expand the capabilities of zap.

These changes are breaking changes, but are incredibly beneficial and expand the capabilities of Zap. This early into development these breaking changes are acceptable.

Reworked Structs

Currently structs are defined as such:

type t = {
	fieldName: fieldTy,
}

The proposed new syntax would add a struct keyword in front of the opening brace:

type t = struct {
	fieldName: fieldTy,
}

While this is a regression in terms of concise syntax, it enables enums to use braces which will remove confusion around parens being used for type constraints.

Reworked Enums

Currently enums are defined as such:

type t = ( one, two }

The proposed new syntax would add an enum keyword and replace the parens with braces:

type t = enum { one, two }

This has the same downsides and positives as the structs change, it also allows us to easily add new syntax for tagged union enums. Having empty enums is currently allowed, after this change it will be a syntax error.

Addition of Tagged Union Enums

Tagged unions are a somewhat commonly used data structure and are extremely useful for some types of data. They are comparable in structure to the Rust enum type. These tagged unions are structs consist of a field which stores the discriminant, or tag, and data that changes depending on the tag. As a luau type:

type t = {
	type: "num",
	value: number,
} | {
	type: "str",
	value: string,
}

In it's current state Zap is unable to serialize these types, although they are possible to serialize. The proposed new syntax for these kinds of types is to add a string value between the enum keyword and opening brace: enum "type" {, used to determine the tag field, and require a struct to be defined after each enum variant:

type t = enum "type" {
	num { value: u8 },
	str { value: String },
}

The tag field cannot be used inside any of the enum variant's structs. This syntax fully enables tagged unions, both the serdes and types.

Alternatives

• The syntax enum (type) { could have been used for tagged unions.
  This syntax has the same issues with using parens for enums in the first place, it can get confused with type constraints.

• Structs could have remained the same while only adding a enum keyword for enums.
  This would create an inconsistency between the two types, and make it slightly harder to read and for beginners to learn. Coming from a language full of inconsistencies (luau), we want to avoid these wherever possible.

• Enums could have been top-level declarations and tagged unions would be formed by indexing enums into structs.
  This still has the problem of needing to union the structs.

Implementation Details

This adds a very significant amount of complexity to the parser, and will need a new internal Ty for tagged union enums. However this shouldn't require much modification of currently existing code.

Additional context

N/A

[FEAT] Clarify output FS errors

Describe your feature

While using the CLI, you can run into filesystem errors which don't clearly describe what the issue is.

These error messages should be more descriptive, pointing users to solutions. As a quality of life feature for output directories, any nonexistent directories should be created during execution.

Examples

For instance, a config that looks like this

opt output_client = ./network/client.luau
opt output_server = ./network/server.luau

...will cause the following generic and unclear filesystem error on Windows if no ./network directory exists:

Error: The system cannot find the path specified. (os error 3)

A similar error,

Error: The system cannot find the file specified. (os error 2)

...is output if the path provided to the config file is invalid.

Alternatives

Both of these are minor issues, so creating the directories in question and/or ensuring file paths are correct both work. The docs could also cover these common mistakes.

[FEAT] Safely handle requiring Zap in the Studio environment

Describe your feature

Zap in its current state fails to account for the case where its modules (most significantly the client's) are required before runtime. When using plugins that may require modules that contain a require() to the client's Zap module you are currently met with an infinite yield.

This is because the client modules call WaitForChild on Zap's remotes:

local reliable = ReplicatedStorage:WaitForChild("ZAP_RELIABLE")
local unreliable = ReplicatedStorage:WaitForChild("ZAP_UNRELIABLE")

These remotes aren't created until Zap's server module is required for the first time.

I propose adding an environment check at the beginning of both modules to safely return some sort of mock table if they are required outside of a server or client context.

Alternatives

The alternative for this is creating the two events manually in ReplicatedStorage, either by requiring the server module or by constructing them yourself. As a temporary solution, in the current version (0.6) of Zap this is fairly simple and doable.

However, going forward with #72 for future versions (0.7+) this will introduce a problem as there will be an arbitrary number of events to keep up with. These events will increase and decrease in number between edits to the zap config.

Implementation Details

Zap should continue to return a table and maintain the typings regardless of whether it's loaded successfully or not. This load status of the module should be determined by whether the module is required in its correct environment (server or client) and not in the Studio environment.

This loaded state should be accessible outside of Zap module so that code that may be dependent on it can maneuver around any fatal errors that may arise from indexing for non-existent events.

An example implementation that should maintain typings:

-- Check context anything else
local loadedSuccessfully: boolean = --> Ensure context

if not loadedSuccessfully then
  local out = { Loaded = loadedSuccessfully }
  return (out :: any) :: ReturnType
end

-- Zap Module Code

-- Create return table:
local returns = {
  Loaded = loadedSuccessfully
  -- Event/Function Tables
}
type ReturnType = typeof(returns)

return returns

[BUG] Luau reports typechecking error in strict mode on Server Output

Describe the bug

Luau reports a type checking error on the server output since variables reliable and unreliable are of type Instance?.

Reproduction

Any server output with an event

Expected behavior

Not produce a type error

[FEAT] Implement Integer Bit Sizes 24, 40, 48, 56, and 64

Describe your feature

Allow for all byte sizes within [1, 8] for unsigned and signed integers. This simply grants finer control over how many bytes network data may use to potentially save a lot of space.

Alternatives

There are no alternatives aside from rounding to 16 or 32 bits.

Implementation Details

Serializing and deserializing a non-power of 2 will require up to 3 buffer reads and writes, and the compiler will need to handle these cases.