Using super class in decoder with tags about msgspec

[jcrist]

Yeah, we don't support implicit unions based on subclasses for a few reasons:

• It can be unclear what is expected. Does the decoder expect exactly the TaggedBase message schema, or any subclass of it? Does the union include the base class, or only its subclasses? What if new subclasses of TaggedBase are defined after a Decoder is created?
• Supporting implicit unions based on subclasses means that you can't look at a message schema and know if a field is a union or a single struct. I think being explicit is better here.
• From an implementation detail, explicit unions are a lot simpler to support using msgspec's internal data model.

From a usability perspective, I wonder what could be done here. Better documentation could certainly help. I don't see an easy way to infer this mistake based on usage to raise a nicer warning.

We could also provide a utility to autogenerate a Union based on subclasses of a type. Something like:

import typing
import msgspec

T = typing.TypeVar("T")

def subclass_union(cls: typing.Type[T]) -> typing.Type[T]:
    """Returns a Union of all subclasses of `cls` (excluding `cls` itself)"""
    classes = set()
    def _add(cls):
        for c in cls.__subclasses__():
            _add(c)
        classes.add(cls)
    for c in cls.__subclasses__():
        _add(c)
    return typing.Union[tuple(classes)]  # type: ignore


class Base(msgspec.Struct, tag=True):
    pass


class Foo(Base):
    x: int


class Bar(Base):
    x: float


class Baz(Bar):
    y: float


decoder = msgspec.json.Decoder(subclass_union(Base))

print(decoder.decode(b'{"type": "Foo", "x": 1}'))
out = decoder.decode(b'{"type": "Bar", "x": 2.0}')
print(out)
if typing.TYPE_CHECKING:
    reveal_type(out)

$ python subclass.py
Foo(x=1)
Bar(x=2.0)
$ mypy subclass.py
subclass.py:41: note: Revealed type is "subclass.Base"
Success: no issues found in 1 source file
$ pyright subclass.py
No configuration file found.
No pyproject.toml file found.
stubPath /home/jcristharif/Code/msgspec/typings is not a valid directory.
Assuming Python platform Linux
Searching for source files
Found 1 source file
/home/jcristharif/Code/msgspec/subclass.py
  /home/jcristharif/Code/msgspec/subclass.py:41:17 - information: Type of "out" is "Base"
0 errors, 0 warnings, 1 information 
Completed in 0.728sec

For reasons that are unclear to me, both mypy and pyright type check properly if the output of subclass_union is passed on to a Decoder/decode function, but only pyright type checks properly if the output of subclass_union is used to define a larger composite type. mypy complains about a variable being used in a typing context. For example:

BaseUnion = subclass_union(Base)

class Message(msgspec.Struct):
    data: BaseUnion

For this reason, I'm reluctant to add this utility to msgspec itself, but if it helps you you're welcome to copy-paste it into your project.

[CamDavidsonPilon]

Thanks for the context, appreciate it. I feel the stance to be explicit here fits nicely with the purpose of the library.

For my own codebase, I'll find a nice compromise. Closing!