supported dtypes about pandas2

[max-sixty]

• different datetime precision & ranges (e.g. ms vs ns)

[chris-b1]

Under possible

• date with no time type (edit: on second thought, is this anything more than a Period[D]?)

[chrisaycock]

Some of the more extreme:

• IP addresses (both v4 and v6)
• fractions, storing numerator and denominator as integers

[datnamer]

@jreback wouldn't we just want a way to have user defined dtypes instead of hardcoding a limited list? Can Dynd help with this?

[jreback]

you certainly can have parameterized types. but completely generic types is a recipe for disaster.
what do you think is missing for primitive / logical typing?

[datnamer]

What do you mean by parameterized? What types can be parameterized and by what? The link is broken.

Sorry I'm a bit lost.

I'm thinking of having a column of distribution objects or linear models or agents with their own attributes.

[jreback]

that's much too high level - though potential for a another library to build on pandas type system is possible

we are taking about columns of primitives

paramterized are things like

datetime64[D]

[datnamer]

gotcha.

[wesm]

@datnamer either way, pandas needs to have its own metadata implementation (see the logical/physical decoupling discussion in https://pydata.github.io/pandas-design/internal-architecture.html#logical-types-and-physical-storage-decoupling). We do not want to delegate metadata details to a third party library. Data structures and computation are another matter on a case by case basis (i.e. assuming a library conforms to our memory representation expectations, we can use its algorithms). The tight coupling between metadata (numpy dtypes), memory representation, and algorithms/computation is part of why we are in the current mess.

[jreback]

maybe thing about this: pandas-dev/pandas#3443, which is about nested dtypes in a single object. On another vein should think about a union type (which is a essentially a restricted looking object dtype); SFrame has these.

[sinhrks]

+1 for sparse.

maybe including subtype in sparse and categorical is useful, like category[int64]