Suggestion for Encoding Categoricals about fes

[topepo]

It would make sense to add a subsection on supervised encodings in that chapter.

[alexpghayes]

I second the request for a section on likelihood encoding. I've been having a hard time finding a canonical reference on them. I'd also be curious if there are any comparisons between standard likelihood encoding and a multilevel encoding scheme like suggested here.

[topepo]

@alexpghayes When I do this for the town variable in the OkC data mentioned in Chapter 5 (glm = "fixed" and Bayesian glm as "pooled"), the log-odds estimates are pretty similar. However, the overall sample size isn't small so it would not shrink as much as smaller data sets

[AllardJM]

@topepo ,

Using a multi-level model can make a large difference when the cardinality is large and the data sparse, in my experience. Think about something like zipcodes and the use of logistic regression versus lmer(outcome~1 + (1|zipcode),data=DAT,family=binomial)
to estimate the encodings.

I like the idea of a supervised encoding section. It would be use mentioning transfer learning as well - since this is the starting point for most these days (e.g. NLP)

[dmi3kno]

I second the target encoding request. Additional useful references can be found in:
https://arxiv.org/abs/1611.09477

Supervised encoding extends beyond categorical features and one can think of, for example "y-aware dimensionality reduction methods" as a wider definition which would also capture "y-aware PCA"
http://www.win-vector.com/blog/2016/05/pcr_part2_yaware/

[alexpghayes]

@topepo Interesting. I'm planning to play around with this some in the next couple of days, I'll let you know if I find a compelling example where pooling makes a big difference.

[alexpghayes]

@topepo One example where partial pooling for the encoding can have an impact:

Code (rough, will improve): https://gist.github.com/alexpghayes/d1ffb5c78b2e063d29b415a1a6bcb44e

[topepo]

A first pass at some code for recipes is the embed package that uses stan or tensorflow. More documentation is here. Feedback is welcome.