ejikeugba / gofcat

Goodness-of-fit tests for categorical response models

gofcat

Overview

Post-estimation (goodness-of-fit) tests for some widely used categorical response models (CRM). Package currently supports inputs from objects of class serp(), clm(), polr(), multinom(), mlogit(), vglm() and glm(). Available tests include the Hosmer-Lemeshow, the Lipsitz and the Pulkstenis-Robinson tests for the binary, multinomial and ordinal logistic regression models. The proportional odds, adjacent category, and constrained continuation-ratio models are particularly supported at the ordinal level. Tests for the proportional odds assumptions in ordinal models are also possible with the Brant and the Likelihood-Ratio tests. Moreover, several summary measures of predictive strength (Pseudo R-squared) and error measurement metrics (brier score, misclassification rate and logloss) are also available for the binary, multinomial and ordinal models.

Example

require(serp)
set.seed(1)
n <- 200
dt <- data.frame(y = ordered(rbinom(n,2,0.5)), x1 = factor(rbinom(n,2,0.7)), x2 = runif(n))
sp <- serp(y ~ x1 + x2, slope="parallel", link = "logit", reverse= TRUE, data = dt)

## Goodness-of-fit
# Hosmer-Lemeshow test
hosmerlem(sp, tables = TRUE)
hosmerlem(sp, tables = TRUE, customFreq = rep(20,10))

# Lipsitz test
lipsitz(sp)
lipsitz(sp, customFreq = rep(20, 10))

# Pulkstenis-Robinson test
pulkroben(sp, test = "chisq", tables = TRUE)
pulkroben(sp, test = "deviance", tables = TRUE)

## Proportional odds test
brant.test(sp)
brant.test(sp, global = TRUE, call = TRUE)
LR.test(sp, call = TRUE)

## Error metrics
erroR(sp, type = "brier")
erroR(sp, type = "logloss")
erroR(sp, type = "misclass")

# with dataframe and custom threshold
df <- data.frame(y, sp$fitted.values)
erroR(df, type = "misclass", thresh = 0.7)

## Summary metrics
Rsquared(sp, measure = "ugba")
Rsquared(sp, measure = "mcfadden")

Installation and Use

Before installing gofcat, it is encouraged to have a recent version of R installed. The released version of gofcat can be installed from CRAN with:

install.packages("gofcat")

or the development version from GitHub with:

if (!require("devtools")) install.packages("devtools")
devtools::install_github("ejikeugba/gofcat")

Load gofcat into R environment with:

library(gofcat)

Community Guidelines

Pull requests are welcomed! Please submit your contributions to gofcat through the list of Pull Requests, following the contributing guidelines. To report issues and/or seek support, please file a new ticket in the issue tracker, and expect a feedback ASAP!

Code of Conduct

Please note that gofcat is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

References

Ugba, E. R. and Gertheiss, J. (2023). A Modification of McFadden's $R^2$ for Binary and Ordinal Response Models. Commun. Stat. Appl. Methods, 30, 49-63. https://doi.org/10.29220/CSAM.2023.30.1.049

Ugba, E. R. (2022). gofcat: An R package for goodness-of-fit of categorical response models. Journal of Open Source Software, 7(76), 4382, https://doi.org/10.21105/joss.04382

Fagerland, M. W. and Hosmer, D. W. (2017). How to test for goodness of fit in ordinal logistic regression models. Stata Journal, 17, 668-686.

gofcat's Issues

`lipsitz()` behaviour when there is a `grp`variable

Hi,
Thanks for this package, I have a question while conducting some tests for the review ( openjournals/joss-reviews#4382 ).

What does happen if you have a variable called 'grp' in the data ? I suspect some results depending on the variable, see the snippet bellow, adapted from the README.

dt <- data.frame(y = ordered(rbinom(n,2,0.5)), x1 = factor(rbinom(n,2,0.7)), x2 = runif(n))
dt$grp = dt$x1
sp <- serp(y ~ x1 +x2, slope="parallel", link = "logit", reverse= TRUE, data = dt)
sp_grp <- serp(y ~ grp +x2, slope="parallel", link = "logit", reverse= TRUE, data = dt)

#Same results
hosmerlem(sp,tables = TRUE)
hosmerlem(sp_grp,tables = TRUE)

#different results
lipsitz(sp)
lipsitz(sp_grp)

Link with `nnet`

Hello,

I am still conducting tests for the JOSS review (openjournals/joss-reviews#4382).
I was testing the use of a nnet fitted model. I just took the example from the help of the function, which is a binary response model but I think it should work ?

library(nnet)
library(MASS)
example(birthwt)
bwt.mu <- multinom(low ~ ., bwt)
erroR(bwt.mu)
Rsquared(bwt.mu)

I obtain the following error:

Error: more than one column of fitted values for a binary model is not allowed