[R] multinom() residual deviance

[Bill.Venables at csiro.au]

The residual deviance from a multinomial model is numerically equal (up to round-off error) to that you would get had you fitted the model as a surrogate Poisson generalized linear model.  Here is a short demo building on your example

> set.seed(101)
> df <- data.frame(f = sample(letters[1:3], 500, rep = TRUE),
+                  x = rnorm(500))
> library(nnet)
> mm <- multinom(f ~ x, df)
# weights:  9 (4 variable)
initial  value 549.306144 
final  value 548.246724 
converged
> 
> X <- with(df, outer(f, levels(f), "==")+0)
> 
> DF <- within(expand.grid(a = 1:500, b = letters[1:3]),
+              a <- factor(a))
> DF <- cbind(DF, f = as.vector(X), x = df$x)
> 
> 
> gm <- glm(f ~ a + b/x, poisson, DF)  ## surrogate Poisson equivalent model
> 
> deviance(gm)  ## surrogate Poisson
[1] 1096.493
> deviance(mm)  ## multinomial from nnet package
[1] 1096.493
> 

Bill Venables
________________________________________
From: Sascha Vieweg [saschaview at gmail.com]
Sent: 09 April 2011 17:06
To: Venables, Bill (CMIS, Dutton Park)
Cc: saschaview at gmail.com; r-help at r-project.org
Subject: RE: [R] multinom() residual deviance

Hello

I am aware of the differences between the two models, excuse me
for being imprecise on that in my first posting. My question only
regards the "nature" or "structure" of the deviance, and thus
whether the residual deviance of the multinomial model is the same
residual deviance as reported by, say, glm. This is particularly
important, since I want to calculate a pseudo R-Squared as
follows (data from below):

library("nnet")
m <- multinom(y ~ ., data=df)
(llnull <- deviance(update(m, . ~ 1, trace=F)))
(llmod <- deviance(m))
(RMcFadden <- 1 - (llmod/llnull))

(I know that many statisticians here highly discourage people from
using the pseudo R-Squareds, however, not many editors read this
mailing list and still insist.)

The "MASS" book warning alerted me that the multinom residual
deviance may be of principally different structure/nature than the
glm one. Thus, while the calculation above holds for a glm, it
does not for a multinom model. Am I right? And how to fix?


On 11-04-09 05:43, Bill.Venables at csiro.au wrote:

> The two models you fit are quite different.  The first is a binomial model equivalent to
>
> fm <- glm(I(y == "a") ~ x, binomial, df)
>
> which you can check leads to the same result.  I.e. this model amalgamates classes "b" and "c" into one.
>
> The second is a multivariate logistic model that considers all three classes defined by your factor y, (and has twice the number of parameters, among other things).  The three clases, "a", "b" and "c" remain separate in the model.
>
> Hence the two models are not directly comparable, so why should the deviance be?
>
> Bill Venables.
> ________________________________________
> From: r-help-bounces at r-project.org [r-help-bounces at r-project.org] On Behalf Of Sascha Vieweg [saschaview at gmail.com]
> Sent: 09 April 2011 01:14
> To: r-help at r-project.org
> Subject: [R] multinom() residual deviance
>
> Running a binary logit model on the data
>
> df <- data.frame(y=sample(letters[1:3], 100, repl=T),
> x=rnorm(100))
>
> reveals some residual deviance:
>
> summary(glm(y ~ ., data=df, family=binomial("logit")))
>
> However, running a multinomial model on that data (multinom, nnet)
> reveals a residual deviance:
>
> summary(multinom(y ~ ., data=df))
>
> On page 203, the MASS book says that "here the deviance is
> comparing with the model that correctly predicts each person, not
> the multinomial response for each cell of the mininal model",
> followed by and instruction how to compare with the saturated
> model.
>
> For me as a beginner, this sounds like an important warning,
> however, I don't know what the warning exactly means and hence do
> not know what the difference between the residual deviance of the
> former (binary) and the latter (multinomial) model is.
>
> (I need the deviances to calculate some of the pseudo R-squares
> with function pR2(), package "pscl".)
>
> Could you give good advice?
>
> Thanks
> *S*
>
> --
> Sascha Vieweg, saschaview at gmail.com
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

--
Sascha Vieweg, saschaview at gmail.com