[R] crr - computationally singular

[Laura Bonnett]

Hi Everyone,

Thank you for all your comments and suggestions.

I determined that I had a full rank model matrix by using the code:
> qr(covaeb)$rank
This is 17 which is equal to the number of covariates in the matrix, covaeb.

I cannot invert the model matrix using 'solve' as my matrix is not
square.  In the matrices ending in a, there are 1677 rows and 15
columns/covariates while in the matrices ending in b, there are 701
rows and 17 columns.

Thank you,

Laura

2009/6/26 Ravi Varadhan <RVaradhan at jhmi.edu>:
>
> How did you determine that you have "full rank" model matrix comprising 17
> predictors?  Are you able to invert the model matrix using `solve'?  If not,
> you still have collinearity problem.
>
> If you are, then the problem might be in the Newton's method used by `crr'
> to solve the partial-likelihood optimization.  The hessian matrix of the
> parameters might be singular during the iterations.  If this is the case,
> your best bet would be to just simplify the model, i.e. use fewer
> predictors.
>
> Ravi.
>
> ----------------------------------------------------------------------------
> -------
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor, The Center on Aging and Health
>
> Division of Geriatric Medicine and Gerontology
>
> Johns Hopkins University
>
> Ph: (410) 502-2619
>
> Fax: (410) 614-9625
>
> Email: rvaradhan at jhmi.edu
>
> Webpage:
> http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
> tml
>
>
>
> ----------------------------------------------------------------------------
> --------
>
>
> -----Original Message-----
> From: Laura Bonnett [mailto:l.j.bonnett at googlemail.com]
> Sent: Friday, June 26, 2009 6:22 AM
> To: Ravi Varadhan
> Cc: r-help at r-project.org
> Subject: Re: [R] crr - computationally singular
>
> Dear Sir,
>
> Thank you for your response.  You were correct, I had 1 linearly dependent
> column.  I have solved this problem and now the rank of 'covaeb' is 17
> (qr(covaeb)$rank = 17).  However, I still get the same error message when I
> use covaeb in the 'crr' function.
>
>> fit=crr(snearmb$with.Withtime,csaeb,covaeb,failcode=2,cencode=0)
> 8 cases omitted due to missing values
> Error in drop(.Call("La_dgesv", a, as.matrix(b), tol, PACKAGE = "base")) :
>  system is computationally singular: reciprocal condition number =
> 3.45905e-25
>
> Are there any other reasons why this may be happening?
>
> Thank you,
>
> Laura
>
> 2009/6/25 Ravi Varadhan <RVaradhan at jhmi.edu>:
>> This means that your design matrix or model matrix is rank deficient,
>> i.e it does not have linearly independent columns.  Your predictors are
> collinear!
>>
>>
>> Just take your design matrices "covaea" or "covaeb" with 17 predcitors
>> and compute their rank or try to invert them.  You will see the problem.
>>
>> Ravi.
>>
>> ----------------------------------------------------------------------
>> ------
>> -------
>>
>> Ravi Varadhan, Ph.D.
>>
>> Assistant Professor, The Center on Aging and Health
>>
>> Division of Geriatric Medicine and Gerontology
>>
>> Johns Hopkins University
>>
>> Ph: (410) 502-2619
>>
>> Fax: (410) 614-9625
>>
>> Email: rvaradhan at jhmi.edu
>>
>> Webpage:
>> http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Vara
>> dhan.h
>> tml
>>
>>
>>
>> ----------------------------------------------------------------------
>> ------
>> --------
>>
>>
>> -----Original Message-----
>> From: r-help-bounces at r-project.org
>> [mailto:r-help-bounces at r-project.org] On Behalf Of Laura Bonnett
>> Sent: Thursday, June 25, 2009 11:39 AM
>> To: r-help at r-project.org
>> Subject: [R] crr - computationally singular
>>
>> Dear R-help,
>>
>> I'm very sorry to ask 2 questions in a week.  I am using the package 'crr'
>> and it does exactly what I need it to when I use the dataset a.
>> However, when I use dataset b I get the following error message:
>> Error in drop(.Call("La_dgesv", a, as.matrix(b), tol, PACKAGE = "base")) :
>>  system is computationally singular: reciprocal condition number =
>> 1.28654e-24
>>
>> This is obviously as a result of a problem with the data but apart
>> from dataset a having 1674 rows and dataset b having 701 rows there is
>> really no difference between them.
>>
>> The code I am using is as follows where covaea and covaeb are matrices
>> of covarites, all coded as binary variables.
>> In case a:
>>> covaea <-
>>> cbind(sexa,fsha,fdra,nsigna,eega,th1a,th2a,stype1a,stype2a,stype3a,pg
>>> u
>>> 1a,pgu2a,log(agea),firstinta/1000,totsezbasea)
>>> fita <- crr(snearma$with.Withtime,csaea,covaea,failcode=2,cencode=0)
>>
>> and in case b:
>>> covaeb <-
>>> cbind(sexb,fshb,fdrb,nsignb,eegb,th1b,th2b,stype1b,stype2b,stype3b,st
>>> y
>>> pe4b,stype5b,pgu1b,pgu2b,(ageb/10)^(-1),firstintb,log(totsezbaseb))
>>> fitb <- crr(snearmb$with.Withtime,csaeb,covaeb,failcode=2,cencode=0)
>>
>> csaea and csaeb are the censoring indicators for a and b respectively
>> which equal 1 for the event of interest, 2 for the competing risks
>> event and 0 otherwise.
>>
>> Can anyone suggest a reason for the error message?  I've tried running
>> fitb with variants of covaeb and irrespective of the order of the
>> covariates in the matrix, the code runs fine with 16 of the 17
>> covariates included but then produces an error message when the 17th is
> added.
>>
>> Thank you for your help,
>>
>> Laura
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>
>