[R] nearest correlation to polychoric

[Ravi Varadhan]

Martin,

I sent you the Matlab code for this which I had obtained from Nich Higham. 

Cheng, Sheung Hun and Higham, Nick (1998)
  A Modified Cholesky Algorithm Based on a Symmetric Indefinite
Factorization;
  \emph{SIAM J. Matrix Anal.\ Appl.}, \bold{19}, 1097--1110.

Do you remember?

Ravi.

----------------------------------------------------------------------------
-------

Ravi Varadhan, Ph.D.

Assistant Professor, The Center on Aging and Health

Division of Geriatric Medicine and Gerontology 

Johns Hopkins University

Ph: (410) 502-2619

Fax: (410) 614-9625

Email: rvaradhan at jhmi.edu

Webpage:  http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html

 

----------------------------------------------------------------------------
--------

-----Original Message-----
From: r-help-bounces at stat.math.ethz.ch
[mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Martin Maechler
Sent: Friday, July 13, 2007 10:29 AM
To: Dimitris Rizopoulos
Cc: "Jens Oehlschlägel"; r-help at stat.math.ethz.ch
Subject: Re: [R] nearest correlation to polychoric

>>>>> "DR" == Dimitris Rizopoulos <dimitris.rizopoulos at med.kuleuven.be>
>>>>>     on Fri, 13 Jul 2007 14:43:08 +0200 writes:

    DR> you could also have a look at function posdefify() from
    DR> package `sfsmisc'.

    DR> I hope it helps.

Yes, thanks, Dimitris; note that my posdefify() function uses a
pretty arbitrary "fudge" value for posdefification, namely
      eps.ev = 1e-07

As a matter of fact, earlier this week (in "the first
international R/Rmetrics workshop"),
I've talked to people from finance who also need that (or
something better?).

Jordi Molins Coronado (Madrid) drew my attention to an idea
he found in the book (English re-edition of French as of 1996)
Jean-Philippe Bouchaud (2000)
    Theory of Financial Risk and Derivative Pricing:
    From Statistical Physics to Risk Management

which supposedly uses theory of random matrices and
the entailing distribution of random eigenvalues in order to
find a more sensible cutoff than my 'eps.ev' default of 1e-7.
Unfortunately that book is checked out from our library and I
can't have a look.   Googling and Wikipedia seem to indicate to
me that most of the random matrix theory does not directly apply
here, since I'm really interested in the spectrum of X'X where X
is a de-meaned n x p random matrix.


OTOH, help(posdefify) already mentions more sophisticated
approaches to the problem, the one I (as I vaguely remember)
should be made available being 

  Cheng, Sheung Hun and Higham, Nick (1998)
  A Modified Cholesky Algorithm Based on a Symmetric Indefinite
Factorization;
  \emph{SIAM J. Matrix Anal.\ Appl.}, \bold{19}, 1097--1110.


Jens, could you make your code (mentioned below) available to
the community, or even donate to be included as a new method of
posdefify() ?

Regards,
Martin Maechler, ETH Zurich


    DR> Best, Dimitris

    DR> ---- Dimitris Rizopoulos Ph.D. Student Biostatistical
    DR> Centre School of Public Health Catholic University of
    DR> Leuven

    DR> Address: Kapucijnenvoer 35, Leuven, Belgium Tel:
    DR> +32/(0)16/336899 Fax: +32/(0)16/337015 Web:
    DR> http://med.kuleuven.be/biostat/
    DR> http://www.student.kuleuven.be/~m0390867/dimitris.htm


    DR> ----- Original Message ----- From: ""Jens Oehlschlägel""
    DR> <oehl_list at gmx.de> To: <r-help at stat.math.ethz.ch> Sent:
    DR> Friday, July 13, 2007 2:25 PM Subject: [R] nearest
    DR> correlation to polychoric


    DR> Dear all,

    DR> Has someone implemented in R (or any other language)

    DR> Knol DL, ten Berge JMF.  Least-squares approximation of
    DR> an improper correlation matrix by a proper one.
    DR> Psychometrika, 1989, 54, 53-61.

    DR> or any other similar algorithm?

    DR> Best regards


    DR> Jens Oehlschlägel


    DR> Background:

    DR> I want to factanal() matrices of polychoric correlations
    DR> which have negative eigenvalue. I coded

    DR> Highham 2002 Computing the nearest correlation matrix -
    DR> a problem from finance, IMA Journal of Numerical
    DR> Analysis (2002), 22, 329-343.

    DR> which basically works but still leaves very small
    DR> negative eigenvalues which causes factanal() to fail
    DR> with

    >> factanal(cov=ncor$cor, factors=2)
    DR> Fehler in optim(start, FAfn, FAgr, method = "L-BFGS-B",
    DR> lower = lower, : L-BFGS-B benötigt endliche Werte von
    DR> 'fn' Zusätzlich: Warning message: NaNs wurden erzeugt
    DR> in: log(x)
    >> version
    DR>                _ platform i386-pc-mingw32 arch i386 os
    DR> mingw32 system i386, mingw32 status major 2 minor 4.1
    DR> year 2006 month 12 day 18 svn rev 40228 language R
    DR> version.string R version 2.4.1 (2006-12-18)

______________________________________________
R-help at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.