[R] Remove highly correlated variables from a data frame or matrix
Jim Lemon
drj|m|emon @end|ng |rom gm@||@com
Thu Nov 14 22:39:25 CET 2019
I thought you were going to trick us. What I think you are asking now
is how to get the variable names in the columns that have at most one
_absolute_ value greater than 0.8. OK:
# I'm not going to try to recreate your correlation matrix
calc.jim<-matrix(runif(100,min=-1,max=1),nrow=10)
for(i in 1:10) calc.jim[i,i]<-1
rownames(calc.jim)<-<-colnames(calc.jim)<-paste0("rs",1:10)
Now that we have a plausible fake correlation matrix, all we have to
do is extract the column names:
colnames(calc.jim)[colSums(abs(calc.jim)>0.8)<2]
Of course, what you really meant could have been, "I want the column
names of the variables with at most one absolute value greater than
0.8 ignoring the diagonal values because I don't care about those". If
so:
colnames(calc.jim)[colSums(abs(calc.jim)>0.8)<3]
Any more tricks?
Jim
On Fri, Nov 15, 2019 at 8:17 AM Ana Marija <sokovic.anamarija using gmail.com> wrote:
>
> what would be the approach to remove variable that has at least 2
> correlation coefficients >0.8?
> this is the whole output of the head()
>
> > head(calc.rho)
> rs56192520 rs3764410 rs145984817 rs1807401 rs1807402 rs35350506
> rs56192520 1.000 0.976 0.927 0.927 0.927 0.927
> rs3764410 0.976 1.000 0.952 0.952 0.952 0.952
> rs145984817 0.927 0.952 1.000 1.000 1.000 1.000
> rs1807401 0.927 0.952 1.000 1.000 1.000 1.000
> rs1807402 0.927 0.952 1.000 1.000 1.000 1.000
> rs35350506 0.927 0.952 1.000 1.000 1.000 1.000
> rs2089177 rs12325677 rs62064624 rs62064631 rs2349295 rs2174369
> rs56192520 0.927 0.927 0.927 0.927 0.709 0.903
> rs3764410 0.952 0.952 0.952 0.952 0.728 0.928
> rs145984817 1.000 1.000 1.000 1.000 0.771 0.975
> rs1807401 1.000 1.000 1.000 1.000 0.771 0.975
> rs1807402 1.000 1.000 1.000 1.000 0.771 0.975
> rs35350506 1.000 1.000 1.000 1.000 0.771 0.975
> rs7218554 rs62064634 rs4360974 rs4527060 rs6502526 rs6502527
> rs56192520 0.903 0.903 0.903 0.903 0.903 0.903
> rs3764410 0.928 0.928 0.928 0.928 0.928 0.928
> rs145984817 0.975 0.975 0.975 0.975 0.975 0.975
> rs1807401 0.975 0.975 0.975 0.975 0.975 0.975
> rs1807402 0.975 0.975 0.975 0.975 0.975 0.975
> rs35350506 0.975 0.975 0.975 0.975 0.975 0.975
> rs9900318 rs8069906 rs9908521 rs9908336 rs9908870 rs9895995
> rs56192520 0.903 0.268 0.327 0.327 0.327 0.582
> rs3764410 0.928 0.276 0.336 0.336 0.336 0.598
> rs145984817 0.975 0.309 0.371 0.371 0.371 0.638
> rs1807401 0.975 0.309 0.371 0.371 0.371 0.638
> rs1807402 0.975 0.309 0.371 0.371 0.371 0.638
> rs35350506 0.975 0.309 0.371 0.371 0.371 0.638
> rs7211086 rs9905280 rs8073305 rs8072086 rs4312350 rs4313843
> rs56192520 0.880 0.268 0.327 0.880 0.880 0.880
> rs3764410 0.905 0.276 0.336 0.905 0.905 0.905
> rs145984817 0.951 0.309 0.371 0.951 0.951 0.951
> rs1807401 0.951 0.309 0.371 0.951 0.951 0.951
> rs1807402 0.951 0.309 0.371 0.951 0.951 0.951
> rs35350506 0.951 0.309 0.371 0.951 0.951 0.951
> rs8069610 rs883504 rs8072394 rs4280293 rs4465638 rs12602378
> rs56192520 0.582 0.903 0.582 0.582 0.811 0.302
> rs3764410 0.598 0.928 0.598 0.598 0.836 0.311
> rs145984817 0.638 0.975 0.638 0.638 0.879 0.344
> rs1807401 0.638 0.975 0.638 0.638 0.879 0.344
> rs1807402 0.638 0.975 0.638 0.638 0.879 0.344
> rs35350506 0.638 0.975 0.638 0.638 0.879 0.344
> rs9899059 rs6502530 rs4380085 rs6502532 rs4792798 rs4792799
> rs56192520 0.302 0.309 0.834 0.251 0.063 0.063
> rs3764410 0.311 0.318 0.858 0.259 0.080 0.080
> rs145984817 0.344 0.352 0.902 0.291 0.086 0.086
> rs1807401 0.344 0.352 0.902 0.291 0.086 0.086
> rs1807402 0.344 0.352 0.902 0.291 0.086 0.086
> rs35350506 0.344 0.352 0.902 0.291 0.086 0.086
> rs4316813 rs148563931 rs74751226 rs8068857 rs8069441 rs77397878
> rs56192520 0.006 0.006 0.006 0.006 0.006 0.006
> rs3764410 0.006 0.006 0.006 0.006 0.006 0.006
> rs145984817 0.006 0.006 0.006 0.006 0.006 0.006
> rs1807401 0.006 0.006 0.006 0.006 0.006 0.006
> rs1807402 0.006 0.006 0.006 0.006 0.006 0.006
> rs35350506 0.006 0.006 0.006 0.006 0.006 0.006
> rs75339756 rs4608391 rs79569548 rs4275914 rs11870422 rs8075751
> rs56192520 0.006 0.006 0.006 0.044 0.007 0.004
> rs3764410 0.006 0.006 0.006 0.042 0.005 0.005
> rs145984817 0.006 0.006 0.006 0.047 0.002 0.015
> rs1807401 0.006 0.006 0.006 0.047 0.002 0.015
> rs1807402 0.006 0.006 0.006 0.047 0.002 0.015
> rs35350506 0.006 0.006 0.006 0.047 0.002 0.015
> rs11658904 rs138437542 rs80344434 rs7222311 rs7221842 rs7223686
> rs56192520 0.003 0.004 0.004 0.033 0.009 0.000
> rs3764410 0.004 0.004 0.004 0.031 0.007 0.000
> rs145984817 0.010 0.004 0.004 0.035 0.011 0.005
> rs1807401 0.010 0.004 0.004 0.035 0.011 0.005
> rs1807402 0.010 0.004 0.004 0.035 0.011 0.005
> rs35350506 0.010 0.004 0.004 0.035 0.011 0.005
> rs78013597 rs74965036 rs78063986 rs118106233 rs117345712
> rs56192520 0.004 0.004 0.004 0.004 0.005
> rs3764410 0.004 0.004 0.004 0.004 0.006
> rs145984817 0.004 0.004 0.004 0.004 0.005
> rs1807401 0.004 0.004 0.004 0.004 0.005
> rs1807402 0.004 0.004 0.004 0.004 0.005
> rs35350506 0.004 0.004 0.004 0.004 0.005
> rs113004656 rs9898995 rs4985718 rs9893911 rs79110942 rs7208929
> rs56192520 0.004 0.033 0.033 0.023 0.004 0.023
> rs3764410 0.004 0.031 0.031 0.021 0.004 0.021
> rs145984817 0.004 0.035 0.035 0.025 0.004 0.025
> rs1807401 0.004 0.035 0.035 0.025 0.004 0.025
> rs1807402 0.004 0.035 0.035 0.025 0.004 0.025
> rs35350506 0.004 0.035 0.035 0.025 0.004 0.025
> rs12601453 rs4078062 rs75129280 rs76664572 rs78961289 rs146364798
> rs56192520 0.004 0.001 0.004 0.004 0.004 0.004
> rs3764410 0.004 0.002 0.004 0.004 0.004 0.004
> rs145984817 0.004 0.001 0.004 0.004 0.004 0.004
> rs1807401 0.004 0.001 0.004 0.004 0.004 0.004
> rs1807402 0.004 0.001 0.004 0.004 0.004 0.004
> rs35350506 0.004 0.001 0.004 0.004 0.004 0.004
> rs76715413 rs4078534 rs79457460 rs74369938 rs76423171 rs74668400
> rs56192520 0 0.004 0.004 0.002 0.004 0.004
> rs3764410 0 0.004 0.004 0.001 0.004 0.004
> rs145984817 0 0.004 0.004 0.005 0.004 0.004
> rs1807401 0 0.004 0.004 0.005 0.004 0.004
> rs1807402 0 0.004 0.004 0.005 0.004 0.004
> rs35350506 0 0.004 0.004 0.005 0.004 0.004
> rs75146120 rs1135237 rs9914671 rs117759512 rs4985696 rs16961340
> rs56192520 0.004 0.003 0.009 0.004 0.009 0.004
> rs3764410 0.004 0.003 0.007 0.004 0.007 0.004
> rs145984817 0.004 0.003 0.011 0.004 0.011 0.004
> rs1807401 0.004 0.003 0.011 0.004 0.011 0.004
> rs1807402 0.004 0.003 0.011 0.004 0.011 0.004
> rs35350506 0.004 0.003 0.011 0.004 0.011 0.004
> rs17794159 rs4247118 rs78572469 rs12601193 rs2349646 rs2090018
> rs56192520 0.001 0.033 0.002 0.004 0.020 0.033
> rs3764410 0.002 0.031 0.001 0.004 0.019 0.031
> rs145984817 0.001 0.035 0.005 0.004 0.022 0.035
> rs1807401 0.001 0.035 0.005 0.004 0.022 0.035
> rs1807402 0.001 0.035 0.005 0.004 0.022 0.035
> rs35350506 0.001 0.035 0.005 0.004 0.022 0.035
> rs12601424 rs4985701 rs8064550 rs2271521 rs2271520 rs11078374
> rs56192520 0.004 0.033 0.033 0.004 0.033 0.014
> rs3764410 0.004 0.031 0.031 0.004 0.031 0.012
> rs145984817 0.004 0.035 0.035 0.004 0.035 0.016
> rs1807401 0.004 0.035 0.035 0.004 0.035 0.016
> rs1807402 0.004 0.035 0.035 0.004 0.035 0.016
> rs35350506 0.004 0.035 0.035 0.004 0.035 0.016
> rs4985702 rs1124961 rs11652674 rs3924340 rs112450164 rs7208973
> rs56192520 0.033 0.003 0.002 0.001 0.004 0.033
> rs3764410 0.031 0.003 0.001 0.002 0.004 0.031
> rs145984817 0.035 0.003 0.005 0.001 0.004 0.035
> rs1807401 0.035 0.003 0.005 0.001 0.004 0.035
> rs1807402 0.035 0.003 0.005 0.001 0.004 0.035
> rs35350506 0.035 0.003 0.005 0.001 0.004 0.035
> rs9910857 rs78574480 rs8072184 rs12602196 rs6502563 rs3744135
> rs56192520 0.006 0.004 0.014 0.004 0.033 0.004
> rs3764410 0.005 0.004 0.012 0.004 0.031 0.004
> rs145984817 0.002 0.004 0.016 0.004 0.035 0.004
> rs1807401 0.002 0.004 0.016 0.004 0.035 0.004
> rs1807402 0.002 0.004 0.016 0.004 0.035 0.004
> rs35350506 0.002 0.004 0.016 0.004 0.035 0.004
> rs148779543 rs77689691 rs41319048 rs117340532 rs78647096 rs77712968
> rs56192520 0 0.004 0.004 0.002 0.004 0.004
> rs3764410 0 0.004 0.004 0.001 0.004 0.004
> rs145984817 0 0.004 0.004 0.005 0.004 0.004
> rs1807401 0 0.004 0.004 0.005 0.004 0.004
> rs1807402 0 0.004 0.004 0.005 0.004 0.004
> rs35350506 0 0.004 0.004 0.005 0.004 0.004
> rs16961396 rs80054920 rs7206981 rs4985740 rs3803762 rs77103270
> rs56192520 0.004 0.004 0.033 0.023 0.004 0.002
> rs3764410 0.004 0.004 0.031 0.021 0.004 0.001
> rs145984817 0.004 0.004 0.035 0.025 0.004 0.005
> rs1807401 0.004 0.004 0.035 0.025 0.004 0.005
> rs1807402 0.004 0.004 0.035 0.025 0.004 0.005
> rs35350506 0.004 0.004 0.035 0.025 0.004 0.005
> rs7207485 rs77342773 rs3826304 rs3744126 rs7210879 rs7211576
> rs56192520 0.029 0.004 0.004 0.004 0.023 0.006
> rs3764410 0.027 0.004 0.004 0.004 0.021 0.005
> rs145984817 0.031 0.004 0.004 0.004 0.025 0.002
> rs1807401 0.031 0.004 0.004 0.004 0.025 0.002
> rs1807402 0.031 0.004 0.004 0.004 0.025 0.002
> rs35350506 0.031 0.004 0.004 0.004 0.025 0.002
> rs117967362 rs75978745 rs6502564 rs9894565 rs36079048 rs8076621
> rs56192520 0.004 0.004 0.007 0.017 0 0.004
> rs3764410 0.004 0.004 0.005 0.015 0 0.004
> rs145984817 0.004 0.004 0.009 0.019 0 0.004
> rs1807401 0.004 0.004 0.009 0.019 0 0.004
> rs1807402 0.004 0.004 0.009 0.019 0 0.004
> rs35350506 0.004 0.004 0.009 0.019 0 0.004
> rs7218795 rs3803761 rs12602675 rs7208065 rs4985705 rs8080386
> rs56192520 0.026 0.032 0 0.018 0.014 0.003
> rs3764410 0.024 0.029 0 0.015 0.011 0.002
> rs145984817 0.028 0.034 0 0.021 0.016 0.004
> rs1807401 0.028 0.034 0 0.021 0.016 0.004
> rs1807402 0.028 0.034 0 0.021 0.016 0.004
> rs35350506 0.028 0.034 0 0.021 0.016 0.004
> rs8065832 rs2018781 rs1736221 rs1736220 rs1736217 rs1708620
> rs56192520 0.008 0.039 0.003 0.003 0.021 0.009
> rs3764410 0.006 0.037 0.002 0.002 0.019 0.007
> rs145984817 0.010 0.042 0.004 0.004 0.024 0.011
> rs1807401 0.010 0.042 0.004 0.004 0.024 0.011
> rs1807402 0.010 0.042 0.004 0.004 0.024 0.011
> rs35350506 0.010 0.042 0.004 0.004 0.024 0.011
> rs1708619 rs1736216 rs76319098 rs1736215 rs1736214 rs1708617
> rs56192520 0.009 0.024 0.017 0.012 0.019 0.029
> rs3764410 0.007 0.021 0.016 0.009 0.016 0.026
> rs145984817 0.011 0.026 0.018 0.014 0.022 0.031
> rs1807401 0.011 0.026 0.018 0.014 0.022 0.031
> rs1807402 0.011 0.026 0.018 0.014 0.022 0.031
> rs35350506 0.011 0.026 0.018 0.014 0.022 0.031
> rs12602831 rs12602871 rs1736213 rs1736212 rs76045368 rs34518797
> rs56192520 0.000 0.000 0.015 0.029 0.001 0.001
> rs3764410 0.001 0.001 0.013 0.026 0.001 0.001
> rs145984817 0.000 0.000 0.018 0.031 0.000 0.000
> rs1807401 0.000 0.000 0.018 0.031 0.000 0.000
> rs1807402 0.000 0.000 0.018 0.031 0.000 0.000
> rs35350506 0.000 0.000 0.018 0.031 0.000 0.000
> rs11078378 rs8079562 rs8065774 rs8066090 rs41337846 rs1736209
> rs56192520 0.043 0.001 0.001 0.029 0.000 0.029
> rs3764410 0.041 0.001 0.001 0.026 0.001 0.026
> rs145984817 0.046 0.000 0.000 0.031 0.000 0.031
> rs1807401 0.046 0.000 0.000 0.031 0.000 0.031
> rs1807402 0.046 0.000 0.000 0.031 0.000 0.031
> rs35350506 0.046 0.000 0.000 0.031 0.000 0.031
> rs1736208 rs12949822 rs76246042 rs12600635 rs55689224 rs1736207
> rs56192520 0.043 0.043 0.000 0.000 0.000 0.015
> rs3764410 0.041 0.041 0.001 0.001 0.001 0.013
> rs145984817 0.046 0.046 0.000 0.000 0.000 0.018
> rs1807401 0.046 0.046 0.000 0.000 0.000 0.018
> rs1807402 0.046 0.046 0.000 0.000 0.000 0.018
> rs35350506 0.046 0.046 0.000 0.000 0.000 0.018
> rs1708626 rs1736206 rs9896078 rs16961474 rs1708627 rs1736205
> rs56192520 0.015 0.015 0.001 0.001 0.017 0.021
> rs3764410 0.013 0.013 0.001 0.001 0.014 0.019
> rs145984817 0.018 0.018 0.000 0.001 0.020 0.024
> rs1807401 0.018 0.018 0.000 0.001 0.020 0.024
> rs1807402 0.018 0.018 0.000 0.001 0.020 0.024
> rs35350506 0.018 0.018 0.000 0.001 0.020 0.024
> rs1708628 rs7220577 rs2294155 rs1736204 rs1736203 rs1736202
> rs56192520 0.011 0.011 0.000 0.021 0.014 0.014
> rs3764410 0.009 0.009 0.000 0.019 0.011 0.011
> rs145984817 0.013 0.013 0.001 0.024 0.016 0.016
> rs1807401 0.013 0.013 0.001 0.024 0.016 0.016
> rs1807402 0.013 0.013 0.001 0.024 0.016 0.016
> rs35350506 0.013 0.013 0.001 0.024 0.016 0.016
> rs12937908 rs1736200 rs1708623 rs1708624 rs9894884 rs9901894
> rs56192520 0.009 0.009 0.008 0.007 0.009 0.009
> rs3764410 0.007 0.007 0.006 0.005 0.007 0.007
> rs145984817 0.011 0.011 0.010 0.008 0.011 0.011
> rs1807401 0.011 0.011 0.010 0.008 0.011 0.011
> rs1807402 0.011 0.011 0.010 0.008 0.011 0.011
> rs35350506 0.011 0.011 0.010 0.008 0.011 0.011
> rs9903294 rs2472689 rs1630656 rs111478970 rs3182911 rs7219012
> rs56192520 0.008 0.011 0.007 0.007 0.008 0.000
> rs3764410 0.006 0.009 0.005 0.005 0.006 0.000
> rs145984817 0.010 0.013 0.008 0.008 0.010 0.001
> rs1807401 0.010 0.013 0.008 0.008 0.010 0.001
> rs1807402 0.010 0.013 0.008 0.008 0.010 0.001
> rs35350506 0.010 0.013 0.008 0.008 0.010 0.001
> rs9890657 rs12453455 rs12947291 rs150267386 rs16961493 rs11652745
> rs56192520 0.009 0.007 0.008 0.013 0.000 0.009
> rs3764410 0.007 0.005 0.006 0.012 0.000 0.007
> rs145984817 0.011 0.008 0.010 0.014 0.001 0.011
> rs1807401 0.011 0.008 0.010 0.014 0.001 0.011
> rs1807402 0.011 0.008 0.010 0.014 0.001 0.011
> rs35350506 0.011 0.008 0.010 0.014 0.001 0.011
> rs9907107 rs8070574 rs4985759 rs3866959 rs7219248 rs6502568
> rs56192520 0.009 0.009 0.009 0.011 0.009 0.011
> rs3764410 0.007 0.007 0.007 0.009 0.007 0.009
> rs145984817 0.011 0.011 0.011 0.013 0.011 0.013
> rs1807401 0.011 0.011 0.011 0.013 0.011 0.013
> rs1807402 0.011 0.011 0.011 0.013 0.011 0.013
> rs35350506 0.011 0.011 0.011 0.013 0.011 0.013
> rs7220275 rs12450037 rs7225876 rs9892352 rs4985760 rs6502569
> rs56192520 0.009 0.008 0.007 0.011 0.011 0.011
> rs3764410 0.007 0.006 0.005 0.009 0.009 0.009
> rs145984817 0.011 0.010 0.008 0.013 0.013 0.013
> rs1807401 0.011 0.010 0.008 0.013 0.013 0.013
> rs1807402 0.011 0.010 0.008 0.013 0.013 0.013
> rs35350506 0.011 0.010 0.008 0.013 0.013 0.013
> rs1029830 rs2012954 rs1029832 rs2270180 rs8072402 rs7221553
> rs56192520 0.009 0.011 0.008 0.000 0.009 0.011
> rs3764410 0.007 0.009 0.006 0.000 0.007 0.009
> rs145984817 0.011 0.013 0.010 0.001 0.011 0.013
> rs1807401 0.011 0.013 0.010 0.001 0.011 0.013
> rs1807402 0.011 0.013 0.010 0.001 0.011 0.013
> rs35350506 0.011 0.013 0.010 0.001 0.011 0.013
> rs145597919 rs150772017 rs2041393 rs6502578 rs11078382 rs9912109
> rs56192520 0.013 0.013 0.005 0.005 0 0.005
> rs3764410 0.012 0.012 0.004 0.004 0 0.004
> rs145984817 0.014 0.014 0.006 0.006 0 0.006
> rs1807401 0.014 0.014 0.006 0.006 0 0.006
> rs1807402 0.014 0.014 0.006 0.006 0 0.006
> rs35350506 0.014 0.014 0.006 0.006 0 0.006
> rs12601631 rs11869054 rs11869079 rs9912599 rs7220057 rs9896970
> rs56192520 0 0 0 0 0 0
> rs3764410 0 0 0 0 0 0
> rs145984817 0 0 0 0 0 0
> rs1807401 0 0 0 0 0 0
> rs1807402 0 0 0 0 0 0
> rs35350506 0 0 0 0 0 0
> rs34121330 rs34668117 rs67773570 rs242252 rs955893 rs28583584
> rs56192520 0.000 0.000 0 0.002 0.001 0.013
> rs3764410 0.001 0.001 0 0.003 0.001 0.005
> rs145984817 0.000 0.000 0 0.002 0.001 0.004
> rs1807401 0.000 0.000 0 0.002 0.001 0.004
> rs1807402 0.000 0.000 0 0.002 0.001 0.004
> rs35350506 0.000 0.000 0 0.002 0.001 0.004
> rs9944423 rs7217764 rs11651957 rs73978990 rs8071007 rs56044345
> rs56192520 0.013 0.011 0.011 0.011 0.011 0.011
> rs3764410 0.005 0.004 0.004 0.004 0.004 0.004
> rs145984817 0.004 0.003 0.003 0.003 0.003 0.003
> rs1807401 0.004 0.003 0.003 0.003 0.003 0.003
> rs1807402 0.004 0.003 0.003 0.003 0.003 0.003
> rs35350506 0.004 0.003 0.003 0.003 0.003 0.003
> rs17804843
> rs56192520 0
> rs3764410 0
> rs145984817 0
> rs1807401 0
> rs1807402 0
> rs35350506 0
>
>
> On Thu, Nov 14, 2019 at 2:59 PM Abby Spurdle <spurdle.a using gmail.com> wrote:
> >
> > That's assuming your data was returned by head().
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list