[Rd] [xts, quantmod] segfault probelm when I work with memcpy function
Daniel Cegiełka
daniel.cegielka at gmail.com
Fri Sep 10 21:11:18 CEST 2010
W dniu 10 września 2010 20:32 użytkownik Brian G. Peterson
<brian at braverock.com> napisał:
> Daniel,
>
> I haven't tried your example, but I wonder why you aren't using C accessor
> methods defined by xts itself or at least derived from the significant
> amounts of C code in xts.
>
> For example, your test code seems to bear a close resemblance in principle
> to coredata.c, but you don't appear to have used or derived from that code.
I want to have a better expertise with SEXP and R and I hope in the
future I want to help to develop some R financial packages (xts, TTR
and blotter). So this is my main motivation.
>
> I understand that your 'real' problem is likely different from your
> contrived example, but it still seems that you should leverage the C code
> already in xts where possible.
I study xts code :) I could write some indicator in C and build xts
object directly from C, so you have only .Call('myind',IBM) and it's
return xts object. On large data and when you need to call this
function with high frequency it's much more faster then
.xts(.Call('myind',Cl(IBM)),getAttrib(IBM,'index')).
regards,
daniel
>
> Regards,
>
> - Brian
>
> On 09/10/2010 12:27 PM, Daniel Cegiełka wrote:
>>
>> Hi,
>>
>> I work with SEXP C code and with xts and quantmod packages. I try to
>> touch how xts internal works.
>>
>> So we have R session and:
>>
>>> ls()
>>
>> character(0)
>>>
>>> getSymbols('AAPL') # quantmod package
>>
>> [1] "AAPL"
>>>
>>> ls()
>>
>> [1] "AAPL"
>>>
>>> str(AAPL)
>>
>> An 'xts' object from 2007-01-03 to 2010-09-09 containing:
>> Data: num [1:929, 1:6] 86.3 84 85.8 86 86.5 ...
>> - attr(*, "dimnames")=List of 2
>> ..$ : NULL
>> ..$ : chr [1:6] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close" ...
>> Indexed by objects of class: [Date] TZ:
>> xts Attributes:
>> List of 2
>> $ src : chr "yahoo"
>> $ updated: POSIXct[1:1], format: "2010-09-10 18:42:10"
>>>
>>> tail(AAPL,5)
>>
>> AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume
>> AAPL.Adjusted
>> 2010-09-02 251.26 252.17 248.57 252.17 14811900
>> 252.17
>> 2010-09-03 255.09 258.78 254.50 258.77 18580100
>> 258.77
>> 2010-09-07 256.64 259.53 256.25 257.81 12234200
>> 257.81
>> 2010-09-08 259.78 264.39 259.10 262.92 18777000
>> 262.92
>> 2010-09-09 265.04 266.52 262.92 263.07 15642700
>> 263.07
>>>
>>> q()
>>
>> Save workspace image? [y/n/c]: y
>> [danice at entropy ~]$
>>
>>
>> It's looks, that xts data in mamory looks like this (Open=o,High=h,Low=l
>> etc):
>>
>> ooooo(...)ohhhhh(...)hlllll(...)lccccc(...)cvvvvv(...)vaaaaa(...)a
>>
>> So if I want to read Open price I need to read this array (from C
>> code) from x[0] to x[nrow(x)-1] where x is the pointer do AAPL.
>>
>>
>> I have to test functions - one based on memcpy and second based on for
>> loop:
>>
>>
>> #include<R.h>
>> #include<Rinternals.h>
>>
>> SEXP open(SEXP x) {
>> int nr=nrows(x);
>> SEXP r;
>> PROTECT(r=allocVector(REALSXP,nr));
>>
>> memcpy(&REAL(r)[0],&REAL(x)[0],nr*sizeof(double));
>>
>> UNPROTECT(1);
>> return(r);
>> }
>>
>>
>> SEXP open2(SEXP x) {
>> int P=0;
>> if (TYPEOF(x) != REALSXP) { PROTECT(x = coerceVector(x,REALSXP));
>> P++; }
>> double *d_x = REAL(x);
>> int nr = nrows(x);
>>
>> SEXP s;
>> PROTECT(s = allocVector(REALSXP,nr));
>> P++;
>> double *d_s = REAL(s);
>>
>> int i;
>> for (i=0;i<nr;i++) d_s[i] = d_x[i];
>>
>> UNPROTECT(P);
>> return(s);
>> }
>>
>> We starts R session again and:
>>
>>
>>> # we have AAPL data in memory
>>>
>>> ls()
>>
>> [1] "AAPL"
>>>
>>> tail(AAPL)
>>
>> AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume
>> AAPL.Adjusted
>> 2010-09-02 251.26 252.17 248.57 252.17 14811900
>> 252.17
>> 2010-09-03 255.09 258.78 254.50 258.77 18580100
>> 258.77
>> 2010-09-07 256.64 259.53 256.25 257.81 12234200
>> 257.81
>> 2010-09-08 259.78 264.39 259.10 262.92 18777000
>> 262.92
>> 2010-09-09 265.04 266.52 262.92 263.07 15642700
>> 263.07
>>>
>>> # now we call do open2 function
>>>
>>> .Call('open2',AAPL)
>>
>> [1] 86.29 84.05 85.77 85.96 86.45 94.75 95.94 94.59 95.68
>> 97.56 92.10 88.63 89.14 85.73 86.68 87.11 87.11 86.30 86.43
>> 84.86 86.23 84.12
>>
>> (...)
>>
>>>
>>> # and now call to open based on memcpy
>>>
>>> .Call('open',AAPL)
>>
>> [1] 86.29 84.05 85.77 85.96 86.45 94.75 95.94 94.59 95.68
>> 97.56 92.10 88.63 89.14 85.73 86.68 87.11 87.11 86.30 86.43
>> 84.86 86.23 84.12
>>
>> (...)
>>
>>
>>
>> AND HERE IS MY PROBLEM:
>>
>> We download new data:
>>
>>> getSymbols('IBM')
>>
>> [1] "IBM"
>>>
>>> #### WE DOWNLOAD NEW DATA
>>>
>>> getSymbols('IBM')
>>
>> [1] "IBM"
>>>
>>> # and try open2 function (based on for loop)
>>>
>>> .Call('open2',AAPL)
>>
>> [1] 86.29 84.05 85.77 85.96 86.45 94.75 95.94 94.59 95.68
>> 97.56 92.10 88.63 89.14
>>
>> (...)
>>
>>>
>>> # now we try open function based on memcpy
>>> # ... and we will have a segfault
>>>
>>> .Call('open',AAPL)
>>
>> *** caught segfault ***
>> address 0x2, cause 'memory not mapped'
>>
>> Possible actions:
>> 1: abort (with core dump, if enabled)
>> 2: normal R exit
>> 3: exit R without saving workspace
>> 4: exit R saving workspace
>> Selection: 3
>>
>>
>> I have this problem only if I download new data...
>>
>> Do someone know how can I solve this memcpy problem? memcpy should be
>> much faster in this kind of area... and I want to write some C based
>> extensions for xts package.
>>
>> Best regards,
>> daniel
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
>
More information about the R-devel
mailing list