[R] The explanation of ns() with df =2

Xing Zhao zhaoxing at uw.edu
Thu Apr 17 07:22:02 CEST 2014 

Dear John,

Thanks for your patience, got your idea

Best,
Xing

On Wed, Apr 16, 2014 at 4:11 AM, John Fox <jfox at mcmaster.ca> wrote:
> Dear Xing,
>
> My point wasn't that you should add 1 df for a constant to each cubic but that you need not subtract 1 for continuity. I'm sorry that I didn't make that clear. When you use the natural spline in a regression, there's a constant in the model as a separate term with 1 df, which in effect adjusts the level of the regression curve (assuming one X for simplicity). If you add a constant to each component cubic, the constant in the model is rendered redundant, accounting for the extra df.
>
> I'm afraid that if this is still not sufficiently clear, I'll have to defer to someone with greater powers of explanation.
>
> Best,
> John
>
> -------------------------------------------------------------
> John Fox
> McMaster University
> Hamilton, Ontario, Canada
> http://socserv.socsci.mcmaster.ca/jfox
>
>> On Apr 16, 2014, at 12:10 AM, Xing Zhao <zhaoxing at uw.edu> wrote:
>>
>> Dear John
>>
>> Sorry I use 3 degree of freedom for  cubic spline. After using 4, it
>> is still not 2. I may make some naive mistake, but I cannot figure
>> out. Where is the problem?
>>
>> 4 (cubic on the right side of the *interior* knot 8)
>> + 4 (cubic on the left side of the *interior* knot 8)
>> - 1 (two curves must be continuous at the *interior* knot 8)
>> - 1 (two curves must have 1st order derivative continuous at the
>> *interior* knot 8)
>> - 1 (two curves must have 2nd order derivative continuous at the
>> *interior* knot 8)
>> - 1 (right side cubic curve must have 2nd order derivative = 0 at the
>> boundary knot 15 due to the linearity constraint)
>> - 1 (similar for the left)
>> = 3, not 2
>>
>> Thanks
>> Xing
>>
>>> On Tue, Apr 15, 2014 at 10:54 AM, John Fox <jfox at mcmaster.ca> wrote:
>>> Dear Xing,
>>>
>>>> -----Original Message-----
>>>> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-
>>>> project.org] On Behalf Of Xing Zhao
>>>> Sent: Tuesday, April 15, 2014 1:18 PM
>>>> To: John Fox
>>>> Cc: r-help at r-project.org; Michael Friendly
>>>> Subject: Re: [R] The explanation of ns() with df =2
>>>>
>>>> Dear Michael and Fox
>>>>
>>>> Thanks for your elaboration. Combining your explanations would, to my
>>>> understanding, lead to the following  calculation of degree of
>>>> freedoms.
>>>>
>>>> 3 (cubic on the right side of the *interior* knot 8)
>>>> + 3 (cubic on the left side of the *interior* knot 8)
>>>> - 1 (two curves must be continuous at the *interior* knot 8)
>>>
>>> You shouldn't subtract 1 for continuity since you haven't allowed a
>>> different level on each side of the knot (that is your initial counting of 3
>>> parameters for the cubic doesn't include a constant).
>>>
>>> Best,
>>> John
>>>
>>>> - 1 (two curves must have 1st order derivative continuous at the
>>>> *interior* knot 8)
>>>> - 1 (two curves must have 2nd order derivative continuous at the
>>>> *interior* knot 8)
>>>> - 1 (right side cubic curve must have 2nd order derivative = 0 at the
>>>> boundary knot 15 due to the linearity constraint)
>>>> - 1 (similar for the left)
>>>> = 1, not 2
>>>>
>>>> Where is the problem?
>>>>
>>>> Best,
>>>> Xing
>>>>
>>>>> On Tue, Apr 15, 2014 at 6:17 AM, John Fox <jfox at mcmaster.ca> wrote:
>>>>> Dear Xing Zhao,
>>>>>
>>>>> To elaborate slightly on Michael's comments, a natural cubic spline
>>>> with 2 df has one *interior* knot and two boundary knots (as is
>>>> apparent in the output you provided). The linearity constraint applies
>>>> beyond the boundary knots.
>>>>>
>>>>> I hope this helps,
>>>>> John
>>>>>
>>>>> ------------------------------------------------
>>>>> John Fox, Professor
>>>>> McMaster University
>>>>> Hamilton, Ontario, Canada
>>>>> http://socserv.mcmaster.ca/jfox/
>>>>>
>>>>> On Tue, 15 Apr 2014 08:18:40 -0400
>>>>> Michael Friendly <friendly at yorku.ca> wrote:
>>>>>> No, the curves on each side of the know are cubics, joined
>>>>>> so they are continuous.  Se the discussion in \S 17.2 in
>>>>>> Fox's Applied Regression Analysis.
>>>>>>
>>>>>>> On 4/15/2014 4:14 AM, Xing Zhao wrote:
>>>>>>> Dear all
>>>>>>>
>>>>>>> I understand the definition of Natural Cubic Splines are those
>>>> with
>>>>>>> linear constraints on the end points. However, it is hard to think
>>>>>>> about how this can be implement when df=2. df=2 implies there is
>>>> just
>>>>>>> one knot, which, according the the definition, the curves on its
>>>> left
>>>>>>> and its right should be both be lines. This means the whole line
>>>>>>> should be a line. But when making some fits. the result still
>>>> looks
>>>>>>> like 2nd order polynomial.
>>>>>>>
>>>>>>> How to think about this problem?
>>>>>>>
>>>>>>> Thanks
>>>>>>> Xing
>>>>>>>
>>>>>>> ns(1:15,df =2)
>>>>>>>               1           2
>>>>>>>  [1,] 0.0000000  0.00000000
>>>>>>>  [2,] 0.1084782 -0.07183290
>>>>>>>  [3,] 0.2135085 -0.13845171
>>>>>>>  [4,] 0.3116429 -0.19464237
>>>>>>>  [5,] 0.3994334 -0.23519080
>>>>>>>  [6,] 0.4734322 -0.25488292
>>>>>>>  [7,] 0.5301914 -0.24850464
>>>>>>>  [8,] 0.5662628 -0.21084190
>>>>>>>  [9,] 0.5793481 -0.13841863
>>>>>>> [10,] 0.5717456 -0.03471090
>>>>>>> [11,] 0.5469035  0.09506722
>>>>>>> [12,] 0.5082697  0.24570166
>>>>>>> [13,] 0.4592920  0.41197833
>>>>>>> [14,] 0.4034184  0.58868315
>>>>>>> [15,] 0.3440969  0.77060206
>>>>>>> attr(,"degree")
>>>>>>> [1] 3
>>>>>>> attr(,"knots")
>>>>>>> 50%
>>>>>>>   8
>>>>>>> attr(,"Boundary.knots")
>>>>>>> [1]  1 15
>>>>>>> attr(,"intercept")
>>>>>>> [1] FALSE
>>>>>>> attr(,"class")
>>>>>>> [1] "ns"     "basis"  "matrix"
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Michael Friendly     Email: friendly AT yorku DOT ca
>>>>>> Professor, Psychology Dept. & Chair, Quantitative Methods
>>>>>> York University      Voice: 416 736-2100 x66249 Fax: 416 736-5814
>>>>>> 4700 Keele Street    Web:   http://www.datavis.ca
>>>>>> Toronto, ONT  M3J 1P3 CANADA
>>>>>>
>>>>>> ______________________________________________
>>>>>> R-help at r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>>> PLEASE do read the posting guide http://www.R-project.org/posting-
>>>> guide.html
>>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide http://www.R-project.org/posting-
>>>> guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.

More information about the R-help mailing list