[R] complicated time series filtering issue

[Cade, Brian S]

Hello:  I have an issue with filtering in a time series of animal growth data that seems conceptually simple but I have not come up with effective code to implement this.  I have temporal sequences of lengths by individuals and I want to retain only those data that are >10 days apart sequentially within an individuals records.  I can readily compute intervals between successive dates by individual using data.table() and its by = INDIVIDUAL functionality.  See example data for one individual below.  But what currently eludes me in processing this is how to recognize for example that deleting the 2nd and 3rd rows is required because the totality of their time interval is 9 days, deleting 8th record with 4 days is required,  deleting 17th record with 1 day is required, deleting 22nd and 23rd records is required because their sum is 2 days, but we do not delete 24th record of 10 days because the sum of previous 2 records deleted and this one is now 12 days.  Each individual can have very different patterns of these sort of sequences.  These sequences are easy to look at and determine what needs to be done but writing effective code to accomplish this filtering seems to require some functionality that I am currently missing.

Any suggestions would be greatly appreciated.

         Date   INDIVIDUAL DATENUMBER LENGTH length.prev interval
228 12-May-04 57084544        133         682.4           NA       NA
229 28-Sep-04 57084544        272         724.8        682.4      139
230 30-Sep-04 57084544        274         740.8        724.8        2
231  7-Oct-04 57084544        281         745.4        740.8        7
232 22-Nov-04 57084544        327         780.2        745.4       46
233 27-Jan-05 57084544        393         817.2        780.2       66
234  8-Mar-05 57084544        433         834.1        817.2       40
235  2-Jul-05 57084544        549         876.3        834.1      116
236  6-Jul-05 57084544        553         871.5        876.3        4
237  4-Aug-05 57084544        582         887.5        871.5       29
238 28-Dec-05 57084544        728         921.8        887.5      146
239 31-Jan-06 57084544        762         936.8        921.8       34
240 27-Feb-06 57084544        789         962.4        936.8       27
241 21-Nov-06 57084544       1056         972.3        962.4      267
242 30-Mar-07 57084544       1185        1007.2        972.3      129
243 23-Apr-07 57084544       1209        1009.1       1007.2       24
244 22-May-07 57084544       1238         991.6       1009.1       29
245 23-May-07 57084544       1239        1015.9        991.6        1
246 16-Jul-07 57084544       1293        1006.5       1015.9       54
247  9-Aug-07 57084544       1317        1013.0       1006.5       24
248 27-Aug-07 57084544       1335        1013.0       1013.0       18
249 29-Jul-08 57084544       1672        1021.5       1013.0      337
250 30-Jul-08 57084544       1673         984.3       1021.5        1
251 31-Jul-08 57084544       1674        1008.5        984.3        1
252 10-Aug-08 57084544       1684        1002.8       1008.5       10
253 22-Oct-08 57084544       1757         977.6       1002.8       73
254  2-Dec-08 57084544       1798        1000.6        977.6       41



Brian



Brian S. Cade, PhD

U. S. Geological Survey
Fort Collins Science Center
2150 Centre Ave., Bldg. C
Fort Collins, CO  80526-8818

email:  cadeb using usgs.gov<mailto:brian_cade using usgs.gov>
tel:  970 226-9326


	[[alternative HTML version deleted]]