# [R] Filtering data

John Fox jfox at mcmaster.ca
Wed Nov 7 13:46:59 CET 2001

```At 08:39 PM 11/6/2001 -0700, Matt Pocernich wrote:

>I am having difficulty filtering data.  I am working with flow data
>collected at a stream gage.  For each record, I have a date and flow
>value.  I have filtered this data to only include days when flow values
>exceed a given threshold.
>
>Here is my problem.  Within this subset of data, I often have several
>consecutive days above the threshold.  From this group of days, I wish to
>select the record (both date and flow) containing the maximum flow.  If an
>exceedance is isolated ( the preceeding and succeeding day is below the
>threshold) I also wish to select that record.
>
>For example from the data set
>
>Day     Flow
>
>1       10
>4       13
>5       20
>6       15
>9       13
>
>I would like the 1st, 3rd and 5th record filered.
>
>Any ideas on how I would write such and algorithm would be appreciated.

Dear Matt,

Here's a function that does what you want with loops. Perhaps someone else
will produce a more elegant solution:

> select.rows <- function(data) {
+     indices <- data[,1]
+     values <- data[,2]
+     n <- length(indices)
+     if (n == 0) stop('no data')
+     if (n == 1) return(data)
+     selection <- rep(0, n)  # so as not to grow the selection vector
+     current <- 1
+     number <- 1
+     for (i in 2:n){
+         if (indices[i] == 1 + indices[i - 1]){
+             if (values[i] > values[current]) current <- i
+             }
+         else {
+             selection[number] <- current
+             number <- number + 1
+             current <- i
+             }
+         }
+     selection[number] <- current
+     data[selection,]
+     }
>
> data <- matrix(c(1,4,5,6,9, 10,13,20,15,13), 5, 2)
> colnames(data) <- c('Day', 'Flow')
> select.rows(data)
Day Flow
[1,]   1   10
[2,]   5   20
[3,]   9   13

I hope that this isn't too inefficient.

John
-----------------------------------------------------
John Fox
Department of Sociology
McMaster University
email: jfox at mcmaster.ca
phone: 905-525-9140x23604
web: www.socsci.mcmaster.ca/jfox
-----------------------------------------------------

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

```