[Rd] the pipe |> and line breaks in pipelines
bbo|ker @end|ng |rom gm@||@com
Wed Dec 9 21:19:19 CET 2020
FWIW there is previous discussion of this in a twitter thread from May:
at the end I suggested defining something like .__END <- identity() as a
On 12/9/20 2:58 PM, Kevin Ushey wrote:
> I agree with Duncan that the right solution is to wrap the pipe
> expression with parentheses. Having the parser treat newlines
> differently based on whether the session is interactive, or on what
> type of operator happens to follow a newline, feels like a pretty big
> can of worms.
> I think this (or something similar) would accomplish what you want
> while still retaining the nice aesthetics of the pipe expression, with
> a minimal amount of syntax "noise":
> result <- (
> |> op1()
> |> op2()
> For interactive sessions where you wanted to execute only parts of the
> pipeline at a time, I could see that being accomplished by the editor
> -- it could transform the expression so that it could be handled by R,
> either by hoisting the pipe operator(s) up a line, or by wrapping the
> to-be-executed expression in parentheses for you. If such a style of
> coding became popular enough, I'm sure the developers of such editors
> would be interested and willing to support this ...
> Perhaps more importantly, it would be much easier to accomplish than a
> change to the behavior of the R parser, and it would be work that
> wouldn't have to be maintained by the R Core team.
> On Wed, Dec 9, 2020 at 11:34 AM Timothy Goodman <timsgoodman using gmail.com> wrote:
>> If I type my_data_frame_1 and press Enter (or Ctrl+Enter to execute the
>> command in the Notebook environment I'm using) I certainly *would* expect R
>> to treat it as a complete statement.
>> But what I'm talking about is a different case, where I highlight a
>> multi-line statement in my notebook:
>> |> filter(some_conditions_1)
>> and then press Ctrl+Enter. Or, I suppose the equivalent would be to run an
>> R script containing those two lines of code, or to run a multi-line
>> statement like that from the console (which in RStudio I can do by pressing
>> Shift+Enter between the lines.)
>> In those cases, R could either (1) Give an error message [the current
>> behavior], or (2) understand that the first line is meant to be piped to
>> the second. The second option would be significantly more useful, and is
>> almost certainly what the user intended.
>> consider the first token of the next line when determining if the previous
>> but a rule like "a pipe following a line break is treated as continuing the
>> previous line" would be much simpler. And while it might be objectionable
>> to treat the operator %>% different from other operators, the addition of
>> |>, which isn't truly an operator at all, seems like the right time to
>> consider it.)
>> On Wed, Dec 9, 2020 at 3:12 AM Duncan Murdoch <murdoch.duncan using gmail.com>
>>> The requirement for operators at the end of the line comes from the
>>> interactive nature of R. If you type
>>> how could R know that you are not done, and are planning to type the
>>> rest of the expression
>>> %>% filter(some_conditions_1)
>>> before it should consider the expression complete? The way languages
>>> like C do this is by requiring a statement terminator at the end. You
>>> can also do it by wrapping the entire thing in parentheses ().
>>> However, be careful: Don't use braces: they don't work. And parens
>>> have the side effect of removing invisibility from the result (which is
>>> a design flaw or bonus, depending on your point of view). So I actually
>>> wouldn't advise this workaround.
>>> Duncan Murdoch
>>> On 09/12/2020 12:45 a.m., Timothy Goodman wrote:
>>>> I'm a data scientist who routinely uses R in my day-to-day work, for
>>>> such as cleaning and transforming data, exploratory data analysis, etc.
>>>> This includes frequent use of the pipe operator from the magrittr and
>>>> libraries, %>%. So, I was pleased to hear about the recent work on a
>>>> native pipe operator, |>.
>>>> This seems like a good time to bring up the main pain point I encounter
>>>> when using pipes in R, and some suggestions on what could be done about
>>>> it. The issue is that the pipe operator can't be placed at the start of
>>>> line of code (except in parentheses). That's no different than any
>>>> operator in R, but I find it's a source of difficulty for the pipe
>>>> of how pipes are often used.
>>>> [I'm assuming here that my usage is fairly typical of a lot of users; at
>>>> any rate, I don't think I'm *too* unusual.]
>>>> === Why this is a problem ===
>>>> It's very common (for me, and I suspect for many users of dplyr) to write
>>>> multi-step pipelines and put each step on its own line for readability.
>>>> Something like this:
>>>> ### Example 1 ###
>>>> my_data_frame_1 %>%
>>>> filter(some_conditions_1) %>%
>>>> inner_join(my_data_frame_2, by = some_columns_1) %>%
>>>> group_by(some_columns_2) %>%
>>>> summarize(some_aggregate_functions_1) %>%
>>>> filter(some_conditions_2) %>%
>>>> left_join(my_data_frame_3, by = some_columns_3) %>%
>>>> group_by(some_columns_4) %>%
>>>> summarize(some_aggregate_functions_2) %>%
>>>> [I guess some might consider this an overly long pipeline; for me it's
>>>> pretty typical. I *could* split it up by assigning intermediate results
>>>> variables, but much of the value I get from the pipe is that it lets my
>>>> code communicate which results are temporary, and which will be used
>>>> later. Assigning variables for single-use results would remove that
>>>> I would prefer (for reasons I'll explain) to be able to write the above
>>>> example like this, which isn't valid R:
>>>> ### Example 2 (not valid R) ###
>>>> %>% filter(some_conditions_1)
>>>> %>% inner_join(my_data_frame_2, by = some_columns_1)
>>>> %>% group_by(some_columns_2)
>>>> %>% summarize(some_aggregate_functions_1)
>>>> %>% filter(some_conditions_2)
>>>> %>% left_join(my_data_frame_3, by = some_columns_3)
>>>> %>% group_by(some_columns_4)
>>>> %>% summarize(some_aggregate_functions_2)
>>>> %>% arrange(some_columns_5)
>>>> One (minor) advantage is obvious: It lets you easily line up the pipes,
>>>> which means that you can see at a glance that the whole block is a single
>>>> pipeline, and you'd immediately notice if you inadvertently omitted a
>>>> which otherwise can lead to confusing output. [It's also aesthetically
>>>> pleasing, especially when %>% is replaced with |>, but that's
>>>> But the bigger issue happens when I want to re-run just *part* of the
>>>> pipeline. I do this often when debugging: if the output of the pipeline
>>>> seems wrong, I re-run the first few steps and check the output, then
>>>> include a little more and re-run again, etc., until I locate my mistake.
>>>> Working in an interactive notebook environment, this involves using the
>>>> cursor to select just the part of the code I want to re-run.
>>>> It's fast and easy to select *entire* lines of code, but unfortunately
>>>> the pipes placed at the end of the line I must instead select everything
>>>> *except* the last three characters of the line (the last two characters
>>>> the new pipe). Then when I want to re-run the same partial pipeline with
>>>> the next line of code included, I can't just press SHIFT+Down to select
>>>> as I otherwise would, but instead must move the cursor horizontally to a
>>>> position three characters before the end of *that* line (which is
>>>> different due to varying line lengths). And so forth each time I want to
>>>> include an additional line.
>>>> Moreover, with the staggered positions of the pipes at the end of each
>>>> line, it's very easy to accidentally select the final pipe on a line, and
>>>> then sit there for a moment wondering if the environment has stopped
>>>> responding before realizing it's just waiting for further input (i.e.,
>>>> the right-hand side). These small delays and disruptions add up over the
>>>> course of a day.
>>>> This desire to select and re-run the first part of a pipeline is also the
>>>> reason why it doesn't suffice to achieve syntax like my "Example 2" by
>>>> wrapping the entire pipeline in parentheses. That's of no use if I want
>>>> re-run a selection that doesn't include the final close-paren.
>>>> === Possible Solutions ===
>>>> I can think of two, but maybe there are others. The first would make
>>>> "Example 2" into valid code, and the second would allow you to run a
>>>> selection that included a trailing pipe.
>>>> Solution 1: Add a special case to how R is parsed, so if the first
>>>> (non-whitespace) token after an end-line is a pipe, that pipe gets moved
>>>> before the end-line.
>>>> - Argument for: This lets you write code like example 2, which
>>>> addresses the pain point around re-running part of a pipeline, and has
>>>> advantages for readability. Also, since starting a line with a pipe
>>>> operator is currently invalid, the change wouldn't break any working
>>>> - Argument against: It would make the behavior of %>% inconsistent
>>>> that of other binary operators in R. (However, this objection might not
>>>> apply to the new pipe, |>, which I understand is being implemented as a
>>>> syntax transformation rather than a binary operator.)
>>>> Solution 2: Ignore the pipe operator if it occurs as the final token
>>>> the code being executed.
>>>> - Argument for: This would mean the user could select and re-run the
>>>> first few lines of a longer pipeline (selecting *entire* lines), avoiding
>>>> the difficulties described above.
>>>> - Argument against: This means that %>% would be valid even if it
>>>> occurred without a right-hand side, which is inconsistent with other
>>>> operators in R. (But, as above, this objection might not apply to |>.)
>>>> Also, this solution still doesn't enable the syntax of "Example 2", with
>>>> its readability benefit.
>>>> Thanks for reading this and considering it.
>>>> - Tim Goodman
>>>> [[alternative HTML version deleted]]
>>>> R-devel using r-project.org mailing list
>> [[alternative HTML version deleted]]
>> R-devel using r-project.org mailing list
> R-devel using r-project.org mailing list
More information about the R-devel