Skip to content

lib/process: always pipe _exactly_ what the command outputs

Administrator requested to merge aharvey/handle-unterminated-pipes into main

Created by: LawnGnome

Right now, we use the default scanner split function, which is bufio.SplitLines. This removes the trailing newline and any preceding \r before it, then returns the remainder of the token. PipeOutput then puts the newline back by invoking fmt.Fprintln.

This has the positive effect that we essentially normalise \r\n and \n into \n, but the negative effect that every line of output becomes newline-terminated, even if it's a final line that doesn't include a newline.

This matters because this output is then used for output variables when executing batch specs, which means that something like echo -n hi will end up with an extraneous newline, even though echo -n has only written two bytes to stdout.

Instead, let's set the split function to one that splits by line, but retains the newline, and then we can write it to the piped writer(s) with fmt.Fprint directly.

The one tradeoff here is that I haven't reimplemented the \r swallowing that bufio.SplitLines does. In practice, I don't believe we ever actually used this behaviour: Docker may return extraneous \r bytes when running with TTY mode enabled, but we never enable that when executing batch specs, and if the user's container actually outputs \r we should actually retain it, rather than trying to filter their output.

Fixes #36562 (closed).

Test plan

Added unit test coverage for this case. Since this is only actually used in src-cli, testing beyond that will occur when that repo gets a PR updating to use this version of lib.

Merge request reports

Loading