Bidirectional pipes in bash

10

bash version 4 has coproc command that allows this done in pure bash without named pipes.

From bash man page about coproc command:

coproc [NAME] command [redirections]

This creates a coprocess named NAME. If NAME is not supplied, the default name is COPROC.

The standard output of command is connected via a pipe to a file descriptor in the executing shell, and that file descriptor is assigned to NAME[0]. The standard input of command is connected via a pipe to a file descriptor in the executing shell, and that file descriptor is assigned to NAME[1].

Start with a version easy to understand (but does not work). Start entire pipeline under coproc and use extra cat for plumbing.

coproc {
  cmd1 | cmd2
}
cat <&${COPROC[0]} >&${COPROC[1]}
Buffering is obviously something that needs to be taken care of. Here plain cat will likely break it due to buffering. So use stdbuf to disable buffering by cat:

# this is same as above
coproc {
  cmd1 | cmd2
}
# replace last line from sample above with:
stdbuf -i0 -o0 cat <&${COPROC[0]} >&${COPROC[1]}
Better still, do away with cat. Break up pipeline into two steps, run 1st section under coproc and 2nd section to connect with 1st.

coproc cmd1
cmd2 <&${COPROC[0]} >&${COPROC[1]}
This still assumes that cmd1 and cmd2 handle buffering correctly to make it work.

Some other shells also can do coproc as well.

That is all the answer above.

Below is just some exploration and tests.

Here example chains three commands only to make it a little more interesting.

# start pipeline
coproc {
    cmd1 | cmd2 | cmd3
}

# reconnect STDOUT from `cmd3` to STDIN of `cmd1`
stdbuf -i0 -o0 /bin/cat <&${COPROC[0]} >&${COPROC[1]}
Get rid of cat and stdbuf and stay with pure bash. Break it up into two parts, launch the first pipeline under coproc, then launch second part (either a single command or a pipeline), reconnecting it to the first:

coproc {
    cmd 1 | cmd2
}
cmd3 <&${COPROC[0]} >&${COPROC[1]}
Proof of concept

File prog - a worker program participating in looped IO. Just a dummy prog to consume, tag and re-print lines. It is using subshells to avoid buffering problems maybe overkill, it's not the point here.

#!/bin/bash

# Start this prog with only param "1" to produce some output

let c=0
sleep 2

[ "$1" == "1" ] && ( echo start )

while : ; do
  read line
  echo "$1:${c} ${line}" 1>&2
  sleep 2
  ( echo "$1:${c} ${line}" )
  let c++
  [ $c -eq 3 ] && exit
done
File start_io_loop_with_cat - a demo launcher. This is a version using bash, cat and stdbuf

#!/bin/bash

# start all 3 commands as one pipeline
coproc {
  ./prog 1 \
  | ./prog 2 \
  | ./prog 3
}

# start cat without buffering to connect IO loop
stdbuf -i0 -o0 /bin/cat <&${COPROC[0]} >&${COPROC[1]}
File start_io_loop_pure_bash - another demo launcher. This is a version using pure bash only.

#!/bin/bash

# start first 2 of 3 commands in the pipeline
coproc {
  ./prog 1 \
  | ./prog 2
}

# start 3rd command connecting IO to 1 and 2 to make the IO loop
./prog 3 <&${COPROC[0]} >&${COPROC[1]}
Output:

> ./start_io_loop_pure_bash
2:0 start
3:0 2:0 start
1:0 3:0 2:0 start
2:1 1:0 3:0 2:0 start
3:1 2:1 1:0 3:0 2:0 start
1:1 3:1 2:1 1:0 3:0 2:0 start
2:2 1:1 3:1 2:1 1:0 3:0 2:0 start
3:2 2:2 1:1 3:1 2:1 1:0 3:0 2:0 start
1:2 3:2 2:2 1:1 3:1 2:1 1:0 3:0 2:0 start
That does it.

Share
Improve this answer
Follow
edited Dec 21, 2021 at 15:26
answered Jul 17, 2016 at 16:07
user avatar
AnyDev
58777 silver badges18