TIP #304 Version 1.10: A Standalone [chan pipe] Primitive for Advanced Child IPC

This is not necessarily the current version of this TIP.

Abstract

Currently, it is not easy to get both (separate) dataflows from the stdout and stderr of a child. BLT's bgexec does this in an extension, and could be added to the core. But the point of this TIP is to show that a much smaller code addition can provide a lower-level primitive with much more potential than bgexec's: a standalone pipe creation tool like TclX's pipe command.

Background

Getting back both stdout and stderr from a child has long been an FAQ on news:comp.lang.tcl, to the point that bgexec has been offered in an extension, BLT, whose main job is very remote from IPC. Now this has been a problem for many, who didn't want to have the problems of distributing a script depending on an extension. Moreover, bgexec does not scale up, in that it cannot bring back the separate stderrs of all four children in:

	set ff [open "|a | b | c | d" r]

A popular workaround for script-only purists is to spawn an external "pump" like cat in an [open ... r+], and redirect the wanted stderr to the write side of the pump. Its output can then be monitored through the read side:

	set pump [open "|cat" r+]
	set f1 [open "|cmd args 2>@ $pump" r]
	fileevent $f1 readable got_stdout
	fileevent $pump readable got_stderr

Now this is all but elegant of course, difficult to deploy on Windows (where you need an extra cat.exe), and not especially efficient since the "pump" consumes context switches and memory bandwidth only to emulate a single OS pipe when Tcl is forced to create two of them via [open ... r+].

For this latter performance issue, a better alternative is a named pipe. But it is even harder to create on Windows, and it is a nightmare to handle its lifecycle properly