From 8057dace35009b3ef6de5ddedf926d2a8c1414a5 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 20 Aug 2026 11:50:38 +0000 Subject: [PATCH 1/2] Give each chain a colour of its own A build with four chains streams four labels down the left of the terminal, and telling them apart meant reading them. Each chain now gets a colour from a palette, in declaration order and round again from the top when a build has more chains than there are colours, and the colour covers the label and the bar in both layouts: the bars make a column in the chain's colour, and what a command wrote goes out exactly as it wrote it. The `[!]` and `[.]` in front of an outcome line stay plain, so the colour says which chain and nothing else. Colour is on where `run` prints to a terminal that says it has colours, and off everywhere else, so a build that pipes its output to a file or greps it for `[!]` gets the same bytes it got before. Four things decide, each one overruling the one before it: - the terminal: stdout is one, and terminfo reports eight colours or more. `TERM` unset or `dumb` counts as none, and where there is no `tput` to ask, a `TERM` that is set and is not `dumb` is taken at its word. - `NO_COLOR`, set to anything at all: no colour. - `FORCE_COLOR`, set to anything but `0`: colour. `FORCE_COLOR=0` is how a good many tools are told to stop, so it is read as a refusal rather than as the name being set. - `COLOR`: `1`, `0`, or `auto` to leave it to the three above. Last, because it is the only one of the four aimed at this library. The question is asked at `run` and not when the library is sourced, since the answer is about the stream `run` prints to and a build script is free to redirect its own output before it gets there. `COLOR_PALETTE` is the colours themselves, as SGR parameters, so `1;36` is bold cyan and `38;5;213` is one of the 256. It defaults to cyan, magenta, green, yellow and blue: the colours a terminal has had since it had eight, less red, which belongs to what a build says about its own failures, and less black and white, which the rest of the line already is. Emptying it is another way of asking for no colour, so the default fills in for a palette that was never set rather than for one that was deliberately emptied. The escape character is written as its bytes, the way the box drawing bar already was: the sources stay ASCII, and `make lint` still says so. Eleven examples cover it, including the precedence between the four and the palette starting again once it runs out. --- README.md | 56 +++++++++++- parallel.sh | 139 +++++++++++++++++++++++++++-- spec/parallel_spec.sh | 200 ++++++++++++++++++++++++++++++++++++++++++ spec/spec_helper.sh | 5 ++ 4 files changed, 391 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 35768e3..9c6d9d8 100644 --- a/README.md +++ b/README.md @@ -87,6 +87,10 @@ composer install │ Generating optimized autoload files [.] composer install cancelled ``` +In a terminal each label is in a colour of its own, one a chain, and +everywhere else the output is the plain text above; [`COLOR`](#color) is the +whole of it. + A line is printed once it is whole, so a command that writes a line in two writes still gets one line, and the chain it belongs to is the only thing that decides where it goes. `STREAM=0` holds the output and groups it @@ -218,6 +222,53 @@ STREAM_SEP='|' ./build.sh Labelled output is read on the same poll that watches for finished chains, so a line can be up to `POLL` behind the command that wrote it. +## COLOR + +Each chain's label is given a colour of its own, so which chain a line came +from reads without reading the label at all. The colour opens before the +label and closes after the bar, in both layouts, so the bars make a column in +the chain's colour and what a command wrote goes out exactly as it wrote it. +The `[!]` and `[.]` in front of an outcome line stay plain: the colour says +which chain, and nothing else. + +Colour is on where `run` prints to a terminal that says it has colours, and +off everywhere else, so a build that pipes its output to a file or greps it +for `[!]` gets the same plain text it got before. Four things have a say, and +each one overrules the one before it: + +- **the terminal** — colour where stdout is one and terminfo reports eight + colours or more. `TERM` unset or `dumb` counts as none, and where there is + no `tput` to ask, a `TERM` that is set and is not `dumb` is taken at its + word. +- **`NO_COLOR`** — set to anything at all, no colour. The convention the rest + of the build already follows. +- **`FORCE_COLOR`** — set to anything but `0`, colour; set to `0`, none. + `FORCE_COLOR=0` is how a good many tools are told to stop, so it is read as + a refusal rather than as the name being set. +- **`COLOR`** — `1` for colour, `0` for none, `auto` to leave it to the three + above. It is last because it is the only one of the four aimed at this + library: + +```sh +COLOR=1 ./build.sh | tee build.log +``` + +`COLOR_PALETTE` is the colours themselves, as SGR parameters, handed out in +declaration order and started again from the top once a build has more chains +than the palette has colours. It defaults to cyan, magenta, green, yellow and +blue: the colours a terminal has had since it had eight of them, less red, +which belongs to what a build says about its own failures, and less black and +white, which the rest of the line already is. + +```sh +COLOR_PALETTE='1;36 1;35 1;32' ./build.sh +``` + +An entry is whatever SGR takes, so `1;36` is bold cyan and `38;5;213` is one +of the 256 a terminal that has them will answer to. Emptying the palette is +another way of asking for no colour at all; leaving it unset is what gets the +default. + ## Benchmarks `bench/bench.sh` runs four builds twice each, once in declaration order the @@ -289,8 +340,9 @@ Fractional `sleep` is not in POSIX, though both GNU coreutils and BSD accept it. Whole seconds are the fallback, not the default, so a build on a shell without fractional sleep finishes up to a second later than it might. -The file itself is ASCII, comments included, and the box drawing bar is -written as its bytes in the one place it is needed. A shell reads a script +The file itself is ASCII, comments included, and the two characters in it +that are not, the box drawing bar and the escape a colour begins with, are +written as their bytes in the places they are needed. A shell reads a script through the locale, and a strict one in the C locale refuses a file carrying a byte sequence that locale cannot make a character of, which is the locale a build container has when nobody has set one. `make lint` checks it. diff --git a/parallel.sh b/parallel.sh index 7557536..5deb306 100644 --- a/parallel.sh +++ b/parallel.sh @@ -20,6 +20,7 @@ POLL=${POLL:-0.1} # seconds between checks POLL_WHOLE=${POLL_WHOLE:-1} # used instead if this sleep rejects fractions STREAM=${STREAM:-1} # 0 to hold the output and print it grouped +COLOR=${COLOR:-auto} # 1 to colour the labels, 0 to leave them plain # How many polls a cancelled chain gets to leave on its own before it is # killed outright, which comes to about GRACE times POLL seconds: ten of them @@ -42,6 +43,19 @@ case ${LC_ALL:-${LC_CTYPE:-${LANG:-}}} in *) STREAM_SEP=${STREAM_SEP:-'|'} ;; esac +# The colours the labels are given, as SGR parameters, one a chain in the +# order the chains were declared and round again from the top once a build has +# more chains than this has colours. +# +# Cyan, magenta, green, yellow, blue: the colours a terminal has had since it +# had eight of them, less red, which belongs to what a build says about its +# own failures, and less black and white, which the rest of the line already +# is. An entry is whatever SGR takes, so `1;36` is bold cyan, and a palette +# with nothing in it is another way of asking for no colour at all: the +# default fills in for a palette that was never set, and not for one that was +# deliberately emptied. +COLOR_PALETTE=${COLOR_PALETTE-'36 35 32 33 34'} + # Somewhere to keep what has to travel between the chains and the script that # started them: a chain's log, and the status it ended on. What only this # shell ever looks at, a chain's label and its pid, is kept in a variable. @@ -62,14 +76,19 @@ _signal=0 # what to exit with if a signal arrives; 0 until one does _fractional='' # whether sleep takes POLL; unknown until the first nap _groups='' # whether a chain can have its own process group; unknown _monitor='' # whether the calling shell had job control on already +_colored='' # whether the labels are coloured; unknown until `run` looks +_esc='' # the escape character, once there is a use for one +_reset='' # what closes a colour, and nothing where there is none # POSIX sh has no arrays, so what a chain has one of is kept in a name with # the chain's number on the end of it, written and read through `eval`: -# _label_1, _pid_1, _prefix_1, _seen_1. These are where one comes back out. +# _label_1, _pid_1, _prefix_1, _seen_1, _color_1. These are where one comes +# back out. _label='' _pid='' _prefix='' _seen='' +_color='' _cleanup() { rm -rf "$_work" || :; } @@ -288,6 +307,8 @@ chain() { } run() { + _probe_color + _colors if _streaming; then _prefixes; fi _await _cancel @@ -390,6 +411,102 @@ _streaming() { esac } +# Whether the labels are coloured, asked here at `run` rather than when the +# library is sourced: the answer is about the stream `run` prints to, and a +# build script is free to redirect its own output before it gets that far. +# +# Four things have a say, and each one overrules the one before it: the +# terminal, then NO_COLOR, then FORCE_COLOR, then COLOR. The two environment +# variables are the convention the rest of the tools in a build already +# follow, and COLOR is last because it is the only one of the four aimed at +# this library in particular. +_probe_color() { + _colored=no + + # A terminal, and one with colours to give. terminfo is what knows how + # many, and `tput` is what asks it; where there is no `tput`, a TERM + # that is set and is not `dumb` is taken at its word. + if [ -t 1 ]; then + if command -v tput >/dev/null 2>&1; then + _ncolors=$(tput colors 2>/dev/null) || _ncolors=0 + else + _ncolors=8 + fi + case ${TERM:-} in '' | dumb) _ncolors=0 ;; esac + # `tput` answers -1 for a terminal with no colours at all and + # nothing whatever for a TERM terminfo has not heard of, and + # `[` either compares numbers or fails, which under set -e + # takes the build with it. + case $_ncolors in '' | *[!0-9]*) _ncolors=0 ;; esac + if [ "$_ncolors" -ge 8 ]; then _colored=yes; fi + fi + + # Set at all, whatever it is set to, which is what everything else + # reading it in the same build takes it as. + if [ -n "${NO_COLOR+set}" ]; then _colored=no; fi + + # The same, but for the 0 that means the opposite: FORCE_COLOR=0 is how + # a good many tools are told to stop, and reading it as `start` on the + # grounds that the name is set would be exactly the wrong way round. + case ${FORCE_COLOR-} in + '') ;; + 0) _colored=no ;; + *) _colored=yes ;; + esac + + case $COLOR in + 1 | yes | on | true | always) _colored=yes ;; + 0 | no | off | false | never) _colored=no ;; + esac + + case $_colored in + no) return 0 ;; + esac + + # Written as its bytes for the same reason the bar above is: the escape + # character is not ASCII either, and this file has to stay readable as + # text by a shell in a locale that can make nothing of it. + _esc=$(printf '\033') + _reset="${_esc}[0m" +} + +# One colour a chain, handed out here rather than at `chain` because whether +# there are any to hand out is not known until `run` has looked at where the +# output is going. +# +# The palette is a list in a string, POSIX sh having nothing better, and the +# positional parameters are where it is worked through: they are the one list +# a function of no arguments has to itself, and `shift` walks it without a +# counter. Refilled from the palette each time it runs out, which is what +# makes a sixth chain cyan again. +_colors() { + case $_colored in + no) return 0 ;; + esac + + # shellcheck disable=SC2086 # the palette is a list, and splitting it is the point + set -- $COLOR_PALETTE + # Nothing to hand out, so nothing is coloured, and the reset that + # closes a colour has nothing left to close. + if [ "$#" -eq 0 ]; then + _colored=no + _reset='' + return 0 + fi + + _i=1 + while [ "$_i" -le "$_count" ]; do + if [ "$#" -eq 0 ]; then + # shellcheck disable=SC2086 # as above + set -- $COLOR_PALETTE + fi + _open="${_esc}[$1m" + shift + eval "_color_$_i=\$_open" + _i=$((_i + 1)) + done +} + # The label a streamed line carries, one per chain, right aligned to the # longest of them so that the bars line up under each other. Every label is # known by the time `run` is called, which is the first moment a width can be @@ -444,18 +561,22 @@ _emit() { _n=$1 eval "_seen=\${_seen_$_n:-0}" eval "_prefix=\$_prefix_$_n" + eval "_color=\${_color_$_n:-}" tail -n "+$((_seen + 1))" "$_work/$_n.log" \ >|"$_work/$_n.chunk" 2>/dev/null || return 0 + # The colour runs from the start of the label to the end of the bar and + # closes there, so the bars make a column in the chain's colour and + # what the command wrote goes out exactly as it wrote it. _line='' while IFS= read -r _line; do - printf '%s %s %s\n' "$_prefix" "$STREAM_SEP" "$_line" + printf '%s%s %s%s %s\n' "$_color" "$_prefix" "$STREAM_SEP" "$_reset" "$_line" _seen=$((_seen + 1)) _line='' done <"$_work/$_n.chunk" if [ -n "$_line" ] && [ "$2" = last ]; then - printf '%s %s %s\n' "$_prefix" "$STREAM_SEP" "$_line" + printf '%s%s %s%s %s\n' "$_color" "$_prefix" "$STREAM_SEP" "$_reset" "$_line" _seen=$((_seen + 1)) fi @@ -488,15 +609,18 @@ _report() { # a chain on this line, so a build that greps for one finds it either way. _outcome() { eval "_label=\$_label_$1" + eval "_color=\${_color_$1:-}" + # The label carries the colour and the mark in front of it does not: + # the colour says which chain, and nothing else, here as above. if [ -f "$_work/$1.cancelled" ]; then - printf '[.] %s cancelled\n' "$_label" + printf '[.] %s%s%s cancelled\n' "$_color" "$_label" "$_reset" elif [ -f "$_work/$1.killed" ]; then - printf '[!] %s killed\n' "$_label" + printf '[!] %s%s%s killed\n' "$_color" "$_label" "$_reset" else _code=$(cat "$_work/$1.code") case $_code in 0) ;; - *) printf '[!] %s exited %s\n' "$_label" "$_code" ;; + *) printf '[!] %s%s%s exited %s\n' "$_color" "$_label" "$_reset" "$_code" ;; esac fi } @@ -513,7 +637,8 @@ _group() { fi eval "_label=\$_label_$1" - printf '\n%s %s\n' "$_mark" "$_label" + eval "_color=\${_color_$1:-}" + printf '\n%s %s%s%s\n' "$_mark" "$_color" "$_label" "$_reset" cat "$_work/$1.log" # A chain whose last line never got a newline would otherwise have the # line below glued onto the end of it. diff --git a/spec/parallel_spec.sh b/spec/parallel_spec.sh index 18ac77c..302d6e5 100644 --- a/spec/parallel_spec.sh +++ b/spec/parallel_spec.sh @@ -480,6 +480,206 @@ Describe 'parallel.sh' End End + # A colour a chain, so a line's chain reads without reading the label. + # Nothing here has a terminal to be asked about, and every example + # below asks for colour outright and reads the escapes back out. + Describe 'colour' + It 'gives each chain a colour of its own' + Data + #|set -eu + #|STREAM_SEP='|' + #|COLOR=1 + #|. "$LIB" + #|chain "one" "printf 'first line\n'" + #|chain "two" "printf 'second line\n'" + #|run + End + + When call driver + The status should equal 0 + # The colour opens before the label and closes after the bar, + # so what the command wrote is left exactly as it wrote it. + The output should include "$(sgr 36)one |$(sgr 0) first line" + The output should include "$(sgr 35)two |$(sgr 0) second line" + End + + # The default, and what a build gets when it pipes its output + # somewhere or reads it back from a file: the plain text, which is + # also what greps for the outcome lines rely on. + It 'leaves the labels plain where the output is not a terminal' + Data + #|set -eu + #|STREAM_SEP='|' + #|. "$LIB" + #|chain "one" "printf 'a line\n'" + #|run + End + + When call driver + The status should equal 0 + The output should equal "one | a line" + End + + It 'starts the palette again once it runs out' + Data + #|set -eu + #|STREAM_SEP='|' + #|COLOR=1 + #|COLOR_PALETTE='31 32' + #|. "$LIB" + #|chain "one" "printf 'a\n'" + #|chain "two" "printf 'b\n'" + #|chain "the" "printf 'c\n'" + #|run + End + + When call driver + The status should equal 0 + The output should include "$(sgr 31)one |$(sgr 0) a" + The output should include "$(sgr 32)two |$(sgr 0) b" + The output should include "$(sgr 31)the |$(sgr 0) c" + End + + # An entry is passed to the terminal as it stands, so anything SGR + # understands is a colour a build can ask for. + It 'takes the colours from COLOR_PALETTE' + Data + #|set -eu + #|STREAM_SEP='|' + #|COLOR=1 + #|COLOR_PALETTE='1;35' + #|. "$LIB" + #|chain "one" "printf 'a line\n'" + #|run + End + + When call driver + The status should equal 0 + The output should equal "$(sgr '1;35')one |$(sgr 0) a line" + End + + # A palette emptied on purpose is a build asking for no colour by + # another name, and there is nothing left for the reset to close. + It 'colours nothing where the palette is empty' + Data + #|set -eu + #|STREAM_SEP='|' + #|COLOR=1 + #|COLOR_PALETTE='' + #|. "$LIB" + #|chain "one" "printf 'a line\n'" + #|run + End + + When call driver + The status should equal 0 + The output should equal "one | a line" + End + + It 'colours the label a chain ended on, and not the mark' + Data + #|set -eu + #|STREAM_SEP='|' + #|COLOR=1 + #|. "$LIB" + #|chain "one" "exit 3" + #|run || printf 'run returned %s\n' "$?" + End + + When call driver + The status should equal 0 + The output should include "[!] $(sgr 36)one$(sgr 0) exited 3" + The output should include "run returned 3" + End + + It 'colours the heading of a grouped chain' + Data + #|set -eu + #|STREAM=0 + #|COLOR=1 + #|. "$LIB" + #|chain "one" "printf 'a line\n'" + #|run + End + + When call driver + The status should equal 0 + The output should include "--- $(sgr 36)one$(sgr 0)" + The output should include "a line" + End + + # The four say so in this order, each overruling the one before it, + # and only the last two can be tested without a terminal to be + # asked about. + Describe 'what decides' + It 'is turned on by FORCE_COLOR where there is no terminal' + Data + #|set -eu + #|STREAM_SEP='|' + #|FORCE_COLOR=1 + #|. "$LIB" + #|chain "one" "printf 'a line\n'" + #|run + End + + When call driver + The status should equal 0 + The output should equal "$(sgr 36)one |$(sgr 0) a line" + End + + # FORCE_COLOR=0 is how a good many tools are told to stop, so + # it is read as a refusal and not as the name being set. + It 'is turned off by FORCE_COLOR=0' + Data + #|set -eu + #|STREAM_SEP='|' + #|COLOR=auto + #|FORCE_COLOR=0 + #|. "$LIB" + #|chain "one" "printf 'a line\n'" + #|run + End + + When call driver + The status should equal 0 + The output should equal "one | a line" + End + + It 'has FORCE_COLOR overrule NO_COLOR' + Data + #|set -eu + #|STREAM_SEP='|' + #|NO_COLOR=1 + #|FORCE_COLOR=1 + #|. "$LIB" + #|chain "one" "printf 'a line\n'" + #|run + End + + When call driver + The status should equal 0 + The output should equal "$(sgr 36)one |$(sgr 0) a line" + End + + It 'has COLOR overrule both' + Data + #|set -eu + #|STREAM_SEP='|' + #|COLOR=0 + #|NO_COLOR=1 + #|FORCE_COLOR=1 + #|. "$LIB" + #|chain "one" "printf 'a line\n'" + #|run + End + + When call driver + The status should equal 0 + The output should equal "one | a line" + End + End + End + # STREAM=0 holds each chain's output and prints it in one block instead, # in declaration order, which is what a build that would rather read the # whole of a chain at once asks for. diff --git a/spec/spec_helper.sh b/spec/spec_helper.sh index 3e47aaa..6d5c2a4 100644 --- a/spec/spec_helper.sh +++ b/spec/spec_helper.sh @@ -230,6 +230,11 @@ no_process_groups() { 2>/dev/null)" != yes ] } +# An SGR escape, written as its bytes because this file is ASCII too: the +# escape character is not, and a spec has to be readable by a shell in a +# locale that can make nothing of it. +sgr() { printf '\033[%sm' "$1"; } + lines_in() { wc -l <"$1" | tr -d ' ' } From 9cf72e6af4d3ab811dd798d6d5825e8c47541df0 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 20 Aug 2026 13:06:30 +0000 Subject: [PATCH 2/2] Take the palette apart without asking the shell to split it `set -- $COLOR_PALETTE` split the palette into colours everywhere the suite runs but zsh, which does not split an unquoted expansion at all: there the whole palette arrived as one colour and every label opened with `ESC[36 35 32 33 34m`. Six examples said so on the zsh and macOS zsh jobs, and nothing said so under the eight other shells. The palette is walked a word at a time now, with the expansions that trim a string, which every shell here does the same way. Leading spaces are trimmed between words, so the spacing in a hand written palette does not matter and a palette of nothing but them still reads as the empty one it is. Verified under all nine Linux shells in the matrix rather than the three that were to hand: dash, bash, bash --posix, zsh, zsh --emulate sh, mksh, ksh93, posh and busybox ash, 53 examples and no failures under each. --- README.md | 4 ++-- parallel.sh | 41 ++++++++++++++++++++++++++++------------- 2 files changed, 30 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 9c6d9d8..da4eb2b 100644 --- a/README.md +++ b/README.md @@ -253,8 +253,8 @@ each one overrules the one before it: COLOR=1 ./build.sh | tee build.log ``` -`COLOR_PALETTE` is the colours themselves, as SGR parameters, handed out in -declaration order and started again from the top once a build has more chains +`COLOR_PALETTE` is the colours themselves, a space separated list of SGR +parameters, handed out in declaration order and started again from the top once a build has more chains than the palette has colours. It defaults to cyan, magenta, green, yellow and blue: the colours a terminal has had since it had eight of them, less red, which belongs to what a build says about its own failures, and less black and diff --git a/parallel.sh b/parallel.sh index 5deb306..cad534f 100644 --- a/parallel.sh +++ b/parallel.sh @@ -474,21 +474,22 @@ _probe_color() { # there are any to hand out is not known until `run` has looked at where the # output is going. # -# The palette is a list in a string, POSIX sh having nothing better, and the -# positional parameters are where it is worked through: they are the one list -# a function of no arguments has to itself, and `shift` walks it without a -# counter. Refilled from the palette each time it runs out, which is what -# makes a sixth chain cyan again. +# The palette is a list in a string, POSIX sh having nothing better, and it is +# taken a word at a time with the expansions that trim a string rather than by +# letting the shell split it on the spaces: zsh does not split an unquoted +# expansion at all, and what that came to was the whole palette arriving as +# one colour. Refilled each time it runs out, which is what makes a sixth +# chain cyan again. _colors() { case $_colored in no) return 0 ;; esac - # shellcheck disable=SC2086 # the palette is a list, and splitting it is the point - set -- $COLOR_PALETTE + _rest=$COLOR_PALETTE + _trim_palette # Nothing to hand out, so nothing is coloured, and the reset that # closes a colour has nothing left to close. - if [ "$#" -eq 0 ]; then + if [ -z "$_rest" ]; then _colored=no _reset='' return 0 @@ -496,17 +497,31 @@ _colors() { _i=1 while [ "$_i" -le "$_count" ]; do - if [ "$#" -eq 0 ]; then - # shellcheck disable=SC2086 # as above - set -- $COLOR_PALETTE + if [ -z "$_rest" ]; then + _rest=$COLOR_PALETTE + _trim_palette fi - _open="${_esc}[$1m" - shift + _sgr=${_rest%% *} + _rest=${_rest#"$_sgr"} + _trim_palette + _open="${_esc}[${_sgr}m" eval "_color_$_i=\$_open" _i=$((_i + 1)) done } +# The spaces off the front of what is left of the palette, so that the next +# word off it is a colour and never the nothing between two spaces, and so +# that a palette of nothing but spaces reads as the empty one it is. +_trim_palette() { + while :; do + case $_rest in + ' '*) _rest=${_rest# } ;; + *) return 0 ;; + esac + done +} + # The label a streamed line carries, one per chain, right aligned to the # longest of them so that the bars line up under each other. Every label is # known by the time `run` is called, which is the first moment a width can be