hyperreal/bash-hackers-wiki
1
0
Fork 0
mirror of https://github.com/flokoe/bash-hackers-wiki.git synced 2024-11-01 06:53:05 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Convert scripting pages to Markdown

Browse Source
This commit is contained in:
flokoe 2023-07-05 11:31:29 +02:00
parent 854c84588c
commit 82096dc0ff
11 changed files with 3329 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

360
docs/scripting/bashbehaviour.md Normal file
View File

@ -0,0 +1,360 @@
# Bash\'s behaviour
![](keywords>bash shell scripting startup files dotfiles modes POSIX)
FIXME incomplete
## Bash startup modes
### Login shell
As a \"login shell\", Bash reads and sets (executes) the user\'s profile
from `/etc/profile` and one of `~/.bash_profile`, `~/.bash_login`, or
`~/.profile` (in that order, using the first one that\'s readable!).
When a login shell exits, Bash reads and executes commands from the file
`~/.bash_logout`, if it exists.
Why an extra login shell mode? There are many actions and variable sets
that only make sense for the initial user login. That\'s why all UNIX(r)
shells have (should have) a \"login\" mode.
[**Methods to start Bash as a login shell:**]{.underline}
- the first character of `argv[0]` is `-` (a hyphen): traditional
UNIX(r) shells start from the `login` binary
- Bash is started with the `-l` option
- Bash is started with the `--login` option
[**Methods to test for login shell mode:**]{.underline}
- the shell option `login_shell` is set
[**Related switches:**]{.underline}
- `--noprofile` disables reading of all profile files
### Interactive shell
When Bash starts as an interactive non-login shell, it reads and
executes commands from `~/.bashrc`. This file should contain, for
example, aliases, since they need to be defined in every shell as
they\'re not inherited from the parent shell.
The feature to have a system-wide `/etc/bash.bashrc` or a similar
system-wide rc-file is specific to vendors and distributors that ship
*their own, patched variant of Bash*. The classic way to have a
system-wide rc file is to `source /etc/bashrc` from every user\'s
`~/.bashrc`.
[**Methods to test for interactive-shell mode:**]{.underline}
- the special parameter `$-` contains the letter `i` (lowercase I)
[**Related switches:**]{.underline}
- `-i` forces the interactive mode
- `--norc` disables reading of the startup files (e.g.
`/etc/bash.bashrc` if supported) and `~/.bashrc`
- `--rcfile` defines another startup file (instead of
`/etc/bash.bashrc` and `~/.bashrc`)
### SH mode
When Bash starts in SH compatiblity mode, it tries to mimic the startup
behaviour of historical versions of `sh` as closely as possible, while
conforming to the POSIX(r) standard as well. The profile files read are
`/etc/profile` and `~/.profile`, if it\'s a login shell.
If it\'s not a login shell, the environment variable
[ENV](/syntax/shellvars#ENV) is evaluated and the resulting filename is
used as the name of the startup file.
After the startup files are read, Bash enters the [POSIX(r) compatiblity
mode (for running, not for starting!)](#posix_run_mode).
[**Bash starts in `sh` compatiblity mode when:**]{.underline}
- the base filename in `argv[0]` is `sh` (:!: NB: `/bin/sh` may be
linked to `/bin/bash`, but that doesn\'t mean it acts like
`/bin/bash` :!:)
### POSIX mode
When Bash is started in POSIX(r) mode, it follows the POSIX(r) standard
for startup files. In this mode, **interactive shells** expand the
[ENV](/syntax/shellvars#ENV) variable and commands are read and executed
from the file whose name is the expanded value.\
No other startup files are read. Hence, a non-interactive shell doesn\'t
read any startup files in POSIX(r) mode.
[**Bash starts in POSIX(r) mode when:**]{.underline}
- the commandline option `--posix` is specified
- the environment variable
[POSIXLY_CORRECT](/syntax/shellvars#POSIXLY_CORRECT) is set
### Quick startup file reference
- Eventual system-wide rc-files are usually read when `~/.bashrc`
would be read (at least Debian GNU/Linux behaves like that)
- Regardless of the system-wide files in `/etc` which are always read,
Bash usually reads the first file found, when multiple choices are
given (for user files in `~/`)
Mode `/etc/profile` `~/.bash_profile` `~/.bash_login` `~/.profile` `~/.bashrc` `${ENV}`
----------------------- ---------------- ------------------- ----------------- -------------- ------------- ----------
Login shell X X X X \- \-
Interactive shell \- \- \- \- X \-
SH compatible login X \- \- X \- \-
SH compatible \- \- \- \- \- X
POSIX(r) compatiblity \- \- \- \- \- X
## Bash run modes
### Normal Bash
### POSIX run mode
In POSIX(r) mode, Bash follows the POSIX(r) standard regarding behaviour
and parsing (excerpt from a Bash maintainer\'s document):
Starting Bash with the `--posix' command-line option or executing `set
-o posix' while Bash is running will cause Bash to conform more closely
to the POSIX standard by changing the behavior to match that specified
by POSIX in areas where the Bash default differs.
When invoked as `sh', Bash enters POSIX mode after reading the startup
files.
The following lists what's changed when Bash is in `POSIX mode':
1. When a command in the hash table no longer exists, Bash will
re-search `$PATH' to find the new location. This is also
available with `shopt -s checkhash'.
2. The message printed by the job control code and builtins when a job
exits with a non-zero status is `Done(status)'.
3. The message printed by the job control code and builtins when a job
is stopped is `Stopped(SIGNAME)', where SIGNAME is, for example,
`SIGTSTP'.
4. The `bg' builtin uses the required format to describe each job
placed in the background, which does not include an indication of
whether the job is the current or previous job.
5. Reserved words appearing in a context where reserved words are
recognized do not undergo alias expansion.
6. The POSIX `PS1' and `PS2' expansions of `!' to the history number
and `!!' to `!' are enabled, and parameter expansion is performed
on the values of `PS1' and `PS2' regardless of the setting of the
`promptvars' option.
7. The POSIX startup files are executed (`$ENV') rather than the
normal Bash files.
8. Tilde expansion is only performed on assignments preceding a
command name, rather than on all assignment statements on the line.
9. The default history file is `~/.sh_history' (this is the default
value of `$HISTFILE').
10. The output of `kill -l' prints all the signal names on a single
line, separated by spaces, without the `SIG' prefix.
11. The `kill' builtin does not accept signal names with a `SIG'
prefix.
12. Non-interactive shells exit if FILENAME in `.' FILENAME is not
found.
13. Non-interactive shells exit if a syntax error in an arithmetic
expansion results in an invalid expression.
14. Redirection operators do not perform filename expansion on the word
in the redirection unless the shell is interactive.
15. Redirection operators do not perform word splitting on the word in
the redirection.
16. Function names must be valid shell names. That is, they may not
contain characters other than letters, digits, and underscores, and
may not start with a digit. Declaring a function with an invalid
name causes a fatal syntax error in non-interactive shells.
17. POSIX special builtins are found before shell functions during
command lookup.
18. If a POSIX special builtin returns an error status, a
non-interactive shell exits. The fatal errors are those listed in
the POSIX standard, and include things like passing incorrect
options, redirection errors, variable assignment errors for
assignments preceding the command name, etc.
19. If `CDPATH' is set, the `cd' builtin will not implicitly append
the current directory to it. This means that `cd' will fail if no
valid directory name can be constructed from any of the entries in
`$CDPATH', even if the a directory with the same name as the name
given as an argument to `cd' exists in the current directory.
20. A non-interactive shell exits with an error status if a variable
assignment error occurs when no command name follows the assignment
statements. A variable assignment error occurs, for example, when
trying to assign a value to a readonly variable.
21. A non-interactive shell exits with an error status if the iteration
variable in a `for' statement or the selection variable in a
`select' statement is a readonly variable.
22. Process substitution is not available.
23. Assignment statements preceding POSIX special builtins persist in
the shell environment after the builtin completes.
24. Assignment statements preceding shell function calls persist in the
shell environment after the function returns, as if a POSIX
special builtin command had been executed.
25. The `export' and `readonly' builtin commands display their output
in the format required by POSIX.
26. The `trap' builtin displays signal names without the leading `SIG'.
27. The `trap' builtin doesn't check the first argument for a possible
signal specification and revert the signal handling to the original
disposition if it is, unless that argument consists solely of
digits and is a valid signal number. If users want to reset the
handler for a given signal to the original disposition, they
should use `-' as the first argument.
28. The `.' and `source' builtins do not search the current directory
for the filename argument if it is not found by searching `PATH'.
29. Subshells spawned to execute command substitutions inherit the
value of the `-e' option from the parent shell. When not in POSIX
mode, Bash clears the `-e' option in such subshells.
30. Alias expansion is always enabled, even in non-interactive shells.
31. When the `alias' builtin displays alias definitions, it does not
display them with a leading `alias ' unless the `-p' option is
supplied.
32. When the `set' builtin is invoked without options, it does not
display shell function names and definitions.
33. When the `set' builtin is invoked without options, it displays
variable values without quotes, unless they contain shell
metacharacters, even if the result contains nonprinting characters.
34. When the `cd' builtin is invoked in LOGICAL mode, and the pathname
constructed from `$PWD' and the directory name supplied as an
argument does not refer to an existing directory, `cd' will fail
instead of falling back to PHYSICAL mode.
35. When the `pwd' builtin is supplied the `-P' option, it resets
`$PWD' to a pathname containing no symlinks.
36. The `pwd' builtin verifies that the value it prints is the same as
the current directory, even if it is not asked to check the file
system with the `-P' option.
37. When listing the history, the `fc' builtin does not include an
indication of whether or not a history entry has been modified.
38. The default editor used by `fc' is `ed'.
39. The `type' and `command' builtins will not report a non-executable
file as having been found, though the shell will attempt to
execute such a file if it is the only so-named file found in
`$PATH'.
40. The `vi' editing mode will invoke the `vi' editor directly when
the `v' command is run, instead of checking `$FCEDIT' and
`$EDITOR'.
41. When the `xpg_echo' option is enabled, Bash does not attempt to
interpret any arguments to `echo' as options. Each argument is
displayed, after escape characters are converted.
There is other POSIX behavior that Bash does not implement by default
even when in POSIX mode. Specifically:
1. The `fc' builtin checks `$EDITOR' as a program to edit history
entries if `FCEDIT' is unset, rather than defaulting directly to
`ed'. `fc' uses `ed' if `EDITOR' is unset.
2. As noted above, Bash requires the `xpg_echo' option to be enabled
for the `echo' builtin to be fully conformant.
Bash can be configured to be POSIX-conformant by default, by specifying
the `--enable-strict-posix-default' to `configure' when building.
FIXME help me to find out what breaks in POSIX(r) mode!
[**The POSIX(r) mode can be switched on by:**]{.underline}
- Bash starting as `sh` (the basename of `argv[0]` is `sh`)
- starting Bash with the commandline option `--posix`
- on startup, the environment variable
[POSIXLY_CORRECT](/syntax/shellvars#POSIXLY_CORRECT) is set
- the command `set -o posix`
[**Tests for the POSIX(r) mode:**]{.underline}
- the variable [SHELLOPTS](/syntax/shellvars#SHELLOPTS) contains
`posix` in its list
### Restricted shell
In restricted mode, Bash sets up (and runs) a shell environment that\'s
far more controlled and limited than the standard shell mode. It acts
like normal Bash with the following restrictions:
- the `cd` command can\'t be used to change directories
- the variables [SHELL](/syntax/shellvars#SHELL),
[PATH](/syntax/shellvars#PATH), [ENV](/syntax/shellvars#ENV) and
[BASH_ENV](/syntax/shellvars#BASH_ENV) can\'t be set or unset
- command names that contain a `/` (slash) can\'t be called (hence
you\'re limited to `PATH`)
- filenames containing a `/` (slash) can\'t be specified as argument
to the `source` or `.` builtin command
- filenames containing a `/` (slash) can\'t be specified as argument
to the `-p` option of the `hash` builtin command
- function definitions are not inherited from the environment at shell
startup
- the environment variable [SHELLOPTS](/syntax/shellvars#SHELLOPTS) is
ignored at startup
- redirecting output using the `>`, `>|`, `<>`, `>&`, `&>`, and `>>`
redirection operators isn\'t allowed
- the `exec` builtin command can\'t replace the shell with another
process
- adding or deleting builtin commands with the `-f` and `-d` options
to the enable builtin command is forbidden
- using the `enable` builtin command to enable disabled shell builtins
doesn\'t work
- the `-p` option to the `command` builtin command doesn\'t work
- turning off restricted mode with `set +r` or `set +o restricted` is
(of course) forbidden
The \"-r\" restrictions are turned on **after** Bash has read its
startup files.
When the command that is run is a shell script, then the restrictions
are **turned off** for the (sub-)shell that runs that shell script.
[**The restricted shell can be switched on by:**]{.underline}
- calling Bash as `rbash` (the basename of `argv[0]` is `rbash`)
- calling Bash with the `-r` option
- calling Bash with the `--restricted` option
[**Tests for restricted mode:**]{.underline}
- the special parameter `$-` contains the letter `r` (lowercase R)
- the shell option `restricted_shell` is set and can be checked by the
`shopt` builtin command

333
docs/scripting/bashchanges.md Normal file
View File

@ -0,0 +1,333 @@
# Bash changes
This article is an **incomplete overview** of changes to Bash over time.
Not all changes are listed, just the ones most likely to be useful for
normal scripting. The overviews are categorized by topic and ordered by
version.
A useful starting point is the [NEWS
file](https://github.com/bminor/bash/blob/master/NEWS) in bash sources.
If you have more detailed information, or historical information about
Bash versions earlier than V2, feel free to mail me, or use the
discussion below.
Status: 5.1 (alpha)
## Shell options
Note that the `shopt` builtin command first appeared in Bash 2.0.
For this topic, see also
- [shell_options](/internals/shell_options)
- [set](/commands/builtin/set)
Feature or change description Appeared in Bash version See also/remarks
--------------------------------- -------------------------- ---------------------------------------------------------------------------------
`posix` (for `set -o`) 1.14.0
`hostcomplete` 2.0-alpha3
`expand_aliases` 2.0
`huponexit` 2.02-alpha1
`nocaseglob` 2.02-alpha1
`extglob` 2.02-alpha1 together with extended globbing, KSH88
`restricted_shell` 2.03-alpha
`xpg_echo` 2.04-beta1
`progcomp` 2.04-alpha1
`no_empty_command_completion` 2.04
`login_shell` 2.05a-alpha1
`nolog` (for `set -o`) 2.05a
`gnu_errfmt` 3.0-alpha
`force_fignore` 3.0-alpha
`failglob` 3.0-alpha
`extquote` 3.0-alpha unsure \-- verify!
`extdebug` 3.0-alpha
`pipefail` (for `set -o`) 3.0
`functrace` (for `set -o`) 3.0
`errtrace` (for `set -o`) 3.0
`nocasematch` 3.1-alpha1
`dirspell` 4.0-alpha
`globstar` 4.0-alpha
`checkjobs` 4.0-alpha
`autocd` 4.0-alpha
`set -e` effects more intuitive 4.0 not directly specified by POSIX, but in consensus with POSIX WG
`compat40` 4.1-beta
`lastpipe` 4.2-alpha only works with job control disabled
`compat41` 4.2-alpha
`globasciiranges` 4.3-alpha enable \"character range globbing\" to always act as if in `C` locale
`compat42` 4.3-alpha
`compat43` 4.4-alpha
`compat44` 5.0-alpha
`localvar_inherit` 5.0-alpha local variables inherit preceeding scope values if they have the same name
`syslog_history` 5.0-alpha send history lines to syslog (undocumented, default off) if syslog is supported
`assoc_expand_once` 5.0-alpha expand associative array subscripts only one
`globasciiranges` 5.0-beta New default: on (default may be configured at compile time)
`localvar_inherit` 5.0-beta guard code against inheriting from an incompatible data type
`checkwinsize` 5.0-beta2 New default: on
`shift_verbose` 5.0-beta2 Default on when in POSIX mode
### General (all/many builtins)
Feature or change description Appeared in Bash version See also/remarks
----------------------------------------------------------------------- -------------------------- -----------------------------------------------------------------
generally return 2 on usage error 2.0
generally accept `--` (end of options) 2.0
(where applicable) implement a `-p` option to produce reusable output 2.0 `shopt` and `umask` builtins were fixed to support that in 2.02
### printf
For this topic, see also
- [printf](/commands/builtin/printf)
Feature or change description Appeared in Bash version See also/remarks
------------------------------------------------------------- -------------------------- --------------------------------------------------------
new `printf` command 2.02-alpha1
respects `0..` and `0x..` prefixed numbers 2.04-beta1 consistency with arithmetic
POSIX(r) length specifiers `j`, `t` and `z` 2.05a-alpha1 ISO C99
POSIX(r) flag `'` 2.05a-alpha1
conversion `a` and `A` 2.05a-rc1 if provided by the underlying printf(3)
conversion `F` 2.05a-rc1
conversion `n` 2.05a-rc1
new option `-v` 3.1-alpha1
escape sequences `\"` and `\?` 3.0-beta1
modified option `-v` to assign to individual array elements 4.1-alpha
conversion `(...)T` 4.2-alpha support stftime(3) date/time format; uses current time
`\uNNNN` and `\UNNNNNNNN` escape sequences 4.2-alpha for: `printf`, `echo -e`, `$'...'`
### Conditional expressions and test command
For this topic, see also
- [conditional_expression](/syntax/ccmd/conditional_expression)
- [classictest](/commands/classictest)
Feature or change description Appeared in Bash version See also/remarks
-------------------------------------------------------------------- -------------------------- --------------------------------------------------------------------------------------
`test`: `-o`, `==`, `<` and `>` 2.0
`test`: `-N` 2.02
`[[...]]`: new 2.02-alpha1 KSH93
`[[...]]`: regex support (`=~`) 3.0-alpha
`[[...]]`: quotable right-hand-side of `=~` forces string matching 3.2-alpha for consistency with pattern matching
`[[...]]`: `<` and `>` operators respect locale 4.1-alpha for consistency, since 4.1-beta: ensure you have set compatiblity to \>4.0 (default)
`test`/`[`/`[[`: `-v` 4.2-alpha check if a variable is set
`test`/`[`/`[[`: `-v` 4.2-alpha support array syntax to check for elements
`test`/`[`/`[[`: `-N` accepts nanoseconds 5.1-alpha
`test`/`[`/`[[`: `-v` accepts positional parameters 5.1-alpha
### Other builtins and keywords
Builtin Feature or change description Appeared in Bash version See also/remarks
----------------------- -------------------------------------------------------------------------------------------------------------------------------------------- -------------------------- -----------------------------------------------------------------------------------------------
`bashbug` new 1.14.0
`select` new 1.14.0
`disown` new 2.0
`shopt` new 2.0 [shopt](/commands/builtin/shopt)
`declare` new options `-a` and `-F` 2.0
`enable` builtin has basic plugin support (dlopen) 2.0
`exec` options `-l`, `-c` and `-a` 2.0
`read` options `-p`, `-e` and `-a` 2.0 [read](/commands/builtin/read)
`readonly` option `-a` 2.0 [arrays](/syntax/arrays)
`time` new keyword 2.0
`shopt` `-p` (reusable output) 2.02
`umask` `-p` (reusable output) 2.02
`complete` new 2.04-devel for and together with support for programmable completion
`compgen` new 2.04-devel for and together with support for programmable completion
`read` options `-t`, `-n`, `-d`, `-s` 2.04-devel [read](/commands/builtin/read)
`for ((...;...;...))` new 2.04-devel KSH93
`set` print shell functions in a format reusable as input 2.05-beta1
`for` allow an empty word list 2.05a-alpha1
`read` new option `-u` 2.05b-alpha1 [read](/commands/builtin/read)
`caller` new 3.0 [caller](/commands/builtin/caller)
`coproc` new 4.0-alpha
`declare` new options `-l` and `-u` 4.0-alpha together with case-changing expansion forms
`case` new action list terminators \'\';;& and \'\';& 4.0-alpha ksh93: only `;&`. zsh and mksh: `;|`. mksh: all 4, (`;;&` is undocumented Bash compatibility)
`read` changed `-t` (fractional seconds) 4.0-alpha
`mapfile` new 4.0-alpha
`read` new option `-i` 4.0-alpha
`compopt` new 4.0-alpha
`read` modified option `-t` to test for data 4.0-beta
`read` new option `-N` 4.1-alpha
`mapfile` changed behaviour regarding history spamming 4.1-alpha
`declare` new option `-g` 4.2-alpha
`mapfile` calls the callback with an additional argument: The line (data) 4.2-alpha
`cd` new option `-e` 4.2-alpha
`echo` `\uNNNN` and `\UNNNNNNNN` escape sequences 4.2-alpha for: `printf`, `echo -e`, `$'...'`
`exec` option `-a` to give a `argv[0]` string 4.2-alpha
`time` allowed as a command by itself to display timing values of the shell and its children 4.2-alpha POSIX change
`help` `help` now searches exact topic-strings (i.e. `help read` won\'t find `readonly` anymore) 4.3-alpha
`return` accept negative values as return value (e.g. `return -1` will show as (8 bit) 255 in the caller) 4.3-alpha
`exit` accept negative values as return value (e.g. `return -1` will show as (8 bit) 255 in the caller) 4.3-alpha
`read` `read` skips `NUL` (ASCII Code 0) in input 4.3-alpha
`declare` new option `-n`/`+n` to support nameref variable type 4.3-alpha
`wait` new option `-n` to wait for the next background job to finish, returning its exit status. 4.3-alpha
`read` `read` checks first variable argument for validity before trying to read inout 4.3-beta
`help` attempts substring matching (as it did through bash-4.2) if exact string matching fails 4.3-beta2
`fc` interprets option `-0` (zero) as the current command line 4.3-beta2
`cd` new option `-@` to browse a file\'s extended attributes (on systems that support `O_XATTR`) 4.3-rc1
`kill` new option `-L` (upper case ell) to list signals like the normal lowercase option `-l` (compatiblity with some standalone `kill` commands) 4.4-beta
`mapfile` new option `-d` 4.4-alpha
`wait` new option `-f` 5.0-alpha
`history` option `-d` allows negative numbers to index from the end of the history list 5.0-alpha
`umask` allows modes greater than octal 777 5.0-alpha
`times` honors current locale settings when printing decimal points 5.0-alpha
`kill` New options `-n SIGNUMBER` and `-s SIGNAME` 5.0-beta2 [kill](/commands/builtin/kill)
`select` Support for an empty wordlist following `in` 5.0-beta2
`read` Option `-e` (use ReadLine to obtain input) now works with arbitrary file descriptors (given by `-u` option) 5.1-alpha
`trap` `-p` option prints signals with SIG_DFL/SIG_IGN on shell start (POSIX mode) 5.1-alpha
`unset` automatically tries to unset a function if the given name is an invalid variable name 5.1-aplha
`wait` option `-n` now accepts a list of jobs 5.1-alpha
`wait` new option `-p NAME` to store PID/JobID (useful when waiting for a list of jobs) 5.1-alpha
`local` new option `-p` to print local variables in the current scope 5.1-alpha
`ulimit` new option `-R` to get/set `RLIMIT_RTTIME` resource 5.1-alpha
## Builtin variables
Feature or change description Appeared in Bash version See also
--------------------------------------------------- -------------------------- ----------------------------------------------------------------------------------
`HISTCMD` 1.14.0 interactive usage
`PS1`, `PS2`, `PATH`, and `IFS` are unsettable 2.0
`DIRSTACK` array variable 2.0
`PIPESTATUS` array variable 2.0
`BASH_VERSINFO` array variable 2.0
`HOSTNAME` 2.0
`SHELLOPTS` 2.0
`MACHTYPE` 2.0
`GLOBIGNORE` 2.0
`HISTIGNORE` 2.0
respect `LC_ALL` 2.0
respect `LC_MESSAGES` 2.0
respect `LC_CTYPE` 2.0
respect `LC_COLLATE` 2.0
respect `LANG` 2.0
`GROUPS` array variable 2.01
`GROUPS` unsettable/takes (discarded) assignments 2.04
`FUNCNAME` 2.04
respect `LC_NUMERIC` 2.04
`TMOUT` 2.05b
`BASH_REMATCH` 3.0 together with regex support in `[[...]]`
`BASH_ARGC` 3.0 debugger support
`BASH_ARGV` 3.0 debugger support
`BASH_SOURCE` 3.0 debugger support
`BASH_LINENO` 3.0 debugger support
`BASH_SUBSHELL` 3.0 debugger support
`BASH_EXECUTION_STRING` 3.0 debugger support
`BASH_COMMAND` 3.0 debugger support
`HISTTIMEFORMAT` 3.0
`COMP_WORDBREAKS` 3.0
respect `LC_TIME` 3.1
`BASHPID` 4.0-alpha Added to mksh R41.
`PROMPT_DIRTRIM` 4.0
`BASH_XTRACEFD` 4.1-alpha
`BASHOPTS` 4.1-alpha
`FUNCNEST` 4.2-alpha
`HISTSIZE` 4.3-alpha can be set to negative values for unlimited history length
`HISTFILESIZE` 4.3-alpha can be set to negative values for unlimit history file size
`CHILD_MAX` 4.3-alpha max. number of exit status of children the shell remembers
`BASH_COMPAT` 4.3-alpha set shell compatiblity levels
`EPOCHSECONDS` 5.0-alpha expands to the time in seconds since Unix epoch
`EPOCHREALTIME` 5.0-alpha expands to the time in seconds since Unix epoch with microsecond granularity
`BASH_ARGV0` 5.0-alpha get/set `$0`
`PATH` 5.0-alpha Possibility to set a static path for use in a restricted shell (at compile time)
`HISTSIZE` 5.0-beta Default can now be set at runtime
`SRANDOM` 5.1-alpha New random generator for 32bit numbers (using various methods in the backend)
`ARGV0` 5.1-alpha Respected when set in initial shell environment, then initially used to set `$0`
`BASH_REMATCH` 5.1-alpha Not readonly anymore
`PROMPT_COMMANDS` 5.1-alpha New array variable. List of commands to be executed like `PROMPT_COMMAND`
`SECONDS` 5.1-alpha Assignment using arithmetic expressions (is nominally an integer variabnle)
`RANDOM` 5.1-alpha Assignment using arithmetic expressions (is nominally an integer variabnle)
`LINENO` 5.1-alpha Not an integer variabe
## Quoting, expansions, substitutions and related
For this topic, see also
- [pe](/syntax/pe).
Feature or change description Appeared in Bash version Remarks
------------------------------------------------------------------------------------------------------ -------------------------- ---------------------------------------------------------------------------------------------------------------
Support for integer-indexed arrays 2.0 relevant builtins also got array support
`${PARAMETER//PATTERN/REPLACEMENT}` 2.0
`${PARAMETER:OFFSET:LENGTH}` 2.0
`${!PARAMETER}` (indirection) 2.0
`$"..."` (localized strings) 2.0
`$'...'` (ANSI-C-like strings) 2.0
`\xNNN` in `$'...'` (and `echo -e`) 2.02-alpha1
`$(< FILENAME)` (file content) 2.02-alpha1
globbing (`fnmatch()`) capable of POSIX(r) character classes etc. 2.02-alpha1
extended globbing 2.02-alpha1 KSH88
globbing inside array mass-assignment: `ARRAY=(*.txt)` 2.03-alpha
`$'...\'...'` escaped single quote inside ANSI-C-like strings 2.04-devel KSH93
`${!PREFIX*}` (parameter name expansion) 2.04 KSH93
`$'...'` expands `\cx` (Control-x) 2.05b
`[:class:]` syntax for pattern matching 2.05b KSH93
`${!ARRAY[@]}` (array index expansion) 3.0-alpha KSH93
`{x..y}` (range brace expansion) 3.0-alpha
`$'...'` expands `\xNNN` (Hexdigits) 3.0
`+=` operator for arrays and strings 3.1-alpha1
`${PARAMETER//PATTERN/REPLACEMENT}` behaviour changed 3.2-alpha anchoring for global substitution is no longer allowed, changes the way old syntax may work
`${@:0:x}` includes `$0` 4.0-alpha
Support for associative arrays 4.0-alpha relevant builtins also got associative array support
case modification operators for expansions 4.0-alpha
`{0x..0y}` (zeropadding brace expansion) 4.0-alpha
numerically indexed arrays can be accessed (on expansion time) from the end using negative indexes 4.1-alpha
`\uNNNN` and `\uNNNNNNNN` in `$'...'` 4.2-alpha for: `printf`, `echo -e`, `$'...'`
`${PARAMETER:OFFSET:LENGTH}`: Negative `LENGTH` values are used as offset from the end of the string 4.2-alpha Substrings only for Bash and ksh93. Works also for argument expansions in zsh. ksh93 can use `${arr[n..-m]}`.
Word expansions like `${foo##bar}` understand indirect variable references 4.3-beta
Transformations 4.4
Process substitution now works in POSIX mode 5.1-alpha
New transformations: `U`, `u`, `L` 5.1-alpha Case-transformation
New transformation: `K` 5.1-alpha Display associative arrays as key/value pairs
## Arithmetic
For this topic, see also
- [arith_expr](/syntax/arith_expr)
- [arith](/syntax/expansion/arith)
Feature or change description Appeared in Bash version Remarks
-------------------------------------------- -------------------------- -------------------------------------------
`((...))` 2.0-beta2 KSH93
ternary operator 2.0
base 64 integer constants 2.0 the max. base before is unknown. Anybody?
deprecated `$[...]` in favor of `$((...))` 2.0
exponentiaition operator (`**`) 2.02-alpha1
comma operator `EXPR,EXPR` 2.04-devel
pre- and postfix operators 2.04-devel
## Redirection and related
For this topic, see also
- [redirection](/syntax/redirection)
Feature or change description Appeared in Bash version Remarks
--------------------------------------------------------------------------------------- -------------------------- ---------
socket redirection (`/dev/tcp/`, `/dev/udp/`) 2.04-devel
OS/filesystem-independent support for `/dev/std(in|out|err)` and `/dev/fd/*` 2.04
socket redirection accepts service names 2.05
`[n]<&word-` and `[n]>&word-` FD-duplicate/closing 2.05b-alpha1 KSH93
Here strings: `<<< WORD` 2.05b-alpha1
`|&` (synonym for `2>&1 |`) 4.0-alpha
`&>>` (equiv. to `>>FILE 2>&1`) 4.0-alpha
`{varname}` style automatic file descriptor allocation 4.1-alpha ksh93
`{varname[idx]}` fd allocation accepts array subscripts and special-meaning variables 4.3-alpha ksh93
## Misc
Feature or change description Appeared in Bash version See also/remarks
------------------------------------------------------------------------------------------------- -------------------------- ------------------------------------------------------------------------------------------------------------------------------------------
`DEBUG` trap 2.0
`ERR` trap 2.05a KSH93
Support for multibyte characters: Unicode / UTF8 2.05b
`RETURN` trap 3.0 ksh93 `EXIT` trap evaluates in caller scope (for `function name {`). Bash `RETURN` in same scope.
`command_not_found_handle` handler function 4.0-alpha
official introduction of switchable \"compatiblity levels\" 4.0-alpha `compat31` was introduced in a 3.2 version, mainly because of the incompatibilities that were introduced by the changed `=~` operator
`[[...]]` and `((...))` conditional commands are subject to the `ERR` trap and `set -e` feature 4.1-alpha
ACL support for file status checks 4.1-alpha
Assignment to negative array indices 4.3-alpha ksh93, zsh
`declare`/`typeset -n` 4.3-alpha Support for nameref variable type, a variable referencing another one by name
shells started to run process substitutions now run any trap set on `EXIT` 4.3-beta
process substitution does not inherit the `v` flag 5.0-alpha
`ERR` trap 5.0-alpha Reports more reliable line numbers
Variable assignment 5.0-beta Assignments preceeding a special builtin that chages variable attributes are not propagated back unless compatiblity mode is 44 or lower

357
docs/scripting/basics.md Normal file
View File

@ -0,0 +1,357 @@
# The basics of shell scripting
![](keywords>bash shell scripting basics learning tutorial)
## Script files
A shell script usually resides inside a file. The file can be
executable, but you can call a Bash script with that filename as a
parameter:
bash ./myfile
There is **no need to add a boring filename extension** like `.bash` or
`.sh`. That is a holdover from UNIX(r), where executables are not tagged
by the extension, but by **permissions** (filemode). The file name can
be any combination of legal filename characters. Adding a proper
filename extension is a convention, nothing else.
chmod +x ./myfile
If the file is executable, and you want to use it by calling only the
script name, the shebang must be included in the file.
## The Shebang
The in-file specification of the interpreter of that file, for example:
``` bash
#!/bin/bash
echo "Hello world..."
```
This is interpreted by the kernel [^1] of your system. In general, if a
file is executable, but not an executable (binary) program, and such a
line is present, the program specified after `#!` is started with the
scriptname and all its arguments. These two characters `#` and `!` must
be **the first two bytes** in the file!
You can follow the process by using `echo` as a fake interpreter:
#!/bin/echo
We don\'t need a script body here, as the file will never be interpreted
and executed by \"`echo`\". You can see what the Operating System does,
it calls \"`/bin/echo`\" with the name of the executable file and
following arguments.
$ /home/bash/bin/test testword hello
/home/bash/bin/test testword hello
The same way, with `#!/bin/bash` the shell \"`/bin/bash`\" is called
with the script filename as an argument. It\'s the same as executing
\"`/bin/bash /home/bash/bin/test testword hello`\"
If the interpreter can be specified with arguments and how long it can
be is system-specific (see
[#!-magic](http://www.in-ulm.de/~mascheck/various/shebang/)). When Bash
executes a file with a #!/bin/bash shebang, the shebang itself is
ignored, since the first character is a hashmark \"#\", which indicates
a comment. The shebang is for the operating system, not for the shell.
Programs that don\'t ignore such lines, may not work as shebang driven
interpreters.
\<WRAP center round important 60%\> [**Attention:**]{.underline}When the
specified interpreter is unavailable or not executable (permissions),
you usually get a \"`bad interpreter`\" error message., If you get
nothing and it fails, check the shebang. Older Bash versions will
respond with a \"`no such file or directory`\" error for a nonexistant
interpreter specified by the shebang. \</WRAP\>
**Additional note:** When you specify `#!/bin/sh` as shebang and that\'s
a link to a Bash, then Bash will run in POSIX(r) mode! See:
- [Bash behaviour](/scripting/bashbehaviour).
A common method is to specify a shebang like
#!/usr/bin/env bash
\...which just moves the location of the potential problem to
- the `env` utility must be located in /usr/bin/
- the needed `bash` binary must be located in `PATH`
Which one you need, or whether you think which one is good, or bad, is
up to you. There is no bulletproof portable way to specify an
interpreter. **It\'s a common misconception that it solves all problems.
Period.**
## The standard filedescriptors
Once Initialized, every normal UNIX(r)-program has *at least 3 open
files*:
- **stdin**: standard input
- **stdout**: standard output
- **stderr**: standard error output
Usually, they\'re all connected to your terminal, stdin as input file
(keyboard), stdout and stderr as output files (screen). When calling
such a program, the invoking shell can change these filedescriptor
connections away from the terminal to any other file (see redirection).
Why two different output filedescriptors? It\'s convention to send error
messages and warnings to stderr and only program output to stdout. This
enables the user to decide if they want to see nothing, only the data,
only the errors, or both - and where they want to see them.
When you write a script:
- always read user-input from `stdin`
- always write diagnostic/error/warning messages to `stderr`
To learn more about the standard filedescriptors, especially about
redirection and piping, see:
- [An illustrated redirection tutorial](/howto/redirection_tutorial)
## Variable names
It\'s good practice to use lowercase names for your variables, as shell
and system-variable names are usually all in UPPERCASE. However, you
should avoid naming your variables any of the following (incomplete
list!):
---------------- ------------- ---------------- --------------- ------------------ -----------------
`BASH` `BASH_ARGC` `BASH_ARGV` `BASH_LINENO` `BASH_SOURCE` `BASH_VERSINFO`
`BASH_VERSION` `COLUMNS` `DIRSTACK` `DISPLAY` `EDITOR` `EUID`
`GROUPS` `HISTFILE` `HISTFILESIZE` `HISTSIZE` `HOME` `HOSTNAME`
`IFS` `LANG` `LANGUAGE` `LC_ALL` `LINES` `LOGNAME`
`LS_COLORS` `MACHTYPE` `MAILCHECK` `OLDPWD` `OPTERR` `OPTIND`
`OSTYPE` `PATH` `PIPESTATUS` `PPID` `PROMPT_COMMAND` `PS1`
`PS2` `PS4` `PS3` `PWD` `SHELL` `SHELLOPTS`
`SHLVL` `TERM` `UID` `USER` `USERNAME` `XAUTHORITY`
---------------- ------------- ---------------- --------------- ------------------ -----------------
This list is incomplete. **The safest way is to use all-lowercase
variable names.**
## Exit codes
Every program you start terminates with an exit code and reports it to
the operating system. This exit code can be utilized by Bash. You can
show it, you can act on it, you can control script flow with it. The
code is a number between 0 and 255. Values from 126 to 255 are reserved
for use by the shell directly, or for special purposes, like reporting a
termination by a signal:
- **126**: the requested command (file) was found, but can\'t be
executed
- **127**: command (file) not found
- **128**: according to ABS it\'s used to report an invalid argument
to the exit builtin, but I wasn\'t able to verify that in the source
code of Bash (see code 255)
- **128 + N**: the shell was terminated by the signal N
- **255**: wrong argument to the exit builtin (see code 128)
The lower codes 0 to 125 are not reserved and may be used for whatever
the program likes to report. A value of 0 means **successful**
termination, a value not 0 means **unsuccessful** termination. This
behavior (== 0, != 0) is also what Bash reacts to in some flow control
statements.
An example of using the exit code of the program `grep` to check if a
specific user is present in /etc/passwd:
``` bash
if grep ^root /etc/passwd; then
echo "The user root was found"
else
echo "The user root was not found"
fi
```
A common decision making command is \"`test`\" or its equivalent
\"`[`\". But note that, when calling test with the name \"`[`\", the
square brackets are not part of the shell syntax, the left bracket
**is** the test command!
``` bash
if [ "$mystring" = "Hello world" ]; then
echo "Yeah dude, you entered the right words..."
else
echo "Eeeek - go away..."
fi
```
Read more about [the test command](/commands/classictest)
A common exit code check method uses the \"`||`\" or \"`&&`\" operators.
This lets you execute a command based on whether or not the previous
command completed successfully:
``` bash
grep ^root: /etc/passwd >/dev/null || echo "root was not found - check the pub at the corner."
which vi && echo "Your favourite editor is installed."
```
Please, when your script exits on errors, provide a \"FALSE\" exit code,
so others can check the script execution.
## Comments
In a larger, or complex script, it\'s wise to comment the code. Comments
can help with debugging or tests. Comments start with the \# character
(hashmark) and continue to the end of the line:
``` bash
#!/bin/bash
# This is a small script to say something.
echo "Be liberal in what you accept, and conservative in what you send" # say something
```
The first thing was already explained, it\'s the so-called shebang, for
the shell, **only a comment**. The second one is a comment from the
beginning of the line, the third comment starts after a valid command.
All three syntactically correct.
### Block commenting
To temporarily disable complete blocks of code you would normally have
to prefix every line of that block with a \# (hashmark) to make it a
comment. There\'s a little trick, using the pseudo command `:` (colon)
and input redirection. The `:` does nothing, it\'s a pseudo command, so
it does not care about standard input. In the following code example,
you want to test mail and logging, but not dump the database, or execute
a shutdown:
``` bash
#!/bin/bash
# Write info mails, do some tasks and bring down the system in a safe way
echo "System halt requested" | mail -s "System halt" netadmin@example.com
logger -t SYSHALT "System halt requested"
##### The following "code block" is effectively ignored
: <<"SOMEWORD"
/etc/init.d/mydatabase clean_stop
mydatabase_dump /var/db/db1 /mnt/fsrv0/backups/db1
logger -t SYSHALT "System halt: pre-shutdown actions done, now shutting down the system"
shutdown -h NOW
SOMEWORD
##### The ignored codeblock ends here
```
What happened? The `:` pseudo command was given some input by
redirection (a here-document) - the pseudo command didn\'t care about
it, effectively, the entire block was ignored.
The here-document-tag was quoted here **to avoid substitutions** in the
\"commented\" text! Check [redirection with
here-documents](/syntax/redirection#tag_heredoc) for more
## Variable scope
In Bash, the scope of user variables is generally *global*. That means,
it does **not** matter whether a variable is set in the \"main program\"
or in a \"function\", the variable is defined everywhere.
Compare the following *equivalent* code snippets:
``` bash
myvariable=test
echo $myvariable
```
``` bash
myfunction() {
myvariable=test
}
myfunction
echo $myvariable
```
In both cases, the variable `myvariable` is set and accessible from
everywhere in that script, both in functions and in the \"main
program\".
**[Attention:]{.underline}** When you set variables in a child process,
for example a *subshell*, they will be set there, but you will **never**
have access to them outside of that subshell. One way to create a
subshell is the pipe. It\'s all mentioned in a small article about [Bash
in the processtree](/scripting/processtree)!
### Local variables
Bash provides ways to make a variable\'s scope *local* to a function:
- Using the `local` keyword, or
- Using `declare` (which will *detect* when it was called from within
a function and make the variable(s) local).
``` bash
myfunc() {
local var=VALUE
# alternative, only when used INSIDE a function
declare var=VALUE
...
}
```
The *local* keyword (or declaring a variable using the `declare`
command) tags a variable to be treated *completely local and separate*
inside the function where it was declared:
``` bash
foo=external
printvalue() {
local foo=internal
echo $foo
}
# this will print "external"
echo $foo
# this will print "internal"
printvalue
# this will print - again - "external"
echo $foo
```
### Environment variables
The environment space is not directly related to the topic about scope,
but it\'s worth mentioning.
Every UNIX(r) process has a so-called *environment*. Other items, in
addition to variables, are saved there, the so-called *environment
variables*. When a child process is created (in Bash e.g. by simply
executing another program, say `ls` to list files), the whole
environment *including the environment variables* is copied to the new
process. Reading that from the other side means: **Only variables that
are part of the environment are available in the child process.**
A variable can be tagged to be part of the environment using the
`export` command:
``` bash
# create a new variable and set it:
# -> This is a normal shell variable, not an environment variable!
myvariable="Hello world."
# make the variable visible to all child processes:
# -> Make it an environment variable: "export" it
export myvariable
```
Remember that the *exported* variable is a **copy**. There is no
provision to \"copy it back to the parent.\" See the article about [Bash
in the process tree](/scripting/processtree)!
[^1]: under specific circumstances, also by the shell itself

374
docs/scripting/debuggingtips.md Normal file
View File

@ -0,0 +1,374 @@
# Debugging a script
![](keywords>bash shell scripting bug debug debugging)
These few lines are not intended as a full-fledged debugging tutorial,
but as hints and comments about debugging a Bash script.
## Use a unique name for your script
Do **not** name your script `test`, for example! *Why?* `test` is the
name of a UNIX(r)-command, and [most likely built into your
shell]{.underline} (it\'s a built-in in Bash) - so you won\'t be able to
run a script with the name `test` in a normal way.
**Don\'t laugh!** This is a classic mistake :-)
## Read the error messages
Many people come into IRC and ask something like *\"Why does my script
fail? I get an error!\"*. And when you ask them what the error message
is, they don\'t even know. Beautiful.
Reading and interpreting error messages is 50% of your job as debugger!
Error messages actually **mean** something. At the very least, they can
give you hints as to where to start debugging. **READ YOUR ERROR
MESSAGES!**
You may ask yourself why is this mentioned as debugging tip? Well, [you
would be surprised how many shell users ignore the text of error
messages!]{.underline} When I find some time, I\'ll paste 2 or 3 IRC
log-snips here, just to show you that annoying fact.
## Use a good editor
Your choice of editor is a matter of personal preference, but one with
**Bash syntax highlighting** is highly recommended! Syntax highlighting
helps you see (you guessed it) syntax errors, such as unclosed quotes
and braces, typos, etc.
From my personal experience, I can suggest `vim` or `GNU emacs`.
## Write logfiles
For more complex scripts, it\'s useful to write to a log file, or to the
system log. Nobody can debug your script without knowing what actually
happened and what went wrong.
An available syslog interface is `logger` ([online
manpage](http://unixhelp.ed.ac.uk/CGI/man-cgi?logger+1)).
## Inject debugging code
Insert **echos** everywhere you can, and print to `stderr`:
echo "DEBUG: current i=$i" >&2
If you read input from **anywhere**, such as a file or [command
substitution](/syntax/expansion/cmdsubst), print the debug output with
literal quotes, to see leading and trailing spaces!
pid=$(< fooservice.pid)
echo "DEBUG: read from file: pid=\"$pid\"" >&2
Bash\'s [printf](/commands/builtin/printf) command has the `%q` format,
which is handy for verifying whether strings are what they appear to be.
foo=$(< inputfile)
printf "DEBUG: foo is |%q|\n" "$foo" >&2
# exposes whitespace (such as CRs, see below) and non-printing characters
## Use shell debug output
There are two useful debug outputs for that task (both are written to
`stderr`):
- `set -v` mode (`set -o verbose`)
- print commands to be executed to `stderr` as if they were read
from input (script file or keyboard)
- print everything **before** any ([substitution and
expansion](/syntax/expansion/intro), \...) is applied
- `set -x` mode (`set -o xtrace`)
- print everything as if it were executed, after [substitution and
expansion](/syntax/expansion/intro) is applied
- indicate the depth-level of the subshell (by default by
prefixing a `+` (plus) sign to the displayed command)
- indicate the recognized words after [word
splitting](/syntax/expansion/wordsplit) by marking them like
`'x y'`
- in shell version 4.1, this debug output can be printed to a
configurable file descriptor, rather than sdtout by setting the
[BASH_XTRACEFD](/syntax/shellvars#BASH_XTRACEFD) variable.
**[Hint:]{.underline}** These modes can be entered when calling Bash:
- from commandline: `bash -vx ./myscript`
- from shebang (OS dependant): `#!/bin/bash -vx`
### Simple example of how to interpret xtrace output
Here\'s a simple command (a string comparison using the [classic test
command](/commands/classictest)) executed while in `set -x` mode:
set -x
foo="bar baz"
[ $foo = test ]
That fails. Why? Let\'s see the `xtrace` output:
+ '[' bar baz = test ']'
And now you see that it\'s (\"bar\" and \"baz\") recognized as two
separate words (which you would have realized if you READ THE ERROR
MESSAGES ;) ). Let\'s check it\...
# next try
[ "$foo" = test ]
`xtrace` now gives
+ '[' 'bar baz' = test ']'
^ ^
word markers!
### Making xtrace more useful
(by AnMaster)
`xtrace` output would be more useful if it contained source file and
line number. Add this assignment [PS4](/syntax/shellvars#PS4) at the
beginning of your script to enable the inclusion of that information:
export PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'
**Be sure to use single quotes here!**
The output would look like this when you trace code *outside a
function*:
+(somefile.bash:412): echo 'Hello world'
\...and like this when you trace code *inside a function*:
+(somefile.bash:412): myfunc(): echo 'Hello world'
That helps a lot when the script is long, or when the main script
sources many other files.
#### Set flag variables with descriptive words
If you test variables that flag the state of options, such as with
`if [[ -n $option ]];`, consider using descriptive words rather than
short codes, such as 0, 1, Y, N, because xtrace will show
`[[ -n word ]]` rather than `[[ -n 1 ]]` when the option is set.
## Debugging commands depending on a set variable
For general debugging purposes you can also define a function and a
variable to use:
debugme() {
[[ $script_debug = 1 ]] && "$@" || :
# be sure to append || : or || true here or use return 0, since the return code
# of this function should always be 0 to not influence anything else with an unwanted
# "false" return code (for example the script's exit code if this function is used
# as the very last command in the script)
}
This function does nothing when `script_debug` is unset or empty, but it
executes the given parameters as commands when `script_debug` is set.
Use it like this:
script_debug=1
# to turn it off, set script_debug=0
debugme logger "Sorting the database"
database_sort
debugme logger "Finished sorting the database, exit code $?"
Of course this can be used to execute something other than echo during
debugging:
debugme set -x
# ... some code ...
debugme set +x
## Dry-run STDIN driven commands
Imagine you have a script that runs FTP commands using the standard FTP
client:
``` bash
ftp user@host <<FTP
cd /data
get current.log
dele current.log
FTP
```
A method to dry-run this with debug output is:
``` bash
if [[ $DRY_RUN = yes ]]; then
sed 's/^/DRY_RUN FTP: /'
else
ftp user@host
fi <<FTP
cd /data
get current.log
dele current.log
FTP
```
This can be wrapped in a shell function for more readable code.
## Common error messages
### Unexpected end of file
script.sh: line 100: syntax error: unexpected end of file
Usually indicates exactly what it says: An unexpected end of file. It\'s
unexpected because Bash waits for the closing of a [compound
command](/syntax/ccmd/intro):
- did you close your `do` with a `done`?
- did you close your `if` with a `fi`?
- did you close your `case` with a `esac`?
- did you close your `{` with a `}`?
- did you close your `(` with a `)`?
**[Note:]{.underline}** It seems that here-documents (tested on versions
`1.14.7`, `2.05b`, `3.1.17` and `4.0`) are correctly terminated when
there is an EOF before the end-of-here-document tag (see
[redirection](/syntax/redirection)). The reason is unknown, but it seems
to be deliberate. Bash 4.0 added an extra message for this:
`` warning: here-document at line <N> delimited by end-of-file (wanted `<MARKER>') ``
### Unexpected end of file while looking for matching \...
script.sh: line 50: unexpected EOF while looking for matching `"'
script.sh: line 100: syntax error: unexpected end of file
This one indicates the double-quote opened in line 50 does not have a
matching closing quote.
These *unmatched errors* occur with:
- double-quote pairs
- single-quote pairs (also `$'string'`!)
- missing a closing `}` with [parameter expansion syntax](/syntax/pe)
### Too many arguments
bash: test: too many arguments
You most likely forgot to quote a variable expansion somewhere. See the
example for `xtrace` output from above. External commands may display
such an error message though in our example, it was the **internal**
test-command that yielded the error.
### !\": event not found
$ echo "Hello world!"
bash: !": event not found
This is not an error per se. It happens in interactive shells, when the
C-Shell-styled history expansion (\"`!searchword`\") is enabled. This is
the default. Disable it like this:
set +H
# or
set +o histexpand
### syntax error near unexpected token \`(\'
When this happens during a script **function definition** or on the
commandline, e.g.
$ foo () { echo "Hello world"; }
bash: syntax error near unexpected token `('
you most likely have an alias defined with the same name as the function
(here: `foo`). Alias expansion happens before the real language
interpretion, thus the alias is expanded and makes your function
definition invalid.
## The CRLF issue
### What is the CRLF issue?
There\'s a big difference in the way that UNIX(r) and Microsoft(r) (and
possibly others) handle the **line endings** of plain text files. The
difference lies in the use of the CR (Carriage Return) and LF (Line
Feed) characters.
- MSDOS uses: `\r\n` (ASCII `CR` #13 `^M`, ASCII LF #10)
- UNIX(r) uses: `\n` (ASCII `LF` #10)
Keep in mind your script is a **plain text file**, and the `CR`
character means nothing special to UNIX(r) - it is treated like any
other character. If it\'s printed to your terminal, a carriage return
will effectively place the cursor at the beginning of the *current*
line. This can cause much confusion and many headaches, since lines
containing CRs are not what they appear to be when printed. In summary,
CRs are a pain.
### How did a CR end up in my file?
Some possible sources of CRs:
- a DOS/Windows text editor
- a UNIX(r) text editor that is \"too smart\" when determining the
file content type (and thinks \"*it\'s a DOS text file*\")
- a direct copy and paste from certain webpages (some pastebins are
known for this)
### Why do CRs hurt?
CRs can be a nuisance in various ways. They are especially bad when
present in the shebang/interpreter specified with `#!` in the very first
line of a script. Consider the following script, written with a
Windows(r) text editor (`^M` is a symbolic representation of the `CR`
carriage return character!):
#!/bin/bash^M
^M
echo "Hello world"^M
...
Here\'s what happens because of the `#!/bin/bash^M` in our shebang:
- the file `/bin/bash^M` doesn\'t exist (hopefully)
- So Bash prints an error message which (depending on the terminal,
the Bash version, or custom patches!) may or may not expose the
problem.
- the script can\'t be executed
The error message can vary. If you\'re lucky, you\'ll get:
bash: ./testing.sh: /bin/bash^M: bad interpreter: No such file or directory
which alerts you to the CR. But you may also get the following:
: bad interpreter: No such file or directory
Why? Because when printed literally, the `^M` makes the cursor go back
to the beginning of the line. The whole error message is *printed*, but
you *see* only part of it!
\<note warning\> It\'s easy to imagine the `^M` is bad in other places
too. If you get weird and illogical messages from your script, rule out
the possibility that`^M` is involved. Find and eliminate it! \</note\>
### How can I find and eliminate them?
**To display** CRs (these are only a few examples)
- in VI/VIM: `:set list`
- with `cat(1)`: `cat -v FILE`
**To eliminate** them (only a few examples)
- blindly with `tr(1)`: `tr -d '\r' <FILE >FILE.new`
- controlled with `recode(1)`: `recode MSDOS..latin1 FILE`
- controlled with `dos2unix(1)`: `dos2unix FILE`
## See also
- [the set builtin command](/commands/builtin/set) (for `-v` and `-x`)
FIXME
- DEBUG trap
- BASH Debugger <http://bashdb.sourceforge.net/>

282
docs/scripting/newbie_traps.md Normal file
View File

@ -0,0 +1,282 @@
# Beginner Mistakes
![](keywords>bash shell scripting pitfalls traps beginners)
Here are some typical traps:
## Script execution
### Your perfect Bash script executes with syntax errors
If you write Bash scripts with Bash specific syntax and features, run
them with [Bash]{.underline}, and run them with Bash in [native
mode]{.underline}.
**Wrong**:
- no shebang
- the interpreter used depends on the OS implementation and
current shell
- **can** be run by calling bash with the script name as an
argument, e.g. `bash myscript`
- `#!/bin/sh` shebang
- depends on what `/bin/sh` actually is, for a Bash it means
compatiblity mode, **not** native mode
See also:
- [Bash startup mode: SH mode](/scripting/bashbehaviour#sh_mode)
- [Bash run mode: POSIX mode](/scripting/bashbehaviour#posix_run_mode)
### Your script named \"test\" doesn\'t execute
Give it another name. The executable `test` already exists.
In Bash it\'s a builtin. With other shells, it might be an executable
file. Either way, it\'s bad name choice!
Workaround: You can call it using the pathname:
/home/user/bin/test
## Globbing
### Brace expansion is not globbing
The following command line is not related to globbing (filename
expansion):
# YOU EXPECT
# -i1.vob -i2.vob -i3.vob ....
echo -i{*.vob,}
# YOU GET
# -i*.vob -i
**Why?** The brace expansion is simple text substitution. All possible
text formed by the prefix, the postfix and the braces themselves are
generated. In the example, these are only two: `-i*.vob` and `-i`. The
filename expansion happens **after** that, so there is a chance that
`-i*.vob` is expanded to a filename - if you have files like
`-ihello.vob`. But it definitely doesn\'t do what you expected.
Please see:
- [brace](/syntax/expansion/brace)
## Test-command
- `if [ $foo ] ...`
- `if [-d $dir] ...`
- \...
Please see:
- [The classic test command -
pitfalls](/commands/classictest#pitfalls_summarized)
## Variables
### Setting variables
#### The Dollar-Sign
There is no `$` (dollar-sign) when you reference the **name** of a
variable! Bash is not PHP!
# THIS IS WRONG!
$myvar="Hello world!"
A variable name preceeded with a dollar-sign always means that the
variable gets **expanded**. In the example above, it might expand to
nothing (because it wasn\'t set), effectively resulting in\...
="Hello world!"
\...which **definitely is wrong**!
When you need the **name** of a variable, you write **only the name**,
for example
- (as shown above) to set variables:
`picture=/usr/share/images/foo.png`
- to name variables to be used by the `read` builtin command:
`read picture`
- to name variables to be unset: `unset picture`
When you need the **content** of a variable, you prefix its name with
**a dollar-sign**, like
- echo \"The used picture is: \$picture\"
#### Whitespace
Putting spaces on either or both sides of the equal-sign (`=`) when
assigning a value to a variable **will** fail.
# INCORRECT 1
example = Hello
# INCORRECT 2
example= Hello
# INCORRECT 3
example =Hello
The only valid form is **no spaces between the variable name and
assigned value**:
# CORRECT 1
example=Hello
# CORRECT 2
example=" Hello"
### Expanding (using) variables
A typical beginner\'s trap is quoting.
As noted above, when you want to **expand** a variable i.e. \"get the
content\", the variable name needs to be prefixed with a dollar-sign.
But, since Bash knows various ways to quote and does word-splitting, the
result isn\'t always the same.
Let\'s define an example variable containing text with spaces:
example="Hello world"
Used form result number of words
-------------- --------------- -----------------
`$example` `Hello world` 2
`"$example"` `Hello world` 1
`\$example` `$example` 1
`'$example'` `$example` 1
If you use parameter expansion, you **must** use the **name** (`PATH`)
of the referenced variables/parameters. i.e. **not** (`$PATH`):
# WRONG!
echo "The first character of PATH is ${$PATH:0:1}"
# CORRECT
echo "The first character of PATH is ${PATH:0:1}"
Note that if you are using variables in [arithmetic
expressions](/syntax/arith_expr), then the bare **name** is allowed:
((a=$a+7)) # Add 7 to a
((a = a + 7)) # Add 7 to a. Identical to the previous command.
((a += 7)) # Add 7 to a. Identical to the previous command.
a=$((a+7)) # POSIX-compatible version of previous code.
Please see:
- [words](/syntax/words)
- [quoting](/syntax/quoting)
- [wordsplit](/syntax/expansion/wordsplit)
- [pe](/syntax/pe)
### Exporting
Exporting a variable means giving **newly created** (child-)processes a
copy of that variable. It does **not** copy a variable created in a
child process back to the parent process. The following example does
**not** work, since the variable `hello` is set in a child process (the
process you execute to start that script `./script.sh`):
$ cat script.sh
export hello=world
$ ./script.sh
$ echo $hello
$
Exporting is one-way. The direction is from parent process to child
process, not the reverse. The above example **will** work, when you
don\'t execute the script, but include (\"source\") it:
$ source ./script.sh
$ echo $hello
world
$
In this case, the export command is of no use.
Please see:
- [processtree](/scripting/processtree)
## Exit codes
### Reacting to exit codes
If you just want to react to an exit code, regardless of its specific
value, you **don\'t need** to use `$?` in a test command like this:
``` bash
grep ^root: /etc/passwd >/dev/null 2>&1
if [ $? -ne 0 ]; then
echo "root was not found - check the pub at the corner"
fi
```
This can be simplified to:
``` bash
if ! grep ^root: /etc/passwd >/dev/null 2>&1; then
echo "root was not found - check the pub at the corner"
fi
```
Or, simpler yet:
``` bash
grep ^root: /etc/passwd >/dev/null 2>&1 || echo "root was not found - check the pub at the corner"
```
If you need the specific value of `$?`, there\'s no other choice. But if
you need only a \"true/false\" exit indication, there\'s no need for
`$?`.
See also:
- [Exit codes](/scripting/basics#exit_codes)
### Output vs. Return Value
It\'s important to remember the different ways to run a child command,
and whether you want the output, the return value, or neither.
When you want to run a command (or a pipeline) and save (or print) the
**output**, whether as a string or an array, you use Bash\'s
`$(command)` syntax:
$(ls -l /tmp)
newvariable=$(printf "foo")
When you want to use the **return value** of a command, just use the
command, or add ( ) to run a command or pipeline in a subshell:
if grep someuser /etc/passwd ; then
# do something
fi
if ( w | grep someuser | grep sqlplus ) ; then
# someuser is logged in and running sqlplus
fi
Make sure you\'re using the form you intended:
# WRONG!
if $(grep ERROR /var/log/messages) ; then
# send alerts
fi
Please see:
- [intro](/syntax/ccmd/intro)
- [cmdsubst](/syntax/expansion/cmdsubst)
- [grouping_subshell](/syntax/ccmd/grouping_subshell)

204
docs/scripting/nonportable.md Normal file
View File

@ -0,0 +1,204 @@
# Portability talk
![](keywords>bash shell scripting portability POSIX portable)
The script programming language of BASH is based on the Bourne Shell
syntax, with some extensions and derivations.
If scripts need to be portable, some of the BASH-specific syntax
elements should be avoided. Others should be avoided for all scripts,
e.g. if there is a corresponding POSIX(r)-compatible syntax (see
[obsolete](/scripting/obsolete)).
Some syntax elements have a BASH-specific, and a portable[^1]) pendant.
In these cases the portable syntax should be preferred.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
construct portable equivalent Description Portability
------------------------------------------ --------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------
`source\ FILE` `. FILE` include a script file Bourne shell (bash, ksh, POSIX(r), zsh, \...)
`declare`\\ keyword `typeset` keyword define local variables (or variables with special attributes) ksh, zsh, \..., **not POSIX!**
`command\ <<<\ WORD` `command <<MARKER WORD MARKER` a here-string, a special form of the here-document, avoid it in portable scripts! POSIX(r)
`export VAR=VALUE` `VAR=VALUE export VAR` Though POSIX(r) allows it, some shells don\'t want the assignment and the exporting in one command POSIX(r), zsh, ksh, \...
`(( MATH ))` `: $(( MATH ))` POSIX(r) does\'t define an arithmetic compund command, many shells don\'t know it. Using the pseudo-command `:` and the arithmetic expansion `$(( ))` is a kind of workaround here. **Attention:** Not all shell support assignment like `$(( a = 1 + 1 ))`! Also see below for a probably more portable solution. all POSIX(r) compatible shells
`[[\ EXPRESSION\ ]]` `[ EXPRESSION ]`\ The Bashish test keyword is reserved by POSIX(r), but not defined. Use the old fashioned way with the `test` command. See [the classic test command](/commands/classictest) POSIX(r) and others
or\
`test EXPRESSION`
`COMMAND\ <\ <(\ ...INPUTCOMMANDS...\ )` `INPUTCOMMANDS\ >\ TEMPFILE COMMAND\ <\ TEMPFILE` Process substitution (here used with redirection); use the old fashioned way (tempfiles) POSIX(r) and others
`((echo X);(echo Y))` `( (echo X); (echo Y) )` Nested subshells (separate the inner `()` from the outer `()` by spaces, to not confuse the shell regarding arithmetic control operators) POSIX(r) and others
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## Portability rationale
Here is some assorted portability information. Take it as a small guide
to make your scripts a bit more portable. It\'s not complete (it never
will be!) and it\'s not very detailed (e.g. you won\'t find information
about how which shell technically forks off which subshell). It\'s just
an assorted small set of portability guidelines. *-Thebonsai*
FIXME UNIX shell gurus out there, please be patient with a newbie like
me and give comments and hints instead of flames.
### Environment (exported) variables
When a new value is assigned to an **existing environment variable**,
there are two possibilities:
The *new value* is seen by subsequent programs
- without any special action (e.g. Bash)
- only after an explicit export with `export VARIABLE` (e.g. Sun\'s
`/bin/sh`)
Since an extra `export` doesn\'t hurt, the safest and most portable way
is to always (re-)export a changed variable if you want it to be seen by
subsequent processes.
### Arithmetics
Bash has a special compound command to do arithmetic without expansion.
However, POSIX has no such command. In the table at the top, there\'s
the `: $((MATH))` construct mentioned as possible alternative. Regarding
the exit code, a 100% equivalent construct would be:
# Bash (or others) compound command
if ((MATH)); then
...
# portable equivalent command
if [ "$((MATH))" -ne 0 ]; then
...
Quotes around the arithmetic expansion `$((MATH))` should not be
necessary as per POSIX, but Bash and AT&T-KSH perform word-splitting on
aritrhmetic expansions, so the most portable is *with quotes*.
### echo command
The overall problem with `echo` is, that there are 2 (maybe more)
mainstream flavours around. The only case where you may safely use an
`echo` on all systems is: Echoing non-variable arguments that don\'t
start with a `-` (dash) and don\'t contain a `\` (backslash).
Why? (list of known behaviours)
- may or may not automatically interpret backslash escpape codes in
the strings
- may or may not automatically interpret switches (like `-n`)
- may or may not ignore \"end of options\" tag (`--`)
- `echo -n` and `echo -e` are neither portable nor standard (**even
within the same shell**, depending on the version or environment
variables or the build options, especially KSH93 and Bash)
For these, and possibly other, reasons, POSIX (SUS) standardized the
existance of [the `printf` command](/commands/builtin/printf).
### Parameter expansions
- `${var:x:x}` is KSH93/Bash specific
- `${var/../..}` and `${var//../..}` are KSH93/Bash specific
- `var=$*` and `var=$@` are not handled the same in all shells if the
first char of IFS is not \" \" (space). `var="$*"` should work
(except the Bourne shell always joins the expansions with space)
### Special variables
#### PWD
[PWD](/syntax/shellvars#PWD) is POSIX but not Bourne. Most shells are
*not POSIX* in that they don\'t ignore the value of the `PWD`
environment variable. Workaround to fix the value of `PWD` at the start
of your script:
pwd -P > dev/null
#### RANDOM
[RANDOM](/syntax/shellvars#RANDOM) is Bash/KSH/ZSH specific variable
that will give you a random number up to 32767 (2\^15-1). Among many
other available external options, you can use awk to generate a random
number. There are multiple implementations of awk and which version your
system uses will depend. Most modern systems will call \'gawk\' (i.e.
GNU awk) or \'nawk\'. \'oawk\' (i.e. Original/Old awk) does not have the
rand() or srand() functions, so is best avoided.
# 'gawk' can produce random numbers using srand(). In this example, 10 integers between 1 and 500:
randpm=$(gawk -v min=1 -v max=500 -v nNum=10 'BEGIN { srand(systime() + PROCINFO["pid"]); for (i = 0; i < nNum; ++i) {print int(min + rand() * (max - min)} }')
# 'nawk' and 'mawk' does the same, but needs a seed to be provided for its rand() function. In this example we use $(date)
randpm=$(mawk -v min=1 -v max=500 -v nNum=10 -v seed="$(date +%Y%M%d%H%M%S)" 'BEGIN { srand(seed); for (i = 0; i < nNum; ++i) {print int(min + rand() * (max - min)} }')
*Yes, I\'m not an `awk` expert, so please correct it, rather than
complaining about possible stupid code!*
# Well, seeing how this //is// BASH-hackers.org I kinda missed the bash way of doing the above ;-)
# print a number between 0 and 500 :-)
printf $(( 500 * RANDOM / 32767 ))
# Or print 30 random numbers between 0 and 10 ;)
X=0; while (( X++ < 30 )); do echo $(( 10 * RANDOM / 32767 )); done
#### SECONDS
[SECONDS](/syntax/shellvars#SECONDS) is KSH/ZSH/Bash specific. Avoid it.
Find another method.
### Check for a command in PATH
The [PATH](/syntax/shellvars#PATH) variable is a colon-delimited list of
directory names, so it\'s basically possible to run a loop and check
every `PATH` component for the command you\'re looking for and for
executability.
However, this method doesn\'t look nice. There are other ways of doing
this, using commands that are *not directly* related to this task.
#### hash
The `hash` command is used to make the shell store the full pathname of
a command in a lookup-table (to avoid re-scanning the `PATH` on every
command execution attempt). Since it has to do a `PATH` search, it can
be used for this check.
For example, to check if the command `ls` is available in a location
accessible by `PATH`:
if hash ls >/dev/null 2>&1; then
echo "ls is available"
fi
Somewhat of a mass-check:
for name in ls grep sed awk; do
if ! hash "$name" >/dev/null 2>&1; then
echo "FAIL: Missing command '$name'"
exit 1
fi
done
Here (bash 3), `hash` also respects builtin commands. I don\'t know if
this works everywhere, but it seems logical.
#### command
The `command` command is used to explicitly call an external command,
rather than a builtin with the same name. For exactly this reason, it
has to do a `PATH` search, and can be used for this check.
For example, to check if the command `sed` is available in a location
accessible by `PATH`:
if command -v sed >/dev/null 2>&1; then
echo "sed is available"
fi
[^1]: \"portable\" doesn\'t necessarily mean it\'s POSIX, it can also
mean it\'s \"widely used and accepted\", and thus maybe more
portable than POSIX(r

80
docs/scripting/obsolete.md Normal file
View File

@ -0,0 +1,80 @@
# Obsolete and deprecated syntax
![](keywords>bash shell scripting obsolete deprecated outdated)
This (incomplete) page describes some syntax and commands considered
obsolete by some measure. A thorough discussion of the rationale is
beyond the scope of this page. See the [portability
page](/scripting/nonportable) for a discussion on portability issues.
This first table lists syntax that is tolerated by Bash but has few if
any legitimate uses. These features exist mostly for Bourne, csh, or
some other backward compatibility with obsolete shells, or were
Bash-specific features considered failed experiments and deprecated or
replaced with a better alternative. These should be irrelevant to most
everyone except maybe code golfers. New scripts should never use them.
None of the items on this list are specified by the most current version
of POSIX, and some may be incompatible with POSIX.
Syntax Replacement Description
---------------------------------- --------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
`&>FILE` and `>&FILE` `>FILE 2>&1` This redirection syntax is short for `>FILE 2>&1` and originates in the C Shell. The latter form is especially uncommon and should never be used, and the explicit form using separate redirections is preferred over both. These shortcuts contribute to confusion about the copy descriptor because the syntax is unclear. They also introduce parsing ambiguity, and conflict with POSIX. Shells without this feature treat `cmd1 &>file cmd2` as: \"background `cmd1` and then execute `cmd2` with its stdout redirected to `file`\", which is the correct interpretation of this expression. See: [redirection](/syntax/redirection) \`\$ { bash; dash \</dev/fd/0; } \<\<\<\'echo foo\>/dev/null&\>/dev/fd/2 echo bar\' foo echo bar bar\`
`$[EXPRESSION]` `$((EXPRESSION))` This undocumented syntax is completely replaced by the POSIX-conforming arithmetic expansion `$((EXPRESSION))`. It is unimplemented almost everywhere except Bash and Zsh. See [arithmetic expansion](/syntax/expansion/arith). [Some discussion](http://lists.gnu.org/archive/html/bug-bash/2012-04/msg00034.html).
`COMMAND\ |&\ COMMAND` `COMMAND 2>&1 | COMMAND` This is an alternate pipeline operator derived from Zsh. Officially, it is not considered deprecated by Bash, but I highly discourage it. It conflicts with the list operator used for [coprocess](/syntax/keywords/coproc) creation in most Korn shells. It also has confusing behavior. The stdout is redirected first like an ordinary pipe, while the stderr is actually redirected last \-- after other redirects preceding the pipe operator. Overall, it\'s pointless syntax bloat. Use an explicit redirect instead.
`function\ NAME()\ COMPOUND-CMD` `NAME()\ COMPOUND-CMD` or `function\ NAME\ {\ CMDS;\ }` This is an amalgamation between the Korn and POSIX style function definitions - using both the `function` keyword and parentheses. It has no useful purpose and no historical basis or reason to exist. It is not specified by POSIX. It is accepted by Bash, mksh, zsh, and perhaps some other Korn shells, where it is treated as identical to the POSIX-style function. It is not accepted by AT&T ksh. It should never be used. See the next table for the `function` keyword. Bash doesn\'t have this feature documented as expressly deprecated.
`for x; { ...;}` `do`, `done`, `in`, `esac`, etc. This undocumented syntax replaces the `do` and `done` reserved words with braces. Many Korn shells support various permutations on this syntax for certain compound commands like `for`, `case`, and `while`. Which ones and certain details like whether a newline or semicolon are required vary. Only `for` works in Bash. Needless to say, don\'t use it.
This table lists syntax that is specified by POSIX (unless otherwise
specified below), but has been superseded by superior alternatives
(either in POSIX, Bash, or both), or is highly discouraged for other
reasons such as encouraging bad practices or dangerous code. Those that
are specified by POSIX may be badly designed and unchangeable for
historical reasons.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Syntax Replacement Description
----------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Unquoted expansions, [wordsplit](/syntax/expansion/wordsplit), and [globs](/syntax/expansion/globs) [Proper quoting](http://mywiki.wooledge.org/Quotes), Ksh/Bash-style [arrays](/syntax/arrays), The \"\$@\" expansion, [read](/commands/builtin/read) *Quoting errors* are a broad category of common mistakes brought about by a few unintuitive features carried over from the Bourne shell due to complaints of broken scripts and changes in previously documented behavior. Most of the important expansions are performed at the same time from left to right. However, a few expansions, most notably word-splitting and globbing, and in shells other than Bash, [brace expansion](/syntax/expansion/brace), are performed **on the results of previous expansions, by default, unless they are quoted.** This means that the act of expanding an unquoted variable in an ordinary argument context, depending on the value of the variable, can yield different results depending on possibly uncontrolled side-effects like the value of `IFS`, and the names of files in the current working directory. You can\'t get globbing without word-splitting, or vice versa (without `set -f`). [You can\'t store a command or character-delimited list in a variable and safely evaluate it with unquoted expansion](http://mywiki.wooledge.org/BashFAQ/050). If possible, always choose a shell that supports Korn shell arrays such as Bash. They are a vital but non-standard feature for writing clean, safe scripts. Well-written scripts don\'t use word-splitting. A few exceptions are listed on the [word splitting page](/syntax/expansion/wordsplit). A significant proportion of the issues on the famous [Pitfalls list](http://mywiki.wooledge.org/BashPitfalls) fall under this category. See also: *[Don\'t read lines with for!](http://mywiki.wooledge.org/DontReadLinesWithFor)*
`` `COMMANDS` `` `$(COMMANDS)` This is the older Bourne-compatible form of the [command substitution](/syntax/expansion/cmdsubst). Both the `` `COMMANDS` `` and `$(COMMANDS)` syntaxes are specified by POSIX, but the latter is [greatly]{.underline} preferred, though the former is unfortunately still very prevalent in scripts. New-style command substitutions are widely implemented by every modern shell (and then some). The only reason for using backticks is for compatibility with a real Bourne shell (like Heirloom). Backtick command substitutions require special escaping when nested, and examples found in the wild are improperly quoted more often than not. See: *[Why is \$(\...) preferred over \`\...\` (backticks)?](http://mywiki.wooledge.org/BashFAQ/082)*.
`[\ EXPRESSION\ ]`\\ and\\ `test\ EXPRESSION` `[[\ EXPRESSION\ ]]` `test` and `[` are the Bourne/POSIX commands for evaluating test expressions (they are almost identical, and `[` is somewhat more common). The expressions consist of regular arguments, unlike the Ksh/Bash `[[` command. While the issue is analogous to `let` vs `((`, the advantages of `[[` vs `[` are even more important because the arguments/expansions aren\'t just concatenated into one expression. With the classic `[` command, the number of arguments is significant. If at all possible, use the [conditional expression](/syntax/ccmd/conditional_expression) (\"new test command\") `[[ EXPRESSION ]]`. Unless there is a need for POSIX compatibility, there are only a few reasons to use `[`. `[[` is one of the most portable and consistent non-POSIX ksh extensions available. See: [conditional_expression](/syntax/ccmd/conditional_expression) and *[What is the difference between test, \[ and \[\[ ?](http://mywiki.wooledge.org/BashFAQ/031)*
`set -e`, `set -o errexit`\ proper control flow and error handling `set -e` causes untested non-zero exit statuses to be fatal. It is a debugging feature intended for use only during development and should not be used in production code, especially init scripts and other high-availability scripts. Do not be tempted to think of this as \"error handling\"; it\'s not, it\'s just a way to find the place you\'ve *forgotten* to put error handling.\
and the `ERR` trap Think of it as akin to `use strict` in Perl or `throws` in C++: tough love that makes you write better code. Many guides recommend avoiding it entirely because of the apparently-complex rules for when non-zero statuses cause the script to abort. Conversely, large software projects with experienced coders may recommend or even mandate its use.\
Because it provides no notification of the location of the error, it\'s more useful combined with `set -x` or the `DEBUG` trap and other Bash debug features, and both flags are normally better set on the command line rather than within the script itself.\
Most of this also applies to the `ERR` trap, though I\'ve seen it used in a few places in shells that lack `pipefail` or `PIPESTATUS`. The `ERR` trap is not POSIX, but `set -e` is. `failglob` is another Bash feature that falls into this category (mainly useful for debugging).\
**The `set -e` feature generates more questions and false bug reports on the Bash mailing list than all other features combined!** Please do not rely on `set -e` for logic in scripts. If you still refuse to take this advice, make sure you understand **exactly** how it works. See: *[Why doesn\'t set -e (or set -o errexit, or trap ERR) do what I expected?](http://mywiki.wooledge.org/BashFAQ/105)* and <http://www.fvue.nl/wiki/Bash:_Error_handling>
`set -u` or `set -o nounset` Proper control flow and error handling `set -u` causes attempts to expand unset variables or parameters as fatal errors. Like `set -e`, it bypasses control flow and exits immediately from the current shell environment. Like non-zero statuses, unset variables are a normal part of most non-trivial shell scripts. Living with `set -u` requires hacks like `${1+"$1"}` for each expansion that might possibly be unset. Only very current shells guarantee that expanding `@` or `*` won\'t trigger an error when no parameters are set (<http://austingroupbugs.net/view.php?id=155>, <http://www.in-ulm.de/~mascheck/various/bourne_args/>). Apparently some find it useful for debugging. See *[How do I determine whether a variable is already defined? Or a function?](http://mywiki.wooledge.org/BashFAQ/083)* for how to properly test for defined variables. Don\'t use `set -u`.
`${var?msg}` or `${var:?msg}` Proper control flow and error handling Like `set -u`, this expansion causes a fatal error which immediately exits the current shell environment if the given parameter is unset or is null. It prints the error message given, to the right of the operator. If a value is expected and you\'d like to create an assertion or cause errors, it\'s better to test for undefined variables using one of [these techniques](http://mywiki.wooledge.org/BashFAQ/083) and handle the error manually, or call a `die` function. This expansion is defined by POSIX. It\'s better than `set -u`, because it\'s explicit, but not by much. It also allows you to accidentally construct hilariously deceptive error messages: `bash -c 'f() { definitely_not_printf "${printf:?"$1" - No such option}"; }; f -v'
bash: printf: -v - No such option`
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
This table lists features that are used only if you have a specific
reason to prefer it over another alternative. These have some legitimate
uses if you know what you\'re doing, such as for those with specific
portability requirements, or in order to make use of some subtle
behavioral differences. These are frequently (mis)used for no reason.
Writing portable scripts that go outside of POSIX features requires
knowing how to account for many (often undocumented) differences across
many shells. If you do happen to know what you\'re doing, don\'t be too
surprised if you run across someone telling you not to use these.
Syntax Replacement Description
------------------------------- --------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
`function\ NAME\ {\ CMDS;\ }` `NAME()\ COMPOUND-CMD` This is the ksh form of function definition created to extend the Bourne and POSIX form with modified behaviors and additional features like local variables. The idea was for new-style functions to be analogous to regular builtins with their own environment and scope, while POSIX-style functions are more like special builtins. `function` is supported by almost every ksh-derived shell including Bash and Zsh, but isn\'t specified by POSIX. Bash treats all function styles the same, but this is unusual. `function` has some preferable characteristics in many ksh variants, making it more portable for scripts that use non-POSIX extensions by some measures. If you\'re going to use the `function` keyword, it implies that you\'re either targeting Ksh specifically, or that you have detailed knowledge of how to compensate for differences across shells. It should always be used consistently with `typeset`, but never used with `declare` or `local`. Also in ksh93, the braces are not a [command group](/syntax/ccmd/grouping_plain), but a required part of the syntax (unlike Bash and others). See [shell function definitions](/syntax/basicgrammar#shell_function_definitions)
`typeset` `declare`, `local`, `export`, `readonly` This is closely related to the above, and should often be used together. `typeset` exists primarily for `ksh` compatibility, but is marked as \"deprecated\" in Bash (though I don\'t entirely agree with this). This makes some sense, because future compatibility can\'t be guaranteed, and any compatibility at all, requires understanding the non-POSIX features of other shells and their differences. Using `declare` instead of `typeset` emphasizes your intention to be \"Bash-only\", and definitely breaks everywhere else (except possibly zsh if you\'re lucky). The issue is further complicated by Dash and the [Debian policy](http://www.debian.org/doc/debian-policy/ch-files.html#s-scripts) requirement for a `local` builtin, which is itself not entirely compatible with Bash and other shells.
\'\'let \'EXPR\' \'\' `((EXPR))` or `[\ $((EXPR))\ -ne\ 0 ]` `let` is the \"simple command\" variant of arithmetic evaluation command, which takes regular arguments. Both `let` and `((expr))` were present in ksh88, and everything that supports one should support the other. Neither are POSIX. The compound variant is preferable because it doesn\'t take regular arguments for [wordsplitting](/syntax/expansion/wordsplit) and [globbing](/syntax/expansion/globs), which makes it safer and clearer. It is also usually faster, especially in Bash, where compound commands are typically significantly faster. Some of the (few) reasons for using `let` are detailed on the [let](/commands/builtin/let) page. See [arithmetic evaluation compound command](/syntax/ccmd/arithmetic_eval)
`eval` Depends. Often code can be restructured to use better alternatives. `eval` is thrown in here for good measure, as sadly it is so often misused that any use of `eval` (even the rare clever one) is immediately dismissed as wrong by experts, and among the most immediate solutions abused by beginners. In reality, there are correct ways to use `eval`, and even cases in which it\'s necessary, even in sophisticated shells like Bash and Ksh. `eval` is unusual in that it is less frequently appropriate in more feature-rich shells than in more minimal shells like Dash, where it is used to compensate for more limitations. If you find yourself needing `eval` too frequently, it might be a sign that you\'re either better off using a different language entirely, or trying to borrow an idiom from some other paradigm that isn\'t well suited to the shell language. By the same token, there are some cases in which working too hard to avoid `eval` ends up adding a lot of complexity and sacrificing all portability. Don\'t substitute a clever `eval` for something that\'s a bit \"too clever\", just to avoid the `eval`, yet, take reasonable measures to avoid it where it is sensible to do so. See: [eval](/commands/builtin/eval) and [Eval command and security issues](http://mywiki.wooledge.org/BashFAQ/048).
## See also
- [Non-portable syntax and command uses](/scripting/nonportable)
- [bashchanges](/scripting/bashchanges)
- [Greg\'s BashFAQ 061: List of essential features added (with the
Bash version tag)](BashFAQ>061)
- [Bash \<-\> POSIX Portability guide with a focus on
Dash](http://mywiki.wooledge.org/Bashism)
- <http://mywiki.wooledge.org/BashPitfalls>

397
docs/scripting/posparams.md Normal file
View File

@ -0,0 +1,397 @@
# Handling positional parameters
![](keywords>bash shell scripting arguments positional parameters options)
## Intro
The day will come when you want to give arguments to your scripts. These
arguments are known as **positional parameters**. Some relevant special
parameters are described below:
Parameter(s) Description
------------------ ------------------------------------------------------------------------------------------------------------------------------------
`$0` the first positional parameter, equivalent to `argv[0]` in C, see [the first argument](/scripting/posparams#the_first_argument)
`$FUNCNAME` the function name ([**attention**]{.underline}: inside a function, `$0` is still the `$0` of the shell, **not** the function name)
`$1 ... $9` the argument list elements from 1 to 9
`${10} ... ${N}` the argument list elements beyond 9 (note the [parameter expansion](/syntax/pe) syntax!)
`$*` all positional parameters except `$0`, see [mass usage](/scripting/posparams#mass_usage)
`$@` all positional parameters except `$0`, see [mass usage](/scripting/posparams#mass_usage)
`$#` the number of arguments, not counting `$0`
These positional parameters reflect exactly what was given to the script
when it was called.
Option-switch parsing (e.g. `-h` for displaying help) is not performed
at this point.
See also [the dictionary entry for
\"parameter\"](/dict/terms/parameter).
## The first argument
The very first argument you can access is referenced as `$0`. It is
usually set to the script\'s name exactly as called, and it\'s set on
shell initialization:
[Testscript]{.underline} - it just echos `$0`:
#!/bin/bash
echo "$0"
You see, `$0` is always set to the name the script is called with (`>`
is the prompt\...):
> ./testscript
./testscript
> /usr/bin/testscript
/usr/bin/testscript
However, this isn\'t true for login shells:
> echo "$0"
-bash
In other terms, `$0` is not a positional parameter, it\'s a special
parameter independent from the positional parameter list. It can be set
to anything. In the **ideal** case it\'s the pathname of the script, but
since this gets set on invocation, the invoking program can easily
influence it (the `login` program does that for login shells, by
prefixing a dash, for example).
Inside a function, `$0` still behaves as described above. To get the
function name, use `$FUNCNAME`.
## Shifting
The builtin command `shift` is used to change the positional parameter
values:
- `$1` will be discarded
- `$2` will become `$1`
- `$3` will become `$2`
- \...
- in general: `$N` will become `$N-1`
The command can take a number as argument: Number of positions to shift.
e.g. `shift 4` shifts `$5` to `$1`.
## Using them
Enough theory, you want to access your script-arguments. Well, here we
go.
### One by one
One way is to access specific parameters:
#!/bin/bash
echo "Total number of arguments: $#"
echo "Argument 1: $1"
echo "Argument 2: $2"
echo "Argument 3: $3"
echo "Argument 4: $4"
echo "Argument 5: $5"
While useful in another situation, this way is lacks flexibility. The
maximum number of arguments is a fixedvalue - which is a bad idea if you
write a script that takes many filenames as arguments.
=\> forget that one
### Loops
There are several ways to loop through the positional parameters.
------------------------------------------------------------------------
You can code a [C-style for-loop](/syntax/ccmd/c_for) using `$#` as the
end value. On every iteration, the `shift`-command is used to shift the
argument list:
numargs=$#
for ((i=1 ; i <= numargs ; i++))
do
echo "$1"
shift
done
Not very stylish, but usable. The `numargs` variable is used to store
the initial value of `$#` because the shift command will change it as
the script runs.
------------------------------------------------------------------------
Another way to iterate one argument at a time is the `for` loop without
a given wordlist. The loop uses the positional parameters as a wordlist:
for arg
do
echo "$arg"
done
[Advantage:]{.underline} The positional parameters will be preserved
------------------------------------------------------------------------
The next method is similar to the first example (the `for` loop), but it
doesn\'t test for reaching `$#`. It shifts and checks if `$1` still
expands to something, using the [test command](/commands/classictest):
while [ "$1" ]
do
echo "$1"
shift
done
Looks nice, but has the disadvantage of stopping when `$1` is empty
(null-string). Let\'s modify it to run as long as `$1` is defined (but
may be null), using [parameter expansion for an alternate
value](/syntax/pe#use_an_alternate_value):
while [ "${1+defined}" ]; do
echo "$1"
shift
done
### Getopts
There is a [small tutorial dedicated to
`getopts`](/howto/getopts_tutorial) (*under construction*).
## Mass usage
### All Positional Parameters
Sometimes it\'s necessary to just \"relay\" or \"pass\" given arguments
to another program. It\'s very inefficient to do that in one of these
loops, as you will destroy integrity, most likely (spaces!).
The shell developers created `$*` and `$@` for this purpose.
As overview:
Syntax Effective result
-------- -----------------------------
`$*` `$1 $2 $3 ... ${N}`
`$@` `$1 $2 $3 ... ${N}`
`"$*"` `"$1c$2c$3c...c${N}"`
`"$@"` `"$1" "$2" "$3" ... "${N}"`
Without being quoted (double quotes), both have the same effect: All
positional parameters from `$1` to the last one used are expanded
without any special handling.
When the `$*` special parameter is double quoted, it expands to the
equivalent of: `"$1c$2c$3c$4c........$N"`, where \'c\' is the first
character of `IFS`.
But when the `$@` special parameter is used inside double quotes, it
expands to the equivanent of\...
`"$1" "$2" "$3" "$4" ..... "$N"`
\...which **reflects all positional parameters as they were set
initially** and passed to the script or function. If you want to re-use
your positional parameters to **call another program** (for example in a
wrapper-script), then this is the choice for you, use double quoted
`"$@"`.
Well, let\'s just say: **You almost always want a quoted `"$@"`!**
### Range Of Positional Parameters
Another way to mass expand the positional parameters is similar to what
is possible for a range of characters using [substring
expansion](/syntax/pe#substring_expansion) on normal parameters and the
mass expansion range of [arrays](/syntax/arrays).
`${@:START:COUNT}`
`${*:START:COUNT}`
`"${@:START:COUNT}"`
`"${*:START:COUNT}"`
The rules for using `@` or `*` and quoting are the same as above. This
will expand `COUNT` number of positional parameters beginning at
`START`. `COUNT` can be omitted (`${@:START}`), in which case, all
positional parameters beginning at `START` are expanded.
If `START` is negative, the positional parameters are numbered in
reverse starting with the last one.
`COUNT` may not be negative, i.e. the element count may not be
decremented.
[**Example:**]{.underline} START at the last positional parameter:
echo "${@: -1}"
[**Attention**]{.underline}: As of Bash 4, a `START` of `0` includes the
special parameter `$0`, i.e. the shell name or whatever \$0 is set to,
when the positional parameters are in use. A `START` of `1` begins at
`$1`. In Bash 3 and older, both `0` and `1` began at `$1`.
## Setting Positional Parameters
Setting positional parameters with command line arguments, is not the
only way to set them. The [builtin command, set](/commands/builtin/set)
may be used to \"artificially\" change the positional parameters from
inside the script or function:
set "This is" my new "set of" positional parameters
# RESULTS IN
# $1: This is
# $2: my
# $3: new
# $4: set of
# $5: positional
# $6: parameters
It\'s wise to signal \"end of options\" when setting positional
parameters this way. If not, the dashes might be interpreted as an
option switch by `set` itself:
# both ways work, but behave differently. See the article about the set command!
set -- ...
set - ...
Alternately this will also preserve any verbose (-v) or tracing (-x)
flags, which may otherwise be reset by `set`
set -$- ...
FIXME continue
## Production examples
### Using a while loop
To make your program accept options as standard command syntax:
`COMMAND [options] <params>` \# Like \'cat -A file.txt\'
See simple option parsing code below. It\'s not that flexible. It
doesn\'t auto-interpret combined options (-fu USER) but it works and is
a good rudimentary way to parse your arguments.
#!/bin/sh
# Keeping options in alphabetical order makes it easy to add more.
while :
do
case "$1" in
-f | --file)
file="$2" # You may want to check validity of $2
shift 2
;;
-h | --help)
display_help # Call your function
# no shifting needed here, we're done.
exit 0
;;
-u | --user)
username="$2" # You may want to check validity of $2
shift 2
;;
-v | --verbose)
# It's better to assign a string, than a number like "verbose=1"
# because if you're debugging the script with "bash -x" code like this:
#
# if [ "$verbose" ] ...
#
# You will see:
#
# if [ "verbose" ] ...
#
# Instead of cryptic
#
# if [ "1" ] ...
#
verbose="verbose"
shift
;;
--) # End of all options
shift
break;
-*)
echo "Error: Unknown option: $1" >&2
exit 1
;;
*) # No more options
break
;;
esac
done
# End of file
### Filter unwanted options with a wrapper script
This simple wrapper enables filtering unwanted options (here: `-a` and
`--all` for `ls`) out of the command line. It reads the positional
parameters and builds a filtered array consisting of them, then calls
`ls` with the new option set. It also respects the `--` as \"end of
options\" for `ls` and doesn\'t change anything after it:
#!/bin/bash
# simple ls(1) wrapper that doesn't allow the -a option
options=() # the buffer array for the parameters
eoo=0 # end of options reached
while [[ $1 ]]
do
if ! ((eoo)); then
case "$1" in
-a)
shift
;;
--all)
shift
;;
-[^-]*a*|-a?*)
options+=("${1//a}")
shift
;;
--)
eoo=1
options+=("$1")
shift
;;
*)
options+=("$1")
shift
;;
esac
else
options+=("$1")
# Another (worse) way of doing the same thing:
# options=("${options[@]}" "$1")
shift
fi
done
/bin/ls "${options[@]}"
### Using getopts
There is a [small tutorial dedicated to
`getopts`](/howto/getopts_tutorial) (*under construction*).
## See also
- Internal: [getopts_tutorial](/howto/getopts_tutorial)
- Internal: [while_loop](/syntax/ccmd/while_loop)
- Internal: [c_for](/syntax/ccmd/c_for)
- Internal: [arrays](/syntax/arrays) (for equivalent syntax for
mass-expansion)
- Internal: [Substring expansion on a
parameter](/syntax/pe#substring_expansion) (for equivalent syntax
for mass-expansion)
- Dictionary, internal: [parameter](/dict/terms/parameter)

179
docs/scripting/processtree.md Normal file
View File

@ -0,0 +1,179 @@
# Bash and the process tree
![](keywords>bash shell scripting processes pipes variables environment)
## The process tree
The processes in UNIX(r) are - unlike other systems - **organized as a
tree**. Every process has a parent process that started, or is
responsible, for it. Every process has its own **context memory** (Not
the memory where the process stores its data, rather, the memory where
data is stored that doesn\'t directly belong to the process, but is
needed to run the process) i.e. [**The environment**]{.underline}.
Every process has its **own** environment space.
The environment stores, among other things, data that\'s useful to us,
the **environment variables**. These are strings in common `NAME=VALUE`
form, but they are not related to shell variables. A variable named
`LANG`, for example, is used by every program that looks it up in its
environment to determinate the current locale.
**[Attention:]{.underline}** A variable that is set, like with
`MYVAR=Hello`, is **not** automatically part of the environment. You
need to put it into the environment with the bash builtin command
`export`:
export MYVAR
Common system variables like [PATH](/syntax/shellvars#PATH) or
[HOME](/syntax/shellvars#HOME) are usually part of the environment (as
set by login scripts or programs).
## Executing programs
All the diagrams of the process tree use names like \"`xterm`\" or
\"`bash`\", but that\'s just to make it easier to understand what\'s
going on, it doesn\'t mean those processes are actually executed.
Let\'s take a short look at what happens when you \"execute a program\"
from the Bash prompt, a program like \"ls\":
$ ls
Bash will now perform **two steps**:
- It will make a copy of itself
- The copy will replace itself with the \"ls\" program
The copy of Bash will inherit the environment from the \"main Bash\"
process: All environment variables will also be copied to the new
process. This step is called **forking**.
For a short moment, you have a process tree that might look like
this\...
xterm ----- bash ----- bash(copy)
\...and after the \"second Bash\" (the copy) replaces itself with the
`ls` program (the copy execs it), it might look like
xterm ----- bash ----- ls
If everything was okay, the two steps resulted in one program being run.
The copy of the environment from the first step (forking) becomes the
environment for the final running program (in this case, `ls`).
[**What is so important about it?**]{.underline} In our example, what
the program `ls` does inside its own environment, it can\'t affect the
environment of its parent process (in this case, `bash`). The
environment was copied when ls was executed. Nothing is \"copied back\"
to the parent environment when `ls` terminates.
## Bash playing with pipes
Pipes are a very powerful tool. You can connect the output of one
process to the input of another process. We won\'t delve into piping at
this point, we just want to see how it looks in the process tree. Again,
we execute some commands, this time, we\'ll run `ls` and `grep`:
$ ls | grep myfile
It results in a tree like this:
+-- ls
xterm ----- bash --|
+-- grep
Note once again, `ls` can\'t influence the `grep` environment, `grep`
can\'t influence the `ls` environment, and neither `grep` nor `ls` can
influence the `bash` environment.
[**How is that related to shell programming?!?**]{.underline}
Well, imagine some Bash code that reads data from a pipe. For example,
the internal command `read`, which reads data from *stdin* and puts it
into a variable. We run it in a loop here to count input lines:
counter=0
cat /etc/passwd | while read; do ((counter++)); done
echo "Lines: $counter"
What? It\'s 0? Yes! The number of lines might not be 0, but the variable
`$counter` still is 0. Why? Remember the diagram from above? Rewriting
it a bit, we have:
+-- cat /etc/passwd
xterm ----- bash --|
+-- bash (while read; do ((counter++)); done)
See the relationship? The forked Bash process will count the lines like
a charm. It will also set the variable `counter` as directed. But if
everything ends, this extra process will be terminated - **your
\"counter\" variable is gone.** You see a 0 because in the main shell it
was 0, and wasn\'t changed by the child process!
[**So, how do we count the lines?**]{.underline} Easy: **Avoid the
subshell.** The details don\'t matter, the important thing is the shell
that sets the counter must be the \"main shell\". For example:
counter=0
while read; do ((counter++)); done </etc/passwd
echo "Lines: $counter"
It\'s nearly self-explanatory. The `while` loop runs in the **current
shell**, the counter is incremented in the **current shell**, everything
vital happens in the **current shell**, also the `read` command sets the
variable `REPLY` (the default if nothing is given), though we don\'t use
it here.
## Actions that create a subshell
Bash creates **subshells** or **subprocesses** on various actions it
performs:
### Executing commands
As shown above, Bash will create subprocesses everytime it executes
commands. That\'s nothing new.
But if your command is a subprocess that sets variables you want to use
in your main script, that won\'t work.
For exactly this purpose, there\'s the `source` command (also: the *dot*
`.` command). Source doesn\'t execute the script, it imports the other
script\'s code into the current shell:
source ./myvariables.sh
# equivalent to:
. ./myvariables.sh
### Pipes
The last big section was about pipes, so no example here.
### Explicit subshell
If you group commands by enclosing them in parentheses, these commands
are run inside a subshell:
(echo PASSWD follows; cat /etc/passwd; echo GROUP follows; cat /etc/group) >output.txt
### Command substitution
With [command substitution](/syntax/expansion/cmdsubst) you re-use the
output of another command as text in your command line, for example to
set a variable. The other command is run in a subshell:
number_of_users=$(cat /etc/passwd | wc -l)
Note that, in this example, a second subshell was created by using a
pipe in the command substitution:
+-- cat /etc/passwd
xterm ----- bash ----- bash (cmd. subst.) --|
+-- wc -l
FIXME to be continued

409
docs/scripting/style.md Normal file
View File

@ -0,0 +1,409 @@
# Scripting with style
FIXME continue
These are some coding guidelines that helped me to read and understand
my own code over the years. They also will help to produce code that
will be a bit more robust than \"if something breaks, I know how to fix
it\".
This is not a bible, of course. But I have seen so much ugly and
terrible code (not only in shell) during all the years, that I\'m 100%
convinced there needs to be *some* code layout and style. No matter
which one you use, use it throughout your code (at least don\'t change
it within the same shellscript file); don\'t change your code layout
with your mood.
Some good code layout helps you to read your own code after a while. And
of course it helps others to read the code.
## Indentation guidelines
Indentation is nothing that technically influences a script, it\'s only
for us humans.
I\'m used to seeing/using indentation of *two space characters* (though
many may prefer 4 spaces, see below in the discussion section):
- it\'s easy and fast to type
- it\'s not a hard-tab that\'s displayed differently in different
environments
- it\'s wide enough to give a visual break and small enough to not
waste too much space on the line
Speaking of hard-tabs: Avoid them if possible. They only make trouble. I
can imagine one case where they\'re useful: Indenting
[here-documents](/syntax/redirection#here_documents).
### Breaking up lines
Whenever you need to break lines of long code, you should follow one of
these two rules:
[**Indention using command width:**]{.underline}
activate some_very_long_option \
some_other_option
[**Indention using two spaces:**]{.underline}
activate some_very_long_option \
some_other_option
Personally, with some exceptions, I prefer the first form because it
supports the visual impression of \"these belong together\".
### Breaking compound commands
[Compound commands](/syntax/ccmd/intro) form the structures that make a
shell script different from a stupid enumeration of commands. Usually
they contain a kind of \"head\" and a \"body\" that contains command
lists. This type of compound command is relatively easy to indent.
I\'m used to (not all points apply to all compound commands, just pick
the basic idea):
- put the introducing keyword and the initial command list or
parameters on one line (\"head\")
- put the \"body-introducing\" keyword on the same line
- the command list of the \"body\" on separate lines, indented by two
spaces
- put the closing keyword on a separated line, indented like the
initial introducing keyword
What?! Well, here again:
##### Symbolic
HEAD_KEYWORD parameters; BODY_BEGIN
BODY_COMMANDS
BODY_END
##### if/then/elif/else
This construct is a bit special, because it has keywords (`elif`,
`else`) \"in the middle\". The visually appealing way is to indent them
like this:
if ...; then
...
elif ...; then
...
else
...
fi
##### for
for f in /etc/*; do
...
done
##### while/until
while [[ $answer != [YyNn] ]]; do
...
done
##### The case construct
The `case` construct might need a bit more discussion here, since its
structure is a bit more complex.
In general, every new \"layer\" gets a new indentation level:
case $input in
hello)
echo "You said hello"
;;
bye)
echo "You said bye"
if foo; then
bar
fi
;;
*)
echo "You said something weird..."
;;
esac
Some notes:
- if not 100% needed, the optional left parenthesis on the pattern is
not used
- the patterns (`hello)`) and the corresponding action terminator
(`;;`) are indented at the same level
- the action command lists are indented one more level (and continue
to have their own indentation, if needed)
- though optional, the very last action terminator is given
## Syntax and coding guidelines
### Cryptic constructs
Cryptic constructs, we all know them, we all love them. If they are not
100% needed, avoid them, since nobody except you may be able to decipher
them.
It\'s - just like in C - the middle ground between smart, efficient and
readable.
If you need to use a cryptic construct, include a comment that explains
what your \"monster\" does.
### Variable names
Since all reserved variables are `UPPERCASE`, the safest way is to only
use `lowercase` variable names. This is true for reading user input,
loop counting variables, etc., \... (in the example: `file`)
- prefer `lowercase` variables
- if you use `UPPERCASE` names, **do not use reserved variable names**
(see
[SUS](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08)
for an incomplete list)
- if you use `UPPERCASE` names, prepend the name with a unique prefix
(`MY_` in the example below)
```{=html}
<!-- -->
```
#!/bin/bash
# the prefix 'MY_'
MY_LOG_DIRECTORY=/var/adm/
for file in "$MY_LOG_DIRECTORY"/*; do
echo "Found Logfile: $file"
done
### Variable initialization
As in C, it\'s always a good idea to initialize your variables, though,
the shell will initialize fresh variables itself (better: Unset
variables will generally behave like variables containing a null
string).
It\'s no problem to pass an **environment variable** to the script. If
you blindly assume that all variables you use for the first time are
**empty**, anybody can **inject** content into a variable by passing it
via the environment.
The solution is simple and effective: **Initialize them**
my_input=""
my_array=()
my_number=0
If you do that for every variable you use, then you also have some
in-code documentation for them.
### Parameter expansion
Unless you are really sure what you\'re doing, **quote every parameter
expansion**.
There are some cases where this isn\'t needed from a technical point of
view, e.g.
- inside `[[ ... ]]` (other than the RHS of the `==`, `!=`, and `=~`
operators)
- the parameter (`WORD`) in `case $WORD in ....`
- variable assignment: `VAR=$WORD`
But quoting these is never a mistake. If you quote every parameter
expansion, you\'ll be safe.
If you need to parse a parameter as a list of words, you can\'t quote,
of course, e.g.
list="one two three"
# you MUST NOT quote $list here
for word in $list; do
...
done
### Function names
Function names should be all `lowercase` and meaningful. The function
names should be human readable. A function named `f1` may be easy and
quick to write down, but for debugging and especially for other people,
it reveals nothing. Good names help document your code without using
extra comments.
**do not use command names for your functions**. e.g. naming a script or
function `test`, will collide with the UNIX `test` command.
Unless absolutely necessary, only use alphanumeric characters and the
underscore for function names. `/bin/ls` is a valid function name in
Bash, but is not a good idea.
### Command substitution
As noted in [the article about command
substitution](/syntax/expansion/cmdsubst), you should use the `$( ... )`
form.
If portability is a concern, use the backquoted form `` ` ... ` ``.
In any case, if other expansions and word splitting are not wanted, you
should quote the command substitution!
### Eval
Well, like Greg says: **\"If eval is the answer, surely you are asking
the wrong question.\"**
Avoid it, unless absolutely neccesary:
- `eval` can be your neckshot
- there are most likely other ways to achieve what you want
- if possible, re-think the way your script works, if it seems you
can\'t avoid `eval` with your current method
- if you really, really, have to use it, then take care, and be sure
about what you\'re doing
## Basic structure
The basic structure of a script simply reads:
#!SHEBANG
CONFIGURATION_VARIABLES
FUNCTION_DEFINITIONS
MAIN_CODE
### The shebang
If possible (I know it\'s not always possible!), use [a
shebang](/dict/terms/shebang).
Be careful with `/bin/sh`: The argument that \"on Linux `/bin/sh` is
Bash\" **is a lie** (and technically irrelevant)
The shebang serves two purposes for me:
- it specifies the interpreter to be used when the script file is
called directly: If you code for Bash, specify `bash`!
- it documents the desired interpreter (so: use `bash` when you write
a Bash-script, use `sh` when you write a general Bourne/POSIX
script, \...)
### Configuration variables
I call variables that are meant to be changed by the user
\"configuration variables\" here.
Make them easy to find (directly at the top of the script), give them
meaningful names and maybe a short comment. As noted above, use
`UPPERCASE` for them only when you\'re sure about what you\'re doing.
`lowercase` will be the safest.
### Function definitions
Unless there are reasons not to, all function definitions should be
declared before the main script code runs. This gives a far better
overview and ensures that all function names are known before they are
used.
Since a function isn\'t parsed before it is executed, you usually don\'t
have to ensure they\'re in a specific order.
The portable form of the function definition should be used, without the
`function` keyword (here using the [grouping compound
command](/syntax/ccmd/grouping_plain)):
getargs() {
...
}
Speaking about the command grouping in function definitions using
`{ ...; }`: If you don\'t have a good reason to use another compound
command directly, you should always use this one.
## Behaviour and robustness
### Fail early
**Fail early**, this sounds bad, but usually is good. Failing early
means to error out as early as possible when checks indicate an error or
unmet condition. Failing early means to error out **before** your script
begins its work in a potentially broken state.
### Availability of commands
If you use external commands that may not be present on the path, or not
installed, check for their availability, then tell the user they\'re
missing.
Example:
my_needed_commands="sed awk lsof who"
missing_counter=0
for needed_command in $my_needed_commands; do
if ! hash "$needed_command" >/dev/null 2>&1; then
printf "Command not found in PATH: %s\n" "$needed_command" >&2
((missing_counter++))
fi
done
if ((missing_counter > 0)); then
printf "Minimum %d commands are missing in PATH, aborting\n" "$missing_counter" >&2
exit 1
fi
### Exit meaningfully
The [exit code](/dict/terms/exit_status) is your only way to directly
communicate with the calling process without any special provisions.
If your script exits, provide a meaningful exit code. That minimally
means:
- `exit 0` (zero) if everything is okay
- `exit 1` - in general non-zero - if there was an error
This, **and only this**, will enable the calling component to check the
operation status of your script.
You know: **\"One of the main causes of the fall of the Roman Empire was
that, lacking zero, they had no way to indicate successful termination
of their C programs.\"** *\-- Robert Firth*
## Misc
### Output and appearance
- if the script is interactive, if it works for you and if you think
this is a nice feature, you can try to [save the terminal content
and restore it](/snipplets/screen_saverestore) after execution
- output clean and understandable screen messages
- if applicable, you can use colors or specific prefixes to tag error
and warning messages
- make it easy for the user to identify those messages
- write normal output to `STDOUT`. write error, warning and diagnostic
messages to `STDERR`
- enables message filtering
- keeps the script from mixing output data with diagnostic, or
error messages
- if the script gives syntax help (`-?` or `-h` or `--help`
arguments), it should go to `STDOUT`
- if applicable, write error/diagnostic messages to a logfile
- avoids screen clutter
- messages are available for diagnostic use
### Input
- never blindly assume anything. If you want the user to input a
number, **check for numeric input, leading zeros**, etc. If you have
specific format or content needs, **always validate the input!**
### Tooling
- some of these guidelines, such as indentation, positioning of
\"body-introducing\" keywords, and portable function declarations,
can be enforced by [shfmt](https://github.com/mvdan/sh)

354
docs/scripting/terminalcodes.md Normal file
View File

@ -0,0 +1,354 @@
# Terminal codes (ANSI/VT100) introduction
![](keywords>bash shell scripting colors cursor control vt100 ansi)
Terminal (control) codes are used to issue specific commands to your
terminal. This can be related to switching colors or positioning the
cursor, i.e. anything that can\'t be done by the application itself.
## How it technically works
A terminal control code is a special sequence of characters that is
printed (like any other text). If the terminal understands the code, it
won\'t display the character-sequence, but will perform some action. You
can print the codes with a simple `echo` command.
[**Note:**]{.underline} I see codes referenced as \"Bash colors\"
sometimes (several \"Bash tutorials\" etc\...): That\'s a completely
incorrect definition.
## The tput command
Because there\'s a large number of different terminal control languages,
usually a system has an intermediate communication layer. The real codes
are looked up in a database **for the currently detected terminal type**
and you give standardized requests to an API or (from the shell) to a
command.
One of these commands is `tput`. Tput accepts a set of acronyms called
*capability names* and any parameters, if appropriate, then looks up the
correct escape sequences for the detected terminal in the `terminfo`
database and prints the correct codes (the terminal hopefully
understands).
## The codes
In this list I\'ll focus on ANSI/VT100 control codes for the most common
actions - take it as quick reference. The documentation of your terminal
or the `terminfo` database is always the preferred source when something
is unclear! Also the `tput` acronyms are usually the ones dedicated for
ANSI escapes!
I listed only the most relevant codes, of course, any ANSI terminal
understands many more! But let\'s keep the discussion centered on common
shell scripting ;-)
If I couldn\'t find a matching ANSI escape, you\'ll see a :?: as the
code. Feel free to mail me or fix it.
The ANSI codes always start with the ESC character. (ASCII 0x1B or octal
033) This isn\'t part of the list, but **you should avoid using the ANSI
codes directly - use the `tput` command!**
All codes that can be used with `tput` can be found in terminfo(5). (on
OpenBSD at least) See [OpenBSD\'s
terminfo(5)](http://www.openbsd.org/cgi-bin/man.cgi?query=terminfo&apropos=0&sektion=5&manpath=OpenBSD+Current&arch=i386&format=html)
under the [Capabilities]{.underline} section. The *cap-name* is the code
to use with tput. A description of each code is also provided.
### General useful ASCII codes
The **Ctrl-Key** representation is simply associating the non-printable
characters from ASCII code 1 with the printable (letter) characters from
ASCII code 65 (\"A\"). ASCII code 1 would be `^A` (Ctrl-A), while ASCII
code 7 (BEL) would be `^G` (Ctrl-G). This is a common representation
(and input method) and historically comes from one of the VT series of
terminals.
Name decimal octal hex C-escape Ctrl-Key Description
------- --------- ------- ------ ---------- ---------- --------------------------------
`BEL` 7 007 0x07 `\a` `^G` Terminal bell
`BS` 8 010 0x08 `\b` `^H` Backspace
`HT` 9 011 0x09 `\t` `^I` Horizontal TAB
`LF` 10 012 0x0A `\n` `^J` Linefeed (newline)
`VT` 11 013 0x0B `\v` `^K` Vertical TAB
`FF` 12 014 0x0C `\f` `^L` Formfeed (also: New page `NP`)
`CR` 13 015 0x0D `\r` `^M` Carriage return
`ESC` 27 033 0x1B `<none>` `^[` Escape character
`DEL` 127 177 0x7F `<none>` `<none>` Delete character
### Cursor handling
-------------------------------------------------------------------------------------------------------------------------------
ANSI terminfo equivalent Description
----------------------------------------- --------------------- ---------------------------------------------------------------
`[ <X> ; <Y> H`\ `cup <X> <Y>` Home-positioning to `X` and `Y` coordinates\
`[ <X> ; <Y> f` :!: it seems that ANSI uses 1-1 as home while `tput` uses 0-0
`[ H` `home` Move cursor to home position (0-0)
`7` `sc` Save current cursor position
`8` `rc` Restore saved cursor position
:?: most likely a normal code like `\b` `cub1` move left one space (backspace)
VT100 `[ ? 25 l` `civis` make cursor invisible
VT100 `[ ? 25 h` `cvvis` make cursor visible
-------------------------------------------------------------------------------------------------------------------------------
### Erasing text
------------------------------------------------------------------------------------------------
ANSI terminfo equivalent Description
--------- --------------------- ----------------------------------------------------------------
`[ K`\ `el` **Clear line** from current cursor position **to end** of line
`[ 0 K`
`[ 1 K` `el1` **Clear line from beginning** to current cursor position
`[ 2 K` `el2`:?: **Clear whole line** (cursor position unchanged)
------------------------------------------------------------------------------------------------
### General text attributes
ANSI terminfo equivalent Description
--------- ----------------------------- ------------------------------------------------
`[ 0 m` `sgr0` Reset all attributes
`[ 1 m` `bold` Set \"bright\" attribute
`[ 2 m` `dim` Set \"dim\" attribute
`[ 3 m` `smso` Set \"standout\" attribute
`[ 4 m` set `smul` unset `rmul` :?: Set \"underscore\" (underlined text) attribute
`[ 5 m` `blink` Set \"blink\" attribute
`[ 7 m` `rev` Set \"reverse\" attribute
`[ 8 m` `invis` Set \"hidden\" attribute
### Foreground coloring
ANSI terminfo equivalent Description
----------- --------------------- ----------------------------------------------
`[ 3 0 m` `setaf 0` Set **foreground** to color #0 - **black**
`[ 3 1 m` `setaf 1` Set **foreground** to color #1 - **red**
`[ 3 2 m` `setaf 2` Set **foreground** to color #2 - **green**
`[ 3 3 m` `setaf 3` Set **foreground** to color #3 - **yellow**
`[ 3 4 m` `setaf 4` Set **foreground** to color #4 - **blue**
`[ 3 5 m` `setaf 5` Set **foreground** to color #5 - **magenta**
`[ 3 6 m` `setaf 6` Set **foreground** to color #6 - **cyan**
`[ 3 7 m` `setaf 7` Set **foreground** to color #7 - **white**
`[ 3 9 m` `setaf 9` Set **default** color as foreground color
### Background coloring
ANSI terminfo equivalent Description
----------- --------------------- ----------------------------------------------
`[ 4 0 m` `setab 0` Set **background** to color #0 - **black**
`[ 4 1 m` `setab 1` Set **background** to color #1 - **red**
`[ 4 2 m` `setab 2` Set **background** to color #2 - **green**
`[ 4 3 m` `setab 3` Set **background** to color #3 - **yellow**
`[ 4 4 m` `setab 4` Set **background** to color #4 - **blue**
`[ 4 5 m` `setab 5` Set **background** to color #5 - **magenta**
`[ 4 6 m` `setab 6` Set **background** to color #6 - **cyan**
`[ 4 7 m` `setab 7` Set **background** to color #7 - **white**
`[ 4 9 m` `setab 9` Set **default** color as background color
### Misc codes
#### Save/restore screen
Used capabilities: `smcup`, `rmcup`
You\'ve undoubtedly already encountered programs that restore the
terminal contents after they do their work (like `vim`). This can be
done by the following commands:
# save, clear screen
tput smcup
clear
# example "application" follows...
read -n1 -p "Press any key to continue..."
# example "application" ends here
# restore
tput rmcup
These features require that certain capabilities exist in your
termcap/terminfo. While `xterm` and most of its clones (`rxvt`, `urxvt`,
etc) will support the instructions, your operating system may not
include references to them in its default xterm profile. (FreeBSD, in
particular, falls into this category.) If \`tput smcup\` appears to do
nothing for you, and you don\'t want to modify your system
termcap/terminfo data, and you KNOW that you are using a compatible
xterm application, the following may work for you:
echo -e '\033[?47h' # save screen
echo -e '\033[?47l' # restore screen
Certain software uses these codes (via their termcap capabilities) as
well. You may have seen the screen save/restore in `less`, `vim`, `top`,
`screen` and others. Some of these applications may also provide
configuration options to \*disable\* this behaviour. For example, `less`
has a `-X` option for this, which can also be set in an environment
variable:
export LESS=X
less /path/to/file
Similarly, `vim` can be configured not to \"restore\" the screen by
adding the following to your `~/.vimrc`:
set t_ti= t_te=
#### Additional colors
Some terminal emulators support additional colors. The most common
extension used by xterm-compatible terminals supports 256 colors. These
can be generated by `tput` with `seta{f,b} [0-255]` when the `TERM`
value has a `-256color` suffix. [Some
terminals](https://gist.github.com/XVilka/8346728#now-supporting-truecolour)
also support full 24-bit colors, and any X11 color code can be written
directly into a special escape sequence. ([More
infos](https://gist.github.com/XVilka/8346728)) Only a few programs make
use of anything beyond 256 colors, and tput doesn\'t know about them.
Colors beyond 16 usually only apply to modern terminal emulators running
in graphical environments.
The Virtual Terminal implemented in the Linux kernel supports only 16
colors, and the usual default terminfo entry for `TERM=linux` defines
only 8. There is sometimes an alternate \"linux-16color\" that you can
switch to, to get the other 8 colors.
## Bash examples
### Hardcoded colors
printf '%b\n' 'It is \033[31mnot\033[39m intelligent to use \033[32mhardcoded ANSI\033[39m codes!'
### Colors using tput
[Directly inside the echo:]{.underline}
echo "TPUT is a $(tput setaf 2)nice$(tput setaf 9) and $(tput setaf 5)user friendly$(tput setaf 9) terminal capability database."
[With preset variables:]{.underline}
COL_NORM="$(tput setaf 9)"
COL_RED="$(tput setaf 1)"
COL_GREEN="$(tput setaf 2)"
echo "It's ${COL_RED}red${COL_NORM} and ${COL_GREEN}green${COL_NORM} - have you seen?"
### Misc
[HOME function]{.underline}
home() {
# yes, actually not much shorter ;-)
tput home
}
### Silly but nice effect
#!/bin/bash
DATA[0]=" _/ _/ _/ _/ "
DATA[1]=" _/_/_/_/_/ _/_/_/ _/_/_/ _/_/_/ _/_/_/ "
DATA[2]=" _/ _/ _/ _/ _/ _/ _/_/ _/ _/"
DATA[3]="_/_/_/_/_/ _/ _/ _/ _/ _/_/ _/ _/ "
DATA[4]=" _/ _/ _/_/_/ _/_/_/ _/_/_/ _/ _/ "
# virtual coordinate system is X*Y ${#DATA} * 5
REAL_OFFSET_X=0
REAL_OFFSET_Y=0
draw_char() {
V_COORD_X=$1
V_COORD_Y=$2
tput cup $((REAL_OFFSET_Y + V_COORD_Y)) $((REAL_OFFSET_X + V_COORD_X))
printf %c ${DATA[V_COORD_Y]:V_COORD_X:1}
}
trap 'exit 1' INT TERM
trap 'tput setaf 9; tput cvvis; clear' EXIT
tput civis
clear
while :; do
for ((c=1; c <= 7; c++)); do
tput setaf $c
for ((x=0; x<${#DATA[0]}; x++)); do
for ((y=0; y<=4; y++)); do
draw_char $x $y
done
done
done
done
### Mandelbrot set
This is a slightly modified version of Charles Cooke\'s colorful
Mandelbrot plot scripts ([original w/
screenshot](http://earth.gkhs.net/ccooke/shell.html)) \-- ungolfed,
optimized a bit, and without hard-coded terminal escapes. The `colorBox`
function is [memoized](http://en.wikipedia.org/wiki/Memoization) to
collect `tput` output only when required and output a new escape only
when a color change is needed. This limits the number of `tput` calls to
at most 16, and reduces raw output by more than half. The `doBash`
function uses integer arithmetic, but is still ksh93-compatible (run as
e.g. `bash ./mandelbrot` to use it). The ksh93-only floating-point
`doKsh` is almost 10x faster than `doBash` (thus the ksh shebang by
default), but uses only features that don\'t make the Bash parser crash.
#!/usr/bin/env ksh
# Charles Cooke's 16-color Mandelbrot
# http://earth.gkhs.net/ccooke/shell.html
# Combined Bash/ksh93 flavors by Dan Douglas (ormaaj)
function doBash {
typeset P Q X Y a b c i v x y
for ((P=10**8,Q=P/100,X=320*Q/cols,Y=210*Q/lines,y=-105*Q,v=-220*Q,x=v;y<105*Q;x=v,y+=Y)); do
for ((;x<P;a=b=i=c=0,x+=X)); do
for ((;a**2+b**2<4*P**2&&i++<99;a=((c=a)**2-b**2)/P+x,b=2*c*b/P+y)); do :
done
colorBox $((i<99?i%16:0))
done
echo
done
}
function doKsh {
integer i
float a b c x=2.2 y=-1.05 X=3.2/cols Y=2.1/lines
while
for ((a=b=i=0;(c=a)**2+b**2<=2&&i++<99&&(a=a**2-b**2+x,b=2*c*b+y);)); do :
done
. colorBox $((i<99?i%16:0))
if ((x<1?!(x+=X):(y+=Y,x=-2.2))); then
print
((y<1.05))
fi
do :
done
}
function colorBox {
(($1==lastclr)) || printf %s "${colrs[lastclr=$1]:=$(tput setaf "$1")}"
printf '\u2588'
}
unset -v lastclr
((cols=$(tput cols)-1, lines=$(tput lines)))
typeset -a colrs
trap 'tput sgr0; echo' EXIT
${KSH_VERSION+. doKsh} ${BASH_VERSION+doBash}
A much more sophisticated version by Roland Mainz can be found
[here](http://svn.nrubsig.org/svn/people/gisburn/scripts/mandelbrotset1.sh)