Because Ubuntu change the default shell from bash to dash, many people encounter syntax errors for using bash specific syntax. In this article we will give a cheatsheet for quick reference for standard POSIX shell syntax. And you can learn how to make a bash script work in dash.
Why standard POSIX shell syntax
As Dash as bin/sh said, in Ubuntu 6.10, the default system shell, /bin/sh
, was changed to dash
(the Debian Almquist Shell) mainly for efficiency; previously it had been bash
(the GNU Bourne-Again Shell).
Therefore those scripts that could run with bash could not run anymore due to this change. If you writes scripts with bash syntax, you probably encounter kinds of compliant issues.
A good philosophy is sticking to the standard POSIX shell syntax to make your scripts available for a variety of UNIX systems.
Check which shell you are using
$ echo $SHELL
/bin/bash
Positional parameters
Positional parameters are initially assigned when the shell is invoked , temporarily replaced when a shell function is invoked (see Function Definition Command), and can be reassigned with the set special built-in command.
# $?
# The exit status of the most recent pipeline.
# $@
# All positional parameters.
# $*
# All positional parameters.
# Differences between $@ and $*
# Loop over $@
for a in "$@"; do
echo "Arg: $a" # Run
done
# Loop over $*,
for a in "$*"; do
echo "Args: $a" # Run one time
do
# $#
# The count of positional parameters.
Variables
Quoted variables
# Single-quotes preserve the literal value of each character.
str1='$'
echo $str1 # $
foo=10
x=abc
y='$'$x
echo $y # $foo
# Double-quotes preserve the literal value of all characters with the exception
# of the characters including dollar-sign($), backquote(`) and bashslash().
str2="today is $(date)"
echo $str2 # Today is Wed Oct 28 13:25:17 UTC 2020
str3=abc
echo $str3 # abc
# Characters needed to be quoted to indicate themselves:
# | & ; ( ) $ ` " '
# Chareacters need to be qoted to indicate themselves in cirtern circumstances:
# * ? [ # ˜ = %
str4="*?[~=%"
Empty string, null
# Set value with empty string explicitly
str4=''
# Set value with empty string
str5=
# null value
b=null
echo $b # null
Unset variables
x=123
set y z
# Once a variable is set, it can only be unset by using the unset special built-in command.
unset x
unset y z
Shell environment variables
- HOME
The pathname of the user’s home directory.
-
PWD
Set by the shell and by the cd utility.
-
IFS
IFS stands for “Input Field Separator”. A string treated as a list of cheracters that is used for filed splitting and to split lines into fileds with the read command. If IFS is not set, the shell shall behave as if the value of IFS is
,
, and “; see Field Splitting. Implementations may ignore the value of IFS in the environment at the time the shell is invoked, treating IFS as if it were not set. -
PATH
-
…
# Update environment variables with export
export HOME=/
# or
HOME=/
PATH=/
export HOME PATH
Expansions
Parameter expansions
$s=abc
echo ${s}abc # abcabc
echo $sabc # Nothing is putput for there is no variable named as sabc
# Use default values
echo ${x:-abc} # abc
# Assign defualt values
echo ${x} # Nothing is output
echo ${x:=abc} # abc
echo ${x} # abc
# String length
echo ${#s} # 3
# Error if null or unset
unset x
echo ${x:?} # x: parameter null or not set
echo ${x:?unset parameter} # x: unset value
Arithmetic expansions
i=1
i=$(($i+1)) # Now i=2
For arithmetic operations, except using $((...))
, you can also use expr
command to evaluate (See utilities part in the bottom).
Commands
Redirection
$ cat <<EOF > file_name
line1
line2
EOF
Pipelines
A sequence of commands can be connected by the control operator |
, the shall shall connect the standard output of a command to the standard input of the next command as if by creating a pipe.
# Get count of files
ls -a | wc -l
# Limit files to be list at a time
ls -alh * | less
Lists (multiple commands )
And lists (&&)
# And list
# Only the first command exit with zero, the second command shall be executed
git checkout topic && git status
Or list (||)
# Or list
# If the first command exits with non-zero, the second command shll be executed,
# otherwise only the first command is executed
cd .. || ls -a
Sequential lists (;)
# Sequential list
# Multiple commands executed sequentially in one line.
cd ..; ls -a
Asynchronous lists (&)
# Asynchronous list
# If a command is terminated by the control operator <ampersand> ( '&' ), the shell shall execute the command asynchronously in a subshell. This means that the shell shall not wait for the command to finish before executing the next command.
# Format: command1 & [command2 & ... ]
mv ./build/ ../ &
Command substitution
Command substitution allows the output of a command to be substituted in place of the command name itself.
echo "Hello $(date -u +'%Y-%m-%d')" # Hello 2020-10-28
# or
echo "Hello `date -u +'%Y-%m-%d'`" # Hello 2020-10-28
# Nested command
echo "next year is $(expr $(date +%Y) + 1)"
Conditions
If statement
# If there are less 2 parameters
if [ $# -lt 2 ]; then exit 1; fi
# Or use test explicitly instead of []
if test $# -lt 2; then exit 1; fi
# Put variable inside double-quotes to prevent expanding its value into different words
if [ "$commit_type" = "feat" ]
then
echo "New feature added"
elif [ "$commit_type" = "fix" ]
echo "A bug is fixed"
else
echo "Something else"
fi
# Multiple conditions
if [ $refname = "refs/heads/master" ] || [ $refname = "refs/heads/main" ]; then
echo "This is the main branch"
fi
Case statement
case $file in
*.txt)
type="txt"
echo "This is a txt file"
;;
*.jpg|*.jpg|*.gif)
type="image"
echo "This is an image"
;;
*) # Match anything
type="other"
echo "This is other type"
;;
esac
Loops
For statement
for i in 1 2
do
if test -d "$i"
then break
fi
done
While statement
i=1
while [ $i -lt 3 ]
do
echo "i=$i"
i=$(($i+1))
done
# output:
# i=1
# i=2
Until statement
i=1
until [ $i -gt 3 ]
do
echo "i=$i"
i=$(($i+1))
done
# output:
# i=1
# i=2
# i=3
# Infinite loop
until false
do
...
done
Loop over the output of a command
# The results of a shell command is splitted by $IFS, the default value is tab, space, newline
# Loop over results of "ls"
ls | while read -r f ; do echo ${#f}; done
Functions
Definition
total_files () {
find $1 -type f | wc -l
}
Redirection
Redirect input, output
# Redirect input format: [n]word
# n, represents the file descriptor number. If the number is omitted,
# the redirection shall refer to standard output (file descriptor 1).
Appending redirected output
# Appending redirected output format: [n]>>word
# Appending "hello" to a.txt
echo "hello" >> a.txt
Here-document `< | Relational | Greater Than |
| >= | Relational | Greater Than or Equal to |
| < | Relational | Less Than |
| Note:
>
The XSI extensions specifying the -a and -o binary primaries and the
'('
and')'
operators have been marked obsolescent.test "$1" -a "$2"
should be written as:
test "$1" && test "$2"
Examples
# Exit if there are not two or three arguments (two variations):
if [ $# -ne 2 ] && [ $# -ne 3 ]; then exit 1; fi
if [ $# -lt 2 ] || [ $# -gt 3 ]; then exit 1; fi
# Perform a mkdir if a directory does not exist:
test ! -d tempdir && mkdir tempdir
# Wait for a file to become non-readable:
while test -r thefile
do
sleep 30
done
echo '"thefile" is no longer readable'
# Perform a command if the argument is one of three strings (two variations):
if [ "$1" = "pear" ] || [ "$1" = "grape" ] || [ "$1" = "apple" ]
then
command
fi
case "$1" in
pear|grape|apple) command ;;
esac
Best practice
# The two commands:
test "$1"
test ! "$1"
# could not be used reliably on some historical systems. Unexpected results would occur if such a string expression were used and $1 expanded to '!', '(', or a known unary primary. Better constructs are:
test -n "$1"
test -z "$1"
true
Return true value.
The true utility shall return with exit code zero.
while true
do
<command>
done
false
Return false value.
alias
Define or display aliases.
# Format: alias [alias-name[=string] ...]
# Change ls to give a columnated, more annotated output:
alias ls="ls -CF"
# Create a simple "redo" command to repeat previous entries in the command history file:
alias r='fc -s'
# Use 1K units for du:
alias du=du -k
# Set up nohup so that it can deal with an argument that is itself an alias name:
alias nohup="nohup "
…
Troubleshooting
Common bashism errors
bashism presents a shell command specific to the Bash interpreter. If people use bashisms (bash extensions) in their scripts, they would have many syntax errors.
There are generally several solutions for these errors, they are from Dash as bin/sh.
- Solution 1. Change
#! /bin/sh
to#! /bin/bash
to use bash as the interpreterIf you have limited files that occur such errors, you can adopt this solution and it is a simplest one.
-
Solution 2. Stick to standard POSIX shell syntax to avoid these issues
Someone would like to stick to stand POSIX shell syntax to avoid bashism errors. Doing so makes your scripts more portable to a variety of Unix systems and bring other goodness such as more maintainable.
-
Solution 3. Change the default system shell to bash
If you have widespread such problems, you can instruct the package management system to stop installing
dash
as/bin/sh
:sudo dpkg-reconfigure dash
Be careful to use this solution for this is a more invasive change and may cause other problems.
bashism error examples
function
: not found-
[[
: not found - Bad substitution
Resources
About bashism, bash vs dash
- Dash as bin/sh. https://wiki.ubuntu.com/DashAsBinSh
It lists some more common bash extensions that are not supported by dash. This list is not complete, but we believe that it covers most of the common extensions found in the wild. You can use
dash -n
to check that a script will run underdash
without actually running it; this is not a perfect test (particularly not ifeval
is used), but is good enough for most purposes. Thecheckbashisms
command in thedevscripts
package may also be helpful (it will output a warningpossible bashism in
for every occurrence).“`shell
[
[[
((
$((n++)), $((–n))
{
$'…'
$"…"
${…}
${parm/?/pat[/str]}
${foo:3[:1]}
${foo:3[:1]}
$LINENO
$PIPESTATUS
$RANDOM
function
echo options
let
local
<< A sub-session or sub-process is little different from sub-shell we have seen before. Any code inside parentheses <code>()</code> runs in sub-shell. A sub-shell is also a separate process (<em>with an exception in</em> <a href="https://docstore.mik.ua/orelly/unix3/korn/ch08_06.htm#FOOTNOTE-125"><em>KSh</em></a>) started by the main shell process but in contrast with sub-session, it is an identical copy of the main shell process. <a href="https://docstore.mik.ua/orelly/unix3/korn/ch08_06.htm"><strong>This</strong></a> article explains the difference between sub-shell and sub-process.It is possible to pass an environmental variable to a process directly from the command which started it using below syntax. This way, our environment variables can be portable and we can avoid writing unnecessary boilerplate.
“`shell
main.sh
MY_IP=’192.168.1.7′ bash ./child.sh
~$ bash main.sh
⥤ MY_IP inside child.sh : 192.168.1.7
“`
If you need to set an environmental variable for all the process started by the current terminal session, users can directly execute export command. But once you open a new terminal, it won’t have that environmental variable. To make environmental variables accessible across all terminal sessions, export them from .bash_profile
or any other startup script.
Code block
If we need to execute some code as a block, then we can put our code in {} curly braces. Code block ({}
block) executes the code in the same shell, hence in the same shell process, while ()
block which executes the code inside it in a sub-shell.
If we want to run some code as a block on a single line, we need to terminate the statements wit ;
character (unlike a sub-shell).
“`shell
MAIN_VAR="main"
{ sleep 1; echo "code-block: $MAIN_VAR"; }
( sleep 1; echo "sub-shell: $MAIN_VAR" )
# Output:
code-block: main
sub-shell: main
“`
In the example, code-block and sub-shell both have access to MAIN_VAR
because code-block runs in the same environment of the main shell while sub-shell might run in the different process but it has an identical copy of main process which also contains the variables from the main process.
The difference comes where we try to set or update a variable in the main process from a sub-shell. Here is a demonstration of that.
“`shell
VAR_CODE_BLOCK="INIT"
VAR_SUB_SHELL="INIT"
{ VAR_CODE_BLOCK="MODIFIED"; echo "code-block: $VAR_CODE_BLOCK"; }
( VAR_SUB_SHELL="MODIFIED"; echo "sub-shell: $VAR_SUB_SHELL" )
echo "main/code-block: $VAR_CODE_BLOCK"
echo "main/sub-shell: $VAR_SUB_SHELL"
# Output:
code-block: MODIFIED
sub-shell: MODIFIED
main/code-block: MODIFIED
main/sub-shell: INIT
“`
We can see that when we try to update or set a variable in the sub-shell, it will set a new variable in its own environment.
A good use case of code block would be to pipe (|)
or redirect (>)
the output of some statements as a whole.
“`shell
{ echo -n “Hello”; sleep 1; echo ” World!”; } > hello.txt
~$ bash main.sh && cat hello.txt
# Output:
# Hello World!
“`
- BASH Programming – Introduction HOW-TO. By by Mike G
mikkey
. - Linux Shell Scripting Tutorial v2.0. By Written by Vivek Gite.
- Bash scripting tutorial for beginners
- gnu: Bash Reference Manual