usage: posixcube.sh -h HOST... [-l] [-c CUBE_DIR...] [OPTION]... COMMAND...
A POSIX compliant, shell script-based server automation framework.
-? Help.
-a If using Bash, asynchronously execute remote CUBEs/COMMANDs.
-b If using Bash, install programmable tab completion for SSH hosts.
-c CUBE Execute a cube. Option may be specified multiple times. If COMMANDS
are also specified, cubes are run first.
-d Print debugging information.
-D CMD Use `CMD` as the superuser command instead of `sudo`.
-e ENVAR Shell script with environment variable assignments which is
uploaded and sourced on each HOST. Option may be specified
multiple times. Files ending with .enc will be decrypted
temporarily. If not specified, defaults to envars*sh envars*sh.enc
-F FILE SSH `-F` option.
-h HOST Target host. Option may be specified multiple times. The HOST may
be preceded with USER@ to specify the remote user. If a host has
a wildcard ('*'), then HOST is interpeted as a regular expression,
with '*' replaced with '.*' and any matching hosts in the following
files are added to the HOST list: /etc/ssh_config,
/etc/ssh/ssh_config, ~/.ssh/config, /etc/ssh_known_hosts,
/etc/ssh/ssh_known_hosts, ~/.ssh/known_hosts, and /etc/hosts.
-i FILE SSH `-i` option for identity file.
-k Keep the cube_exec.sh generated script.
-l Local execution. Instead of (or in addition to) `-h HOST`s or a
sub-COMMAND, run the `-c` cubes or `COMMAND`s locally.
-o K=V SSH `-o` option. Option may be specified multiple times. Defaults
to `-o ConnectTimeout=5`.
-O P=V Set the specified variable P with the value V. Option may be
specified multiple times. Do not put double quotes around V. If
V contains *, replace with matching hosts per the -h algorithm.
-p PORT SSH `-p` option.
-P PWD Password for decrypting .enc ENVAR files.
-q Quiet; minimize output.
-r ROLE Role name. Option may be specified multiple times.
-R Use `rsync` instead of scp.
-s Skip remote host initialization (making ~/.posixcube, uploading
posixcube.sh, etc.). Assumes at least one run completed without -s.
Does not support encrypted ENVAR files.
-S Run cube_package and cube_service APIs as superuser.
-t SSH `-t` option.
-u USER SSH user. Defaults to ${USER}. This may also be specified in HOST.
-U CUBE Upload a CUBE but do not execute it. This is needed when one CUBE
includes this CUBE using cube_include.
-v Show version information.
-w PWDF File that contains the password for decrypting .enc ENVAR files.
Defaults to ~/.posixcube.pwd
-z SPEC Use the SPEC set of options from the ./cubespecs.ini file
-y If a HOST returns a non-zero code, continue processing other HOSTs.
COMMAND Remote command to run on each HOST. Option may be specified
multiple times. If no HOSTs are specified, available sub-commands:
edit: Decrypt, edit, and re-encrypt ENVAR file with $EDITOR.
show: Decrypt and print ENVAR file.
source: Source all ENVAR files. Must be run with
POSIXCUBE_SOURCED (see Public Variables section below).
Description:
posixcube.sh is used to execute CUBEs and/or COMMANDs on one or more HOSTs.
A CUBE is a shell script or directory containing shell scripts. The CUBE
is transferred to each HOST. If CUBE is a shell script, it's executed. If
CUBE is a directory, a shell script of the same name in that directory
is executed. In both cases, the directory is changed to the directory
containing the script before execution so that you may reference files
such as templates using relative paths.
An ENVAR script is encouraged to use environment variable names of the form
cubevar_${uniquecontext}_envar="value". If a CUBE directory contains the
file `envars.sh`, it's sourced before anything else (including `-e ENVARs`).
Both CUBEs and COMMANDs may execute any of the functions defined in the
"Public APIs" in the posixcube.sh script. Short descriptions of the
functions are in the APIs section below. See the source comments above each
function for details.
Examples (assuming posixcube.sh is on ${PATH}, or executed absolutely):
posixcube.sh -h socrates uptime
Run the `uptime` command on host `socrates`. This is not very different
from ssh ${USER}@socrates uptime, except that COMMANDs (`uptime`) have
access to the cube_* public functions.
posixcube.sh -h socrates -c test.sh
Run the `test.sh` script (CUBE) on host `socrates`. The script has
access to the cube_* public functions.
posixcube.sh -h socrates -c test
Upload the entire `test` directory (CUBE) to the host `socrates` and
then execute the `test.sh` script within that directory (the name
of the script is expected to be the same as the name of the CUBE). This
allows for easily packaging other scripts and resources needed by
`test.sh`.
posixcube.sh -S -h plato@socrates cube_package install atop
As the remote user `plato` on the host `socrates`, install the package
`atop`. The `-S` option is required to run the commands within
cube_package as the superuser (see the Philosophy section, #3).
posixcube.sh -h root@socrates -h seneca uptime
Run the `uptime` command on hosts `socrates` and `seneca`
as the user `root`.
posixcube.sh -h web*.test.com uptime
Run the `uptime` command on all hosts matching the regular expression
web.*.test.com in the SSH configuration files.
sudo posixcube.sh -b && . /etc/bash_completion.d/posixcube_completion.sh
For Bash users, install a programmable completion script to support tab
auto-completion of hosts from SSH configuration files.
posixcube.sh -e production.sh.enc show
Decrypt and show the contents of production.sh
posixcube.sh -e production.sh.enc edit
Decrypt, edit, and re-encrypt the contents of production.sh with $EDITOR
posixcube.sh -i ~/.ssh/plato -o LogLevel=VERBOSE -o ConnectTimeout=30 \
-h plato@socrates uptime
Run the `uptime` command on host `socrates` as the remote user `plato`
using the SSH identity file ~/.ssh/plato and specifying the SSH options
LogLevel=VERBOSE and ConnectTimeout=30
Philosophy:
1. Fail hard and fast. In principle, a well written script would check ${?}
after each command and either gracefully handle it, or report an error.
Few people write scripts this well, so we enforce this check (using
`cube_check_return` within all APIs) and we encourage you to do the same;
for example, `touch /etc/fstab || cube_check_return`. All cube_*
APIs are guaranteed to do their own checks, so you don't have to do this
for those calls; however, note that if you're executing a cube_* API in a
sub-shell, although any failures will be reported by cube_check_return,
the script will continue unless you also check the return of the sub-shell.
For example: $(cube_readlink /etc/localtime) || cube_check_return
With this strategy, unfortunately, piping becomes more difficult. There are
non-standard mechanisms like pipefail and PIPESTATUS, but the standardized
approach is to run each command separately and check the status. For example:
cubevar_app_x="$(cmd1)" || cube_check_return
cubevar_app_y="$(printf '%s' "${cubevar_app_x}" | cmd2)" || cube_check_return
2. We don't use `set -e` because some functions may handle all errors
internally (with `cube_check_return`) and use a positive return code as a
"benign" result (e.g. `cube_set_file_contents`). We don't use `set -u`
because we source in the user's scripts and they may not want this behavior.
3. Recent versions of many distributions encourage running most commands
as a non-superuser, and then using `sudo` if needed, with some distributions
disallowing remote SSH using the `root` account by default. First, the `sudo`
command is not standardized (see https://unix.stackexchange.com/a/48553).
Moreoever, it is not enough to prefix posixcube APIs with `sudo` because
`sudo` doesn't pass along functions (even if they're exported and `sudo` is
executed with --preserve-env). `su -c` may be used but it requires
password input. For the most common use case of requiring `sudo` for
cube_package and cube_service, if `-S` is specified, the commands within
those APIs are executed using `sudo`. If you need to run something else as a
superuser and you need access to the posixcube APIs, see the `cube_sudo` API.
Frequently Asked Questions:
* Why is there a long delay between "Preparing hosts" and the first remote
execution?
You can see details of what's happening with the `-d` flag. By default,
the script first loops through every host and ensures that ~/.posixcube/
exists, then it transfers itself to the remote host. These two actions
may be skipped with the `-s` parameter if you've already run the script
at least once and your version of this script hasn't been updated. Next,
the script loops through every host and transfers any CUBEs and a script
containing the CUBEs and COMMANDs to run (`cube_exec.sh`). If the shell
is detected to be `bash`, then the above occurs asynchronously across the
HOSTs. Finally, you'll see the "Executing on HOST..." line and the real
execution starts.
Cube Development:
Shell scripts don't have scoping, so to reduce the chances of function name
conflicts, name functions cube_${cubename}_${function} and name variables
cubevar_${cubename}_${var}.
Public APIs:
* cube_echo
Print ${@} to stdout prefixed with ([$(date)] [$(hostname)]) and
suffixed with a newline.
Example: cube_echo "Hello World"
* cube_printf
Print $1 to stdout prefixed with ([$(date)] [$(hostname)]) and
suffixed with a newline (with optional printf arguments in $@).
Example: cube_printf "Hello World from PID %5s" $$
* cube_error_echo
Same as cube_echo except output to stderr and include a red "Error: "
message prefix.
Example: cube_error_echo "Goodbye World"
* cube_error_printf
Same as cube_printf except output to stderr and include a red "Error: "
message prefix.
Example: cube_error_printf "Goodbye World from PID %5s" $$
* cube_warning_echo
Same as cube_echo except output to stderr and include a yellow
"Warning: " message prefix.
Example: cube_warning_echo "Watch out, World"
* cube_warning_printf
Same as cube_printf except output to stderr and include a yellow
"Warning: " message prefix.
Example: cube_warning_printf "Watch out, World from PID %5s" $$
* cube_throw
Same as cube_error_echo but also print a stack of functions and processes
(if available) and then call `exit 1`.
Example: cube_throw "Expected some_file."
* cube_check_return
Check if $? is non-zero and call cube_throw if so.
Example: some_command || cube_check_return
* cube_include
Include the ${1} cube
Example: cube_include core_cube
* cube_check_numargs
Call cube_throw if there are less than $1 arguments in $@
Example: cube_check_numargs 2 "${@}"
* cube_service
Run the $1 action on the $2 service.
Example: cube_service start crond
* cube_package
Pass $@ to the package manager. Implicitly passes the parameter
to say yes to questions. On Debian-based systems, uses --force-confold
Example: cube_package install python
* cube_package_uninstall
Same as cube_package, except that you only specify the packages
to uninstall, and this will figure out the right operation to
pass to the package manager (e.g. dnf=remove, apt=purge).
Example: cube_package_uninstall python
* cube_append_str
Print $1 to stdout with $2 appended after a space if $1 was not blank.
Example: cubevar_app_str=$(cube_append_str "${cubevar_app_str}" "Test")
* cube_element_exists
Check if $1 contains the $2 element with the $3 delimiter (default space).
Example: cube_element_exists "${cubevar_str}" "X"
* cube_elements_count
Print to stdout the number of elements in the string $1 as separated by
the delimeter $2 (default space).
Example: cubevar_app_count=$(cube_elements_count "${cubevar_str}")
* cube_command_exists
Check if $1 command or function exists in the current context.
Example: cube_command_exists systemctl
* cube_dir_exists
Check if $1 exists as a directory.
Example: cube_dir_exists /etc/cron.d/
* cube_file_exists
Check if $1 exists as a file with read access.
Example: cube_file_exists /etc/cron.d/0hourly
* cube_operating_system
Detect operating system and return one of the POSIXCUBE_OS_* values.
Example: [ $(cube_operating_system) -eq ${POSIXCUBE_OS_LINUX} ] && ...
* cube_operating_system_version_major
Returns the major version of the operating system distribution.
Example: [ $(cube_operating_system_version_major) -gt 10 ] && ...
* cube_operating_system_version_minor
Returns the minor version of the operating system distribution.
Example: [ $(cube_operating_system_version_minor) -gt 10 ] && ...
* cube_operating_system_has_flavor
Check if the operating system flavor includes the flavor specified in $1
by one of the POSIXCUBE_OS_FLAVOR_* values.
Example: cube_operating_system_has_flavor ${POSIXCUBE_OS_FLAVOR_FEDORA} \
&& ...
* cube_shell
Detect running shell and return one of the CUBE_SHELL_* values.
Example: [ $(cube_shell) -eq ${POSIXCUBE_SHELL_BASH} ] && ...
* cube_current_script_name
Print to stdout the basename of the currently executing script.
Example: script_name=$(cube_current_script_name)
* cube_current_script_abs_path
Print to stdout the absolute path the currently executing script.
Example: script_name=$(cube_current_script_abs_path)
* cube_file_size
Print to stdout the size of a file $1 in bytes
Example: cube_file_size some_file
* cube_set_file_contents
Copy the contents of $2 on top of $1 if $1 doesn't exist or the contents
are different than $2. If $2 ends with ".template", first evaluate all
${VARIABLE} expressions (except for \${VARIABLE}).
Example: cube_set_file_contents "/etc/npt.conf" "templates/ntp.conf"
* cube_set_file_contents_string
Set the contents of $1 to the string $@. Create file if it doesn't exist.
Example: cube_set_file_contents_string ~/.info "Hello World"
* cube_expand_parameters
Print stdin to stdout with all ${VAR}'s evaluated (except for \${VAR})
Example: cube_expand_parameters < template > output
* cube_readlink
Print to stdout the absolute path of $1 without any symbolic links.
Example: cube_readlink /etc/localtime
* cube_random_number
Print to stdout a random number between 1 and $1
Example: cube_random_number 10
* cube_tmpdir
Print to stdout a temporary directory
Example: cube_tmpdir
* cube_total_memory
Print to stdout total system memory in bytes
Example: cube_total_memory
* cube_ensure_directory
Ensure directory $1, and parent directories, exist. Return true if the
directory is created; otherwise, false.
Example: cube_ensure_directory ~/.ssh/
* cube_ensure_file
Ensure file $1, and parent directories, exist. Return true if the file is
created; otherwise, false.
Example: cube_ensure_file ~/.ssh/authorized_keys
* cube_pushd
Add the current directory to a stack of directories and change directory
to ${1}
Example: cube_pushd ~/.ssh/
* cube_popd
Pop the top of the stack of directories from `cube_pushd` and change
directory to that directory.
Example: cube_popd
* cube_has_role
Return true if the role $1 is set.
Example: cube_has_role "database_backup"
* cube_file_contains
Check if the file $1 contains $2. Uses grep with default arguments
(e.g. regex).
Example: cube_file_contains /etc/fstab nfsmount
* cube_stdin_contains
Check if stdin contains $1
Example: echo "Hello World" | cube_stdin_contains "Hello"
* cube_interface_ipv4_address
Print to stdout the IPv4 address of interface $1
Example: cube_interface_ipv4_address eth0
* cube_interface_ipv6_address
Print to stdout the IPv6 address of interface $1
Example: cube_interface_ipv6_address eth0
* cube_prompt
Prompt the question $1 followed by " (y/N)" and prompt for an answer.
A blank string answer is equivalent to No. Return true if yes, false
otherwise.
Example: cube_prompt "Are you sure?"
* cube_hostname
Print to stdout the full hostname.
Example: cube_hostname
* cube_string_contains
Return true if $1 contains $2; otherwise, false.
Example: cube_string_contains "${cubevar_app_str}" "@" && ...
* cube_string_substring_before
Print to stdout a substring of $1 strictly before the first match of the
regular expression $2.
Example: cubevar_app_x="$(cube_string_substring_before "${somevar}" "@")"
* cube_string_substring_after
Print to stdout a substring of $1 strictly after the first match of the
regular expression $2.
Example: cubevar_app_x="$(cube_string_substring_after "${somevar}" "@")"
* cube_sudo
Execute $* as superuser with all posixcube APIs available
(see Philosophy #3).
Example: cube_sudo cube_ensure_file /etc/app.txt
* cube_read_stdin
Read stdin (e.g. HEREDOC) into the variable named by the first argument.
* cube_ensure_user
Create the user $1 if it doesn't already exist (with group name $1)
Example: cube_ensure_user nginx
* cube_user_exists
Check if the $1 user exists
Example: cube_user_exists nginx
* cube_create_user
Create the user $1
Example: cube_create_user nginx
* cube_group_exists
Check if the $1 group exists
Example: cube_group_exists nginx
* cube_create_group
Create the group $1
Example: cube_create_group nginx
* cube_group_contains_user
Check if the $1 group contains the user $2
Example: cube_group_contains_user nginx nginx
* cube_add_group_user
Add the user $2 to group $1
Example: cube_add_group_user nginx nginx
* cube_current_user
Print to stdout the current user
Example: cube_current_user
* cube_user_home_dir
Print to stdout the home directory of user $1 (defaults to
`cube_current_user`)
Example: cube_user_home_dir root
* cube_user_ensure_private_key
Ensure the SSH private key contents $1 are placed in a file named $2
(defaults to `id_rsa`) for the user $3 (defaults to `cube_current_user`)
Example: cube_user_ensure_private_key "${cubevar_app_key1_private}"
* cube_user_ensure_authorized_public_key
Ensure the SSH public key contents $1 exist in the `.ssh/authorized_keys`
file in the home directory of user $2 (defaults to `cube_current_user`)
Example: cube_user_ensure_authorized_public_key "${cubevar_app_key1_pub}"
* cube_user_authorize_known_host
Ensure the public key(s) of the host $1 are in the `.ssh/known_hosts` file
in the home directory of user $2 (defaults to `cube_current_user`). Note:
the fingerprints are currently not checked, but you may review STDOUT.
To reduce the risks of MITM attacks, you can call this only after a
successful call to `cube_user_ensure_private_key` so that the scan is
only done once on private key setup.
Example: cube_user_ensure_private_key "${cubevar_app_host_privkey}" && \
cube_user_authorize_known_host some_host
Public Variables:
* POSIXCUBE_APIS_ONLY
Set this to any value to only source the public APIs in posixcube.sh.
Example: POSIXCUBE_APIS_ONLY=true . posixcube.sh && cube_echo \
$(cube_random_number 10)
* POSIXCUBE_SOURCED
Set this to any value to only run a sub-COMMAND, most commonly `source`,
to source in all ENVAR files, but skip actual execution of posixcube.
Example: POSIXCUBE_SOURCED=true . posixcube.sh source; \
POSIXCUBE_SOURCED= ; cube_echo Test
Source: https://github.com/myplaceonline/posixcube
- https://github.com/myplaceonline/myplaceonline_posixcubes: Cubes that build a full Ruby on Rails stack with haproxy load balancer (frontend), nginx+passenger Rails servers (web), postgresql database (database) and more (elasticsearch, database backup, rsyslog server, etc.).
