regpg - safely store server secrets
regpg help
regpg check [options] <cryptfile.asc>...
- keys:
regpg lskeys [options] [cryptfile.asc]
regpg addself [options]
regpg addkey [options] <keyname>...
regpg delkey [options] <keyname>...
regpg exportkey [options] [keyname]...
regpg importkey [options] [keyfile]...
- secrets:
regpg encrypt [options] [[clearfile] cryptfile.asc]
regpg decrypt [options] [cryptfile.asc [clearfile]]
regpg recrypt [options] <cryptfile.asc>...
- helpers:
regpg depipe cryptfile.asc fifo
regpg edit [options] <cryptfile.asc>
regpg pbcopy [options] [cryptfile.asc]
regpg pbpaste [options] [cryptfile.asc]
regpg shred [options] <clearfile>...
- generators:
regpg dnssec [opts] <action> [flags] <dnskey>
regpg gencrt [opts] <days> [<cakey> <cacrt>] <priv> <cnf> <crt>
regpg gencsrcnf [options] [<certfile>|<hostname> [csr.cnf]]
regpg gencsr [options] <private.asc> <csr.cnf> [csr]
regpg genkey [options] <algorithm> <private.asc> [ssh.pub]
regpg genpwd [options] [cryptfile.asc]
regpg genspkifp [options] [priv|crt|csr|host]
- setup:
regpg init [options] [hook]...
regpg conv <command> [options] <args>...
The regpg program is a thin wrapper around gpg for looking after secrets that need to be stored encrypted in a version control system and deployed to servers with a configuration management system.
At the root of your project you have a pubring.gpg file which lists the set of people who can decrypt the secrets. Elsewhere in that directory and its subdirectories you have encrypted secret.asc files. (The .asc extension is short for ASCII-armored PGP message.) You can use a different layout, by regpg works best if you follow the usual conventions.
You use the regpg *keys subcommands to maintain pubring.gpg. By default, regpg expects to find pubring.gpg in the current working directory.
You use the regpg *crypt subcommands to manage encrypted files. The "recipients" who can decrypt the files are all the keys in the public key ring. Decryption is non-interactive, using gpg-agent.
You use the regpg gen* subcommands to create encrypted secrets to be used by other software.
The regpg check subcommand verifies that the encrypted files and public keyring are consistent with each other.
The regpg init subcommand helps you to hook up regpg with Ansible and git.
The regpg subcommands all take the same options.
Specify the name of the public key ring file, to override the default ./pubring.gpg.
Do nothing, but show what would have been done.
Quiet mode. This affects the check, conv, genpwd, genspkifp, init, and shred subcommands.
For the addkey, delkey, import, and recrypt subcommands, recrypt all files found by the check subcommand. For the shred subcommand, destroy all cleartext files found by the check subcommand.
Verbose mode. This mainly prints the gpg commands.
Several subcommands have abbreviated synonyms.
Display this documentation.
Check cryptfiles for consistency.
If no arguments are given, check recursively finds and lists all encrypted files. These are the files that are recryped by the -r option.
With the -q option, files are only listed if regpg check finds a problem with them.
If a cryptfile has a .asc or a .gpg extension, and an adjacent file exists without the extension, it is called out as a potential cleartext file. You can use regpg shred to destroy cleartext files.
Keys that can decrypt a cryptfile but are not present in the -k pubring.gpg are listed in red with - markers.
Keys are present in the -k pubring.gpg but cannot decrypt a cryptfile are listed in green with + markers.
If one argument is given then key fingerprints are printed in full, otherwise the diffs just list bare key IDs.
These differences can be resolved by the recrypt subcommand.
The following subcommands manage the contents of the publig key ring file, by default pubring.gpg.
With no argument, list the keys in the regpg keyring.
With a cryptfile.asc argument, list the keys that are able to decrypt the file.
Add your own key(s) to the regpg keyring.
A key is "yours" if you have its secret key and it has an identity matching your login name.
It is a good idea to generate a new gpg key specifically for use with regpg. If you have multiple keys, addself will add them all. You can then use delkey to remove the unwanted ones.
If none of your keys were previously on the regpg keyring then you will not be able to recrypt the secrets. You will need to get one of the existing keyholders to do that for you.
Export keys from your default gpg public keyring, and import them into the regpg keyring.
A keyname can be a key fingerprint or ID or a person's email address.
If the -r option is given, all files are recrypted after the key(s) are added.
Delete keys from the regpg keyring.
A keyname can be a key fingerprint or ID or a person's email address.
If the -r option is given, all files are recrypted after the key(s) are deleted.
Export keys from the regpg keyring.
If no keynames are given then all keys are exported.
Import keys into the regpg keyring that have previously been exported by gpg.
If no keyfiles are given then keys are read from stdin.
If the -r option is given, all files are recrypted after the key(s) are added.
The following are the core secret encryption and decryption subcommands.
Encrypt clearfile to produce cryptfile.asc. The encryption recipients are all the keys in the public key ring.
If clearfile is - or there is one argument then the cleartext is read from stdin.
If cryptfile is - or there are no arguments then the ciphertext is written to stdout.
Note: conventionally the cryptfile has a .asc extention, short for ASCII-armored PGP message.
Decrypt cryptfile.asc to produce clearfile. The output file is created with mode 0600, i.e. the permissions deny access to group and other users. You must be running gpg-agent which will be used to gain access to your private key for decryption.
If cryptfile is - or there are no arguments then the ciphertext is read from stdin.
If clearfile is - or there is one argument then the cleartext is written to stdout.
Note: You can also just use gpg --decrypt. The regpg decrypt subcommand requires the GPG agent so it has consistent behaviour in bulk operations such as regpg recrypt -r.
Decrypt and re-encrypt cryptfiles. If the -r option is given, all files are re-encrypted.
You should use this after using addkey or delkey, if you did not pass the -r option.
The following subcommands provide more convenient access to secrets, at the cost of some safety.
Create a temporary named pipe called fifo. The cryptfile is decrypted and written to the fifo in the background. In the foreground you can start a program which reads the cleartext from the fifo. When it has finished, the fifo is removed.
This subcommand is for use with programs that can't read from stdin but can read from a named pipe.
Decrypt cryptfile.asc (if it exists) and run $EDITOR on the cleartext file. The cleartext is placed on a tmpfs in RAM when possible. When you have finished editing, the file is re-encrypted and the cleartext is shredded.
Decrypt cryptfile.asc and copy the cleartext to the clipboard. When you have used the secret, press ^C and regpg will clear the clipboard.
This uses pbcopy on macOS or xclip on X11.
If cryptfile is missing or - then it is read from stdin.
Encrypt the contents of the clipboard and paste the ciphertext into cryptfile.asc. The clipboard is cleared afterwards.
This uses pbpaste on macOS or xclip on X11.
If cryptfile is missing or - then it is written to stdout.
Destroy cleartext files and any related editor backup files. If shred(1) is not available, the files are just deleted.
If the -r option is given, all cleartext files found by regpg check are shredded.
Note: You might want to test shred with the -n option first!
An alias for regpg shred -r, an effective cleaner that is particularly handy for users of glass TTYs.
The following subcommands combine the core regpg encrypt / decrypt subcommands with secret handling tools from BIND, OpenSSL, OpenSSH, etc.
Create a DNSSEC key using BIND's dnssec-keygen utility, encrypt private key as per regpg dnssec recrypt, then shred it.
The opts are regpg options. The flags and name are passed to dnssec-keygen.
Use the regpg gendnskey wrapper to generate keys for a zone with the recommended setup.
Re-encrypt a DNSSEC private key if necessary. The dnskey can name any of the four files that this subcommand works with:
K*.key - the public keyK*.private - the private key cleartext, possibly modifiedK*.private.asc - the encrypted private keyK*.private.sha256 - fingerprint of decrypted keyIf the K*.private.sha256 file is missing or does not match the contents of the K*.private file, then the K*.private is encrypted and the K*.private.asc file is overwritten.
Update the timing parameters on a DNSSEC key using BIND's dnssec-settime utility.
The private key is decrypted if necessary before running dnssec-settime, then re-encrypted. The private key is shredded unless it was present un-encrypted before.
The opts are regpg options. The flags and dnskey are passed to dnssec-settime. The dnskey can name any of the key files listed under regpg dnssec recrypt.
Create an X.509 certificate with a validity period given by days, and write it to crt. If you provide cakey and cacrt, the certificate will be signed by them, otherwise it will be self-signed.
The certificate's encrypted private key is read from priv, and the certificate parameters (distinguished name, subjectAltName, etc) are given in the OpenSSL configuration file cnf. (See the req(1ssl) man page for details.) If priv does not exist, it is created as if by regpg genkey rsa priv.
In the self-signed case the OpenSSL configuration file cnf should contain all the X.508v3 extension attributes you require. (See the x509v3_config(5ssl) man page for details.) In the CA-signed case, regpg will add the subjectKeyIdentifier and authorityKeyIdentifier, so you should omit them.
See the "EXAMPLES" below for how to use this for a private internal certificate authority.
Convert an X.509 certificate into an openssl req configuration file which can be used with regpg gencsr.
You can use gencsrcnf with an existing certificate file certfile to help with renewals, or you can fetch a web server's certificate from hostname to create an example configuration to adapt for a new certificate request. See the "EXAMPLES" below.
If certfile is - or there are no arguments then it is read from stdin.
If csr.cnf is - or there is one argument then it is written to stdout.
Generate an X.509 certificate signing request for an encrypted private key.
The CSR parameters (distinguished name, subjectAltName, etc) are given in the OpenSSL configuration file csr.cnf. (See the req(1ssl) man page for details.)
The private key private.asc should have been generated with regpg genkey rsa. If it does not exist, it is created for you.
If csr is - or is omitted then it is written to stdout.
As well as being written to csr, the CSR is printed in text form if you give the -v option.
Create recommended DNSSEC keys for the zone if they do not already exist, with a 24 hour TTL, using algorithm 13 (ECDSA P256 SHA-256), with separate ZSK and KSK, and with CDS and CDNSKEY records. This is a shortcut for:
$ regpg dnssec keygen -L 86400 -a 13 $zone
$ regpg dnssec keygen -L 86400 -a 13 -f KSK -Psync now $zoneGenerate a cryptographic key pair, for use with OpenSSL or OpenSSH. The PEM private key is encrypted and written to the file private.asc.
If an ssh.pub filename is given, an ssh public key is written there. If private.asc already exists, genkey will convert it to an ssh public key rather than generating a new key.
The algorithm can be one of:
Unlike the other algorithms, ed25519 is ssh-only, and requires puttygen.
If cryptfile.asc already exists, decrypt it and print a SHA256 (type $5$) crypt(3) hash of the password.
Otherwise, generate a 16 character password, encrypt it, store it in cryptfile.asc, and (unless the -q option is given) print a crypt(3) hash of the password.
If cryptfile.asc is - or is omitted then the encrypted password is written to stdout.
The ReGPG::Login(3pm) module helps you to associate metadata such as username and login URL with an encrypted password. It uses a YAML file conventionally named like cryptfile.yml alonside cryptfile.asc.
The regpg genpwd subcommand will create cryptfile.asc, then report an error instead of printing the password hash when crypt(3) lacks support for SHA256, such as on Mac OS X.
Generate an X.509 subject public key information SHA-256 fingerprint, suitable for use with HTTPS public key pinning (HPKP). The public key can be obtained from a gpg-encrypted private key, a certificate file, a certificate signing request, or by fetching a server's certificate. In the latter three cases, the subject's distinguished name is printed to stderr unless the -q option was given.
Easy initialization of regpg and its hooks for other utilities. It is safe to re-run regpg init since it is idempotent.
Every init run ensures the regpg public keyring exists. If it doesn't, the keyring is created using regpg addself.
The hooks can be zero or more of:
Configure git to provide human-readable diffs of the regpg public keyring, and keys that can decrypt each secret.
Note: You must have run git init first. The git diff configuration is not propagated with the contents of the repository, so you need to re-run regpg init git in each fresh clone.
Note: this does not provide human-readable diffs of the cleartext contents of encrypted files, because that would expose secrets dangerously. Instead it diffs the keys that are able to decrypt each secret. This is intended to help you audit changes to the list of people that can access the secrets.
Configure Ansible for use with regpg. An ansible.cfg file is created if necessary.
This installs three things:
The gpg_d module works like the copy module, except that the src: file is decrypted before being transferred to the remote target. The decrypted contents can be binary.
The gpg_d Jinja2 filter inserts decrypted files into templates. The decrypted contents must be plain text.
To ensure that gpg-agent is not confused by concurrent passphrase requests, you can include gpg-preload.yml at the start of your playbook.
See the "EXAMPLES" below for how to use this setup.
Note: these Ansible hooks only use gpg, so you don't need regpg to run your playbooks. You only need regpg for altering the public keyring and encrypted files.
Note: The plugins/action/gpg_d.py file is distributed under the terms of the GNU General Public License version 3 or later. The other hook files are public domain (CC0).
Configure Ansible Vault for use with regpg. An ansible.cfg file is created if necessary.
This creates an encrypted vault password vault.pwd.asc and a wrapper script vault.open to decrypt the password using gpg.
You can then use ansible-vault as usual, and when it needs the vault password it will be automatically decrypted.
Note: the decryption script only uses gpg, so you don't need regpg to run ansible-playbook nor ansible-vault. You only need regpg for altering the public keyring.
The conv subcommand has a number of sub-sub-commands which help you convert from other setups to regpg.
Convert an ansible-gpg setup for use with regpg, as if you had used regpg init ansible-vault.
This copies the keys from .ansible-gpg/pubring.gpg into the regpg keyring, renames .ansible-gpg/vault_passphrase.gpg to vault.pwd.gpg, reconfigures Ansible, and finally removes the remaining ansible-gpg files.
Decrypt vaultfile using ansible-vault and re-encrypt the cleartext using regpg.
If cryptfile is - or omitted then the ciphertext is written to stdout.
If both arguments are omitted then regpg searches for ansible-vault files that may need conversion.
Convert from a symmetrically-encrypted secrets.tar.gz.asc using its regpg-encrypted passphrase from secrets.pwd.asc. (These filenames are hard-coded to match old tooling.)
The member is the name of a file inside secrets.tar.gz.asc to extract and re-encrypt with regpg.
If cryptfile is - or omitted then the ciphertext is written to stdout.
With no argument, regpg lists the contents of secrets.tar.gz.asc.
When decrypting, regpg tells gpg to use gpg-agent which (in older versions of gpg) is located using $GPG_AGENT_INFO.
By default regpg sets gpg's home directory to ~/.regpg. You can set $REGPGHOME to override this.
The default regpg public keyring.
Dummy $GNUPGHOME directory to isolate regpg from problematic gpg.conf settings. You can set the $REGPGHOME environment variable to change this. When starting, regpg automatically ensures this directory exists.
Get a template OpenSSL CSR configuration file based on a certificate similar to the one you want. This example gets the request details from the certificate for dotat.at, which it downloads from the web server:
$ regpg gencsrcnf dotat.at tls.cnfEdit tls.cnf to the correct details for your server, then you can generate a key and CSR:
$ regpg gencsr tls.pem.asc tls.cnf tls.csrTo create the root certificate, you'll need an OpenSSL configuration file similar to this, but with a more suitable distinguished name:
[ req ]
prompt = no
distinguished_name = distinguished_name
req_extensions = extensions
x509_extensions = extensions
[ extensions ]
keyUsage = critical, keyCertSign, cRLSign
basicConstraints = critical, CA:TRUE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always, issuer:always
[ distinguished_name ]
commonName = "Honest Achmed's Used Cars and Certificates"Then make your CA root private key and certificate:
$ regpg gencrt 3650 root.pem.asc root.cnf root.crtThen to make a certificate, generate a configuration file using gencsrcnf as in the previous example. You may want to add the following to the [extensions] section:
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuthThen run the following command:
$ regpg gencrt 365 root.pem.asc root.crt tls.pem.asc tls.cnf tls.crtNote: For simple self-signed certificates, it's easiest to omit the key usage restrictions.
After running regpg init ansible, here are a couple of ways you can use gpg in your Ansible playbooks.
This Ansible task installs ssh host private keys that have been encrypted with regpg. The when: condition on the last line allows you to avoid decrypting secrets except when necessary.
- name: install ssh host keys
gpg_d:
src="{{item}}.asc"
dest="/etc/ssh/{{item}}"
mode=0600
with_items:
- ssh_host_dsa_key
- ssh_host_ecdsa_key
- ssh_host_rsa_key
when: secrets | default(all) | default()There can be a problem when Ansible invokes gpg_d multiple times concurrently, that you may be asked to enter your passphrase more than once. You can avoid this by forcing Ansible to preload gpg-agent immediately at startup. Do this by including gpg-preload.yml at the start of your main playbook:
---
- include: gpg-preload.yml
when: secrets | default(all) | default()
# etc...The gpg-preload.yml playbook uses the gpg_d filter like this:
assert:
that: "{{ 'gpg-preload.asc' | gpg_d }}"This is regpg-1.12.3 <https://dotat.at/prog/regpg/>
Written by Tony Finch <fanf2@cam.ac.uk> <dot@dotat.at>
at Cambridge University Information Services
and distributed under the terms of the GNU General Public License
version 3 or later. <https://www.gnu.org/licenses/gpl.html>Thanks to Jon Warbrick who gave me the idea for regpg's key management; and David Carter, Ben Harris, Paul Haughton, Ian Lewis, David McBride, mchubby, and Matthew Vernon for helpful bug reports and discussions.
gpg(1), gpg-agent(1), ansible(1), git(1), openssl(1), ReGPG::Login(3pm), puttygen(1) shred(1), ssh-keygen(1)