Git-Mirrors/qubes-doc
1
0
Fork 0
mirror of https://github.com/QubesOS/qubes-doc.git synced 2024-12-29 01:06:24 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'SamuelHentschel-master'

Browse Source
This commit is contained in:
Andrew David Wong 2017-06-26 21:43:13 -05:00
commit aa5149c19d
No known key found for this signature in database
GPG Key ID: 8CE137352A019A17
1 changed files with 198 additions and 180 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

378
configuration/salt.md
View File

@ -3,86 +3,96 @@ layout: doc
title: Management stack title: Management stack
permalink: /doc/salt/ permalink: /doc/salt/
--- ---
# Management infrastructure
Since Qubes R3.1 release we have included `salt` (also called SaltStack) # Management Infrastructure
management engine in dom0 as default with some states already configured. salt
allows administrators to easily configure their systems. In this guide we will Since the Qubes R3.1 release we have included the Salt (also called SaltStack)
show how it is set up and how you can modify it for your own purpose. management engine in dom0 as default (with some states already configured).
Salt allows administrators to easily configure their systems.
In this guide we will show how it is set up and how you can modify it for your
own purpose.
In the current form the **API is provisional** and subject to change between In the current form the **API is provisional** and subject to change between
*minor* releases. *minor* releases.
## Understanding `salt` ## Understanding Salt
This document is not meant to be comprehensive salt documentation, however This document is not meant to be comprehensive Salt documentation; however,
before writing anything it is required you have at least *some* understanding of before writing anything it is required you have at least *some* understanding of
basic salt-related vocabulary. For more exhaustive documentation, visit basic Salt-related vocabulary.
[official site][salt-doc], though we must warn you that it is not easy to read For more exhaustive documentation, visit [official site][salt-doc], though we
if you just start working with salt and know nothing. must warn you that it is not easy to read if you just start working with Salt
and know nothing.
### The architecture ### The Salt Architecture
Salt has client-server architecture, where server (called *master*) manages its Salt is a client-server model, where the server (called *master*) manages
clients (called *minions*). In typical situation it is intended that its clients (called *minions*).
administrator interacts only with master and keeps the configuration there. In In typical situations, it is intended that the administrator interacts only
Qubes OS we don't have master though, since we have only one minion, which with the master and keeps the configurations there.
resides in `dom0` and manages domains from there. This is also supported by In Qubes, we don't have a master.
salt. Instead we have one minion which resides in `dom0` and manages domains from
there.
This setup is also supported by Salt.
Salt is a management engine, that enforces particular state of the system, where Salt is a management engine (similar to Ansible, Puppet, and Chef), that
minion runs. A *state* is an end effect *declaratively* expressed by the enforces a particular state of a minion system.
administrator. This is the most important concept in the whole package. All A *state* is an end effect *declaratively* expressed by the administrator.
configuration (ie. the states) are written in YAML. This is the most important concept in the entire engine.
All configurations (i.e., the states) are written in YAML.
A *pillar* is a data back-end declared by administrator. When states became A *pillar* is a data back-end declared by the administrator.
repetitive, instead of pure YAML they can be written with help of some template When states become repetitive, instead of pure YAML they can be written using a
engine (preferably jinja2), which can use data structures specified in pillars. template engine (preferably Jinja2); which can use data structures specified in
pillars.
A *formula* is a ready to use, packaged solution that combines state and pillar, A *formula* is a ready to use, packaged solution that combines a state and a
possibly with some file templates and other auxiliary files. There are many of pillar (possibly with some file templates and other auxiliary files).
those made by helpful people all over the Internet. There are many formulas made by helpful people all over the Internet.
A *grain* is some data that is also available in templates, but its value is not A *grain* is some data that is also available in templates, but its value is not
directly specified by administrator. For example the distribution (like directly specified by administrator.
`"Debian"` or `"Gentoo"`) is a value of the grain `"os"`. It also contains other For example, the distribution (e.g., `"Debian"` or `"Gentoo"`) is a value of
info about kernel, hardware etc. the grain `"os"`. It also contains other information about the kernel,
hardware, etc.
A *module* is a Python extension to salt that is responsible for actually A *module* is a Python extension to Salt that is responsible for actually
enforcing the state in a particular area. It exposes some *imperative* functions enforcing the state in a particular area.
for administrator. For example there is `system` module that has `system.halt` It exposes some *imperative* functions for the administrator.
function that, when issued, will immediately halt the computer. There is another For example, there is a `system` module that has a `system.halt` function that,
function called `state.highstate` which will synchronize the state of the system when issued, will immediately halt a domain.
with the administrator's will. There is another function called `state.highstate` which will synchronize the
state of the system with the administrator's configuration/desires.
### Configuration ### Configuration
#### States #### States
The smallest unit of configuration is a state. The smallest unit of configuration is a state.
A state is written in yaml and looks like this: A state is written in YAML and looks like this:
stateid: stateid:
cmd.run: #this is the execution module. in this case it will execute a command on the shell cmd.run: #this is the execution module. in this case it will execute a command on the shell
- name: echo 'hello world' #this is a parameter of the state. - name: echo 'hello world' #this is a parameter of the state.
The stateid has to be unique over all states running for a minion and can be used The stateid has to be unique throughout all states running for a minion and can
to order the execution of states. be used to order the execution of the references state.
`cmd.run` is the execution module. It decides which action will be executed. `cmd.run` is an execution module.
`name: echo 'hello world'` is a parameter for the execution module. It depends on It executes a command on behalf of the administrator.
the module which parameters are accepted. `name: echo 'hello world'` is a parameter for the execution module `cmd.run`.
The module used defines which parameters can be passed to it.
There is list of [officially available states][salt-doc-states]. There is a list of [officially available states][salt-doc-states].
There are many very useful states: There are many very useful states:
* For [managing files][salt-doc-states-file]: Use this to create files or * For [managing files][salt-doc-states-file]: Use this to create files or
directories and change them (append lines, replace text, set their content etc.) directories and change them (append lines, replace text, set their content etc.)
* For [installing and uninstalling][salt-doc-states-pkg] packages. * For [installing and uninstalling][salt-doc-states-pkg] packages.
* To [execute shell commands][salt-doc-states-cmd]. * For [executing shell commands][salt-doc-states-cmd].
With these three states you can do most of the configuration inside of a vm. With these three states you can define most of the configuration of a VM.
You also can [order the execution][salt-doc-states-order] of your states: You can also [order the execution][salt-doc-states-order] of your states:
D: D:
cmd.run: cmd.run:
@ -104,20 +114,20 @@ You also can [order the execution][salt-doc-states-order] of your states:
- order: 1 - order: 1
The order of execution will be `A, B, C, D`. The order of execution will be `A, B, C, D`.
The official documentation has more details on the [require][salt-doc-states-req] and The official documentation has more details on the
[order][salt-doc-states-ord] arguments. [require][salt-doc-states-req] and [order][salt-doc-states-ord] arguments.
#### State files #### State Files
When configuring a system you will write one or several state files (`*.sls`) and When configuring a system you will write one or more state files (`*.sls`) and
put (or symlink) them in the salt main directory `/srv/salt/`. put (or symlink) them into the main Salt directory `/srv/salt/`.
Each state file contains one multiple states and should describe some unit of Each state file contains multiple states and should describe some unit of
configuration (e.g.: A state file `mail.sls` could setup a vm for mailing). configuration (e.g., a state file `mail.sls` could setup a VM for e-mail).
#### Top files #### Top Files
After you have state several state files, you need something to assign them to a After you have several state files, you need something to assign them to a VM.
vm. This is done by `*.top` files ([official documentation][salt-doc-top]). This is done by `*.top` files ([official documentation][salt-doc-top]).
Their structure looks like this: Their structure looks like this:
environment: environment:
@ -125,11 +135,11 @@ Their structure looks like this:
- statefile1 - statefile1
- folder2.statefile2 - folder2.statefile2
The environment will be in most cases `base`. In most cases, the environment will be called `base`.
The `target_matching_clause` will be used to select your minions (vms). The `target_matching_clause` will be used to select your minions (VMs).
It can be just the name of a vm or a regular expression. It can be either the name of a VM or a regular expression.
If you are using a regular expression, you need to give salt a hint you are doing If you are using a regular expressions, you need to give Salt a hint you are
so: doing so:
environment: environment:
^app-(work|(?!mail).*)$: ^app-(work|(?!mail).*)$:
@ -137,99 +147,104 @@ so:
- statefile - statefile
For each target you can write a list of state files. For each target you can write a list of state files.
Each line is a path to a state file (without the `.sls`) relative to the main Each line is a path to a state file (without the `.sls` extension) relative to
directory. Each `/` is exchanged by a dot, so you can't reference files or the main directory.
directories with a dot in their name. Each `/` is exchanged with a `.`, so you can't reference files or directories
with a `.` in their name.
### Enabling top files and applying the configuration ### Enabling Top Files and Applying the States
Now because we use custom extension to manage top files (instead of just Now, because we use custom extensions to manage top files (instead of just
enabling them all) to enable the particular top file you should issue command: enabling them all), to enable a particular top file you should issue command:
qubesctl top.enable my-new-vm $ qubesctl top.enable my-new-vm
To list all enabled tops: To list all enabled top files:
qubesctl top.enabled $ qubesctl top.enabled
And to disable one: And to disable one:
qubesctl top.disable my-new-vm $ qubesctl top.disable my-new-vm
To actually apply the states to dom0 and all vms: To apply the states to dom0 and all VMs:
qubesctl --all state.highstate $ qubesctl --all state.highstate
(More information on the command is further down.) (More information on the `qubesctl` command further down.)
### Templating files ### Template Files
You will sometimes find your self writing repetitive states. To solve this, You will sometimes find yourself writing repetitive states.
there is the ability to template files or states. To solve this, there is the ability to template files or states.
This can be done with [jinja][jinja]. This is most commonly done with [Jinja][jinja].
Jinja is similar to python and behaves in many cases similar, but there Jinja is similar to Python and in many cases behaves in a similar fashion, but
sometimes are differences (e.g. If you set some variable inside a loop, there are sometimes differences when, for example, you set some variable inside
the variable outside will not get changed. Unless you use a do statement). a loop: the variable outside will not get changed.
So you should take a look at the [jinja api documentation][jinja-tmp]. Instead, to get this behavior, you would use a `do` statement.
How you can use jinja to directly call salt functions and get data about So you should take a look at the [Jinja API documentation][jinja-tmp].
your system is documented in the [salt documentation][jinja-call-salt-functions]. Documentation about using Jinja to directly call Salt functions and get data
about your system can be found in the official
[Salt documentation][jinja-call-salt-functions].
## Salt configuration, Qubes OS layout ## Salt Configuration, QubesOS layout
All salt configuration in `/srv/` directory, as usual. The main directory is All Salt configuration files are in the `/srv/` directory, as usual.
`/srv/salt/` where all state files reside. States are contained in `*.sls` The main directory is `/srv/salt/` where all state files reside.
files. However the states that are part of standard Qubes distribution are States are contained in `*.sls` files.
mostly templates and the configuration is done in pillars from formulas. However, the states that are part of the standard Qubes distribution are mostly
templates and the configuration is done in pillars from formulas.
The formulas are in `/srv/formulas`, including stock formula for domains in The formulas are in `/srv/formulas`, including stock formulas for domains in
`/srv/formulas/dom0/virtual-machines-formula/qvm`, which are used by firstboot. `/srv/formulas/dom0/virtual-machines-formula/qvm`, which are used by firstboot.
Because we use some code that is not found in older versions of salt, there is Because we use some code that is not found in older versions of Salt, there is
a tool called `qubesctl` that should be run instead of `salt-call --local`. It a tool called `qubesctl` that should be run instead of `salt-call --local`.
accepts all arguments of the vanilla tool. It accepts all the same arguments of the vanilla tool.
## Configuring a VM's System from Dom0
## Configuring system inside of VMs Starting with Qubes R3.2, Salt in Qubes can be used to configure VMs from dom0.
Simply set the VM name as the target minion name in the top file.
Starting with Qubes 3.2, Salt in Qubes can be used to configure VMs. Salt You can also use the `qubes` pillar module to select VMs with a particular
formulas can be used normal way. Simply set VM name as target minion name in property (see below).
top file. You can also use `qubes` pillar module to select VMs with a If you do so, then you need to pass additional arguments to the `qubesctl` tool:
particular property (see below). Then you need to pass additional arguments to
`qubesctl` tool:
usage: qubesctl [-h] [--show-output] [--force-color] [--skip-dom0] usage: qubesctl [-h] [--show-output] [--force-color] [--skip-dom0]
[--targets TARGETS | --templates | --app | --all] [--targets TARGETS | --templates | --app | --all]
... ...
positional arguments: positional arguments:
command Salt command to execute (for example: state.highstate) command Salt command to execute (e.g., state.highstate)
optional arguments: optional arguments:
-h, --help show this help message and exit -h, --help show this help message and exit
--show-output Show output of management commands --show-output Show output of management commands
--force-color Force color output, allow control characters from VM, --force-color Force color output, allow control characters from VM,
UNSAFE UNSAFE
--skip-dom0 Skip dom0 condifuration (VM creation etc) --skip-dom0 Skip dom0 configuration (VM creation etc)
--targets TARGETS Coma separated list of VMs to target --targets TARGETS Coma separated list of VMs to target
--templates Target all templates --templates Target all templates
--app Target all AppVMs --app Target all AppVMs
--all Target all non-disposable VMs (TemplateVMs and AppVMs) --all Target all non-disposable VMs (TemplateVMs and AppVMs)
To apply the configuration to all the templates, call `qubesctl --templates To apply a state to all templates, call `qubesctl --templates state.highstate`.
state.highstate`.
Actual configuration is applied using `salt-ssh` (running over `qrexec` instead The actual configuration is applied using `salt-ssh` (running over `qrexec`
of `ssh`). Which means you don't need to install anything special in a VM you instead of `ssh`).
want to manage. Additionally for each target VM, `salt-ssh` is started from a Which means you don't need to install anything special in a VM you want to
temporary VM. This way dom0 doesn't directly interact with potentially manage.
malicious target VM. Additionally, for each target VM, `salt-ssh` is started from a temporary VM.
This way dom0 doesn't directly interact with potentially malicious target VMs;
and in the case of a compromised Salt VM, because they are temporary, the
compromise cannot spread from one VM to another.
## Writing your own configuration ## Writing Your Own Configurations
Let's start with quick example: Let's start with quick a example:
my new and shiny vm: my new and shiny VM:
qvm.present: qvm.present:
- name: salt-test # can be omitted when same as ID - name: salt-test # can be omitted when same as ID
- template: fedora-21 - template: fedora-21
@ -239,75 +254,82 @@ Let's start with quick example:
- flags: - flags:
- proxy - proxy
It uses Qubes-specific `qvm.present` state, which ensures that domain is It uses the Qubes-specific `qvm.present` state, which ensures that the domain is
created. The name should be `salt-test` (and not `my new and shiny vm`), present (if not, it creates it).
the rest are domains properties, same as in `qvm-prefs`. `proxy` flag informs
salt that the domain should be a ProxyVM.
This should be put in `/srv/salt/my-new-vm.sls` or another `.sls` file. Separate * The `name` flag informs Salt that the domain should be named `salt-test` (not
`*.top` file should be also written: `my new and shiny VM`).
* The `template` flag informs Salt which template should be used for the domain.
* The `label` flag informs Salt what color the domain should be.
* The `mem` flag informs Salt how much RAM should be allocated to the domain.
* The `vcpus` flag informs Salt how many Virtual CPUs should be allocated to the
domain
* The `proxy` flag informs Salt that the domain should be a ProxyVM.
As you will notice, the options are the same (or very similar) to those used in
`qvm-prefs`.
This should be put in `/srv/salt/my-new-vm.sls` or another `.sls` file.
A separate `*.top` file should be also written:
base: base:
dom0: dom0:
- my-new-vm - my-new-vm
The third line should contain the name of the previous file, without `.sls`. **Note** The third line should contain the name of the previous state file,
without the `.sls` extension.
To enable the particular top file you should issue command: To enable the particular top file you should issue the command:
qubesctl top.enable my-new-vm $ qubesctl top.enable my-new-vm
To actually apply the state: To apply the state:
qubesctl state.highstate $ qubesctl state.highstate
### Example of Configuring a VM's System from Dom0
### Example of VM system configuration Lets make sure that the `mc` package is installed in all templates.
Similar to the previous example, you need to create a state file
It is also possible to configure system inside the VM. Lets make sure that `mc` (`/srv/salt/mc-everywhere.sls`):
package is installed in all the templates. Similar to previous example, you
need to create state file (`/srv/salt/mc-everywhere.sls`):
mc: mc:
pkg.installed: [] pkg.installed: []
Then appropriate top file (`/srv/salt/mc-everywhere.top`): Then the appropriate top file (`/srv/salt/mc-everywhere.top`):
base: base:
qubes:type:template: qubes:type:template:
- match: pillar - match: pillar
- mc-everywhere - mc-everywhere
Now you need to enable the configuration: Now you need to enable the top file:
qubesctl top.enable mc-everywhere $ qubesctl top.enable mc-everywhere
And apply the configuration: And apply the configuration:
qubesctl --all state.highstate $ qubesctl --all state.highstate
## All Qubes-specific States
## All Qubes-specific states ### `qvm.present`
As in the example above, it creates a domain and sets its properties.
### qvm.present ### `qvm.prefs`
As in example above, it creates domain and sets its properties. You can set properties of an existing domain:
### qvm.prefs
You can set properties of existing domain:
my preferences: my preferences:
qvm.prefs: qvm.prefs:
- name: salt-test2 - name: salt-test2
- netvm: sys-firewall - netvm: sys-firewall
Note that `name:` is a matcher, ie. it says the domain which properties will be ***Note*** The `name:` option will not change the name of a domain, it will only
manipulated is called `salt-test2`. The implies that you currently cannot rename be used to match a domain to apply the configurations to it.
domains this way.
### qvm.service ### `qvm.service`
services in my domain: services in my domain:
qvm.service: qvm.service:
@ -321,83 +343,79 @@ domains this way.
- default: - default:
- service5 - service5
This enables, disables, or sets to default, the services as in qvm-service. This enables, disables, or sets to default, services as in `qvm-service`.
### qvm.running ### `qvm.running`
Ensures the specified domain is running:
Ensures the domain is running:
domain is running: domain is running:
qvm.running: qvm.running:
- name: salt-test4 - name: salt-test4
## qubes pillar module ## The `qubes` Pillar Module
Additional pillar data is available to ease targeting configuration (for Additional pillar data is available to ease targeting configurations (for
example all the templates). List here may be subject to changes in future example all templates).
releases. ***Note*** List here may be subject to changes in future releases.
### qubes:type ### `qubes:type`
VM type. Possible values: VM type. Possible values:
- `admin` - administration domain (`dom0`) - `admin` - Administration domain (`dom0`)
- `template` - Template VM - `template` - Template VM
- `standalone` - Standalone VM - `standalone` - Standalone VM
- `app` - template based AppVM - `app` - Template based AppVM
### qubes:template ### `qubes:template`
Template name on which given VM is based (if any). Template name on which a given VM is based (if any).
### qubes:netvm ### `qubes:netvm`
VM which provides network to the given VM VM which provides network to the given VM
## Debugging ## Debugging
The output for each vm is logged in `/var/log/qubes/mgmt-VM_NAME.log`.
If the log does not contain useful information, you can stop `qubesctl` by The output for each VM is logged in `/var/log/qubes/mgmt-VM_NAME.log`.
pressing `ctrl+z`.
You need to: If the log does not contain useful information:
1. run `sudo qubesctl --skip-dom0 --target=VM_NAME state.highstate` 1. Run `sudo qubesctl --skip-dom0 --target=VM_NAME state.highstate`
2. When your vm is being started (yellow) press Ctrl-Z on qubesctl. 2. When your VM is being started (yellow) press Ctrl-z on qubesctl.
3. Open terminal in disp-mgmt-VM_NAME. 3. Open terminal in disp-mgmt-VM_NAME.
4. Look at /etc/qubes-rpc/qubes.SaltLinuxVM - this is what is 4. Look at /etc/qubes-rpc/qubes.SaltLinuxVM - this is what is
executed in the management vm. executed in the management VM.
5. Get the last two lines: 5. Get the last two lines:
export PATH="/usr/lib/qubes-vm-connector/ssh-wrapper:$PATH" $ export PATH="/usr/lib/qubes-vm-connector/ssh-wrapper:$PATH"
salt-ssh "$target_vm" $salt_command $ salt-ssh "$target_vm" $salt_command
Adjust $target_vm (VM_NAME) and $salt_command (state.highstate). Adjust $target_vm (VM_NAME) and $salt_command (state.highstate).
6. Execute them, fix problems, repeat. 6. Execute them, fix problems, repeat.
## Known pitfalls ## Known Pitfalls
### Using fedora-24-minimal ### Using fedora-24-minimal
The fedora-24-minimal package is missing the sudo package.
The fedora-24-minimal package is missing the `sudo` package.
You can install it via: You can install it via:
qvm-run -p vmname 'dnf install -y sudo' $ qvm-run -p -u root fedora-24-minimal-template 'dnf install -y sudo'
The `-p` is will cause the execution to wait until the package is installed. The `-p` will cause the execution to wait until the package is installed.
This is important when using a state with `cmd.run`. Having the `-p` flag is important when using a state with `cmd.run`.
### Disk Quota Exceeded (When Installing Templates)
### Disk quota exceeded (when installing templates)
If you install multiple templates you may encounter this error. If you install multiple templates you may encounter this error.
The solution is to shut down the updatevm between each install. The solution is to shut down the updateVM between each install:
E.g.:
{% raw %} install template and shutdown updateVM:
install template and shutdown updatevm:
cmd.run: cmd.run:
- name: sudo qubes-dom0-update -y fedora-24; qvm-shutdown {{salt.cmd.run(qubes-prefs updatevm) }} - name: sudo qubes-dom0-update -y fedora-24; qvm-shutdown {{salt.cmd.run(qubes-prefs updateVM) }}
{% endraw %}
## Further reading ## Further Reading
* [Salt documentation][salt-doc] * [Salt documentation][salt-doc]
* [Salt states][salt-doc-states] ([files][salt-doc-states-file], [commands][salt-doc-states-cmd], * [Salt states][salt-doc-states] ([files][salt-doc-states-file], [commands][salt-doc-states-cmd],
@ -405,7 +423,7 @@ E.g.:
* [Top files][salt-doc-top] * [Top files][salt-doc-top]
* [Jinja templates][jinja] * [Jinja templates][jinja]
* [Qubes specific modules][salt-qvm-doc] * [Qubes specific modules][salt-qvm-doc]
* [Formula for default Qubes VMs][salt-virtual-machines-doc] ([and actual states][salt-virtual-machines-states]) * [Formulas for default Qubes VMs][salt-virtual-machines-doc] ([and actual states][salt-virtual-machines-states])
[salt-doc]: https://docs.saltstack.com/en/latest/ [salt-doc]: https://docs.saltstack.com/en/latest/
[salt-qvm-doc]: https://github.com/QubesOS/qubes-mgmt-salt-dom0-qvm/blob/master/README.rst [salt-qvm-doc]: https://github.com/QubesOS/qubes-mgmt-salt-dom0-qvm/blob/master/README.rst