Alexander Schäferdiek
9c9dfee661
All checks were successful
continuous-integration/drone/push Build is passing
111 lines
No EOL
4.4 KiB
Markdown
111 lines
No EOL
4.4 KiB
Markdown
# README
|
|
|
|
system-helpers - Collection of helper systemd services, systemd timers and
|
|
shell scripts for common tasks.
|
|
|
|
It can help you out for small servers where you think a proper monitoring and
|
|
alerting system like [Prometheus](https://prometheus.io) and
|
|
[AlertManager](https://prometheus.io/docs/alerting/latest/alertmanager/) is too much.
|
|
|
|
The **notifier** script can send notifications via mail or [gotify](https://gotify.net).
|
|
|
|
**system-helpers** collection provides the following helpers
|
|
|
|
- notifier which sends notifications via mail or gotify (see **see notifier**)
|
|
- checking if systemd services and timers are running (see **man systemd_check**)
|
|
- borgmatic backup (see exposed systemd services *borgmatic@* which require a valid *borgmatic.yml* file in */etc/borgmatic* or your **$HOME/.config/borgmatic**)
|
|
- restic backup (see */usr/share/doc/system-helpers/restic* for an example *mybackup*)
|
|
- checking if docker containers are running (see **man docker_check**)
|
|
- batch docker-compose upgrades (see **man docker_compose_update**)
|
|
- dynamic dns update (see **man docker_compose_update**)
|
|
- memory and disk checks (see **man disk_space_alert** and **man memory_usage_alert**)
|
|
- docker housekeeping for removing unused docker containers, volumes and images
|
|
- update check for pacman (ArchLinux specific, see **man check_updates**)
|
|
|
|
Provided systemd services mostly support mail notifications if they have failed
|
|
via `systemd_failure_notify`.
|
|
|
|
See `/usr/share/doc/system-helpers` for example configurations which should
|
|
be copied inside $HOME or /etc depending on the helper script.
|
|
|
|
## Installation
|
|
|
|
### Package repository for Arch Linux
|
|
|
|
Use the custom provided ArchLinux repository hosted at `aur.myservermanager.com`
|
|
|
|
```shell
|
|
[repo-aur-myservermanager-com]
|
|
SigLevel = Never
|
|
Server = https://aur.myservermanager.com
|
|
```
|
|
|
|
### Package build for Arch Linux
|
|
|
|
1. Clone this repository
|
|
2. Build a `pacman` package for ArchLinux via `makepkg -csi` inside the `_pkg/` folder
|
|
|
|
### system-helpers-sync script to synchronize files to locations
|
|
|
|
1. Clone this repository
|
|
2. Run `./system-helpers-sync` and pick an action which should be self-explaining
|
|
|
|
## Usage
|
|
|
|
Services and scripts don't depend on each other, except for `notifier` which is commonly used. See more about `notifier` in the next section. All scripts include hints that they can be combined (out-commented line in service files for `systemd_failure_notify@.service` for example).
|
|
|
|
Examples are given inside `usr/share/doc/system-helpers` and *man pages* are also available.
|
|
|
|
### Making use of notifier
|
|
|
|
Scripts make heavy use the bundled **notifier** script (see `man notifier`) which should be configured properly, e.g. add a `/etc/notifier.conf`:
|
|
|
|
```shell
|
|
NOTIFIER_MAIL_ADDRESS=monitoring@yourdomain.tld
|
|
NOTIFIER_MAIL_ENABLED=true
|
|
# Enable gotify by running gotify init first, you need a custom serverfor this
|
|
NOTIFIER_GOTIFY_ENABLED=false
|
|
```
|
|
|
|
**Important:** You can also change notifier behavior on a per-command basis, e.g. disable mail or gotify by providing ALL notifier variables as environment variables where the command is issued, for example in a systemd .service file.
|
|
|
|
```shell
|
|
# ...
|
|
|
|
[Service]
|
|
# ...
|
|
Environment="NOTIFIER_GOTIFY_ENABLED=false"
|
|
Environment="NOTIFIER_MAIL_ENABLED=true"
|
|
Environment="NOTIFIER_MAIL_ADDRESS=another-mail-different-from-your-global-config@domain.tld"
|
|
|
|
# ...
|
|
```
|
|
|
|
Another option is to set those inside the respective config files of another script. Let's have an example with the `check_updates` script.
|
|
|
|
```shell
|
|
# /etc/check_updates.conf
|
|
NOTIFIER_GOTIFY_ENABLED=false
|
|
NOTIFIER_MAIL_ENABLED=true
|
|
NOTIFIER_MAIL_ADDRESS=updatemail@domain.tld
|
|
```
|
|
|
|
This will not send gotify messages for `check_updates` and send a mail to a different address `updatemail@domain.tld`.
|
|
|
|
## FAQ
|
|
|
|
### User cannot use network-online.target
|
|
|
|
By default, no user unit can depend on system events like `network-online.target`.
|
|
To enable this, you need to link the proper `.target` which you like to use inside
|
|
the user service before.
|
|
|
|
```shell
|
|
systemctl --user link /usr/lib/systemd/system/network-online.target
|
|
```
|
|
|
|
This would make `network-online.target` available after a _reboot_ to the user who issued the command.
|
|
|
|
### `network-online.target` doesn't recognize DNS correctly
|
|
|
|
If you like to set `Persistent=true`, then you should probably equip the related `.service` file with a `ExecStartPre=/bin/bash -c 'until host captiveportal.myservermanager.com; do sleep 1; done'` or any domain you think is 100% reachable. |