system-helpers/README.md

# README

system-helpers - Collection of helper systemd services, systemd timers and
shell scripts for common tasks.

It can help you out for small servers where you think a proper monitoring and
alerting system like [Prometheus](https://prometheus.io) and 
[AlertManager](https://prometheus.io/docs/alerting/latest/alertmanager/) is too much.

The **notifier** script can send notifications via mail or [gotify](https://gotify.net).

**system-helpers** collection provides the following helpers

- notifier which sends notifications via mail or gotify (see **see notifier**)
- checking if systemd services and timers are running (see **man systemd_check**)
- borgmatic backup (see exposed systemd services *borgmatic@* which require a valid *borgmatic.yml* file in */etc/borgmatic* or your **$HOME/.config/borgmatic**)
- restic backup (see */usr/share/doc/system-helpers/restic* for an example *mybackup*)
- checking if docker containers are running (see **man docker_check**)
- batch docker-compose upgrades (see **man docker_compose_update**)
- dynamic dns update (see **man docker_compose_update**)
- memory and disk checks (see **man disk_space_alert** and **man memory_usage_alert**)
- docker housekeeping for removing unused docker containers, volumes and images
- update check for pacman (ArchLinux specific, see **man check_updates**)

Provided systemd services mostly support mail notifications if they have failed
via `systemd_failure_notify`.

See `/usr/share/doc/system-helpers` for example configurations which should 
be copied inside $HOME or /etc depending on the helper script.

## Installation

### Package manager for Arch Linux

1. Build a `pacman` package for ArchLinux via `makepkg -csi` inside the `_pkg/` folder
2. Alternatively, use `./sync` from the git clone and pick an action which should be self-explaining
3. Use the custom provided ArchLinux repository hosted at `aur.myservermanager.com`

```shell
[repo-aur-myservermanager-com]
SigLevel = Never
Server = https://aur.myservermanager.com
```

### system-helpers-sync script to synchronize files to locations

1. Clone this repository
2. Run `./system-helpers-sync` and use the option you like to use

## Usage

Services and scripts don't depend on each other but have hints that they can be 
combined (out-commented line in service files for `systemd_failure_notify@.service` 
for example).

Examples are given inside `usr/share/doc/system-helpers`.

## FAQ

### User cannot use network-online.target

By default, no user unit can depend on system events like `network-online.target`. 
To enable this, you need to link the proper `.target` which you like to use inside
the user service before.

```shell
systemctl --user link /usr/lib/systemd/system/network-online.target
```

This would make `network-online.target` available after a _reboot_ to the user who issued the command.

### `network-online.target` doesn't recognize DNS correctly

If you like to set `Persistent=true`, then you should probably equip the related `.service` file with a `ExecStartPre=/bin/bash -c 'until host captiveportal.myservermanager.com; do sleep 1; done'` or any domain you think is 100% reachable.
Add script to notify about systemd service failures; add a README 2019-01-26 09:39:02 +00:00			`# README`

Make check updates more generic, fix failure notify requirements and update docs 2023-01-25 08:26:00 +00:00			`system-helpers - Collection of helper systemd services, systemd timers and`
Align examples 2023-01-25 22:31:14 +00:00			`shell scripts for common tasks.`
Make check updates more generic, fix failure notify requirements and update docs 2023-01-25 08:26:00 +00:00
Polish and align all scripts; add generic notifier script 2023-01-25 22:27:38 +00:00			`It can help you out for small servers where you think a proper monitoring and`
			`alerting system like [Prometheus](https://prometheus.io) and`
			`[AlertManager](https://prometheus.io/docs/alerting/latest/alertmanager/) is too much.`

			`The notifier script can send notifications via mail or [gotify](https://gotify.net).`

Make check updates more generic, fix failure notify requirements and update docs 2023-01-25 08:26:00 +00:00			`system-helpers collection provides the following helpers`

Polish and align all scripts; add generic notifier script 2023-01-25 22:27:38 +00:00			`- notifier which sends notifications via mail or gotify (see see notifier)`
Make check updates more generic, fix failure notify requirements and update docs 2023-01-25 08:26:00 +00:00			`- checking if systemd services and timers are running (see man systemd_check)`
			`- borgmatic backup (see exposed systemd services borgmatic@ which require a valid borgmatic.yml file in /etc/borgmatic or your $HOME/.config/borgmatic)`
			`- restic backup (see /usr/share/doc/system-helpers/restic for an example mybackup)`
			`- checking if docker containers are running (see man docker_check)`
			`- batch docker-compose upgrades (see man docker_compose_update)`
			`- dynamic dns update (see man docker_compose_update)`
			`- memory and disk checks (see man disk_space_alert and man memory_usage_alert)`
			`- docker housekeeping for removing unused docker containers, volumes and images`
			`- update check for pacman (ArchLinux specific, see man check_updates)`
Add man pages and separate properly between global systemd and user systemd services and timers 2022-06-13 09:31:59 +00:00
			`Provided systemd services mostly support mail notifications if they have failed`
Make check updates more generic, fix failure notify requirements and update docs 2023-01-25 08:26:00 +00:00			via `systemd_failure_notify`.
Add man pages and separate properly between global systemd and user systemd services and timers 2022-06-13 09:31:59 +00:00
Rename to system-helpers 2022-09-17 18:28:53 +00:00			See `/usr/share/doc/system-helpers` for example configurations which should
Add man pages and separate properly between global systemd and user systemd services and timers 2022-06-13 09:31:59 +00:00			`be copied inside $HOME or /etc depending on the helper script.`
Improve new restic scripts and remove resticwrapper, borgwrapper; provide a PKGBUILD for ArchLinux 2022-06-13 00:21:33 +00:00
			`## Installation`

Make check updates more generic, fix failure notify requirements and update docs 2023-01-25 08:26:00 +00:00			`### Package manager for Arch Linux`

Update README 2022-07-11 20:35:41 +00:00			1. Build a `pacman` package for ArchLinux via `makepkg -csi` inside the `_pkg/` folder
			2. Alternatively, use `./sync` from the git clone and pick an action which should be self-explaining
Add steps for creating ArchLinux repository database and add description how to use in the README 2022-07-11 20:29:51 +00:00			3. Use the custom provided ArchLinux repository hosted at `aur.myservermanager.com`

			```shell
			`[repo-aur-myservermanager-com]`
			`SigLevel = Never`
ArchLinux repository without any $arch 2022-07-11 20:49:31 +00:00			`Server = https://aur.myservermanager.com`
Add steps for creating ArchLinux repository database and add description how to use in the README 2022-07-11 20:29:51 +00:00			```
Improve new restic scripts and remove resticwrapper, borgwrapper; provide a PKGBUILD for ArchLinux 2022-06-13 00:21:33 +00:00
Make check updates more generic, fix failure notify requirements and update docs 2023-01-25 08:26:00 +00:00			`### system-helpers-sync script to synchronize files to locations`

			`1. Clone this repository`
			2. Run `./system-helpers-sync` and use the option you like to use

Improve new restic scripts and remove resticwrapper, borgwrapper; provide a PKGBUILD for ArchLinux 2022-06-13 00:21:33 +00:00			`## Usage`
Add script to notify about systemd service failures; add a README 2019-01-26 09:39:02 +00:00
Document user unit system targets 2021-12-27 10:24:05 +00:00			`Services and scripts don't depend on each other but have hints that they can be`
			combined (out-commented line in service files for `systemd_failure_notify@.service`
			`for example).`
Add script to notify about systemd service failures; add a README 2019-01-26 09:39:02 +00:00
Rename to system-helpers 2022-09-17 18:28:53 +00:00			Examples are given inside `usr/share/doc/system-helpers`.
Improve new restic scripts and remove resticwrapper, borgwrapper; provide a PKGBUILD for ArchLinux 2022-06-13 00:21:33 +00:00
			`## FAQ`
Document user unit system targets 2021-12-27 10:24:05 +00:00
Improve new restic scripts and remove resticwrapper, borgwrapper; provide a PKGBUILD for ArchLinux 2022-06-13 00:21:33 +00:00			`### User cannot use network-online.target`
Document user unit system targets 2021-12-27 10:24:05 +00:00
			By default, no user unit can depend on system events like `network-online.target`.
			To enable this, you need to link the proper `.target` which you like to use inside
			`the user service before.`

Improve new restic scripts and remove resticwrapper, borgwrapper; provide a PKGBUILD for ArchLinux 2022-06-13 00:21:33 +00:00			```shell
Document user unit system targets 2021-12-27 10:24:05 +00:00			`systemctl --user link /usr/lib/systemd/system/network-online.target`
			```

Add optional DNS check for network-online.target to user service files 2022-08-23 16:55:27 +00:00			This would make `network-online.target` available after a _reboot_ to the user who issued the command.

			### `network-online.target` doesn't recognize DNS correctly

			If you like to set `Persistent=true`, then you should probably equip the related `.service` file with a `ExecStartPre=/bin/bash -c 'until host captiveportal.myservermanager.com; do sleep 1; done'` or any domain you think is 100% reachable.