alexander.schaeferdiek/system-helpers
1
0
Fork 0
Code Pull requests Activity Actions
system-helpers/README.md

3.9 KiB
Raw Blame History

README

system-helpers - Collection of helper systemd services, systemd timers and shell scripts for common tasks.

It can help you out for small servers where you think a proper monitoring and alerting system like Prometheus and AlertManager is too much.

The notifier script can send notifications via mail or gotify.

system-helpers collection provides the following helpers

  • notifier which sends notifications via mail or gotify (see see notifier)
  • checking if systemd services and timers are running (see man systemd_check)
  • borgmatic backup (see exposed systemd services borgmatic@ which require a valid borgmatic.yml file in /etc/borgmatic or your $HOME/.config/borgmatic)
  • restic backup (see /usr/share/doc/system-helpers/restic for an example mybackup)
  • checking if docker containers are running (see man docker_check)
  • batch docker-compose upgrades (see man docker_compose_update)
  • dynamic dns update (see man docker_compose_update)
  • memory and disk checks (see man disk_space_alert and man memory_usage_alert)
  • docker housekeeping for removing unused docker containers, volumes and images
  • update check for pacman (ArchLinux specific, see man check_updates)

Provided systemd services mostly support mail notifications if they have failed via systemd_failure_notify.

See /usr/share/doc/system-helpers for example configurations which should be copied inside $HOME or /etc depending on the helper script.

Installation

Package repository for Arch Linux

Use the custom provided ArchLinux repository hosted at aur.myservermanager.com

[repo-aur-myservermanager-com]
SigLevel = Never
Server = https://aur.myservermanager.com

Package build for Arch Linux

  1. Clone this repository
  2. Build a pacman package for ArchLinux via makepkg -csi inside the _pkg/ folder

system-helpers-sync script to synchronize files to locations

  1. Clone this repository
  2. Run ./system-helpers-sync and pick an action which should be self-explaining

Usage

Scripts make heavy use the bundled notifier script (see man notifier) which should be configured properly, e.g. add a /etc/notifier.conf:

NOTIFIER_MAIL_ADDRESS=monitoring@yourdomain.tld
NOTIFIER_MAIL_ENABLED=true
# Enable gotify by running gotify init first, you need a custom serverfor this
NOTIFIER_GOTIFY_ENABLED=false

Important: You can also change notifier behavior on a per-command basis, e.g. disable mail or gotify by providing ALL notifier variables as environment variables where the command is issued, for example in a systemd .service file.

# ...

[Service]
# ...
Environment="NOTIFIER_GOTIFY_ENABLED=false"
Environment="NOTIFIER_MAIL_ENABLED=true"
Environment="NOTIFIER_MAIL_ADDRESS=another-mail-different-from-your-global-config@domain.tld"

# ...

Otherwise, services and scripts don't depend on each other, but have hints that they can be combined (out-commented line in service files for systemd_failure_notify@.service for example).

Examples are given inside usr/share/doc/system-helpers and man pages are also available.

FAQ

User cannot use network-online.target

By default, no user unit can depend on system events like network-online.target. To enable this, you need to link the proper .target which you like to use inside the user service before.

systemctl --user link /usr/lib/systemd/system/network-online.target

This would make network-online.target available after a reboot to the user who issued the command.

network-online.target doesn't recognize DNS correctly

If you like to set Persistent=true, then you should probably equip the related .service file with a ExecStartPre=/bin/bash -c 'until host captiveportal.myservermanager.com; do sleep 1; done' or any domain you think is 100% reachable.