Compare commits

...

28 Commits

Author SHA1 Message Date
Daniele Verducci (Slimpenguin)
998315ef05 Match different server successful response, Fixed error log, Added date and time on logs 2024-01-11 07:59:01 +01:00
Daniele Verducci (Slimpenguin)
07e1126a58 Updated readme 2023-12-21 08:38:35 +01:00
Daniele Verducci (Slimpenguin)
a146b668f9 Added LCD description in main README 2023-12-21 08:30:52 +01:00
Daniele Verducci (Slimpenguin)
43a5402706 Refinements in readme 2023-12-21 08:16:41 +01:00
8efa0966df ESP32-LCD 2023-12-10 11:48:50 +01:00
Daniele Verducci (Slimpenguin)
cbc3331482 WIP ESP32-LCD: Very dirty but working code written at the LUG 2023-12-10 07:51:17 +01:00
Daniele Verducci (Slimpenguin)
099fc25352 Better shutdown on battery low regexp 2023-04-27 08:10:27 +02:00
Daniele Verducci (Slimpenguin)
6d47db391f Shutwodn on battery low command example 2023-04-26 08:34:44 +02:00
7d624f7a33 Merge branch 'master' of github.com:penguin86/selfhost-utils 2023-04-25 09:47:02 +02:00
8b5b2c9bda Managed connection error during current IP retrieval 2023-04-25 09:20:18 +02:00
Daniele Verducci
5e2e5ba080
Fix 2023-04-20 11:01:37 +02:00
Daniele Verducci (Slimpenguin)
52d195be09 Fixed typo in cpu fan speed check 2022-10-09 12:45:49 +02:00
Daniele Verducci (Slimpenguin)
0adfd32db1 Added UPS example configuration 2022-10-06 09:37:12 +02:00
Daniele Verducci (Slimpenguin)
d7042fb8f1 Cute icons 2022-06-01 07:58:29 +02:00
Daniele Verducci (Slimpenguin)
8317a99826 Completed mddclient 2022-06-01 07:45:19 +02:00
c05a6717e4 Mddclient: structure 2022-05-28 13:32:50 +02:00
a9f0b79fb9 Better updates check 2022-05-28 12:41:04 +02:00
Daniele Verducci (Slimpenguin)
9c1d598f64 Fixed security updates check 2022-05-20 09:05:55 +02:00
Daniele Verducci (Slimpenguin)
1b0cbfb27a Config: added apt security updates check 2022-05-06 08:57:41 +02:00
Daniele Verducci (Slimpenguin)
952741a4ec Typo 2022-04-16 08:30:31 +02:00
Daniele Verducci (Slimpenguin)
ff7c049cb2 Screenshot updated with new notification title 2022-04-16 08:28:16 +02:00
Daniele Verducci (Slimpenguin)
b93b5eb958 Updated readme, enabled by default universal checks 2022-04-16 00:15:49 +02:00
Daniele Verducci (Slimpenguin)
780b2ac5b3 Email titles with emojis 2022-04-16 00:06:21 +02:00
Daniele Verducci (Slimpenguin)
56c1e01856 Fixed dummy check 2022-04-15 08:59:10 +02:00
Daniele Verducci (Slimpenguin)
cd146018de Notification policy 2022-04-15 08:23:40 +02:00
Daniele Verducci
e24383728a
Updated global readme 2022-04-09 22:48:28 +02:00
Daniele Verducci
7f6fa1d0fa
Updated healthcheck readme 2022-04-09 22:47:39 +02:00
Daniele Verducci (Slimpenguin)
af9cbbf393 Added two checks, better error reporting 2022-04-07 08:50:54 +02:00
15 changed files with 849 additions and 40 deletions

1
.gitignore vendored
View File

@ -1 +1,2 @@
*.cfg
healthcheck/healthcheck-virtualenv

View File

@ -1,17 +1,37 @@
# Selfhost utilities
A collection of utilities for self hosters.
Every utility is in a folder with its relevant configuration and is completely separated from the other, so you can install only the ones you need.
## HEALTHCHECK
## 🚨 HEALTHCHECK
A simple server health check.
Allows to keep under control the machine vitals (cpu usage, raid status, thermals...) and alert the sysadmin in case of anomalies.
Sends an email and/or executes a command in case of alarm (high temperature, RAID disk failed etc...).
As an example, the command may be a ntfy call to obtain a notification on a mobile phone or desktop computer.
Meant to be run with a cron (see healthcheck.cron.example).
Tested on Debian 11, but should run on almost any standard linux box.
![Email](images/healthcheck_email_notification.png) ![Ntfy](images/healthcheck_ntfy_notification.png)
![Email](images/healthcheck_email_notification.png)
![Ntfy](images/healthcheck_ntfy_notification.png)
Please see [healthcheck documentation](healthcheck/README.md)
## 🖥 MDDCLIENT
A DynDns2 client supporting multiple domains with individual API calls. Developed to allow updating multiple (sub)domains on Infomaniak dynamic DNS, that supports only one domain per request. Works with any provider supporting DynDns2 protocol.
Please see [mddclient documentation](mddclient/README.md)
## 📟 ESP32-LCD
A status LCD for your homelab! Low cost (about 20€ in parts), simple to build (no circuit board, no components, only an LCD, a ESP32 and, if needed, a potentiometer). Connects to your local wifi and receives HTTP requests from your homelab machines, and shows them to the screen.
![ESP32-LCD prototype](images/esp32-lcd.jpg)
Please see [ESP32-LCD documentation](esp32-lcd/README.md)
# License
This whole repository is released under GNU General Public License version 3: see http://www.gnu.org/licenses/

56
esp32-lcd/README.md Normal file
View File

@ -0,0 +1,56 @@
# ESP32-LCD
A status LCD for your homelab.
![ESP32-LCD prototype](../images/esp32-lcd.jpg)
## BOM
- Any HD44780-based character LCD display, up to 20x4 characters. The most common are 16x2 characters (16 characters per row, 2 rows).
- An ESP-32 board
- A potentiometer, usually a 10 or 100k one, for controlling contrast (a couple of resistances arranged as a voltage divider may also be fine)
## Assembly
Connect the LCD to the board:
**LCD Pin -> ESP32 Pin**
- PIN01-VSS -> GND
- PIN02-VDD -> 5V
- PIN03 V0 -> 10K Pot (Middle pin)
- PIN04 RS -> GPIO19
- PIN05 RW -> GND
- PIN06 E -> GPIO23
- PIN07 D0 -> NOT USED
- PIN08 D1 -> NOT USED
- PIN09 D2 -> NOT USED
- PIN10 D3 -> NOT USED
- PIN11 D4 -> GPIO18
- PIN12 D5 -> GPIO17
- PIN13 D6 -> GPIO16
- PIN14 D7 -> GPIO15
- PIN15 A -> 5V
- PIN16 K -> GND
Connect the potentiometer lateral pins to VCC and GND. Use the potentiometer to set the screen contrast.
Open config.h file and set display size and your wifi access data.
Flash the code to the ESP32. If you use the Arduino ide to do it, just open the esp32-lcd.ino file with the Arduino ide and follow [this instructions](https://randomnerdtutorials.com/getting-started-with-esp32/)
Restart the ESP32. The display shows "Conn to wifi..." with the WIFI name in the second line (if using a two or more lines display) and then will show the IP address.
## Use
- Turn on the circuit, wait for connection and note down the IP address shown on the screen.
- Make a GET request to the same IP address with a parameter "message" containing some text
> Example: to make the request using CURL from command line, try something along this lines (replace the IP addr with the one shown in the display):
> `curl -G http://192.168.1.78 --data-urlencode "message=Something interesting happened!"`
## Troubleshooting
The ESP32 logs are written in the serial monitor at 115200 baud. Just open the Arduino ide Serial Monitor from Tools menu and look at the logs.
If the screen is supplied with power but not initialized (maybe due to bad contacts or non working esp32 firmware), it should show some black blocks on the first line. If you canot see those (nor any other text), first of all check the contrast using the potentiometer.

2
esp32-lcd/esp32-lcd/.gitignore vendored Normal file
View File

@ -0,0 +1,2 @@
config.h

View File

@ -0,0 +1,15 @@
/**
CONFIGURATION FILE
Change the values in this file and rename it "config.h" before uploading the code to the board.
*/
#ifndef CONFIG_H
#define CONFIG_H
const char* WIFI_SSID = "Squicky";
const char* WIFI_PASSWORD = "SediaChinita@Terrazzo2017";
const unsigned int DISPLAY_WIDTH = 16;
const unsigned int DISPLAY_HEIGHT = 2;
#endif

View File

@ -0,0 +1,129 @@
#include <WiFi.h>
#include <WiFiClient.h>
#include <WebServer.h>
#include <ESPmDNS.h>
#include <LiquidCrystal.h>
#include "config.h"
// ------- Configuration is in config.h file -------
const int WEBSERVER_PORT = 80;
const char* WEBSERVER_MESSAGE_PARAM = "message";
/*
LCD Pin >ESP32 Pins
PIN01-VSS -> GND
PIN02-VDD -> 5V
PIN03 V0-> 10K Pot (Middle pin)
PIN04 RS-> GPIO19
PIN05 RW-> GND
PIN06 E -> GPIO23
PIN07 D0-> NOT USED
PIN08 D1-> NOT USED
PIN09 D2-> NOT USED
PIN10 D3-> NOT USED
PIN11 D4-> GPIO18
PIN12 D5-> GPIO17
PIN13 D6-> GPIO16
PIN14 D7-> GPIO15
PIN15 A-> 5V
PIN16 K-> GND
*/
WebServer server(WEBSERVER_PORT); // Server on port 80
LiquidCrystal lcd(19, 23, 18, 17, 16, 15);
const int led = 13;
void lcdPrintMultilineMessage(String message) {
lcd.clear();
int startFrom = 0;
for (int i=0; i<DISPLAY_HEIGHT; i++) {
lcd.setCursor(0,i);
lcd.print(
message.substring(startFrom, startFrom + DISPLAY_WIDTH)
);
startFrom += DISPLAY_WIDTH;
}
}
void handleRoot() {
digitalWrite(led, 1);
lcdPrintMultilineMessage(server.arg(WEBSERVER_MESSAGE_PARAM));
server.send(200, "text/plain", "ok");
digitalWrite(led, 0);
}
void handleNotFound() {
digitalWrite(led, 1);
String message = "File Not Found\n\n";
message += "URI: ";
message += server.uri();
message += "\nMethod: ";
message += (server.method() == HTTP_GET) ? "GET" : "POST";
message += "\nArguments: ";
message += server.args();
message += "\n";
for (uint8_t i = 0; i < server.args(); i++) {
message += " " + server.argName(i) + ": " + server.arg(i) + "\n";
}
server.send(404, "text/plain", message);
digitalWrite(led, 0);
}
void setup(void) {
// LCD: set up
lcd.begin(DISPLAY_WIDTH, DISPLAY_HEIGHT);
// SERIAL: set up
Serial.begin(115200);
Serial.println("");
// STATUS LED: set up
pinMode(led, OUTPUT);
digitalWrite(led, 0);
// LCD: Show SSID
lcd.print("Conn to wifi...");
lcd.setCursor(0,1);
lcd.print(WIFI_SSID);
// Connect to wifi
WiFi.mode(WIFI_STA);
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
// Wait for connection
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.print("Connected to ");
Serial.println(WIFI_SSID);
Serial.print("IP address: ");
Serial.println(WiFi.localIP());
// Print IP addr to LCD
lcd.clear();
lcd.print("Connected! IP:");
lcd.setCursor(0,1);
lcd.print(WiFi.localIP());
if (MDNS.begin("esp32")) {
Serial.println("MDNS responder started");
}
server.on("/", handleRoot);
server.onNotFound(handleNotFound);
server.begin();
Serial.println("HTTP server started");
}
void loop(void) {
server.handleClient();
delay(2);//allow the cpu to switch to other tasks
}

View File

@ -1,11 +1,15 @@
# HEALTHCHECK
# 🚨 HEALTHCHECK
A simple server health check.
Allows to keep under control the machine vitals (cpu usage, raid status, thermals...) and alert the sysadmin in case of anomalies.
Sends an email and/or executes a command in case of alarm.
As an example, the command may be a ntfy call to obtain a notification on a mobile phone or desktop computer.
Meant to be run with a cron (see healthcheck.cron.example).
Tested on Debian 11, but should run on almost any standard linux box.
![Email](../images/healthcheck_email_notification.png) ![Ntfy](../images/healthcheck_ntfy_notification.png)
![Email](../images/healthcheck_email_notification.png)
![Ntfy](../images/healthcheck_ntfy_notification.png)
## Alarms
Provided ready-to-use alarms in config file:
@ -40,9 +44,13 @@ Copy the script and the config file into the system to check:
cp healthcheck.py /usr/local/bin/healthcheck.py
cp healthcheck.cfg.example /usr/local/etc/healthcheck.cfg
```
Make the script executable:
```
chmod +x /usr/local/bin/healthcheck.py
```
Edit `/usr/local/etc/healthcheck.cfg` enabling the checks you need and configuring email settings.
Run `/usr/local/bin/healthcheck.py /usr/local/etc/healthcheck.cfg` to check it is working. If needed, change the config to make a check fail and see if the notification mail is delivered. If you need to do some testing without spamming emails, run with the parameter `--dry-run`.
Now copy the cron file:
Now copy the cron file (it runs healthcheck every minute):
```
cp healthcheck.cron.example /etc/cron.d/healthcheck
```

View File

@ -1,3 +1,5 @@
# The DEFAULT section contains the global configuration applied to all checks.
# You can re-define this variables in a check to override the global one.
[DEFAULT]
#### EMAIL NOTIFICATIONS ####
@ -39,10 +41,27 @@ MAILTO=root@localhost, user@localhost
#ALARM_COMMAND=curl -H "%%CHECKNAME%% alarm on %%HOSTNAME%%" -d "%%ERROR%% on %%DATETIME%%" ntfy.sh/my-unique-topic-name
#### NOTIFICATION POLICY ###
# Defines when to send the email and/or execute ALARM_COMMAND. Useful to avoid email flooding.
# Possible values:
# EVERY_RUN In case of alarm, sends a mail every time the script is run
# START Sends a mail only when an alarm starts
# ONCE_IN_MINUTES In case of alarm, resends a mail only if NOTIFY_MINUTES has passed
NOTIFY=EVERY_RUN
# Used only if NOTIFY=ONCE_IN_MINUTES. A mail is sent only if NOTIFY_MINUTES has passed from the previous one
NOTIFY_MINUTES=60
# Sends a mail when the alarm has ended
NOTIFY_ALARM_END=TRUE
#### HEALTH CHECKS ####
# Every health check is based on a command being executed, its result being parsed with a regexp
# to extract (as a single group) the numeric or string value, and the value being compared with
# a configured value. This checks are ready to be used, just enable the ones you need.
#
# CUSTOM CHECKS:
# You can add your own custom check declaring another section like this:
#
# [my_custom_check_name]
@ -55,28 +74,37 @@ MAILTO=root@localhost, user@localhost
# ALARM_VALUE_LESS_THAN=12
# COMMAND=/my/custom/binary --with parameters
# REGEXP=my regex to parse (awesome|disappointing) command output
#
# First test your custom command executing it in the command line
# Take the text output and write a regex to match it. Check every case:
# success result, error result, command failure. Then paste the command
# and regex in this config, enable the check and run to verify is working.
[system_load_1min]
# The system load average in the last minute
DISABLED=True
DISABLED=False
ALARM_VALUE_MORE_THAN=1.0
COMMAND=uptime
REGEXP=.*load average: (\d+[,.]\d+), \d+[,.]\d+, \d+[,.]\d+
[system_load_5min]
# The system load average in the last 5 minutes
DISABLED=True
DISABLED=False
ALARM_VALUE_MORE_THAN=1.0
COMMAND=uptime
REGEXP=.*load average: \d+[,.]\d+, (\d+[,.]\d+), \d+[,.]\d+
[system_load_15min]
# The system load average in the last 15 minutes
DISABLED=True
DISABLED=False
ALARM_VALUE_MORE_THAN=1.0
COMMAND=uptime
REGEXP=.*load average: \d+[,.]\d+, \d+[,.]\d+, (\d+[,.]\d+)
[used_disk_space]
# Used disk space (in percent, i.e. ALARM_VALUE_MORE_THAN=75 -> alarm if disk is more than 75% full)
DISABLED=True
@ -84,6 +112,7 @@ ALARM_VALUE_MORE_THAN=75
COMMAND=df -h /dev/sda1
REGEXP=(\d{1,3})%
[raid_status]
# Issues an alarm when the raid is corrupted
# Checks this part of the /proc/mdstat file:
@ -95,6 +124,7 @@ ALARM_STRING_NOT_EQUAL=UU
COMMAND=cat /proc/mdstat
REGEXP=.*\] \[([U_]+)\]\n
[battery_level]
# Issues an alarm when battery is discharging below a certain level (long blackout, pulled power cord...)
# For laptops used as servers, apparently common among the self hosters. Requires acpi package installed.
@ -104,6 +134,7 @@ COMMAND=acpi -b
REGEXP=Battery \d: .*, (\d{1,3})%
ALARM_VALUE_LESS_THAN=90
[laptop_charger_disconnected]
# Issues an alarm when laptop charger is disconnected
# For laptops used as servers, apparently common among the self hosters. Requires acpi package installed.
@ -112,20 +143,29 @@ COMMAND=acpi -a
REGEXP=Adapter \d: (.+)
ALARM_STRING_EQUAL=off-line
[free_ram]
# Free ram in %
# Shows another approach: does all the computation in the command and picks up
# all the output (by not declaring a regexp).
[shutdown_on_battery_low]
# For laptops used as a a server. Requires acpi package installed.
# When the battery is low, shuts down cleanly the system instead of waiting for it
# to shut down itself leaving all filesystems dirty.
# ALARM_COMMAND is the command executed when this check fails. Shuts down the system in
# 15 mins to allow for logging in and cancel the command. If you want to shut down
# immediately, replace the ALARM_COMMAND with "shutdown now".
# To cancel the shutdown, log in and "shutdown -c".
DISABLED=True
COMMAND=free | grep Mem | awk '{print int($4/$2 * 100.0)}'
ALARM_VALUE_LESS_THAN=20
COMMAND=acpi -b
REGEXP=REGEXP=Battery \d: Discharging, (\d{1,3})%
ALARM_VALUE_LESS_THAN=50
ALARM_COMMAND=shutdown +15 "Shutdown in 15 mins due to battery low!"
[available_ram]
# Like Free ram, but shows available instead of free. You may want to use this if you use a memcache.
DISABLED=True
# Shows available ram in %.
DISABLED=False
COMMAND=free | grep Mem | awk '{print int($7/$2 * 100.0)}'
ALARM_VALUE_LESS_THAN=20
[cpu_temperature]
# CPU Temperature alarm: requires lm-sensors installed and configured (check your distribution's guide)
# The regexp must be adapted to your configuration: run `sensors` in the command line
@ -136,6 +176,7 @@ ALARM_VALUE_MORE_THAN=80
COMMAND=sensors
REGEXP=Core 0: +\+?(-?\d{1,3}).\d°[CF]
[fan_speed]
# Fan speed alarm: requires lm-sensors installed and configured (check your distribution's guide)
# The regexp must be adapted to your configuration: run `sensors` in the command line
@ -143,4 +184,80 @@ REGEXP=Core 0: +\+?(-?\d{1,3}).\d°[CF]
DISABLED=True
ALARM_VALUE_LESS_THAN=300
COMMAND=sensors
REGEXP=cpu_fan: +(\d) RPM
REGEXP=cpu_fan: +(\d+) RPM
[host_reachability]
# Check if a remote host is alive with Ping. You can replace the ip with a domain name (e.g. COMMAND=ping debian.org -c 1)
#
# Shows another approach: uses the return value to print a string. Leverages ping's ability to return different error codes:
# 0 = success
# 1 = the host is unreachable
# 2 = an error has occurred (and will be logged to stderr)
# We are throwing away stdout and replacing it with a custom text.
# If there is a different text (the stderr), something bad happened, and it will be reported in the mail.
DISABLED=True
ALARM_STRING_NOT_EQUAL=Online
COMMAND=ping 192.168.1.123 -c 1 > /dev/null && echo "Online" || echo "Offline"
[service_webserver]
# Check if a webserver is running on port 80. You can replace the ip with a domain name.
# You can check different services changing the port number. Some examples:
# 80 HTTP Webserver
# 443 HTTPS Webserver
# 21 FTP
# 22 SSH
# 5900 VNC (Linux remote desktop)
# 3389 RDP (Windows remote desktop)
DISABLED=True
ALARM_STRING_NOT_EQUAL=Online
COMMAND=nc -z -w 3 192.168.1.123 80 > /dev/null && echo "Online" || echo "Offline"
[dummy_always_alarm]
# A dummy check that is always in alarm. Useful for testing notifications.
DISABLED=True
ALARM_STRING_EQUAL=Core meltdown!
COMMAND=echo "Core meltdown!"
[security_updates_available]
# Checks for security updates via apt (works on Debian and derivatives, like Ubuntu).
# Needs the repositories to be updated with `apt update`, but is an heavy command, so it may
# be configured to be executed daily in a command in the same cron of healthcheck.
# E.g.: place this string in /etc/cron.d/healthcheck, before the healthcheck command:
# 1 1 * * * root apt update
DISABLED=True
ALARM_STRING_EQUAL=security updates available
REGEXP=(security updates available|NO security updates available)
COMMAND=apt list --upgradable 2>/dev/null | grep -e "-security" && echo "security updates available" || echo "NO security updates available"
NOTIFY=START
[ups_power]
# Raises an alarm when UPS runs on battery.
# Requires NUT installed and configured on the system
# See complete documentation and support lists: https://networkupstools.org
# See simple start-up guide for Debian: https://wiki.debian.org/nut
# This config is for usbhid-ups driver. If you use a different driver, you may need
# to change the REGEXP to fit your output.
DISABLED=True
ALARM_STRING_NOT_EQUAL=OL
COMMAND=upsc eaton1600 2> /dev/null
REGEXP=^ups\.status: (OL|OB)$
NOTIFY=START
[ups_battery]
# Raises an alarm when UPS battery is discharged below 50%.
# Requires NUT installed and configured on the system
# See complete documentation and support lists: https://networkupstools.org
# See simple start-up guide for Debian: https://wiki.debian.org/nut
# This config is for usbhid-ups driver. If you use a different driver, you may need
# to change the REGEXP to fit your output.
DISABLED=True
ALARM_VALUE_LESS_THAN=50
COMMAND=upsc eaton1600 2> /dev/null
REGEXP=^battery\.charge: (\d{1,3})$
NOTIFY=ONCE_IN_MINUTES
NOTIFY_MINUTES=15

View File

@ -46,13 +46,18 @@ import socket
import getpass
import re
import locale
import json
NAME = 'healthcheck'
VERSION = '0.1'
DESCRIPTION = 'A simple server monitoring software'
EMAIL_SUBJECT_TPL = 'Host {} failed health check for {}'
EMAIL_MESSAGE_TPL = 'Alarm for sensor {} on host {} on {}: {}'
EMAIL_START_SUBJECT_TPL = '\U0001F6A8 {}: {} ALARM!'
EMAIL_START_MESSAGE_TPL = 'Alarm for sensor {} on host {} on {}: {}'
EMAIL_END_SUBJECT_TPL = '\u2705 {}: {} OK'
EMAIL_END_MESSAGE_TPL = 'Alarm ceased for sensor {} on host {} on {}'
# Healthcheck saves the current status (alarms triggered, last run... in this file)
STATUS_FILE = '/tmp/healthcheck.tmp'
class Main:
@ -62,7 +67,7 @@ class Main:
systemLocale = os.getenv('LANG')
if not systemLocale:
raise ValueError('System environment variabile $LANG is not set!')
locale.setlocale(locale.LC_ALL, systemLocale)
''' Reads the config '''
@ -78,6 +83,10 @@ class Main:
def run(self, dryRun):
''' Runs the health checks '''
# Load status
status = Status()
# Run checks based o the config
for section in self.config:
if section == 'DEFAULT':
continue
@ -85,6 +94,7 @@ class Main:
s = Settings(section, self.config)
if s.disabled:
self._log.info('Ignoring disabled check "{}"'.format(section))
status.unsetAlarm(section)
continue
self._log.info('Checking "{}"'.format(section))
@ -93,11 +103,37 @@ class Main:
if error:
# Alarm!
logging.warning('Alarm for {}: {}!'.format(section, error))
if not dryRun:
if s.mailto:
self.sendMail(s, error)
if s.alarmCommand:
self.executeAlarmCommand(s, error)
if self.shouldNotify(section, s, status):
status.setAlarm(section)
if not dryRun:
if s.mailto:
self.sendAlmStartMail(s, error)
if s.alarmCommand:
self.executeAlarmCommand(s, error)
elif status.getAlarmTriggeredTimestamp(section) is not None:
logging.info('Alarm ceased for {}: OK!'.format(section))
if s.notify_alarm_end and not dryRun and s.mailto:
self.sendAlmEndMail(s)
status.unsetAlarm(section)
# Save updated status
status.save()
def shouldNotify(self, section, settings, status):
almTriggeredTime = status.getAlarmTriggeredTimestamp(section)
# Notify if alarm just started
if almTriggeredTime is None:
return True
# Notify if NOTIFY=EVERY_RUN
if settings.notify == 'EVERY_RUN':
return True
# Notify if time elapsed
if settings.notify == 'ONCE_IN_MINUTES' and (time.time() - almTriggeredTime) > (settings.notify_minutes * 60):
return True
return False
# Calls the provided command, checks the value parsing it with the provided regexp
# and returns an error string, or null if the value is within its limits
@ -107,18 +143,21 @@ class Main:
return "bad config: COMMAND is mandatory"
if not config.regexp:
return "bad config: REGEXP is mandatory"
# Run command
stdout = ""
ret = subprocess.run(config.command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
if ret.stderr:
self._log.info('{} subprocess stderr:\n{}', config.command, ret.stderr.decode())
self._log.info('{} subprocess stderr:\n{}'.format(config.command, ret.stderr.decode()))
if ret.stdout:
stdout = ret.stdout.decode()
self._log.debug('{} subprocess stdout:\n{}', config.command, stdout)
self._log.debug('{} subprocess stdout:\n{}'.format(config.command, stdout))
if ret.returncode != 0:
return 'subprocess {} exited with error code {}'.format(config.command, ret.returncode)
return 'the command exited with error code {} {}'.format(
ret.returncode,
'and error message "{}"'.format(ret.stderr.decode().strip()) if ret.stderr else ''
)
# Parse result with regex
match = re.search(config.regexp, stdout, re.MULTILINE)
if not match:
@ -143,15 +182,34 @@ class Main:
if config.alarm_value_less_than and locale.atof(detectedValue) < float(config.alarm_value_less_than):
return 'value is {}, but should be greater than {}'.format(locale.atof(detectedValue), config.alarm_value_less_than)
def sendMail(self, s, error):
def sendAlmStartMail(self, s, error):
subject = EMAIL_START_SUBJECT_TPL.format(self.hostname, s.name)
body = EMAIL_START_MESSAGE_TPL.format(
s.name,
self.hostname,
time.strftime("%a, %d %b %Y %H:%M:%S"),
error
)
self.sendMail(s, subject, body)
def sendAlmEndMail(self, s):
subject = EMAIL_END_SUBJECT_TPL.format(self.hostname, s.name)
body = EMAIL_END_MESSAGE_TPL.format(
s.name,
self.hostname,
time.strftime("%a, %d %b %Y %H:%M:%S")
)
self.sendMail(s, subject, body)
def sendMail(self, s, subject, body):
if s.smtphost:
logging.info("Sending alarm email to %s via %s", s.mailto, s.smtphost)
logging.info("Sending email to %s via %s", s.mailto, s.smtphost)
else:
logging.info("Sending alarm email to %s using local smtp", s.mailto)
logging.info("Sending email to %s using local smtp", s.mailto)
# Create main message
msg = MIMEMultipart()
msg['Subject'] = EMAIL_SUBJECT_TPL.format(self.hostname, s.name)
msg['Subject'] = subject
if s.mailfrom:
m_from = s.mailfrom
else:
@ -161,12 +219,6 @@ class Main:
msg.preamble = 'This is a multi-part message in MIME format.'
# Add base text
body = EMAIL_MESSAGE_TPL.format(
s.name,
self.hostname,
time.strftime("%a, %d %b %Y %H:%M:%S"),
error
)
txt = MIMEText(body)
msg.attach(txt)
@ -204,6 +256,34 @@ class Main:
self._log.error('subprocess {} exited with error code {}'.format(cmdToRun, ret.returncode))
class Status:
''' Represents the current status (alarms triggered, last run...) '''
def __init__(self):
try:
with open(STATUS_FILE, 'r') as openfile:
self.status = json.load(openfile)
except FileNotFoundError:
self.status = {
'lastRun': 0, # unix time in seconds
'alarms': {}, # key-value, alarmName : alarmTriggeredTimestamp
}
def save(self):
self.status['lastRun'] = time.time()
jo = json.dumps(self.status)
with open(STATUS_FILE, "w") as outfile:
outfile.write(jo)
def setAlarm(self, almName):
self.status['alarms'][almName] = time.time()
def unsetAlarm(self, almName):
self.status['alarms'].pop(almName, None)
def getAlarmTriggeredTimestamp(self, almName):
return self.status['alarms'].get(almName, None)
class Settings:
''' Represents settings for a check '''
@ -241,6 +321,10 @@ class Settings:
self.alarm_value_not_equal = self.getStr(name, 'ALARM_VALUE_NOT_EQUAL', None)
self.alarm_value_more_than = self.getStr(name, 'ALARM_VALUE_MORE_THAN', None)
self.alarm_value_less_than = self.getStr(name, 'ALARM_VALUE_LESS_THAN', None)
## Notification policy
self.notify = self.getEnum(name, 'NOTIFY', 'EVERY_RUN', ['EVERY_RUN', 'START', 'ONCE_IN_MINUTES'])
self.notify_minutes = self.getInt(name, 'NOTIFY_MINUTES', 0)
self.notify_alarm_end = self.getBoolean(name, 'NOTIFY_ALARM_END', True)
## Command to obtain the value for comparation
self.command = self.getStr(name, 'COMMAND', None)
## Regexp to extract value from command output (default to match full string)
@ -252,12 +336,20 @@ class Settings:
except configparser.NoOptionError:
return defaultValue
def getInt(self, name, key, defaultValue):
return int(self.getStr(name, key, defaultValue))
def getBoolean(self, name, key, defaultValue):
try:
return self.config.getboolean(name, key)
except configparser.NoOptionError:
return defaultValue
def getEnum(self, name, key, defaultValue, values):
val = self.getStr(name, key, defaultValue)
if not val in values:
raise ValueError("Invalid value {} for configuration {}: expected one of {}".format(val, key, ', '.join(values)))
return val
if __name__ == '__main__':

BIN
images/esp32-lcd.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 39 KiB

After

Width:  |  Height:  |  Size: 51 KiB

53
mddclient/README.md Normal file
View File

@ -0,0 +1,53 @@
# 🖥 MDDCLIENT
It's a client for `dyndns2` protocol: it is used to interact with dynamic DNS services like Dyn or Infomaniak's.
Updates the IP assigned for a domain, allows the use of dynamic ip internet access (commin in residential contexts like DSL, Fiber, 5G...) for hosting services.
Like [ddclient](https://github.com/ddclient/ddclient), but updates multiple domaing with multiple REST calls.
I developed this because [Infomaniak's DynDns APIS](https://www.infomaniak.com/en/support/faq/2376/dyndns-updating-a-dynamic-dns-via-the-api) doesn't support multiple domains updates on the same call. So, even if ddclient was supported for a single domain, in case of multiple (sub)domains it was needed to setup multiple instances to update them all on the same machine.
## Compatibility
Works for any provider supporting `dyndns2` protocol. Was implemented following (this documentation)[https://help.dyn.com/remote-access-api/].
I wrote this to solve the multiple domain update problem on Infomainak.
## Use case
Let's say we have our Nextcloud instance on `https://mysite.cloud`. As self hosters, we run this instance on our server at home, behind our fiber or DSL provider with a dynamic IP. We need to use ddclient to keep the dynamic DNS updated, so the requests to `https://mysite.cloud` are sent to our current IP, that may change in any moment.
Now we decide to host a Matrix chat instance on `https://matrix.mysite.cloud` and a Mastodon social instance on `https://mastodon.mysite.cloud`. We configure ddclient adding the new subdomains in `/etc/ddclient.conf`, but the requests begin to fail with a 400 http code. This happens because our Dynamic DNS provider doesn't support multiple updates on the same request.
So we need a script that issues an update request for every (sub)domain. This is mddclient.
## Setup
Copy the script and the config file into the system to check:
```
cp mddclient.py /usr/local/bin/mddclient.py
cp mddclient.cfg.example /usr/local/etc/mddclient.cfg
```
Make the script executable:
```
chmod +x /usr/local/bin/mddclient.py
```
Edit `/usr/local/etc/mddclient.cfg` setting up the server url, username, password and the domains to update. If you have multiple domains or subdomains running on the same fiber/dsl, configure them, one for subsection as shown in the config example.
Run `/usr/local/bin/mddclient.py /usr/local/etc/mddclient.cfg` to check it is working.
Now copy the cron file (it runs mddclient every 5 minutes):
```
cp mddclient.cron.example /etc/cron.d/mddclient
```
For increased safety, edit the cron file placing your email address in MAILTO var to be notified in case of mddclient.py catastrophic failure.
Setup is now complete: the cron runs the script every five minutes and updates the dns.
## Check status
Some status informations are available with the -s flag:
```
root@myhost# /usr/local/bin/mddclient.py /usr/local/etc/mddclient.cfg -s
lastRun: 2022-05-31 10:40:17.006371
lastRunSuccess: True
lastUpdate: 2022-05-31 10:23:38.510386
lastIpAddr: 151.41.52.133
```
## Optimization
As the update requests are subject to rate limit, the script checks the current IP against Dyn's checkip tool and updates only when necessary. To force an update, use the -f flag.
## Thanks
Thanks to `dyndns.org` for the (checkip)[https://help.dyn.com/remote-access-api/checkip-tool/] tool returning current public IP address.

View File

@ -0,0 +1,28 @@
# The DEFAULT section contains the config common to all subsections.
# Redefining a value in a subsection overrides the DEFAULT's one.
[DEFAULT]
# Dynamic DNS Server
SERVER=infomaniak.com
LOGIN=myUserName
PASSWORD=mySuperSecretPassword
[mysite]
# Main domain
DOMAIN=mysite.cloud
[matrix]
# The matrix.mysite.cloud subdomain
DOMAIN=matrix.mysite.cloud
[mastodon]
# The mastodon.mysite.cloud subdomain
DOMAIN=mastodon.mysite.cloud
[mysite2]
# Another domain, but this is provided by another dynamicdns service
# so we re-define server, login and password to override the DEFAULT ones
SERVER=anotherdyndns.com
LOGIN=anotherUserName
PASSWORD=anotherSuperSecretPassword
DOMAIN=mysite2.io

View File

@ -0,0 +1,6 @@
# Cron to execute mddclient
# As a security measure, the address in MAILTO will be notified
# if the mddclient.py script crashes.
MAILTO="your-email-address"
*/5 * * * * root /usr/local/bin/mddclient.py /usr/local/etc/mddclient.cfg -q >> /var/log/mddclient.log 2>&1

282
mddclient/mddclient.py Executable file
View File

@ -0,0 +1,282 @@
#!/usr/bin/env python3
""" @package docstring
Mddclient
A DynamicDns client like ddclient, but supporting multiple (sub)domains,
every one in its autonomous request.
Installation:
- Copy mddclient.cfg in /usr/local/etc/mddclient.cfg and customize it
- Copy mddclient.py in /usr/local/bin/mddclient.py
Usage:
Place a cron entry like this one:
* * * * * root python3 /usr/local/bin/mddclient.py /usr/local/etc/mddclient.cfg
Exit codes:
0: success
1: unmanaged error (crash)
2: managed error (could not update due to bad config/server error)
For more informations on dyndns2 protocol: https://help.dyn.com/remote-access-api/
@author Daniele Verducci <daniele.verducci@ichibi.eu>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
import os
import sys
import logging
import configparser
import traceback
import requests
import re
import datetime
import json
NAME = 'mddclient'
VERSION = '0.2'
DESCRIPTION = 'A DynamicDns client like ddclient, but supporting multiple (sub)domains'
STATUS_FILE = '/tmp/mddclient.tmp'
CHECKIP_REQUEST_ADDR = 'http://checkip.dyndns.org'
CHECKIP_RESPONSE_PARSER = '<body>Current IP Address: (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})</body>'
DDCLIENT2_REQUEST_ADDR = "https://{}/nic/update?system=dyndns&hostname={}&myip={}"
DDCLIENT2_RESPONSE_PARSER = '^(nochg|no_change|good) (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})$'
USER_AGENT = 'Selfhost Utils Mddclient ' + VERSION
class Main:
def __init__(self, configPath):
''' Reads the config '''
self._log = logging.getLogger('main')
if not os.path.exists(configPath) or not os.path.isfile(configPath):
raise ValueError('configPath must be a file')
self.config = configparser.ConfigParser()
self.config.read(configPath)
def run(self, force, printStatusAndExit):
''' Makes the update requests '''
# Load status
status = Status()
if printStatusAndExit:
status.print()
return True
# Check current ip
currentIp = self.getCurrentIp()
if (currentIp == None):
return False
self._log.info('Current ip is {}'.format(currentIp))
if currentIp == status.getIp():
self._log.info('Ip is up-to-date.')
if force:
self._log.info('User requested forced refresh.')
else:
self._log.info('Nothing to do.')
status.save(True, False)
return
success = True
updated = False
for section in self.config:
s = Settings(section, self.config)
if s.name == 'DEFAULT':
continue
self._log.info('Updating "{}"'.format(section))
try:
newIpAddr = self.update(s.ddserver, s.dduser, s.ddpass, s.domain, currentIp)
self._log.info('Success update {} to addr {}'.format(s.domain, newIpAddr))
updated = True
except Exception as e:
self._log.error('Error while updating {}: {}'.format(s.domain, e))
success = False
# Save current ip
if success:
status.setIp(currentIp)
status.save(success, updated)
return success
def getCurrentIp(self):
'''Obtains current IP from checkip.dyndns.org'''
response = requests.get(CHECKIP_REQUEST_ADDR)
match = re.search(CHECKIP_RESPONSE_PARSER, response.text, re.MULTILINE)
if not match:
self._log.error('Unable to obtain new IP addr: Response format not valid: {}'.format(response.text))
return
groups = match.groups()
if len(groups) != 1:
self._log.error('Unable to obtain new IP addr: Unable to parse response: {}'.format(response.text))
return
return groups[0]
def update(self, server, user, password, domain, ip):
apiUrl = DDCLIENT2_REQUEST_ADDR.format(server, domain, ip)
try:
response = requests.get(apiUrl, auth=(user, password), headers={"User-Agent": USER_AGENT})
except requests.ConnectionError:
raise Exception('Server {} is unreachable'.format(server))
match = re.search(DDCLIENT2_RESPONSE_PARSER, response.text)
if not match:
raise Exception('Response format not valid: {}'.format(response.text))
groups = match.groups()
if len(groups) != 2:
raise Exception('Unable to parse response: {}'.format(response.text))
operationResult = groups[0]
ipAddr = groups[1]
# Check operation result and return appropriate errors
if operationResult == 'good':
# Success!
return ipAddr
elif operationResult == 'nochg' or operationResult == 'no_change':
# Should not happen: IP didn't need update
self._log.warning('Ip addres didn\'t need update: this should happen only at first run')
return ipAddr
elif operationResult == 'badauth':
raise Exception('The username and password pair do not match a real user')
elif operationResult == '!donator':
raise Exception('Option available only to credited users, but the user is not a credited user')
elif operationResult == 'notfqdn':
raise Exception('The hostname specified is not a fully-qualified domain name (not in the form hostname.dyndns.org or domain.com).')
elif operationResult == 'nohost':
raise Exception('The hostname specified does not exist in this user account')
elif operationResult == 'numhost':
raise Exception('Too many hosts specified in an update')
elif operationResult == 'abuse':
raise Exception('The hostname specified is blocked for update abuse')
elif operationResult == 'badagent':
raise Exception('The user agent was not sent or HTTP method is not permitted')
elif operationResult == 'dnserr':
raise Exception('DNS error encountered')
elif operationResult == '911':
raise Exception('There is a problem or scheduled maintenance on server side')
else:
raise Exception('Server returned an unknown result code: {}'.format(operationResult))
class Status:
''' Represents the current status '''
def __init__(self):
try:
with open(STATUS_FILE, 'r') as openfile:
self.status = json.load(openfile)
except FileNotFoundError:
self.status = {
'lastRun': None,
'lastRunSuccess': None,
'lastUpdate': None,
'lastIpAddr': None,
}
def save(self, success, updated):
self.status['lastRun'] = str(datetime.datetime.now())
self.status['lastRunSuccess'] = success
if updated:
self.status['lastUpdate'] = str(datetime.datetime.now())
jo = json.dumps(self.status)
with open(STATUS_FILE, "w") as outfile:
outfile.write(jo)
def setIp(self, ip):
self.status['lastIpAddr'] = ip
def getIp(self):
return self.status['lastIpAddr']
def print(self):
for k in self.status:
print('{}: {}'.format(k, self.status[k]))
class Settings:
''' Represents settings for a domain '''
def __init__(self, name, config):
self.config = config
## Section name
self.name = name
## DynDNS server data
self.ddserver = self.getStr(name, 'SERVER', None)
self.dduser = self.getStr(name, 'LOGIN', None)
self.ddpass = self.getStr(name, 'PASSWORD', None)
## Domain to update
self.domain = self.getStr(name, 'DOMAIN', False)
def getStr(self, name, key, defaultValue):
try:
return self.config.get(name, key)
except configparser.NoOptionError:
return defaultValue
def getBoolean(self, name, key, defaultValue):
try:
return self.config.getboolean(name, key)
except configparser.NoOptionError:
return defaultValue
if __name__ == '__main__':
import argparse
parser = argparse.ArgumentParser(
prog = NAME + '.py',
description = NAME + ' ' + VERSION + '\n' + DESCRIPTION,
formatter_class = argparse.RawTextHelpFormatter
)
parser.add_argument('configFile', help="configuration file path")
parser.add_argument('-q', '--quiet', action='store_true', help="suppress non-essential output")
parser.add_argument('-f', '--force', action='store_true', help="force update")
parser.add_argument('-s', '--status', action='store_true', help="print current status and exit doing nothing")
args = parser.parse_args()
if args.quiet:
level = logging.WARNING
else:
level = logging.INFO
logging.basicConfig(
format='%(asctime)s %(levelname)-8s %(message)s',
level=level,
datefmt='%Y-%m-%d %H:%M:%S'
)
try:
main = Main(args.configFile)
if not main.run(args.force, args.status):
sys.exit(2)
except Exception as e:
logging.critical(traceback.format_exc())
print('FATAL ERROR: {}'.format(e))
sys.exit(1)
sys.exit(0)