chris/system2mqtt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added different intervals for collectors and made them configurable.

Browse Source
This commit is contained in:
Christian Busch 2025-05-22 09:57:21 +02:00
parent 85154fcd60
commit 6e6f22f5d4
6 changed files with 186 additions and 67 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

146
README.md
View File

@ -1,79 +1,115 @@
# system2mqtt
Ein System zur Überwachung von Hosts durch Sammeln von Metriken und Senden via MQTT an Home Assistant.
A system for monitoring hosts by collecting metrics and sending them to Home Assistant via MQTT.
## Features
- Modulare Struktur für Metrik-Sammler
- Einfache Erweiterbarkeit durch neue Sammler
- Automatische Erkennung in Home Assistant
- Verschlüsselte MQTT-Kommunikation
- Detaillierte Geräteinformationen in Home Assistant
- Modular structure for metric collectors
- Easy extensibility through new collectors
- Automatic discovery in Home Assistant
- Encrypted MQTT communication
- Detailed device information in Home Assistant
- Individual update intervals per collector
## Installation
1. Repository klonen:
1. Clone the repository:
```bash
git clone https://github.com/yourusername/system2mqtt.git
cd system2mqtt
```
2. Python-Abhängigkeiten installieren:
2. Install Python dependencies:
```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
3. Konfiguration anpassen:
3. Configure the system:
```yaml
mqtt:
host: "mqtt.example.com" # MQTT Broker Adresse
host: "mqtt.example.com" # MQTT Broker Address
port: 1883 # MQTT Port
username: "your_username"
password: "your_password"
client_id: "system2mqtt"
discovery_prefix: "homeassistant" # Home Assistant Discovery Prefix
collectors:
# Default interval for all collectors (in seconds)
default_interval: 60
# Specific intervals for individual collectors
intervals:
zfs_pools: 300 # ZFS Pools every 5 minutes
cpu_temperature: 30 # CPU Temperature every 30 seconds
system_metrics: 60 # System Metrics every minute
```
## Verwendung
## Usage
Das System wird über das `run.sh` Skript gesteuert:
The system is controlled via the `run.sh` script:
```bash
# Starten des Systems
# Start the system
./run.sh start
# Stoppen des Systems
# Stop the system
./run.sh stop
# MQTT Topics aufräumen
# Clean up MQTT topics
./run.sh cleanup
```
## Sammler
## Collectors
### System Metrics
Sammelt grundlegende Systemmetriken:
- Last Boot Zeit
- Load Average (1, 5, 15 Minuten)
- Speichernutzung (Gesamt, Verfügbar, Verwendet)
- Swap-Nutzung (Gesamt, Verfügbar, Verwendet)
- CPU-Auslastung
- Speicherauslastung
- Swap-Auslastung
Collects basic system metrics:
- Last Boot Time
- Load Average (1, 5, 15 minutes)
- Memory Usage (Total, Available, Used)
- Swap Usage (Total, Available, Used)
- CPU Usage
- Memory Usage
- Swap Usage
Default Update Interval: 60 seconds
### CPU Temperature
Sammelt CPU-Temperaturdaten:
- Unterstützt Linux und FreeBSD
- Automatische Erkennung des Betriebssystems
- Korrekte Einheit (°C) und Device Class (temperature)
Collects CPU temperature data:
- Supports Linux and FreeBSD
- Automatic OS detection
- Correct unit (°C) and device class (temperature)
## Datenformat
Default Update Interval: 30 seconds
Das Datenaustauschformat ist versioniert und folgt den Home Assistant Spezifikationen. Jeder Collector gibt ein JSON-Objekt zurück mit folgender Struktur:
### ZFS Pools
Collects information about ZFS pools:
- Pool Health
- Total Size
- Used Space
- Free Space
- Usage Percentage
- Additional Attributes (readonly, dedup, altroot)
Default Update Interval: 300 seconds (5 minutes)
## Update Intervals
Each collector has a predefined default update interval that can be overridden in the configuration file:
1. Default intervals are defined in the collector files
2. These intervals can be customized per collector in `config.yaml`
3. If no specific interval is defined in the configuration, the collector's default interval is used
4. If no default interval is defined in the collector, the global `default_interval` from the configuration is used
## Data Format
The data exchange format is versioned and follows Home Assistant specifications. Each collector returns a JSON object with the following structure:
```json
{
@ -81,46 +117,44 @@ Das Datenaustauschformat ist versioniert und folgt den Home Assistant Spezifikat
{
"sensor_id": "unique_sensor_id",
"name": "Sensor Name",
"value": "Sensor Value",
"state_class": "measurement|total|total_increasing",
"unit_of_measurement": "Unit",
"device_class": "temperature|humidity|pressure|...",
"icon": "mdi:icon-name",
"entity_category": "diagnostic|config|system",
"value": "sensor_value",
"state_class": "measurement",
"unit_of_measurement": "unit",
"device_class": "device_class",
"icon": "mdi:icon",
"attributes": {
"friendly_name": "Friendly Name",
"source": "Data Source",
"additional_info": "Additional Information"
"additional_attributes": "values"
}
}
]
}
```
### Felder
### Fields
- `sensor_id`: Eindeutige ID für den Sensor (wird für MQTT-Topics verwendet)
- `name`: Anzeigename des Sensors
- `value`: Aktueller Wert des Sensors
- `state_class`: Art der Messung (measurement, total, total_increasing)
- `unit_of_measurement`: Einheit der Messung
- `device_class`: Art des Sensors (temperature, humidity, pressure, etc.)
- `icon`: Material Design Icon Name (mdi:...)
- `entity_category`: Kategorie des Sensors (diagnostic, config, system)
- `attributes`: Zusätzliche Informationen als Key-Value-Paare
- `sensor_id`: Unique ID for the sensor (used for MQTT topics)
- `name`: Display name of the sensor
- `value`: Current value of the sensor
- `state_class`: Type of measurement (measurement, total, total_increasing)
- `unit_of_measurement`: Unit of measurement
- `device_class`: Type of sensor (temperature, humidity, pressure, etc.)
- `icon`: Material Design Icon name (mdi:...)
- `entity_category`: Category of the sensor (diagnostic, config, system)
- `attributes`: Additional information as key-value pairs
### Versionierung
### Versioning
Das Format ist versioniert, um zukünftige Erweiterungen zu ermöglichen. Die aktuelle Version ist 1.0.
The format is versioned to allow for future extensions. The current version is 1.0.
## Home Assistant Integration
Das System nutzt die MQTT Discovery-Funktion von Home Assistant. Die Sensoren werden automatisch erkannt und erscheinen in Home Assistant mit:
- Korrektem Namen und Icon
- Aktuellen Werten
- Historischen Daten
- Detaillierten Geräteinformationen
The system uses Home Assistant's MQTT Discovery feature. Sensors are automatically detected and appear in Home Assistant with:
- Correct name and icon
- Current values
- Historical data
- Detailed device information
## Lizenz
## License
MIT License

3
collectors/cpu_temperature.py
View File

@ -7,6 +7,9 @@ import glob
from typing import Dict, Any, Optional, List, Tuple
import sys
# Default update interval in seconds
DEFAULT_INTERVAL = 30 # 30 seconds
def get_temperature_linux_coretemp() -> List[Tuple[float, str]]:
"""Get CPU temperatures using coretemp module."""
temps = []

3
collectors/system_metrics.py
View File

@ -5,6 +5,9 @@ import time
from datetime import datetime
from typing import Dict, Any
# Default update interval in seconds
DEFAULT_INTERVAL = 60 # 1 minute
def collect_metrics() -> Dict[str, Any]:
"""Collect system metrics and return them in the required format."""
# Get system metrics

3
collectors/zfs_pools.py
View File

@ -4,6 +4,9 @@ import subprocess
from typing import Dict, Any, List
import json
# Default update interval in seconds
DEFAULT_INTERVAL = 300 # 5 minutes
def get_zfs_pools() -> List[Dict[str, Any]]:
"""Get information about ZFS pools."""
try:

50
config.yaml.example
View File

@ -1,8 +1,54 @@
# MQTT Configuration
mqtt:
# MQTT Broker Address
host: "mqtt.example.com"
# MQTT Port (Default: 1883 for unencrypted, 8883 for TLS)
port: 1883
username: "system2mqtt"
password: "your_secure_password"
# MQTT Username
username: "your_username"
# MQTT Password
password: "your_password"
# MQTT Client ID (will be extended with hostname)
client_id: "system2mqtt_{hostname}"
# Home Assistant Discovery Prefix
discovery_prefix: "homeassistant"
# MQTT State Prefix for sensors
state_prefix: "system2mqtt"
# Collector Configuration
collectors:
# Default interval for all collectors (in seconds)
# Used when no specific interval is defined
default_interval: 60
# Specific intervals for individual collectors
# These override the collector's default intervals
intervals:
# ZFS Pools are updated every 5 minutes
zfs_pools: 300
# CPU Temperature is updated every 30 seconds
cpu_temperature: 30
# System Metrics are updated every minute
system_metrics: 60
# Notes:
# 1. The default intervals for collectors are:
# - zfs_pools: 300 seconds (5 minutes)
# - cpu_temperature: 30 seconds
# - system_metrics: 60 seconds (1 minute)
#
# 2. These intervals can be overridden here
#
# 3. If no specific interval is defined, the collector's
# default interval will be used
#
# 4. If no default interval is defined in the collector,
# the global default_interval will be used

46
system2mqtt.py
View File

@ -8,9 +8,10 @@ import time
import yaml
import platform
import paho.mqtt.client as mqtt
from typing import Dict, Any
from typing import Dict, Any, List
import importlib.util
import glob
from datetime import datetime, timedelta
class System2MQTT:
def __init__(self, config_path: str = "config.yaml"):
@ -20,6 +21,7 @@ class System2MQTT:
self.connected = False
self.device_info = self._get_device_info()
self.collectors = self._load_collectors()
self.last_run = {} # Speichert den Zeitpunkt des letzten Laufs für jeden Sammler
def _load_config(self, config_path: str) -> Dict[str, Any]:
"""Load configuration from YAML file."""
@ -74,7 +76,7 @@ class System2MQTT:
"""Generate state topic from sensor_id."""
return f"system2mqtt/{self.hostname}/{sensor_id}/state"
def _load_collectors(self) -> list:
def _load_collectors(self) -> List[Dict[str, Any]]:
"""Load all collector modules from the collectors directory."""
collectors = []
collector_dir = os.path.join(os.path.dirname(__file__), 'collectors')
@ -92,8 +94,19 @@ class System2MQTT:
spec.loader.exec_module(module)
if hasattr(module, 'collect_metrics'):
collectors.append(module)
print(f"Loaded collector: {module_name}")
# Get interval from config or use collector's default
default_interval = getattr(module, 'DEFAULT_INTERVAL', self.config['collectors']['default_interval'])
interval = self.config['collectors']['intervals'].get(
module_name,
default_interval
)
collectors.append({
'module': module,
'name': module_name,
'interval': interval
})
print(f"Loaded collector: {module_name} (interval: {interval}s)")
return collectors
@ -144,14 +157,30 @@ class System2MQTT:
retain=True
)
def should_run_collector(self, collector: Dict[str, Any]) -> bool:
"""Check if a collector should run based on its interval."""
now = datetime.now()
last_run = self.last_run.get(collector['name'])
if last_run is None:
return True
interval = timedelta(seconds=collector['interval'])
return (now - last_run) >= interval
def collect_and_publish(self):
"""Collect metrics from all collectors and publish them."""
for collector in self.collectors:
if not self.should_run_collector(collector):
continue
try:
data = collector.collect_metrics()
data = collector['module'].collect_metrics()
self.process_collector_data(data)
self.last_run[collector['name']] = datetime.now()
print(f"Updated {collector['name']} metrics")
except Exception as e:
print(f"Error collecting metrics from {collector.__name__}: {e}")
print(f"Error collecting metrics from {collector['name']}: {e}")
def connect(self):
"""Connect to MQTT broker."""
@ -179,10 +208,11 @@ def main():
# Initial collection
system2mqtt.collect_and_publish()
# Keep collecting metrics every 60 seconds
# Main loop - check every second if any collector needs to run
while True:
time.sleep(60)
system2mqtt.collect_and_publish()
time.sleep(1) # Check every second
except KeyboardInterrupt:
print("\nShutting down...")
finally: