feat: implement Textual TUI and SQLite database for sensor management
This commit is contained in:
846
COMPLETED_TASKS.md
Normal file
846
COMPLETED_TASKS.md
Normal file
@@ -0,0 +1,846 @@
|
||||
# Tasks
|
||||
|
||||
## Task: Text UI for sensor management (Phase 1)
|
||||
|
||||
**Status**: DONE (2025-12-29)
|
||||
**Priority**: High
|
||||
**Estimated Effort**: 8-10 hours
|
||||
**Actual Effort**: ~6 hours
|
||||
|
||||
### Implementation Summary
|
||||
|
||||
Successfully implemented a modern, full-screen Textual TUI for managing Bluetooth sensors and migrated discovery data to a SQLite database for better persistence and metadata tracking.
|
||||
|
||||
### Key Features Implemented
|
||||
|
||||
✅ **SQLite Database Migration**:
|
||||
- Replaced `discovered_sensors.json` with `discovered_sensors.db`.
|
||||
- Implemented `DatabaseManager` for robust data handling.
|
||||
- Added tracking for RSSI, appearance count, and last seen timestamps.
|
||||
- Created migration script for existing JSON data.
|
||||
|
||||
✅ **Textual TUI Application**:
|
||||
- **Discovery View**: Real-time list of pending sensors with "Approve" and "Ignore" actions.
|
||||
- **Configured View**: Management of `sensors.json` with "Edit" (rename) and "Remove" actions.
|
||||
- **Ignored View**: List of ignored sensors with "Unignore" capability.
|
||||
- **Interactive Modals**: User-friendly dialogs for entering sensor names and ignore reasons.
|
||||
- **Responsive Design**: Full-screen layout with Header, Footer, and Tabbed navigation.
|
||||
|
||||
✅ **Integration & Modernization**:
|
||||
- Added `sensorpajen-tui` entry point for easy access.
|
||||
- Updated `README.md` with TUI usage instructions and keybindings.
|
||||
- Followed TDD approach with unit tests for database and TUI initialization.
|
||||
- Developed in a dedicated `feature/tui-management` branch.
|
||||
|
||||
### Files Created/Modified
|
||||
|
||||
- `src/sensorpajen/db.py`: SQLite database abstraction layer.
|
||||
- `src/sensorpajen/discovery_manager.py`: Refactored to use SQLite.
|
||||
- `src/sensorpajen/tui/app.py`: Main Textual TUI application.
|
||||
- `src/sensorpajen/tui/modals.py`: Modal dialogs for user input.
|
||||
- `src/sensorpajen/migrate_to_db.py`: Migration utility.
|
||||
- `tests/test_db.py`: Unit tests for database logic.
|
||||
- `tests/test_tui.py`: Unit tests for TUI initialization.
|
||||
- `pyproject.toml`: Added `textual` dependency and `sensorpajen-tui` script.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
# Launch the TUI
|
||||
sensorpajen-tui
|
||||
```
|
||||
|
||||
**Keybindings:**
|
||||
- `a`: Approve selected sensor
|
||||
- `i`: Ignore selected sensor
|
||||
- `e`: Edit sensor name
|
||||
- `u`: Unignore sensor
|
||||
- `Delete`: Remove sensor from monitoring
|
||||
- `r`: Refresh data
|
||||
- `q`: Quit
|
||||
|
||||
---
|
||||
|
||||
## Task: Debian Package Creation
|
||||
|
||||
**Status**: DONE (2025-12-27)
|
||||
**Priority**: Medium
|
||||
**Estimated Effort**: 4-6 hours
|
||||
**Actual Effort**: ~5 hours
|
||||
|
||||
### Implementation Summary
|
||||
|
||||
Successfully created a complete Debian package infrastructure for system-wide installation on Raspberry Pi and Debian-based systems. The package provides:
|
||||
|
||||
- **System-wide installation** to `/opt/sensorpajen/` with dedicated user
|
||||
- **Configuration management** via `/etc/sensorpajen/` (preserved on upgrades)
|
||||
- **Automatic setup** including Python venv, dependencies, and Bluetooth capabilities
|
||||
- **Dual-mode operation** supporting both system and development installations
|
||||
- **Build verification** with automated script
|
||||
|
||||
### Files Created
|
||||
|
||||
#### Debian Package Files (debian/)
|
||||
- `control` - Package metadata, dependencies, maintainer info
|
||||
- `compat` - Debhelper compatibility (v13)
|
||||
- `changelog` - Version history and release notes
|
||||
- `rules` - Build instructions (Makefile)
|
||||
- `install` - File installation mappings
|
||||
- `postinst` - Post-installation script (creates user, venv, sets capabilities)
|
||||
- `prerm` - Pre-removal script (stops service)
|
||||
- `postrm` - Post-removal script (cleanup, preserves config)
|
||||
- `sensorpajen.service` - System-wide systemd unit file
|
||||
|
||||
#### Updated Code
|
||||
- `src/sensorpajen/config.py` - Auto-detects system vs development installation
|
||||
- `scripts/approve-sensors.sh` - Supports both installation modes
|
||||
- `scripts/verify-deb.sh` - Automated build and verification script (NEW)
|
||||
|
||||
### Installation Paths
|
||||
|
||||
**System Installation (via .deb):**
|
||||
- Application: `/opt/sensorpajen/`
|
||||
- Python venv: `/opt/sensorpajen/venv/`
|
||||
- Configuration: `/etc/sensorpajen/`
|
||||
- Service: `/etc/systemd/system/sensorpajen.service`
|
||||
- Examples: `/usr/share/doc/sensorpajen/examples/`
|
||||
- User: `sensorpajen` (system account, no login)
|
||||
|
||||
**Development Installation:**
|
||||
- Application: `<project-root>/`
|
||||
- Python venv: `<project-root>/.venv/`
|
||||
- Configuration: `<project-root>/config/`
|
||||
- Service: `~/.config/systemd/user/sensorpajen.service`
|
||||
|
||||
### Key Features Implemented
|
||||
|
||||
✅ System-wide installation with dedicated user
|
||||
✅ Python venv created automatically in postinst
|
||||
✅ All dependencies installed from PyPI
|
||||
✅ Bluetooth capabilities set automatically (setcap)
|
||||
✅ Systemd service enabled but not started (waits for config)
|
||||
✅ Configuration preserved on upgrade/remove/purge
|
||||
✅ Example configs copied to /etc/sensorpajen on first install
|
||||
✅ Dual-mode code (auto-detects system vs dev)
|
||||
✅ Automated verification script
|
||||
✅ Full lintian compliance
|
||||
|
||||
### Build and Install
|
||||
|
||||
```bash
|
||||
# Verify and build
|
||||
./scripts/verify-deb.sh
|
||||
|
||||
# Or manually
|
||||
dpkg-buildpackage -us -uc -b
|
||||
lintian ../sensorpajen_*.deb
|
||||
|
||||
# Install on Raspberry Pi
|
||||
scp ../sensorpajen_*.deb pi@raspberrypi:~/
|
||||
ssh pi@raspberrypi
|
||||
sudo apt install ./sensorpajen_*.deb
|
||||
|
||||
# Configure
|
||||
sudo nano /etc/sensorpajen/sensorpajen.env
|
||||
sudo nano /etc/sensorpajen/sensors.json
|
||||
|
||||
# Start service
|
||||
sudo systemctl start sensorpajen
|
||||
sudo journalctl -u sensorpajen -f
|
||||
```
|
||||
|
||||
### Testing Results
|
||||
|
||||
✅ Package builds successfully with `dpkg-buildpackage`
|
||||
✅ Lintian passes without errors (warnings acceptable)
|
||||
✅ Files installed to correct locations
|
||||
✅ System user created automatically
|
||||
✅ Python venv created with all dependencies
|
||||
✅ Bluetooth capabilities set correctly
|
||||
✅ Service enabled but not started before config
|
||||
✅ Configuration preserved on upgrade/remove/purge
|
||||
✅ Service runs as sensorpajen user (not root)
|
||||
✅ Logs appear in `journalctl -u sensorpajen`
|
||||
✅ Dual-mode operation works correctly
|
||||
|
||||
### Overview
|
||||
Create a Debian `.deb` package for system-wide installation of sensorpajen on Raspberry Pi OS and other Debian-based systems. This enables easy distribution and installation via `apt`/`dpkg` instead of manual git clone + pip install.
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
1. **System-Wide Installation**
|
||||
- Install application to `/opt/sensorpajen/`
|
||||
- Create Python virtual environment in `/opt/sensorpajen/venv/`
|
||||
- Install systemd service file to `/etc/systemd/system/`
|
||||
- Place configuration in `/etc/sensorpajen/`
|
||||
- Put example configs in `/usr/share/doc/sensorpajen/examples/`
|
||||
|
||||
2. **Dedicated Service User**
|
||||
- Create `sensorpajen` system user if not exists
|
||||
- Service runs as `sensorpajen:sensorpajen`
|
||||
- User has no login shell, no home directory (system account)
|
||||
|
||||
3. **Automatic Service Configuration**
|
||||
- Auto-enable systemd service on installation
|
||||
- Configure Bluetooth capabilities (setcap) automatically
|
||||
- Service starts after installation if config exists
|
||||
|
||||
4. **Configuration Management**
|
||||
- Install example configs to `/usr/share/doc/sensorpajen/examples/`:
|
||||
- `sensorpajen.env.example`
|
||||
- `sensors.json.example`
|
||||
- `discovered_sensors.json.example`
|
||||
- Actual config expected in `/etc/sensorpajen/`:
|
||||
- `sensorpajen.env`
|
||||
- `sensors.json`
|
||||
- Do NOT overwrite existing config on upgrade
|
||||
- Preserve config on package removal
|
||||
- Keep config even on purge (user explicitly chooses)
|
||||
- Postinst should copy the examples into `/etc/sensorpajen/` only if they are missing, leaving any existing config untouched
|
||||
- Upgrades should refresh `/usr/share/doc/sensorpajen/examples/` with new defaults but never alter live configs under `/etc/sensorpajen/`
|
||||
|
||||
5. **Dependency Management**
|
||||
- Depend on system packages: `python3`, `python3-venv`, `python3-pip`, `bluetooth`, `bluez`
|
||||
- Create venv and install Python deps from PyPI in postinst script
|
||||
- Use `pyproject.toml` for Python dependency specification
|
||||
|
||||
6. **Package Metadata**
|
||||
- Package name: `sensorpajen`
|
||||
- Section: `misc`
|
||||
- Priority: `optional`
|
||||
- Architecture: `all`
|
||||
- Maintainer: Fredrik (fredrik@wahlberg.se)
|
||||
- Homepage: Repository URL
|
||||
- Description: "Raspberry Pi Bluetooth temperature sensor monitor"
|
||||
- Depends: System packages
|
||||
- Recommends: `mosquitto-clients` (optional)
|
||||
- **Version Source**: Extract version from `pyproject.toml` during build process.
|
||||
|
||||
7. **Files to Include**
|
||||
- All Python source code from `src/sensorpajen/`
|
||||
- Scripts from `scripts/` (approve-sensors.sh)
|
||||
- Systemd service file (system service, not user service)
|
||||
- Example configuration files
|
||||
- Documentation: `README.md`, `INSTALL.md`
|
||||
- License file
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- [ ] Package builds successfully with `dpkg-buildpackage -us -uc -b`
|
||||
- [ ] Can install on fresh Raspberry Pi OS with `sudo apt install ./sensorpajen_*.deb`
|
||||
- [ ] Service user `sensorpajen` created automatically
|
||||
- [ ] Python venv created in `/opt/sensorpajen/venv/` with all dependencies
|
||||
- [ ] Bluetooth capabilities set on Python executable
|
||||
- [ ] Systemd service enabled but not started (waits for config)
|
||||
- [ ] After copying examples to `/etc/sensorpajen/` and editing, service starts successfully
|
||||
- [ ] Service runs as `sensorpajen` user, not root
|
||||
- [ ] Logs appear in `journalctl -u sensorpajen`
|
||||
- [ ] Package upgrade preserves `/etc/sensorpajen/` config files
|
||||
- [ ] Package removal (`dpkg -r`) stops service but keeps config
|
||||
- [ ] Package purge (`dpkg -P`) keeps config (user explicitly deletes if wanted)
|
||||
- [ ] `lintian` passes with no errors (warnings acceptable)
|
||||
- [ ] Automated verification script exists that builds the `.deb` and runs `lintian`
|
||||
|
||||
### Implementation Details
|
||||
|
||||
#### 1. Create `debian/` Directory Structure
|
||||
|
||||
```
|
||||
debian/
|
||||
├── control # Package metadata and dependencies
|
||||
├── rules # Build instructions (Makefile)
|
||||
├── install # Files to install and destinations
|
||||
├── postinst # Post-installation script
|
||||
├── prerm # Pre-removal script
|
||||
├── postrm # Post-removal script
|
||||
├── changelog # Required for native build (minimal entry)
|
||||
└── sensorpajen.service # Systemd service file (system-wide)
|
||||
```
|
||||
|
||||
#### 2. `debian/control` File
|
||||
|
||||
```
|
||||
Source: sensorpajen
|
||||
Section: misc
|
||||
Priority: optional
|
||||
Maintainer: Fredrik <fredrik@wahlberg.se>
|
||||
Build-Depends: debhelper-compat (= 13)
|
||||
Standards-Version: 4.5.0
|
||||
Homepage: https://git.example.com/fredrik/sensorpajen
|
||||
|
||||
Package: sensorpajen
|
||||
Architecture: all
|
||||
Depends: python3 (>= 3.9), python3-venv, python3-pip, bluetooth, bluez, ${misc:Depends}
|
||||
Recommends: mosquitto-clients
|
||||
Description: Raspberry Pi Bluetooth temperature sensor monitor
|
||||
Monitors Xiaomi Mijia LYWSD03MMC temperature sensors via Bluetooth Low Energy
|
||||
and publishes readings to MQTT broker. Supports ATC firmware with automatic
|
||||
sensor discovery and approval workflow.
|
||||
```
|
||||
|
||||
#### 3. `debian/install` File
|
||||
|
||||
```
|
||||
src/sensorpajen/* opt/sensorpajen/src/sensorpajen/
|
||||
scripts/approve-sensors.sh opt/sensorpajen/scripts/
|
||||
pyproject.toml opt/sensorpajen/
|
||||
README.md usr/share/doc/sensorpajen/
|
||||
INSTALL.md usr/share/doc/sensorpajen/
|
||||
config/*.example usr/share/doc/sensorpajen/examples/
|
||||
```
|
||||
|
||||
#### 4. `debian/rules` File
|
||||
|
||||
```makefile
|
||||
#!/usr/bin/make -f
|
||||
|
||||
%:
|
||||
dh $@
|
||||
|
||||
override_dh_auto_build:
|
||||
# No build step needed for pure Python
|
||||
|
||||
override_dh_auto_install:
|
||||
# Installation handled by debian/install file
|
||||
|
||||
override_dh_auto_clean:
|
||||
# Clean build artifacts
|
||||
rm -rf build/ dist/ *.egg-info
|
||||
```
|
||||
|
||||
#### 5. `debian/postinst` Script
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# Create sensorpajen system user
|
||||
if ! getent passwd sensorpajen > /dev/null; then
|
||||
useradd --system --no-create-home --shell /usr/sbin/nologin sensorpajen
|
||||
fi
|
||||
|
||||
# Create config directory
|
||||
mkdir -p /etc/sensorpajen
|
||||
chown sensorpajen:sensorpajen /etc/sensorpajen
|
||||
|
||||
# Create virtual environment
|
||||
cd /opt/sensorpajen
|
||||
python3 -m venv venv
|
||||
venv/bin/pip install --upgrade pip
|
||||
venv/bin/pip install .
|
||||
|
||||
# Set Bluetooth capabilities
|
||||
PYTHON_PATH=$(readlink -f venv/bin/python3)
|
||||
setcap cap_net_raw,cap_net_admin+eip "$PYTHON_PATH" || echo "Warning: setcap failed, install libcap2-bin and rerun"
|
||||
|
||||
# Install systemd service
|
||||
cp debian/sensorpajen.service /etc/systemd/system/
|
||||
systemctl daemon-reload
|
||||
|
||||
# Enable service (but don't start - needs config first)
|
||||
systemctl enable sensorpajen.service || echo "Warning: systemctl enable failed, enable manually"
|
||||
|
||||
# Check if config exists, if so restart service
|
||||
if [ -f /etc/sensorpajen/sensorpajen.env ] && [ -f /etc/sensorpajen/sensors.json ]; then
|
||||
systemctl restart sensorpajen.service
|
||||
echo "sensorpajen service started"
|
||||
else
|
||||
echo "Configuration needed: Copy examples from /usr/share/doc/sensorpajen/examples/ to /etc/sensorpajen/"
|
||||
echo "Then run: sudo systemctl start sensorpajen"
|
||||
fi
|
||||
|
||||
# Copy example configs if they're missing (never overwrite live config)
|
||||
for sample in sensorpajen.env.example sensors.json.example discovered_sensors.json.example; do
|
||||
target="/etc/sensorpajen/${sample%.example}"
|
||||
if [ ! -f "$target" ]; then
|
||||
cp "/usr/share/doc/sensorpajen/examples/$sample" "$target"
|
||||
chown sensorpajen:sensorpajen "$target"
|
||||
echo "Copied $sample to /etc/sensorpajen/"
|
||||
fi
|
||||
done
|
||||
|
||||
exit 0
|
||||
```
|
||||
|
||||
#### 6. `debian/prerm` Script
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# Stop service before removal
|
||||
if systemctl is-active --quiet sensorpajen.service; then
|
||||
systemctl stop sensorpajen.service
|
||||
fi
|
||||
|
||||
# Disable service
|
||||
systemctl disable sensorpajen.service || true
|
||||
|
||||
exit 0
|
||||
```
|
||||
|
||||
#### 7. `debian/postrm` Script
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
case "$1" in
|
||||
remove)
|
||||
# Service removed but config preserved
|
||||
echo "sensorpajen removed, config preserved in /etc/sensorpajen/"
|
||||
;;
|
||||
purge)
|
||||
# Even on purge, keep config (user choice to delete manually)
|
||||
echo "Config preserved in /etc/sensorpajen/ - delete manually if needed"
|
||||
# Could optionally remove user here, but safer to keep
|
||||
;;
|
||||
esac
|
||||
|
||||
# Clean up systemd
|
||||
systemctl daemon-reload || true
|
||||
|
||||
exit 0
|
||||
```
|
||||
|
||||
#### 8. `debian/sensorpajen.service` File
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Sensorpajen Bluetooth Temperature Monitor
|
||||
Documentation=https://github.com/fredrik/sensorpajen
|
||||
After=bluetooth.target network.target
|
||||
Wants=bluetooth.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=sensorpajen
|
||||
Group=sensorpajen
|
||||
WorkingDirectory=/opt/sensorpajen
|
||||
EnvironmentFile=/etc/sensorpajen/sensorpajen.env
|
||||
ExecStart=/opt/sensorpajen/venv/bin/python -m sensorpajen.main
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
# Bluetooth capabilities require this to be false
|
||||
NoNewPrivileges=false
|
||||
|
||||
# Hardening (where possible with BT requirements)
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/etc/sensorpajen
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
#### 9. Build Process
|
||||
|
||||
```bash
|
||||
# From repository root
|
||||
dpkg-deb --build debian sensorpajen_2.0.0_armhf.deb
|
||||
|
||||
# Check package contents
|
||||
dpkg-deb -c sensorpajen_2.0.0_armhf.deb
|
||||
|
||||
# Check for issues
|
||||
lintian sensorpajen_2.0.0_armhf.deb
|
||||
|
||||
> On every upgrade, rewrite `/usr/share/doc/sensorpajen/examples/` with the new package-provided examples so admins always have the latest defaults, but never overwrite existing files under `/etc/sensorpajen/`.
|
||||
|
||||
### Automated Verification
|
||||
|
||||
Provide a script (e.g., `scripts/verify-deb.sh`) that runs the build and linting steps in a clean environment. The script should:
|
||||
|
||||
```
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
./ci/build-debian.sh # builds the deb into a temp directory
|
||||
lintian sensorpajen_*.deb
|
||||
|
||||
echo "Package verification succeeded"
|
||||
```
|
||||
|
||||
Acceptable tooling: `bash`, `lintian`, `dpkg-deb`. If lintian reports errors, the script should fail and print the diagnostics so you can triage the issue.
|
||||
```
|
||||
|
||||
#### 10. Installation Test
|
||||
|
||||
```bash
|
||||
# Install
|
||||
sudo dpkg -i sensorpajen_2.0.0_armhf.deb
|
||||
|
||||
# Copy and edit config
|
||||
sudo cp /usr/share/doc/sensorpajen/examples/sensorpajen.env.example /etc/sensorpajen/sensorpajen.env
|
||||
sudo cp /usr/share/doc/sensorpajen/examples/sensors.json.example /etc/sensorpajen/sensors.json
|
||||
sudo nano /etc/sensorpajen/sensorpajen.env
|
||||
|
||||
# Start service
|
||||
sudo systemctl start sensorpajen
|
||||
|
||||
# Check status
|
||||
sudo systemctl status sensorpajen
|
||||
sudo journalctl -u sensorpajen -f
|
||||
|
||||
# Test upgrade
|
||||
# (make changes, rebuild, reinstall - config should persist)
|
||||
|
||||
# Test removal
|
||||
sudo dpkg -r sensorpajen # Config stays
|
||||
sudo dpkg -P sensorpajen # Config still stays
|
||||
```
|
||||
|
||||
### File Paths Reference
|
||||
|
||||
| Purpose | Path |
|
||||
|---------|------|
|
||||
| Application code | `/opt/sensorpajen/src/sensorpajen/` |
|
||||
| Python venv | `/opt/sensorpajen/venv/` |
|
||||
| Scripts | `/opt/sensorpajen/scripts/` |
|
||||
| Systemd service | `/etc/systemd/system/sensorpajen.service` |
|
||||
| Active config | `/etc/sensorpajen/sensorpajen.env`, `/etc/sensorpajen/sensors.json` |
|
||||
| Discovery data | `/etc/sensorpajen/discovered_sensors.json` |
|
||||
| Example configs | `/usr/share/doc/sensorpajen/examples/*.example` |
|
||||
| Documentation | `/usr/share/doc/sensorpajen/` |
|
||||
| Approve script | `/opt/sensorpajen/scripts/approve-sensors.sh` |
|
||||
|
||||
### Configuration Updates Needed
|
||||
|
||||
When implementing, update these to use `/etc/sensorpajen`:
|
||||
|
||||
**`src/sensorpajen/config.py`**:
|
||||
```python
|
||||
# Change PROJECT_ROOT logic for system installation
|
||||
if Path('/opt/sensorpajen').exists():
|
||||
# System installation
|
||||
PROJECT_ROOT = Path('/opt/sensorpajen')
|
||||
CONFIG_DIR = Path('/etc/sensorpajen')
|
||||
else:
|
||||
# Development installation
|
||||
PROJECT_ROOT = Path(__file__).parent.parent.parent
|
||||
CONFIG_DIR = PROJECT_ROOT / "config"
|
||||
```
|
||||
|
||||
**`scripts/approve-sensors.sh`**:
|
||||
```bash
|
||||
# Update paths for system installation
|
||||
if [ -d "/opt/sensorpajen" ]; then
|
||||
cd /opt/sensorpajen
|
||||
source /etc/sensorpajen/sensorpajen.env
|
||||
source venv/bin/activate
|
||||
else
|
||||
# Development mode
|
||||
cd "$(dirname "$0")/.."
|
||||
source config/sensorpajen.env
|
||||
source .venv/bin/activate
|
||||
fi
|
||||
```
|
||||
|
||||
### Notes
|
||||
|
||||
- Package is **system-wide**, not user-scoped
|
||||
- Config in `/etc/sensorpajen/` is **never** auto-deleted
|
||||
- Service runs as dedicated `sensorpajen` user for security
|
||||
- Virtual environment created post-install to handle PyPI dependencies
|
||||
- Bluetooth capabilities set automatically
|
||||
- Service enabled but not started until config exists
|
||||
- Follow Debian package naming: `sensorpajen_2.0.0_armhf.deb`
|
||||
- Test on fresh Pi before considering complete
|
||||
|
||||
---
|
||||
|
||||
## Task: Add Auto-Discovery and Approval Flow for Sensors
|
||||
|
||||
### Problem Statement
|
||||
|
||||
Adding new sensors currently requires manually editing `sensors.json`, which is error-prone and inconvenient.
|
||||
The system should automatically detect new sensors and provide a controlled way for users to approve or ignore them.
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
Implement **automatic sensor discovery** with a **user approval workflow** that:
|
||||
|
||||
* Detects new sensors automatically
|
||||
* Notifies the user when new sensors are discovered
|
||||
* Allows the user to approve or ignore sensors via a script
|
||||
* Automatically updates `sensors.json` for approved sensors
|
||||
* Restarts the service after configuration changes
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
* Sensor auto-discovery
|
||||
* Tracking newly discovered sensors
|
||||
* Notification via `ntfy`
|
||||
* Interactive user script for approving/ignoring sensors
|
||||
* Updating `sensors.json`
|
||||
* Restarting the service via systemd
|
||||
|
||||
### Out of Scope
|
||||
|
||||
* Web UI
|
||||
* Authentication mechanisms beyond existing system access
|
||||
* Changes to sensor hardware or firmware
|
||||
* Long-term sensor management (removal, editing, etc.)
|
||||
|
||||
---
|
||||
|
||||
## Functional Requirements
|
||||
|
||||
### 1. Sensor Auto-Discovery
|
||||
|
||||
* The service must detect sensors that are not present in `sensors.json`
|
||||
* Each newly discovered sensor must have a stable unique identifier
|
||||
* Discovered-but-unapproved sensors must **not** be added automatically
|
||||
|
||||
---
|
||||
|
||||
### 2. Discovered Sensor Storage
|
||||
|
||||
* Newly discovered sensors must be stored in `config/discovered_sensors.json`
|
||||
* Stored data must include:
|
||||
* `mac` - MAC address (unique identifier)
|
||||
* `name` - Advertised device name (e.g., "ATC_1234AB")
|
||||
* `rssi` - Signal strength in dBm
|
||||
* `first_seen` - ISO timestamp of first discovery
|
||||
* `last_seen` - ISO timestamp of most recent advertisement
|
||||
* `sample_reading` - One example reading with temperature, humidity, battery data
|
||||
* `status` - One of: "pending", "approved", "ignored"
|
||||
* `ignored_at` - ISO timestamp when ignored (if status is "ignored")
|
||||
* `ignore_reason` - Optional user-provided reason for ignoring
|
||||
* Approved sensors must have their status updated to "approved"
|
||||
* Ignored sensors must remain in the file with status "ignored"
|
||||
|
||||
---
|
||||
|
||||
### 3. Notification via ntfy
|
||||
|
||||
* When a new sensor is discovered:
|
||||
* Send a notification to the configured `ntfy` topic via curl
|
||||
* Include at least:
|
||||
* Sensor MAC address
|
||||
* Sensor name
|
||||
* Last seen timestamp
|
||||
* Instruction that user action is required
|
||||
* Configuration (in `config/sensorpajen.env`):
|
||||
* `NTFY_ENABLED` - true/false to enable/disable notifications
|
||||
* `NTFY_URL` - ntfy server URL (e.g., "https://ntfy.sh")
|
||||
* `NTFY_TOPIC` - Topic to publish to
|
||||
* `NTFY_TOKEN` - Authentication token (sent in header)
|
||||
* ntfy is optional - system must work without it:
|
||||
* If `NTFY_ENABLED=false`, skip notifications
|
||||
* If ntfy is unreachable, log error and continue
|
||||
* Discovery and approval must work even if ntfy fails
|
||||
* The user must only be notified once per discovered sensor
|
||||
---
|
||||
|
||||
### 4. User Approval Script
|
||||
|
||||
Provide a CLI command `sensorpajen approve-sensors` that:
|
||||
|
||||
* Lists all sensors with status "pending" or "ignored"
|
||||
* For each sensor, displays:
|
||||
* MAC address
|
||||
* Advertised name (e.g., "ATC_1234AB")
|
||||
* Last seen timestamp
|
||||
* Sample reading (temperature, humidity, battery)
|
||||
* Current status (pending/ignored)
|
||||
* For each sensor, allows the user to:
|
||||
* Approve the sensor (add to `sensors.json`)
|
||||
* Ignore the sensor (mark as ignored)
|
||||
* Skip (leave as pending for later)
|
||||
* If approving:
|
||||
* Prompt for a sensor name (required, human-readable)
|
||||
* Pre-fill comment field with extended metadata (MAC, device name, last seen, sample reading)
|
||||
* Allow user to edit or keep the pre-filled comment (optional)
|
||||
* If ignoring:
|
||||
* Prompt for optional reason
|
||||
* Update status to "ignored" with timestamp
|
||||
* Interactive mode only (no batch/automated approval)
|
||||
|
||||
---
|
||||
|
||||
### 5. Updating sensors.json
|
||||
|
||||
* When a sensor is approved:
|
||||
* Add it to `sensors.json` (only if MAC doesn't already exist)
|
||||
* Include:
|
||||
* `mac` - MAC address from discovery
|
||||
* `name` - User-provided human-readable name
|
||||
* `comment` - User-edited comment (pre-filled with metadata)
|
||||
* The file must remain valid JSON
|
||||
* Existing sensors must not be modified
|
||||
* If MAC already exists in `sensors.json`, skip adding (renaming is done manually in the file)
|
||||
* Update status to "approved" in `discovered_sensors.json`
|
||||
|
||||
---
|
||||
|
||||
### 6. Configuration Reload
|
||||
|
||||
* The service must automatically reload `sensors.json` every 15 minutes
|
||||
* No service restart required after approval
|
||||
* If `sensors.json` is modified:
|
||||
* Load new sensor list
|
||||
* Start monitoring newly added sensors
|
||||
* Continue monitoring existing sensors without interruption
|
||||
* Log configuration reload events
|
||||
|
||||
---
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
* Must be safe to run on a Raspberry Pi
|
||||
* Must not require a GUI
|
||||
* Must fail gracefully if:
|
||||
|
||||
* `ntfy` is unreachable
|
||||
* The user aborts the approval script
|
||||
* Logging must clearly indicate:
|
||||
|
||||
* Discovery events
|
||||
* Notifications sent
|
||||
* Approval or ignore decisions
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
* A new sensor is automatically detected and added to `discovered_sensors.json` with status "pending"
|
||||
* Extended metadata (MAC, name, RSSI, timestamps, sample reading) is stored
|
||||
* A notification is sent via `ntfy` when a sensor is discovered (if enabled)
|
||||
* The approval CLI command (`sensorpajen approve-sensors`) lists pending and ignored sensors
|
||||
* The CLI displays MAC, name, last seen, and sample reading for each sensor
|
||||
* The user can approve a sensor with a custom name
|
||||
* The comment field is pre-filled with metadata and user can edit it
|
||||
* The user can ignore a sensor with an optional reason
|
||||
* Previously ignored sensors can be approved in a later CLI run
|
||||
* Approved sensors appear correctly in `sensors.json` (mac + name + comment only)
|
||||
* Sensors already in `sensors.json` are not added again (no duplicates)
|
||||
* The service automatically reloads `sensors.json` every 15 minutes
|
||||
* New sensors are monitored without service restart
|
||||
* Ignored sensors are stored with `ignored_at` timestamp and optional `ignore_reason`
|
||||
* ntfy failures do not prevent discovery or approval workflow
|
||||
|
||||
---
|
||||
|
||||
## Notes for Implementation
|
||||
|
||||
* Prefer environment-based configuration (no `.ini` files)
|
||||
* Keep the discovery logic separate from user interaction logic
|
||||
* Avoid race conditions between discovery and approval
|
||||
* Assume multiple sensors may be discovered before user action
|
||||
* Use MAC address as unique identifier for sensors
|
||||
* ntfy notification format: `curl -H "Authorization: Bearer $NTFY_TOKEN" -d "message" $NTFY_URL/$NTFY_TOPIC`
|
||||
* Config reload: Use a timer thread that checks file mtime or reloads every 15 minutes
|
||||
* Pre-filled comment example: `"MAC: A4:C1:38:12:34:56, Name: ATC_1234AB, Last seen: 2025-12-27T14:30:00, Temp: 21.5°C, Humidity: 45%, Battery: 87%"`
|
||||
|
||||
---
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### File Locations
|
||||
* Discovered sensors: `config/discovered_sensors.json`
|
||||
* Known sensors: `config/sensors.json` (existing)
|
||||
* Configuration: `config/sensorpajen.env` (add ntfy settings)
|
||||
|
||||
### New CLI Command
|
||||
* Entry point: `sensorpajen approve-sensors`
|
||||
* Add to `pyproject.toml` under `[project.scripts]`
|
||||
|
||||
### Configuration Variables (add to sensorpajen.env)
|
||||
```bash
|
||||
# ntfy notifications (optional)
|
||||
NTFY_ENABLED=true
|
||||
NTFY_URL=https://ntfy.sh
|
||||
NTFY_TOPIC=sensorpajen
|
||||
NTFY_TOKEN=tk_xxxxxxxxxxxxx
|
||||
|
||||
# Config reload interval (seconds)
|
||||
CONFIG_RELOAD_INTERVAL=900 # 15 minutes
|
||||
```
|
||||
|
||||
### discovered_sensors.json Structure
|
||||
```json
|
||||
[
|
||||
{
|
||||
"mac": "A4:C1:38:12:34:56",
|
||||
"name": "ATC_1234AB",
|
||||
"rssi": -65,
|
||||
"first_seen": "2025-12-27T14:30:15",
|
||||
"last_seen": "2025-12-27T14:35:42",
|
||||
"sample_reading": {
|
||||
"temperature": 21.5,
|
||||
"humidity": 45,
|
||||
"battery_percent": 87,
|
||||
"battery_voltage": 2950
|
||||
},
|
||||
"status": "pending"
|
||||
},
|
||||
{
|
||||
"mac": "A4:C1:38:AB:CD:EF",
|
||||
"name": "ATC_ABCDEF",
|
||||
"rssi": -72,
|
||||
"first_seen": "2025-12-27T15:00:00",
|
||||
"last_seen": "2025-12-27T15:10:00",
|
||||
"sample_reading": {
|
||||
"temperature": 19.8,
|
||||
"humidity": 52,
|
||||
"battery_percent": 65,
|
||||
"battery_voltage": 2800
|
||||
},
|
||||
"status": "ignored",
|
||||
"ignored_at": "2025-12-27T15:15:00",
|
||||
"ignore_reason": "Test sensor, not needed"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### sensors.json Entry (after approval)
|
||||
```json
|
||||
{
|
||||
"mac": "A4:C1:38:12:34:56",
|
||||
"name": "Living Room",
|
||||
"comment": "MAC: A4:C1:38:12:34:56, Name: ATC_1234AB, Last seen: 2025-12-27T14:35:42, Temp: 21.5°C, Humidity: 45%, Battery: 87%"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
If you want, I can also:
|
||||
|
||||
* Split this into **multiple smaller tasks**
|
||||
* Add a **definition of done** section
|
||||
* Provide a **suggested file/module structure**
|
||||
* Write a **follow-up roadmap entry** for sensor management
|
||||
|
||||
Just tell me how you want to evolve it next.
|
||||
|
||||
## Task: Add tests
|
||||
|
||||
**Status**: DONE (2025-12-29)
|
||||
**Priority**: High
|
||||
**Estimated Effort**: 2-3 hours
|
||||
**Actual Effort**: ~2 hours
|
||||
|
||||
### Implementation Summary
|
||||
Implemented a comprehensive test suite using `pytest` and `pytest-mock`. The tests cover all core modules of the application, ensuring reliability and making future refactoring (like the TUI migration) safer.
|
||||
|
||||
### Key Features Implemented
|
||||
- **Unit Tests for config.py**: Validates environment variable loading, default values, and sensor configuration parsing.
|
||||
- **Unit Tests for mqtt_publisher.py**: Verifies MQTT client initialization, connection handling, and message publishing for all metrics (temp, humidity, battery).
|
||||
- **Unit Tests for sensor_reader.py**: Tests BLE packet handling, ATC format parsing, and measurement creation using mocked Bluetooth hardware.
|
||||
- **Unit Tests for discovery_manager.py**: Ensures discovered sensors are correctly tracked, updated, and persisted to JSON.
|
||||
- **Test Infrastructure**: Added `conftest.py` for global mocks (Bluetooth, Environment) and configured `pyproject.toml` with dev dependencies.
|
||||
|
||||
### Testing Results
|
||||
- ✅ 17 tests passed across 4 test files.
|
||||
- ✅ Mocked all external dependencies (Bluetooth, MQTT Broker, File System).
|
||||
- ✅ Verified correct handling of both known and unknown sensors.
|
||||
|
||||
Reference in New Issue
Block a user