# Tasks ## Task: Debian Package Creation **Status**: DONE (2025-12-27) **Priority**: Medium **Estimated Effort**: 4-6 hours **Actual Effort**: ~5 hours ### Implementation Summary Successfully created a complete Debian package infrastructure for system-wide installation on Raspberry Pi and Debian-based systems. The package provides: - **System-wide installation** to `/opt/sensorpajen/` with dedicated user - **Configuration management** via `/etc/sensorpajen/` (preserved on upgrades) - **Automatic setup** including Python venv, dependencies, and Bluetooth capabilities - **Dual-mode operation** supporting both system and development installations - **Build verification** with automated script ### Files Created #### Debian Package Files (debian/) - `control` - Package metadata, dependencies, maintainer info - `compat` - Debhelper compatibility (v13) - `changelog` - Version history and release notes - `rules` - Build instructions (Makefile) - `install` - File installation mappings - `postinst` - Post-installation script (creates user, venv, sets capabilities) - `prerm` - Pre-removal script (stops service) - `postrm` - Post-removal script (cleanup, preserves config) - `sensorpajen.service` - System-wide systemd unit file #### Updated Code - `src/sensorpajen/config.py` - Auto-detects system vs development installation - `scripts/approve-sensors.sh` - Supports both installation modes - `scripts/verify-deb.sh` - Automated build and verification script (NEW) ### Installation Paths **System Installation (via .deb):** - Application: `/opt/sensorpajen/` - Python venv: `/opt/sensorpajen/venv/` - Configuration: `/etc/sensorpajen/` - Service: `/etc/systemd/system/sensorpajen.service` - Examples: `/usr/share/doc/sensorpajen/examples/` - User: `sensorpajen` (system account, no login) **Development Installation:** - Application: `/` - Python venv: `/.venv/` - Configuration: `/config/` - Service: `~/.config/systemd/user/sensorpajen.service` ### Key Features Implemented ✅ System-wide installation with dedicated user ✅ Python venv created automatically in postinst ✅ All dependencies installed from PyPI ✅ Bluetooth capabilities set automatically (setcap) ✅ Systemd service enabled but not started (waits for config) ✅ Configuration preserved on upgrade/remove/purge ✅ Example configs copied to /etc/sensorpajen on first install ✅ Dual-mode code (auto-detects system vs dev) ✅ Automated verification script ✅ Full lintian compliance ### Build and Install ```bash # Verify and build ./scripts/verify-deb.sh # Or manually dpkg-buildpackage -us -uc -b lintian ../sensorpajen_*.deb # Install on Raspberry Pi scp ../sensorpajen_*.deb pi@raspberrypi:~/ ssh pi@raspberrypi sudo apt install ./sensorpajen_*.deb # Configure sudo nano /etc/sensorpajen/sensorpajen.env sudo nano /etc/sensorpajen/sensors.json # Start service sudo systemctl start sensorpajen sudo journalctl -u sensorpajen -f ``` ### Testing Results ✅ Package builds successfully with `dpkg-buildpackage` ✅ Lintian passes without errors (warnings acceptable) ✅ Files installed to correct locations ✅ System user created automatically ✅ Python venv created with all dependencies ✅ Bluetooth capabilities set correctly ✅ Service enabled but not started before config ✅ Configuration preserved on upgrade/remove/purge ✅ Service runs as sensorpajen user (not root) ✅ Logs appear in `journalctl -u sensorpajen` ✅ Dual-mode operation works correctly ### Overview Create a Debian `.deb` package for system-wide installation of sensorpajen on Raspberry Pi OS and other Debian-based systems. This enables easy distribution and installation via `apt`/`dpkg` instead of manual git clone + pip install. ### Functional Requirements 1. **System-Wide Installation** - Install application to `/opt/sensorpajen/` - Create Python virtual environment in `/opt/sensorpajen/venv/` - Install systemd service file to `/etc/systemd/system/` - Place configuration in `/etc/sensorpajen/` - Put example configs in `/usr/share/doc/sensorpajen/examples/` 2. **Dedicated Service User** - Create `sensorpajen` system user if not exists - Service runs as `sensorpajen:sensorpajen` - User has no login shell, no home directory (system account) 3. **Automatic Service Configuration** - Auto-enable systemd service on installation - Configure Bluetooth capabilities (setcap) automatically - Service starts after installation if config exists 4. **Configuration Management** - Install example configs to `/usr/share/doc/sensorpajen/examples/`: - `sensorpajen.env.example` - `sensors.json.example` - `discovered_sensors.json.example` - Actual config expected in `/etc/sensorpajen/`: - `sensorpajen.env` - `sensors.json` - Do NOT overwrite existing config on upgrade - Preserve config on package removal - Keep config even on purge (user explicitly chooses) - Postinst should copy the examples into `/etc/sensorpajen/` only if they are missing, leaving any existing config untouched - Upgrades should refresh `/usr/share/doc/sensorpajen/examples/` with new defaults but never alter live configs under `/etc/sensorpajen/` 5. **Dependency Management** - Depend on system packages: `python3`, `python3-venv`, `python3-pip`, `bluetooth`, `bluez` - Create venv and install Python deps from PyPI in postinst script - Use `pyproject.toml` for Python dependency specification 6. **Package Metadata** - Package name: `sensorpajen` - Section: `misc` - Priority: `optional` - Architecture: `all` - Maintainer: Fredrik (fredrik@wahlberg.se) - Homepage: Repository URL - Description: "Raspberry Pi Bluetooth temperature sensor monitor" - Depends: System packages - Recommends: `mosquitto-clients` (optional) - **Version Source**: Extract version from `pyproject.toml` during build process. 7. **Files to Include** - All Python source code from `src/sensorpajen/` - Scripts from `scripts/` (approve-sensors.sh) - Systemd service file (system service, not user service) - Example configuration files - Documentation: `README.md`, `INSTALL.md` - License file ### Acceptance Criteria - [ ] Package builds successfully with `dpkg-buildpackage -us -uc -b` - [ ] Can install on fresh Raspberry Pi OS with `sudo apt install ./sensorpajen_*.deb` - [ ] Service user `sensorpajen` created automatically - [ ] Python venv created in `/opt/sensorpajen/venv/` with all dependencies - [ ] Bluetooth capabilities set on Python executable - [ ] Systemd service enabled but not started (waits for config) - [ ] After copying examples to `/etc/sensorpajen/` and editing, service starts successfully - [ ] Service runs as `sensorpajen` user, not root - [ ] Logs appear in `journalctl -u sensorpajen` - [ ] Package upgrade preserves `/etc/sensorpajen/` config files - [ ] Package removal (`dpkg -r`) stops service but keeps config - [ ] Package purge (`dpkg -P`) keeps config (user explicitly deletes if wanted) - [ ] `lintian` passes with no errors (warnings acceptable) - [ ] Automated verification script exists that builds the `.deb` and runs `lintian` ### Implementation Details #### 1. Create `debian/` Directory Structure ``` debian/ ├── control # Package metadata and dependencies ├── rules # Build instructions (Makefile) ├── install # Files to install and destinations ├── postinst # Post-installation script ├── prerm # Pre-removal script ├── postrm # Post-removal script ├── changelog # Required for native build (minimal entry) └── sensorpajen.service # Systemd service file (system-wide) ``` #### 2. `debian/control` File ``` Source: sensorpajen Section: misc Priority: optional Maintainer: Fredrik Build-Depends: debhelper-compat (= 13) Standards-Version: 4.5.0 Homepage: https://git.example.com/fredrik/sensorpajen Package: sensorpajen Architecture: all Depends: python3 (>= 3.9), python3-venv, python3-pip, bluetooth, bluez, ${misc:Depends} Recommends: mosquitto-clients Description: Raspberry Pi Bluetooth temperature sensor monitor Monitors Xiaomi Mijia LYWSD03MMC temperature sensors via Bluetooth Low Energy and publishes readings to MQTT broker. Supports ATC firmware with automatic sensor discovery and approval workflow. ``` #### 3. `debian/install` File ``` src/sensorpajen/* opt/sensorpajen/src/sensorpajen/ scripts/approve-sensors.sh opt/sensorpajen/scripts/ pyproject.toml opt/sensorpajen/ README.md usr/share/doc/sensorpajen/ INSTALL.md usr/share/doc/sensorpajen/ config/*.example usr/share/doc/sensorpajen/examples/ ``` #### 4. `debian/rules` File ```makefile #!/usr/bin/make -f %: dh $@ override_dh_auto_build: # No build step needed for pure Python override_dh_auto_install: # Installation handled by debian/install file override_dh_auto_clean: # Clean build artifacts rm -rf build/ dist/ *.egg-info ``` #### 5. `debian/postinst` Script ```bash #!/bin/bash set -e # Create sensorpajen system user if ! getent passwd sensorpajen > /dev/null; then useradd --system --no-create-home --shell /usr/sbin/nologin sensorpajen fi # Create config directory mkdir -p /etc/sensorpajen chown sensorpajen:sensorpajen /etc/sensorpajen # Create virtual environment cd /opt/sensorpajen python3 -m venv venv venv/bin/pip install --upgrade pip venv/bin/pip install . # Set Bluetooth capabilities PYTHON_PATH=$(readlink -f venv/bin/python3) setcap cap_net_raw,cap_net_admin+eip "$PYTHON_PATH" || echo "Warning: setcap failed, install libcap2-bin and rerun" # Install systemd service cp debian/sensorpajen.service /etc/systemd/system/ systemctl daemon-reload # Enable service (but don't start - needs config first) systemctl enable sensorpajen.service || echo "Warning: systemctl enable failed, enable manually" # Check if config exists, if so restart service if [ -f /etc/sensorpajen/sensorpajen.env ] && [ -f /etc/sensorpajen/sensors.json ]; then systemctl restart sensorpajen.service echo "sensorpajen service started" else echo "Configuration needed: Copy examples from /usr/share/doc/sensorpajen/examples/ to /etc/sensorpajen/" echo "Then run: sudo systemctl start sensorpajen" fi # Copy example configs if they're missing (never overwrite live config) for sample in sensorpajen.env.example sensors.json.example discovered_sensors.json.example; do target="/etc/sensorpajen/${sample%.example}" if [ ! -f "$target" ]; then cp "/usr/share/doc/sensorpajen/examples/$sample" "$target" chown sensorpajen:sensorpajen "$target" echo "Copied $sample to /etc/sensorpajen/" fi done exit 0 ``` #### 6. `debian/prerm` Script ```bash #!/bin/bash set -e # Stop service before removal if systemctl is-active --quiet sensorpajen.service; then systemctl stop sensorpajen.service fi # Disable service systemctl disable sensorpajen.service || true exit 0 ``` #### 7. `debian/postrm` Script ```bash #!/bin/bash set -e case "$1" in remove) # Service removed but config preserved echo "sensorpajen removed, config preserved in /etc/sensorpajen/" ;; purge) # Even on purge, keep config (user choice to delete manually) echo "Config preserved in /etc/sensorpajen/ - delete manually if needed" # Could optionally remove user here, but safer to keep ;; esac # Clean up systemd systemctl daemon-reload || true exit 0 ``` #### 8. `debian/sensorpajen.service` File ```ini [Unit] Description=Sensorpajen Bluetooth Temperature Monitor Documentation=https://github.com/fredrik/sensorpajen After=bluetooth.target network.target Wants=bluetooth.target [Service] Type=simple User=sensorpajen Group=sensorpajen WorkingDirectory=/opt/sensorpajen EnvironmentFile=/etc/sensorpajen/sensorpajen.env ExecStart=/opt/sensorpajen/venv/bin/python -m sensorpajen.main Restart=always RestartSec=10 # Bluetooth capabilities require this to be false NoNewPrivileges=false # Hardening (where possible with BT requirements) PrivateTmp=true ProtectSystem=strict ProtectHome=true ReadWritePaths=/etc/sensorpajen [Install] WantedBy=multi-user.target ``` #### 9. Build Process ```bash # From repository root dpkg-deb --build debian sensorpajen_2.0.0_armhf.deb # Check package contents dpkg-deb -c sensorpajen_2.0.0_armhf.deb # Check for issues lintian sensorpajen_2.0.0_armhf.deb > On every upgrade, rewrite `/usr/share/doc/sensorpajen/examples/` with the new package-provided examples so admins always have the latest defaults, but never overwrite existing files under `/etc/sensorpajen/`. ### Automated Verification Provide a script (e.g., `scripts/verify-deb.sh`) that runs the build and linting steps in a clean environment. The script should: ``` #!/bin/bash set -e ./ci/build-debian.sh # builds the deb into a temp directory lintian sensorpajen_*.deb echo "Package verification succeeded" ``` Acceptable tooling: `bash`, `lintian`, `dpkg-deb`. If lintian reports errors, the script should fail and print the diagnostics so you can triage the issue. ``` #### 10. Installation Test ```bash # Install sudo dpkg -i sensorpajen_2.0.0_armhf.deb # Copy and edit config sudo cp /usr/share/doc/sensorpajen/examples/sensorpajen.env.example /etc/sensorpajen/sensorpajen.env sudo cp /usr/share/doc/sensorpajen/examples/sensors.json.example /etc/sensorpajen/sensors.json sudo nano /etc/sensorpajen/sensorpajen.env # Start service sudo systemctl start sensorpajen # Check status sudo systemctl status sensorpajen sudo journalctl -u sensorpajen -f # Test upgrade # (make changes, rebuild, reinstall - config should persist) # Test removal sudo dpkg -r sensorpajen # Config stays sudo dpkg -P sensorpajen # Config still stays ``` ### File Paths Reference | Purpose | Path | |---------|------| | Application code | `/opt/sensorpajen/src/sensorpajen/` | | Python venv | `/opt/sensorpajen/venv/` | | Scripts | `/opt/sensorpajen/scripts/` | | Systemd service | `/etc/systemd/system/sensorpajen.service` | | Active config | `/etc/sensorpajen/sensorpajen.env`, `/etc/sensorpajen/sensors.json` | | Discovery data | `/etc/sensorpajen/discovered_sensors.json` | | Example configs | `/usr/share/doc/sensorpajen/examples/*.example` | | Documentation | `/usr/share/doc/sensorpajen/` | | Approve script | `/opt/sensorpajen/scripts/approve-sensors.sh` | ### Configuration Updates Needed When implementing, update these to use `/etc/sensorpajen`: **`src/sensorpajen/config.py`**: ```python # Change PROJECT_ROOT logic for system installation if Path('/opt/sensorpajen').exists(): # System installation PROJECT_ROOT = Path('/opt/sensorpajen') CONFIG_DIR = Path('/etc/sensorpajen') else: # Development installation PROJECT_ROOT = Path(__file__).parent.parent.parent CONFIG_DIR = PROJECT_ROOT / "config" ``` **`scripts/approve-sensors.sh`**: ```bash # Update paths for system installation if [ -d "/opt/sensorpajen" ]; then cd /opt/sensorpajen source /etc/sensorpajen/sensorpajen.env source venv/bin/activate else # Development mode cd "$(dirname "$0")/.." source config/sensorpajen.env source .venv/bin/activate fi ``` ### Notes - Package is **system-wide**, not user-scoped - Config in `/etc/sensorpajen/` is **never** auto-deleted - Service runs as dedicated `sensorpajen` user for security - Virtual environment created post-install to handle PyPI dependencies - Bluetooth capabilities set automatically - Service enabled but not started until config exists - Follow Debian package naming: `sensorpajen_2.0.0_armhf.deb` - Test on fresh Pi before considering complete --- ## Task: Add Auto-Discovery and Approval Flow for Sensors ### Problem Statement Adding new sensors currently requires manually editing `sensors.json`, which is error-prone and inconvenient. The system should automatically detect new sensors and provide a controlled way for users to approve or ignore them. --- ## Goal Implement **automatic sensor discovery** with a **user approval workflow** that: * Detects new sensors automatically * Notifies the user when new sensors are discovered * Allows the user to approve or ignore sensors via a script * Automatically updates `sensors.json` for approved sensors * Restarts the service after configuration changes --- ## Scope ### In Scope * Sensor auto-discovery * Tracking newly discovered sensors * Notification via `ntfy` * Interactive user script for approving/ignoring sensors * Updating `sensors.json` * Restarting the service via systemd ### Out of Scope * Web UI * Authentication mechanisms beyond existing system access * Changes to sensor hardware or firmware * Long-term sensor management (removal, editing, etc.) --- ## Functional Requirements ### 1. Sensor Auto-Discovery * The service must detect sensors that are not present in `sensors.json` * Each newly discovered sensor must have a stable unique identifier * Discovered-but-unapproved sensors must **not** be added automatically --- ### 2. Discovered Sensor Storage * Newly discovered sensors must be stored in `config/discovered_sensors.json` * Stored data must include: * `mac` - MAC address (unique identifier) * `name` - Advertised device name (e.g., "ATC_1234AB") * `rssi` - Signal strength in dBm * `first_seen` - ISO timestamp of first discovery * `last_seen` - ISO timestamp of most recent advertisement * `sample_reading` - One example reading with temperature, humidity, battery data * `status` - One of: "pending", "approved", "ignored" * `ignored_at` - ISO timestamp when ignored (if status is "ignored") * `ignore_reason` - Optional user-provided reason for ignoring * Approved sensors must have their status updated to "approved" * Ignored sensors must remain in the file with status "ignored" --- ### 3. Notification via ntfy * When a new sensor is discovered: * Send a notification to the configured `ntfy` topic via curl * Include at least: * Sensor MAC address * Sensor name * Last seen timestamp * Instruction that user action is required * Configuration (in `config/sensorpajen.env`): * `NTFY_ENABLED` - true/false to enable/disable notifications * `NTFY_URL` - ntfy server URL (e.g., "https://ntfy.sh") * `NTFY_TOPIC` - Topic to publish to * `NTFY_TOKEN` - Authentication token (sent in header) * ntfy is optional - system must work without it: * If `NTFY_ENABLED=false`, skip notifications * If ntfy is unreachable, log error and continue * Discovery and approval must work even if ntfy fails * The user must only be notified once per discovered sensor --- ### 4. User Approval Script Provide a CLI command `sensorpajen approve-sensors` that: * Lists all sensors with status "pending" or "ignored" * For each sensor, displays: * MAC address * Advertised name (e.g., "ATC_1234AB") * Last seen timestamp * Sample reading (temperature, humidity, battery) * Current status (pending/ignored) * For each sensor, allows the user to: * Approve the sensor (add to `sensors.json`) * Ignore the sensor (mark as ignored) * Skip (leave as pending for later) * If approving: * Prompt for a sensor name (required, human-readable) * Pre-fill comment field with extended metadata (MAC, device name, last seen, sample reading) * Allow user to edit or keep the pre-filled comment (optional) * If ignoring: * Prompt for optional reason * Update status to "ignored" with timestamp * Interactive mode only (no batch/automated approval) --- ### 5. Updating sensors.json * When a sensor is approved: * Add it to `sensors.json` (only if MAC doesn't already exist) * Include: * `mac` - MAC address from discovery * `name` - User-provided human-readable name * `comment` - User-edited comment (pre-filled with metadata) * The file must remain valid JSON * Existing sensors must not be modified * If MAC already exists in `sensors.json`, skip adding (renaming is done manually in the file) * Update status to "approved" in `discovered_sensors.json` --- ### 6. Configuration Reload * The service must automatically reload `sensors.json` every 15 minutes * No service restart required after approval * If `sensors.json` is modified: * Load new sensor list * Start monitoring newly added sensors * Continue monitoring existing sensors without interruption * Log configuration reload events --- ## Non-Functional Requirements * Must be safe to run on a Raspberry Pi * Must not require a GUI * Must fail gracefully if: * `ntfy` is unreachable * The user aborts the approval script * Logging must clearly indicate: * Discovery events * Notifications sent * Approval or ignore decisions --- ## Acceptance Criteria * A new sensor is automatically detected and added to `discovered_sensors.json` with status "pending" * Extended metadata (MAC, name, RSSI, timestamps, sample reading) is stored * A notification is sent via `ntfy` when a sensor is discovered (if enabled) * The approval CLI command (`sensorpajen approve-sensors`) lists pending and ignored sensors * The CLI displays MAC, name, last seen, and sample reading for each sensor * The user can approve a sensor with a custom name * The comment field is pre-filled with metadata and user can edit it * The user can ignore a sensor with an optional reason * Previously ignored sensors can be approved in a later CLI run * Approved sensors appear correctly in `sensors.json` (mac + name + comment only) * Sensors already in `sensors.json` are not added again (no duplicates) * The service automatically reloads `sensors.json` every 15 minutes * New sensors are monitored without service restart * Ignored sensors are stored with `ignored_at` timestamp and optional `ignore_reason` * ntfy failures do not prevent discovery or approval workflow --- ## Notes for Implementation * Prefer environment-based configuration (no `.ini` files) * Keep the discovery logic separate from user interaction logic * Avoid race conditions between discovery and approval * Assume multiple sensors may be discovered before user action * Use MAC address as unique identifier for sensors * ntfy notification format: `curl -H "Authorization: Bearer $NTFY_TOKEN" -d "message" $NTFY_URL/$NTFY_TOPIC` * Config reload: Use a timer thread that checks file mtime or reloads every 15 minutes * Pre-filled comment example: `"MAC: A4:C1:38:12:34:56, Name: ATC_1234AB, Last seen: 2025-12-27T14:30:00, Temp: 21.5°C, Humidity: 45%, Battery: 87%"` --- ## Implementation Details ### File Locations * Discovered sensors: `config/discovered_sensors.json` * Known sensors: `config/sensors.json` (existing) * Configuration: `config/sensorpajen.env` (add ntfy settings) ### New CLI Command * Entry point: `sensorpajen approve-sensors` * Add to `pyproject.toml` under `[project.scripts]` ### Configuration Variables (add to sensorpajen.env) ```bash # ntfy notifications (optional) NTFY_ENABLED=true NTFY_URL=https://ntfy.sh NTFY_TOPIC=sensorpajen NTFY_TOKEN=tk_xxxxxxxxxxxxx # Config reload interval (seconds) CONFIG_RELOAD_INTERVAL=900 # 15 minutes ``` ### discovered_sensors.json Structure ```json [ { "mac": "A4:C1:38:12:34:56", "name": "ATC_1234AB", "rssi": -65, "first_seen": "2025-12-27T14:30:15", "last_seen": "2025-12-27T14:35:42", "sample_reading": { "temperature": 21.5, "humidity": 45, "battery_percent": 87, "battery_voltage": 2950 }, "status": "pending" }, { "mac": "A4:C1:38:AB:CD:EF", "name": "ATC_ABCDEF", "rssi": -72, "first_seen": "2025-12-27T15:00:00", "last_seen": "2025-12-27T15:10:00", "sample_reading": { "temperature": 19.8, "humidity": 52, "battery_percent": 65, "battery_voltage": 2800 }, "status": "ignored", "ignored_at": "2025-12-27T15:15:00", "ignore_reason": "Test sensor, not needed" } ] ``` ### sensors.json Entry (after approval) ```json { "mac": "A4:C1:38:12:34:56", "name": "Living Room", "comment": "MAC: A4:C1:38:12:34:56, Name: ATC_1234AB, Last seen: 2025-12-27T14:35:42, Temp: 21.5°C, Humidity: 45%, Battery: 87%" } ``` --- If you want, I can also: * Split this into **multiple smaller tasks** * Add a **definition of done** section * Provide a **suggested file/module structure** * Write a **follow-up roadmap entry** for sensor management Just tell me how you want to evolve it next.