@ohand/eos-connect

GitHub Copilot Instructions for EOS Connect

instructionscopilot

Install

agr install @ohand/eos-connect --target copilot

Writes 1 file into .github/copilot-instructions.md, pinned to git-6d3e22d6.

.github/copilot-instructions.md

Document

GitHub Copilot Instructions for EOS Connect

⚠️ CRITICAL WORKFLOW REQUIREMENTS:

Code changes ALWAYS require documentation updates — no exceptions
"prepare for commit" triggers mandatory documentation checklist — see Commit Preparation section
NEVER auto-commit — always present changes to user for review and approval
GitHub Pages is primary documentation — README.md is entry point only

⚠️ GIT WORKFLOW - MANDATORY RULES (APPLIES TO ALL COMMITS)

These rules apply ALWAYS - regardless of whether user said "prepare for commit" or not.

❌ ABSOLUTELY FORBIDDEN

❌ NEVER run git commit without explicit user approval ("yes", "approved", "confirmed")
❌ NEVER assume user wants to commit
❌ NEVER auto-commit changes silently
❌ NEVER commit based on inferred intent without explicit confirmation

✅ MANDATORY PROCESS (REQUIRED ORDER)

✅ Make all code changes and run tests
✅ Show user:
- Files modified/created (with line counts and paths)
- Test results (all test names and pass/fail status)
- Proposed commit message (exact text, ready to copy-paste)
- Summary of changes
✅ Present changes in clear format with section "## ❓ AWAITING YOUR APPROVAL"
✅ WAIT FOR EXPLICIT USER RESPONSE:
- "yes" / "approved" / "confirmed" → Proceed to step 5
- "no" / "modify" / feedback → Ask what to change, do NOT commit
- Any other response → Clarify what user wants, do NOT commit
✅ ONLY after explicit approval, execute git commands:
```
git add <files>
git commit -m "message"
```
✅ Optional: Only if user asks, also git push or create PR

🔴 THE RULE THAT WAS VIOLATED

"NEVER run git commit without explicit user approval"

This was violated when attempting to commit without waiting for user confirmation. This is now documented explicitly so it cannot be missed.

Project Guidelines

Icon Usage

Always use FontAwesome icons (free tier only) for all documentation and web interfaces
Never use emoji icons - they have been replaced with FontAwesome for consistency and professionalism
The main application icon is located in /docs/assets/images/icon.png and /docs/assets/images/logo.png
FontAwesome CDN: https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.7.2/css/all.min.css

Design Style

Follow the dark theme established in src/web/css/style.css
Color scheme:
- Primary background: rgb(54, 54, 54)
- Secondary background: rgb(78, 78, 78)
- Accent color: #4a9eff
- Border radius: 10px
Maintain responsive design patterns

Documentation

Structure

GitHub Pages documentation is in /docs folder
Structure: 4 main sections (what-is, user-guide, advanced, developer)
Use HTML for documentation pages (better styling control than Markdown)
README.md: Concise quick-start with links to full GitHub Pages docs (primary entry point)
GitHub Pages (/docs): Complete, detailed documentation for all features

Documentation Update Workflow

When preparing to commit (NEVER stage changed files and commit automatically - always review changes first and ask for confirmation):

Update README.md - Concise quick-start only (1-3 sentences per feature), links to full docs
- ⚠️ Key Features Only: README.md lists only user-facing, high-impact features that differentiate EOS Connect
- NOT in README: Implementation details, internal validation logic, bug fixes, or features users interact with indirectly
- In README: User-visible capabilities (battery control, price optimization, PV forecasting, integrations, automation, dashboards)
- Guide: Ask "Would a new user care about this on first read?" If NO → leave it for GitHub Pages only
Update GitHub Pages (/docs folder) - Complete, detailed documentation
- Always write from user perspective (except developer section)
- Main focus: "Easy entry for new and existing users"
- Keep all pages current with latest features and changes
- Use clear, practical examples with code blocks and screenshots

Documentation Perspective

what-is/, user-guide/, advanced/: Write for end users (clear, accessible language)
developer/: Write for contributors (technical details, architecture)
All documentation should help users quickly understand and use EOS Connect
Avoid jargon unless necessary; explain technical concepts simply

Project Role Clarity

EOS Connect is an integration and control platform, NOT an optimizer
The optimization calculations are performed by external servers:
- Akkudoktor EOS Server (https://github.com/Akkudoktor-EOS/EOS)
- EVopt (https://github.com/thecem/hassio-evopt)
Always clarify this distinction in documentation and code comments

Code Style

Follow existing Python conventions in the codebase
Use type hints where appropriate
Include docstrings for classes and functions
Follow pylint recommendations for formatting

Code Changes & Documentation Alignment

MANDATORY: Every code change, new feature, or bugfix MUST be reflected in documentation

When making ANY code changes:

Identify Documentation Impact: Determine which doc sections are affected
- New features → Update what-is, user-guide, and advanced pages
- Configuration changes → Update user-guide/configuration.html
- API changes → Update advanced/index.html (REST API & MQTT sections)
- Bug fixes → Update troubleshooting in user-guide if user-facing
Update All Affected Pages: Changes must be synchronized across:
- /docs GitHub Pages (primary documentation, contains all details)
- README.md (if quick start or core features affected, keep concise)
Maintain Accuracy: Documentation must match actual code behavior
- Verify API endpoint responses match code
- Confirm MQTT topic names and payloads match implementation
- Validate configuration parameter names, types, and valid values
- Update examples to reflect current best practices
Version Consistency: When src/version.py is updated, ensure version display is current on all doc pages

Failure to update documentation is considered incomplete work

Commit Preparation

⚠️ CRITICAL: Triggered by keywords: "prepare for commit", "ready to commit", "commit prep", "ready to merge", "prepare to commit"

MANDATORY CHECKLIST - ALWAYS EXECUTE IN THIS ORDER:

IMPORTANT: "Prepare for commit" means review ALL changes since last commit (not just recent ones). Use git diff and git status to see complete scope. The commit message should reflect the entire change set, not just the most recent fix.

✅ Review ALL changes since last commit (PRIMARY STEP)
- Run: git status (see all modified files)
- Run: git diff --stat (see change scope)
- Verify: changes are logically related (if not, break into multiple commits)
- The most recent change should not dominate the commit message if earlier changes are more significant
✅ Verify code changes are complete and tested
- All tests pass
- No breaking changes
- All new functionality implemented
✅ Update README.md (ALWAYS REQUIRED)
- Add 1-3 sentence mention of the feature/fix
- Include link to full documentation on GitHub Pages
- Keep concise - no lengthy explanations
- Check: Does it follow existing README style?
✅ Update GitHub Pages documentation (ALWAYS REQUIRED unless bugfix with no user-facing changes)
- Identify all affected doc sections (user-guide, advanced, what-is, etc.)
- Update with complete details, examples, best practices
- Write from user perspective - explain "why" and "how", not just "what"
- Verify accuracy: all config names, types, defaults match code
- Add code examples with syntax highlighting where applicable
✅ Run schema export (if config changes)
- Execute: python scripts/export_config_schema.py
- Verify: docs/assets/data/config_schema.json updated
✅ Generate summary and present to user
- List all files modified (from git status)
- List all tests passing
- Link to GitHub Pages sections updated
- WAIT FOR USER APPROVAL before any commits
✅ NEVER stage or commit automatically
- Only prepare and present changes for review
- User must explicitly approve before committing
✅ Include Conventional Commit message in response
- Keep it SHORT: 50 char max for subject line
- Format: <type>: <brief description>
- Body (optional): Why this change, not what (code shows what)
- Footer (optional): Fixes: #123 for issue references
- Make it ready to copy-paste
- Rule: If you're writing multiple paragraphs, the message is too long. Simplify.

If ANY of these steps are skipped, the preparation is INCOMPLETE.

Testing Phase Documentation

ENERGYFORECAST_TESTING.md: Temporary file for develop branch testing
- Contains Smart Price Prediction testing guide
- MUST BE DELETED when merging to main
- Full documentation already exists in /docs/user-guide/configuration.html#energyforecast
- Purpose: Provide accessible docs while feature is on develop (GitHub Pages shows main only)
- Reminder: Check for and remove any similar *_TESTING.md files before merging features to main

Config Schema Maintenance

Every new config field MUST be added to src/config_web/schema.py
Schema fields: key, type, default, section, level, label, description, help_url, validation, depends_on, hot_reload, display_group
After schema changes, run: python scripts/export_config_schema.py
The Config Schema is the SINGLE SOURCE OF TRUTH for field metadata
Web UI and GitHub Pages docs both consume the exported JSON (docs/assets/data/config_schema.json)
New fields must specify a level: getting_started, standard, or expert
New experimental features should use the label "experimental"
Hot-reloadable fields (applied without restart) must set hot_reload=True and have corresponding logic in src/config_web/hot_reload.py

Section Ordering & Wizard Flow

SECTION_META dict order in schema.py defines:

Configuration menu section order in the web UI (Settings)
Setup Wizard step order (recommended sequence for new users)
Both frontend JS (config.js, wizard.js) automatically use this ordering via the API response (section_order array)
Current order (recommended setup flow): eos → evcc → inverter → data_source → battery → load → price → pv_forecast_source → pv_forecast → mqtt → system

Rationale for this order:

Optimizer (eos) — Core backend selection
EVCC (evcc) — Optional, but must configure before Inverter (if used as controller)
Inverter (inverter) — Can reference EVCC as controller type
Data Source (data_source) — Load/battery data collection
Battery (battery) → Load (load) → Price (price) — Hardware setup
PV Source (pv_forecast_source) → PV Installations (pv_forecast) — Forecast configuration
MQTT (mqtt) → System (system) — Integration & system settings

When reordering sections, update SECTION_META in schema.py — it drives all three implementations: config UI, wizard, and exported docs JSON.

Field Dependencies (Cross-Field Validation)

Use depends_on field attribute to enforce cross-field dependencies:

Current dependency rules:

inverter.type → evcc.url: If user selects "evcc" as inverter controller but evcc.url is empty or default, the option is greyed out with tooltip "Configure EVCC URL first". API validation returns unmet_dependencies error and prevents save.
pv_forecast_source.source → evcc.url: If user selects "evcc" as PV source but evcc.url is empty or default, the option is greyed out. API validation blocks save with "EVCC selected as PV source but EVCC URL not configured".
pv_forecast_source.api_key: Only visible if source is "solcast" or "victron"
mqtt.broker: Only visible when mqtt.enabled is true

Dependency validation across UI layers:

Frontend - Config Screen (src/web/js/config.js):
- Greyed out/disabled options with visual styling:
  - Color: #888 (dark grey)
  - Font: italic
  - Label suffix: (not available)
  - Tooltip: "Configure EVCC URL first"
- User cannot click disabled option
- Dependency check on field change
Frontend - Wizard (src/web/js/wizard.js):
- Same visual styling as config screen (grey, italic, "(not available)")
- Dependencies re-evaluated when moving between steps
- Handles both scenarios: empty EVCC URL OR skipped EVCC step
- Disabled options prevent selection but remain visible for user awareness
Backend API (src/config_web/api.py — /api/config/ PUT endpoint):
- _check_dependencies() validates cross-field rules at save time
- Returns unmet_dependencies array with details: {"field": "...", "reason": "...", "requires": "...", "blocking": True}
- Prevents save without frontend intervention
- Enables backend-only validation and audit
UI Response (src/web/js/config.js - _showUnmetDependencies()):
- Shows red error banner: "Cannot save: required dependencies not configured"
- Lists blocking dependencies with field names and links
- Auto-dismisses when user corrects the issue

To add new dependency:

Add depends_on={"field_name": "condition"} to target field in schema.py
Backend validation automatically picked up at save time
Add conditional disabling logic to _renderSelect() in both config.js and wizard.js if visual UI disabling needed (frontend pre-validation)
Update this documentation

Config Web Module Architecture

The web-based configuration system lives in src/config_web/ as a self-contained module. Understanding this architecture is essential for any config-related work.

Module Structure

File	Purpose
`__init__.py`	`ConfigWebModule` facade — single entry point for the main app
`schema.py`	SPOT — All field definitions (`FieldDef`), `BOOTSTRAP_KEYS`, `SECTION_META`
`store.py`	SQLite persistence (WAL mode, thread-safe, change callbacks)
`migration.py`	config.yaml → SQLite and HA options.json → SQLite migration
`merger.py`	Builds merged config dict in same shape as old `config_manager.config`
`api.py`	Flask Blueprint with 10 REST endpoints at `/api/config/`
`hot_reload.py`	`HotReloadAdapter` — applies live changes to running interfaces

Key Design Principles

Zero interface changes: Interfaces receive the same dict shape they always did. The merger produces an identical structure.
SPOT (Single Point of Truth): schema.py defines all field metadata. The web UI, REST API validation, docs export, and merger all consume it.
Bootstrap vs Store: ~5 bootstrap keys (BOOTSTRAP_KEYS in schema.py) stay in config.yaml/ENV/HA options. Everything else lives in SQLite.
Section metadata: SECTION_META in schema.py defines icons + labels + order for all sections. Frontend automatically reads order via section_order array in /api/config/schema response — never hardcode section display info elsewhere.
Section ordering propagation: SECTION_META order flows through: schema.py → api.py (/schema endpoint) → config.js/wizard.js → user-visible UI order.
Dependencies: Cross-field validation at API layer with frontend pre-validation for UX. Disabled options show as greyed with visual indicators so users understand they're available but currently blocked.

Adding a New Config Field (Checklist)

Add FieldDef(...) to _ALL_FIELDS in src/config_web/schema.py
Run python scripts/export_config_schema.py to update docs JSON
Done — Web UI, API validation, docs table, migration, and merger all pick it up automatically
If hot-reloadable: also add to _PRICE_FIELD_MAP or _BATTERY_SOC_FIELDS in hot_reload.py
If field depends on another field: add depends_on param; API validation and UI greyout handle it automatically

Adding a New Config Section

Add fields with the new section name in schema.py
Add entry to SECTION_META dict in schema.py (icon + label), placing in desired order
Run python scripts/export_config_schema.py
Done — Frontend falls back gracefully, section appears in correct order in UI + wizard

REST API Endpoints (all under `/api/config/`)

Method	Path	Purpose
GET	`/schema`	Full schema JSON (fields + section metadata + dependencies)
GET	`/`	Current config values (passwords masked)
PUT	`/`	Partial update; returns `unmet_dependencies` if cross-field check fails
GET	`/section/<name>`	Single section values
POST	`/validate`	Validate without saving
GET	`/restart-required`	Pending restart-required fields
GET	`/export`	Export all settings as flat JSON
POST	`/import`	Import settings from JSON
GET	`/wizard-status`	Setup wizard completion state
POST	`/wizard-complete`	Mark wizard as completed

SPOT Pipeline Flow

schema.py (Python) → export_config_schema.py → config_schema.json (docs)
                   → /api/config/schema (live API) → config.js (web UI)
                   → /api/config/schema (live API) → wizard.js (setup wizard)

Testing

Tests are located in /tests folder
Mirror the source structure in test organization
Use pytest for all testing

Design Rules & Lessons Learned

These rules emerged from comprehensive manual testing (154 test cases) and must be followed in all future development.

Configuration Architecture

config.yaml is bootstrap-only: After migration, config.yaml contains only eos_connect_web_port, time_zone, log_level, and optionally data_path. All other settings live in SQLite and are managed via the web UI.
Never add non-bootstrap keys to config.yaml: New config fields go into schema.py only. The web UI, API, migration, and merger all pick them up automatically.
ConfigManager defaults must use valid values: create_default_config() in config.py must use values that pass schema validation. The placeholder "default" caused a crash when OptimizationInterface received it as eos.source. Always use a real schema-valid default.
Schema choices are authoritative: The validation.choices list in FieldDef is the single source of truth. Migration coercion validates against it and falls back to field_def.default for invalid values.

Fresh Install vs Migration Detection

_has_user_configured_values() distinguishes real user configs from ConfigManager defaults by checking sentinel fields (source values, sensor names)
Wizard flag logic: _wizard_completed is only set when real user config is detected during migration. Fresh installs (all defaults) leave this unset so the setup wizard appears.
Never hardcode wizard completion in migration without checking for real values

Web UI Integration Rules

1-second polling loop: main.js runs init() every second via setInterval. Any check triggered from init() must be guarded to avoid re-triggering (e.g., _wizardCheckDone flag in wizard.js)
Error overlay interaction: The startup error overlay (#overlay) blocks the full page. Any overlay (wizard, config) that needs to appear on top must hide #overlay first.
Restart guidance: When config changes require restart, show clear visual hints (amber banner) on both the wizard completion screen and the dashboard error overlay.
z-index management: Wizard and full-screen overlays must have higher z-index than the startup overlay

Hot Reload Design

Schema flag drives behavior: Set hot_reload=True in FieldDef to mark a field as live-reloadable
Adapter pattern: HotReloadAdapter receives change callbacks from ConfigStore and applies them to running interface instances
Attribute mapping: Each hot-reload field maps to a specific interface attribute name + coercion function
Side-effects: Some changes trigger recalculations (e.g., feed-in price change triggers __create_feedin_prices())
Config dict sync: For battery SOC fields, update both the interface method (set_min_soc()) AND the battery_data dict to prevent clamping against stale values

Validation & Security

Max length validation: All string fields should have reasonable max_length validation to prevent abuse
HTML injection prevention: User input displayed in the web UI must be text-only (no innerHTML with user data)
Latin-1 password encoding: Passwords containing non-Latin-1 characters cause Modbus/network errors. Validate at the API layer.
Unicode safety: All text fields must handle Unicode correctly (UTF-8 throughout)

Hot Reload — Current State & Expansion Priority

Currently Hot-Reloadable (21 fields, applied without restart)

Price fields (4) — via _PRICE_FIELD_MAP in hot_reload.py:

price.fixed_price_adder_ct → PriceInterface.fixed_price_adder_ct
price.relative_price_multiplier → PriceInterface.relative_price_multiplier
price.feed_in_price → PriceInterface.feed_in_tariff_price (+ recalculates feed-in prices)
price.negative_price_switch → PriceInterface.negative_price_switch (+ recalculates feed-in prices)

Battery fields (2) — via _BATTERY_SOC_FIELDS in hot_reload.py:

battery.min_soc_percentage → BatteryInterface.set_min_soc() + battery_data dict
battery.max_soc_percentage → BatteryInterface.set_max_soc() + battery_data dict

Optimizer fields (3) — via _OPTIMIZER_FIELD_MAP in hot_reload.py:

eos.timeout → OptimizationInterface.timeout
eos.dyn_override_discharge_allowed_pv_greater_load → OptimizationInterface.dyn_override_discharge_allowed
eos.pv_battery_charge_control_enabled → OptimizationInterface.pv_battery_charge_control_enabled

PV fields (12) — via debounced PvInterface.reload_config() in hot_reload.py:

pv_forecast_source.source
pv_forecast_source.api_key
pv_forecast.name
pv_forecast.lat
pv_forecast.lon
pv_forecast.azimuth
pv_forecast.tilt
pv_forecast.power
pv_forecast.powerInverter
pv_forecast.inverterEfficiency
pv_forecast.horizon
pv_forecast.resource_id

Notes:

PV updates are debounced/coalesced: multiple pv_forecast* key writes during one Save trigger one live reload.
Reload path re-validates config and safely restarts PV update service without restarting the app.
Optimizer fields use direct attribute updates with type coercion; changes apply immediately to the optimization process.

Expansion Priority List

Fields grouped by predicted code complexity and user impact. Each group shares interface patterns, so implementing one makes the rest in that group trivial.

Priority 1 — Simple attribute swaps (low effort, high user value)

These fields are simple instance attributes that can be set at runtime:

Group	Fields	Interface	Status
EOS tuning	`eos.timeout`, `eos.dyn_override_discharge_allowed_pv_greater_load`, `eos.pv_battery_charge_control_enabled`	OptimizationInterface	✅ IMPLEMENTED
System timing	`refresh_time`	OptimizationScheduler	Next to implement
System timing	`eos.time_frame` (900 or 3600)	All interfaces	Requires cache invalidation (see Priority 1.5)
Inverter limits	`inverter.max_grid_charge_rate`, `inverter.max_pv_charge_rate`	BaseInverter subclass	Could implement next
System	`request_timeout`	All interfaces	Lower priority

Priority 1.5 — Attribute swap + cache clear (medium effort, high user value)

Simple attribute updates but require recalculation or cache invalidation:

Group	Fields	Interface	Change Required
EOS time slot	`eos.time_frame`	OptimizationInterface + all data providers	Update timeframe on all interfaces + clear forecast caches

Priority 2 — Requires recalculation or reconnect (medium effort)

These need more than a simple attribute swap:

Group	Fields	Interface	Change Required
Battery capacity	`battery.capacity_wh`, `battery.charge_efficiency`, `battery.discharge_efficiency`, `battery.max_charge_power_w`	BatteryInterface	Update battery_data dict + recalc charging curve
Battery price calc	`battery.price_update_interval`, `battery.price_history_lookback_hours`, `battery.price_euro_per_wh_accu`	BatteryPriceHandler	Restart timer or update interval
Price fixed array	`price.fixed_24h_array`	PriceInterface	Re-parse array + recalc prices
EOS time slot	`eos.time_frame` (see Priority 1.5)	Multiple	Debounced reload with cache clear

Priority 3 — Requires interface reconstruction (high effort, rare changes)

These change fundamental interface identity (source, URL, credentials). Users rarely change these after initial setup:

Group	Fields	Interface	Change Required
Data source	`data_source.type`, `data_source.url`, `data_source.access_token`	Load, Battery	Full interface re-init (different API client)
Sensor names	`load.load_sensor`, `battery.soc_sensor`, all sensor fields	Load, Battery	Could swap attrs, but untested behavior
MQTT connection	`mqtt.broker`, `mqtt.port`, `mqtt.user`, `mqtt.password`, `mqtt.tls`	MqttInterface	Disconnect + reconnect
MQTT features	`mqtt.ha_mqtt_auto_discovery`, `mqtt.ha_mqtt_auto_discovery_prefix`	MqttInterface	Re-publish discovery messages
Inverter type	`inverter.type`, `inverter.address`, `inverter.user`, `inverter.password`	InverterFactory	Full reconstruction via factory
PV sources	Implemented	PvInterface	Hot-reload via debounced `reload_config()`
Price source	`price.source`, `price.token`	PriceInterface	Different API client
EOS backend	`eos.source`, `eos.server`, `eos.port`	OptimizationInterface	Different backend class
EVCC	`evcc.url`	EvccInterface	New HTTP client

Priority 4 — Bootstrap keys (never hot-reloadable)

These affect the application infrastructure itself:

eos_connect_web_port — Flask server port (requires process restart)
time_zone — System-wide timezone (affects all timestamp handling)
log_level — Logging configuration (could be hot-reloaded but low priority)

Adding a New Hot-Reload Field (Checklist)

Set hot_reload=True in the FieldDef in schema.py
Add field handling in hot_reload.py:
- Simple attribute: add to _PRICE_FIELD_MAP or create new map

Method call: add elif branch in the appropriate _apply_* method
Full interface reconfigure: add prefix/group handling + debounced reload path

If side-effects needed (recalculation), add trigger set like _FEEDIN_TRIGGERS
Add tests in tests/config_web/test_hot_reload.py
Run python scripts/export_config_schema.py

Interface Creation & Startup Error Handling

Two new modules work together to provide centralized interface creation with integrated startup validation and user-visible error handling.

InterfaceFactory (src/interface_factory.py)

Purpose: Centralized factory for interface instantiation with integrated startup validation.

Responsibilities:

Instantiate all interface types (Load, Battery, Price, PV, MQTT, EVCC, Inverter, Optimization)
Catch errors during instantiation
Register errors with StartupValidator for visibility in web UI startup panel
Distinguish critical interfaces (halt startup on failure) from non-critical (use fallbacks)
Track created interfaces for lifecycle management

Usage in eos_connect.py:

from interface_factory import InterfaceFactory
from startup_validator import StartupValidator

# Initialize at startup
validator = StartupValidator()
factory = InterfaceFactory(validator)

# Create interfaces with automatic error handling
battery_interface = factory.create_battery_interface(
    config=config['battery'],
    time_zone=time_zone,
    critical=True,  # Startup halts on failure
)

load_interface = factory.create_load_interface(
    config=config['load'],
    time_frame_base=config['refresh_time'],
    time_zone=time_zone,
    critical=False,  # Uses default on failure
)

Benefits:

Eliminates boilerplate try/except blocks in main app
Consistent error categorization across all interface types
Centralized startup error collection for web UI visibility
Easy to extend with new interface types

StartupValidator (src/startup_validator.py)

Purpose: Lightweight facade for registering startup errors directly to the logging system.

Responsibilities:

Register startup errors with structured metadata
Write ERROR/WARNING logs captured by MemoryLogHandler
Embed metadata markers in log messages for frontend parsing
Act as single source of truth for startup errors via /logs/alerts endpoint

Method signature:

validator.add_error(
    category="connectivity",        # initialization, configuration, connectivity
    component="battery_interface",  # Component name
    severity="error",              # error or warning
    title="Battery unavailable",   # Short, user-friendly title
    message="Connection timeout",  # Detailed message
    action_required=True,          # Flag for ACTION REQUIRED badge
    config_link="#battery",        # Link to config section (e.g., #eos, #battery)
)

Frontend Integration:

Errors are fetched via: GET /logs/alerts?startup_only=1&limit=20
Errors with metadata markers are parsed by main.js → parseAlertMeta()
Startup panel renders with:
- Component name (extracted from [component] prefix)
- Timestamp and occurrence count
- ACTION REQUIRED badge (yellow) if flagged
- "Open Configuration" link pointing to config section

Startup Error Flow

Startup:
  InterfaceFactory.create_*_interface()
    ├─ Try to create interface
    │   ├─ Success → return interface (silent)
    │   └─ Failure → catch exception
    │       └─ validator.add_error(...)
    │           └─ MemoryLogHandler captures ERROR/WARNING log
    │               ├─ Metadata extracted: Config link, ACTION REQUIRED flag
    │               └─ Stored in log buffer, visible via /logs/alerts

Runtime (user views dashboard):
  fetch /logs/alerts?startup_only=1
    └─ Frontend renderAlertSection()
       ├─ Deduplicates by title, counts occurrences
       ├─ Sorts by severity (ACTION REQUIRED first)
       ├─ Shows: timestamp, occurrence count, config link button
       └─ User clicks → showConfigurationMenu(section)

Log Message Format

Error messages should include metadata markers for frontend parsing:

[component] Title: Message | Config: #section | ACTION REQUIRED

Examples:
[eos_backend] EOS Connection failed: Connection timeout | Config: #eos | ACTION REQUIRED
[battery_interface] Battery SOC error: Authentication failed | Config: #battery | ACTION REQUIRED
[load_interface] Load data unavailable: Request timeout | Config: #load

Frontend parsing:

Matches Config: (#\w+) for config section link
Checks for "ACTION REQUIRED" string to show badge
Extracts component name from [component] prefix

Extending with New Interface Types

To add a new interface creation method:

Add method to InterfaceFactory following the pattern of existing methods
Specify error category (connectivity, initialization, configuration), component name, config link
Mark as critical or non-critical
Call from eos_connect.py during startup
On instantiation failure, StartupValidator.add_error() is called automatically
Errors appear in startup panel within 1-2 seconds

Example:

def create_my_new_interface(self, config: Dict[str, Any], critical: bool = True):
    return self._create_interface(
        component_name="my_new_interface",
        category="connectivity",
        critical=critical,
        title="My New Interface unavailable",
        error_message="Failed to initialize",
        config_link="#my_section",
        creator_func=lambda: self._import_and_create(
            "interfaces.my_new_interface",
            "MyNewInterface",
            config,
            request_timeout=10,
        ),
    )

HA Addon Integration

Bootstrap Contract with ha_addons Repo

The HA addon (ohAnd/ha_addons) must provide exactly these options in its config.yaml:

options:
  web_port: 8081
  time_zone: "Europe/Berlin"
  log_level: "INFO"

schema:
  web_port: int
  time_zone: str
  log_level: list(DEBUG|INFO|WARNING|ERROR)

These map to EOS Connect's BOOTSTRAP_KEYS via _HA_BOOTSTRAP_MAP in config.py.

What the Addon Must NOT Do

Do not include non-bootstrap config options in config.yaml/options.json — all other settings are managed by the web UI and stored in SQLite
Do not mount config.yaml into the container — it's optional; if absent, defaults are used and the wizard guides setup
Do not write to /data/eos_connect.db — EOS Connect owns this file exclusively

What the Addon Must Provide

Persistent /data/ volume: SQLite DB lives at /data/eos_connect.db
Network access: Port forwarding for the web UI (default 8081)
HASSIO or HASSIO_TOKEN env var: Set by HA automatically, triggers addon mode
/data/options.json: HA writes this with the 3 bootstrap values
webui declaration: http://[HOST]:[PORT:8081] for HA sidebar integration

Legacy Migration Path

When upgrading from an old addon version that had full config in options.json:

EOS Connect detects non-bootstrap keys in /data/options.json
migrate_ha_options_to_store() imports them into SQLite
Old options.json keys are left in place (HA manages that file)
Wizard is marked complete — user sees their existing config in the web UI

Appendix: ha_addons Repo Change Checklist for Web Config Migration

Context: The ohAnd/ha_addons repo currently passes the FULL EOS Connect configuration through options.json and symlinks it as config.yaml. With the web config migration, only 3 bootstrap keys remain in options.json. All other settings are managed via the built-in web UI and stored in SQLite at /data/eos_connect.db.

Applies to: Both eos_connect/ and eos_connect_develop/ addon directories.

1. `config.yaml` (addon manifest) — MAJOR CHANGES

Current state: ~100 options covering load, eos, price, battery, pv_forecast, inverter, mqtt, evcc, refresh_time, request_timeout, etc.

Target state: Only 3 bootstrap options + metadata.

name: "EOS connect"
description: "Tool to optimize energy usage with EOS system data."
version: "X.Y.Z"
slug: "eos_connect"
url: "https://github.com/ohAnd/EOS_connect"

arch:
  - aarch64
  - amd64

panel_icon: mdi:home-battery
panel_admin: false

ingress: true
ingress_port: 8081
init: false

image: "ghcr.io/ohand/ha-addon-eos_connect_{arch}"

# Persistent /data/ volume for SQLite database
map:
  - type: addon_config
    read_only: false
    path: /app/addon_config

ports:
  8081/tcp: 8081
ports_description:
  8081/tcp: "EOS Connect web server"

# ===== CHANGED: Only bootstrap options =====
options:
  web_port: 8081
  time_zone: "Europe/Berlin"
  log_level: "info"

schema:
  web_port: int(1,65535)
  time_zone: str
  log_level: list(debug|info|warning|error)

What was removed: ALL load:, eos:, price:, battery:, pv_forecast_source:, pv_forecast:, inverter:, evcc:, mqtt:, refresh_time, request_timeout, eos_connect_web_port option blocks — both from options: and schema: sections.

Why eos_connect_web_port is removed from options: The addon uses web_port which maps to eos_connect_web_port internally via _HA_BOOTSTRAP_MAP in config.py. Keep only web_port in the addon manifest.

2. `translations/en.yaml` — MAJOR CHANGES

Current state: ~200 lines describing every config field in all sections.

Target state: Only 3 bootstrap fields + a note about the web UI.

configuration:
  web_port:
    name: Web Port
    description: >-
      Port for the EOS Connect web interface. All other settings are
      configured through the built-in web UI after startup.
  time_zone:
    name: Time Zone
    description: >-
      System time zone (e.g., Europe/Berlin). Used for scheduling
      and timestamp display.
  log_level:
    name: Log Level
    description: >-
      Logging verbosity: debug, info, warning, error.

What was removed: ALL translation entries for load, eos, price, battery, pv_forecast_source, pv_forecast, inverter, evcc, mqtt, refresh_time, request_timeout, eos_connect_web_port.

3. `Dockerfile` — CRITICAL CHANGE

Current state (the critical line):

# Copy application and finalize
COPY ./src /app
WORKDIR /app/src
RUN echo "::group::Finalizing Application" && \
    ln -sf /data/options.json /app/src/config.yaml && \
    echo "=== APPLICATION COPIED ===" && \
    ...

The ln -sf /data/options.json /app/src/config.yaml symlink makes options.json appear as config.yaml — this is how the full config was passed to EOS Connect.

New state: Remove the symlink. EOS Connect now reads /data/options.json directly for bootstrap values (when is_ha_addon is detected) and uses SQLite for everything else.

# Copy application and finalize
COPY ./src /app
WORKDIR /app/src
RUN echo "::group::Finalizing Application" && \
    echo "=== APPLICATION COPIED ===" && \
    echo "Virtual environment size: $(du -sh /opt/venv)" && \
    echo "=== BUILD COMPLETED: $(date) ===" && \
    echo "::endgroup::"

Why: The symlink overwrites any config.yaml with options.json content. Since options.json now only has bootstrap keys, this would make ConfigManager see only 3 values and generate 100% defaults for everything else — breaking existing users. Without the symlink, EOS Connect:

Uses its internal defaults from create_default_config()
Overlays bootstrap values from /data/options.json via load_ha_bootstrap()
Migrates any legacy full options.json values to SQLite on first run
Subsequent runs load from SQLite (already migrated)

No other Dockerfile changes needed — all pip dependencies are the same.

4. `build.yaml` — NO CHANGES NEEDED

The build base images, args, and labels remain unchanged:

build_from:
  aarch64: "ghcr.io/home-assistant/aarch64-base-python:3.13-alpine3.22"
  amd64: "ghcr.io/home-assistant/amd64-base-python:3.13-alpine3.22"

5. GitHub Actions Workflows — NO CHANGES NEEDED

The CI/CD workflows that build and publish Docker images don't need modification. The build process is the same — only the Dockerfile content changed.

6. `CHANGELOG.md` / Release Notes — ADD MIGRATION NOTE

Include a clear migration note for existing users:

## Breaking Change: Web-Based Configuration

EOS Connect now uses a built-in web UI for all configuration.

**For existing users (upgrading)**:

- Your current settings are automatically migrated to the new system on first startup
- The addon config panel now only shows: Web Port, Time Zone, Log Level
- All other settings are managed through the EOS Connect web UI (Settings icon)
- No action needed — your configuration is preserved

**For new users**:

- After installation, open the EOS Connect web UI
- A Setup Wizard will guide you through initial configuration
- The addon config panel only needs Web Port, Time Zone, and Log Level

7. Migration Timing Strategy

Recommended rollout order:

eos_connect_develop/ first — Push the changes to the develop addon
Test with develop users (smaller audience, expects instability)
Verify: existing users get auto-migration, new installs get wizard
eos_connect/ after validation — Push to the stable addon

Critical test scenarios for the addon:

Scenario	Expected Result
Existing user upgrades (full options.json)	Auto-migration imports all settings to SQLite, wizard skipped, app works immediately
Existing user with only default values	Migration stores defaults, wizard appears for real configuration
New install (fresh options.json with 3 keys)	Fresh install, wizard appears, user configures via web UI
User downgrades to old addon version	Old version reads options.json which still has bootstrap keys — but all other settings are lost (document as one-way migration)

8. Summary of File Changes

File	Change Type	Effort
`config.yaml`	Rewrite — reduce from ~100 to 3 options	Medium (careful removal)
`translations/en.yaml`	Rewrite — reduce from ~200 to 3 entries	Low
`Dockerfile`	One-line removal — delete `ln -sf` symlink	Trivial
`build.yaml`	No changes	—
`.github/workflows/`	No changes	—
CHANGELOG / release notes	Add migration note	Low

Important: Apply identical changes to both eos_connect/ and eos_connect_develop/ directories. The only differences between them should be name, version, slug, and image fields in config.yaml.

Commit Message Guidelines

All commits follow Conventional Commits format for clear, scannable history.

Format

<type>: <description>

[optional body with details]

[optional footer with issue references]

Types

feat: New feature (e.g., feat: add battery forecast smoothing)
fix: Bug fix (e.g., fix: prevent startup crash on incomplete config)
docs: Documentation changes (README.md or GitHub Pages /docs folder)
test: Test additions or fixes (no production code change)
refactor: Code restructuring without feature/fix change
perf: Performance improvement
chore: Build, tooling, dependencies

Examples

Feature:

feat: implement two-tier PV config validation

- Lenient startup mode to prevent app crash on incomplete config
- Strict hot-reload mode to validate all changes
- Add configuration_state tracking for web UI visibility

Bug Fix:

fix: prevent startup crash on incomplete config - enable web UI access

- Implement two-tier validation (lenient startup, strict hot-reload)
- Remove sys.exit(1) calls on validation errors
- Graceful degradation: start in DEGRADED mode instead of crashing

Fixes: #259

Documentation:

docs: update configuration guide with new PV source options

Best Practices

Keep subject line ≤ 50 characters
Use imperative mood ("add", "fix", not "added", "fixed")
Include issue reference in footer: Fixes: #123 or Resolves: #123
Use body to explain why, not what (code shows what)
Link related issues: Related: #456, #789
If multiple fixes/features, break into separate commits for clarity

Trust

Not scanned yet. Artifacts are graded after they are crawled, so a recently discovered one may have no result for a while.

Versions

git-6d3e22d6ec842026-08-04

@ohand/eos-connect

Install

Document

GitHub Copilot Instructions for EOS Connect

⚠️ GIT WORKFLOW - MANDATORY RULES (APPLIES TO ALL COMMITS)

❌ ABSOLUTELY FORBIDDEN

✅ MANDATORY PROCESS (REQUIRED ORDER)

🔴 THE RULE THAT WAS VIOLATED

Project Guidelines

Icon Usage

Design Style

Documentation

Structure

Documentation Update Workflow

Documentation Perspective

Project Role Clarity

Code Style

Code Changes & Documentation Alignment

Commit Preparation

Testing Phase Documentation

Config Schema Maintenance

Section Ordering & Wizard Flow

Field Dependencies (Cross-Field Validation)

Config Web Module Architecture

Module Structure

Key Design Principles

Adding a New Config Field (Checklist)

Adding a New Config Section

REST API Endpoints (all under /api/config/)

SPOT Pipeline Flow

Testing

Design Rules & Lessons Learned

Configuration Architecture

Fresh Install vs Migration Detection

Web UI Integration Rules

Hot Reload Design

Validation & Security

Hot Reload — Current State & Expansion Priority

Currently Hot-Reloadable (21 fields, applied without restart)

Expansion Priority List

Adding a New Hot-Reload Field (Checklist)

Interface Creation & Startup Error Handling

InterfaceFactory (src/interface_factory.py)

StartupValidator (src/startup_validator.py)

Startup Error Flow

Log Message Format

Extending with New Interface Types

HA Addon Integration

Bootstrap Contract with ha_addons Repo

What the Addon Must NOT Do

What the Addon Must Provide

Legacy Migration Path

Appendix: ha_addons Repo Change Checklist for Web Config Migration

1. config.yaml (addon manifest) — MAJOR CHANGES

2. translations/en.yaml — MAJOR CHANGES

3. Dockerfile — CRITICAL CHANGE

4. build.yaml — NO CHANGES NEEDED

5. GitHub Actions Workflows — NO CHANGES NEEDED

6. CHANGELOG.md / Release Notes — ADD MIGRATION NOTE

7. Migration Timing Strategy

8. Summary of File Changes

Commit Message Guidelines

Format

Types

Examples

Best Practices

Trust

Versions

REST API Endpoints (all under `/api/config/`)

1. `config.yaml` (addon manifest) — MAJOR CHANGES

2. `translations/en.yaml` — MAJOR CHANGES

3. `Dockerfile` — CRITICAL CHANGE

4. `build.yaml` — NO CHANGES NEEDED

6. `CHANGELOG.md` / Release Notes — ADD MIGRATION NOTE