From a04804beb292d8a364c3b0871567e36f0a13ee51 Mon Sep 17 00:00:00 2001 From: root Date: Sun, 26 Apr 2026 18:48:48 +0800 Subject: [PATCH] docs: add database backup and snapshot design spec --- ...6-04-26-database-backup-snapshot-design.md | 239 ++++++++++++++++++ 1 file changed, 239 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-26-database-backup-snapshot-design.md diff --git a/docs/superpowers/specs/2026-04-26-database-backup-snapshot-design.md b/docs/superpowers/specs/2026-04-26-database-backup-snapshot-design.md new file mode 100644 index 00000000..9e3c8c8d --- /dev/null +++ b/docs/superpowers/specs/2026-04-26-database-backup-snapshot-design.md @@ -0,0 +1,239 @@ +# Database Backup & Snapshot Design + +Date: 2026-04-26 +Related Module: database, web, x-ui.sh +Change Type: Feature + +## Overview + +Add backup, scheduled snapshot, export (download), and restore functionality for both SQLite and MariaDB databases to the 3X-UI panel. Operable via the web panel UI and x-ui.sh CLI. + +## Scope + +- MariaDB backup/restore (primary goal) +- SQLite backup/restore (unified with MariaDB under the new system) +- Scheduled automatic snapshots (configurable frequency) +- Manual ad-hoc backups +- Backup export/download via panel +- Restore from backup via panel or CLI +- Backup retention policy (keep last N backups) +- Existing `getDb`/`importDB` endpoints remain unchanged for SQLite raw .db file operations + +## Architecture + +### New Files + +| File | Purpose | +|------|---------| +| `web/service/backup.go` | Backup service layer: mysqldump/sqlite3 dump execution, gzip, file management, scheduling | +| `web/controller/backup.go` | Backup API controller (HTTP handlers) | +| `web/html/settings/backup.html` | Backup management page (new tab in settings) | +| `web/job/backup_job.go` | Scheduled backup cron job | + +### Modified Files + +| File | Change | +|------|--------| +| `web/html/settings.html` | Add "Backup" tab entry | +| `web/job/job.go` | Register new backup cron job | +| `web/service/setting.go` | Add backup config read/write to AllSetting | +| `web/entity/entity.go` | Add backup config fields to AllSetting | +| `x-ui.sh` | Add `backup`/`restore`/`list-backups` subcommands; add backup menu items to `db_menu` | +| `main.go` | Register `backup` and `restore` CLI subcommands | + +### Dependency Graph + +``` +Settings Page (backup.html) + ↓ POST/GET +BackupController (web/controller/backup.go) + ↓ +BackupService (web/service/backup.go) + ↓ exec.Command("mysqldump") / exec.Command("sqlite3", ".dump") +MariaDB / SQLite Server + ↓ file I/O +/etc/x-ui/backups/ (tar.gz files) +``` + +Scheduling: +``` +web/job/backup_job.go → BackupService.CreateSnapshot() + ↑ reads schedule config + web/service/setting.go (backup config) +``` + +## API Endpoints + +All endpoints under `/panel/api/server`, registered in `web/controller/backup.go`. Admin authentication required for all. + +| Method | Route | Purpose | +|--------|-------|---------| +| POST | `/panel/api/server/backup` | Create an immediate manual backup | +| POST | `/panel/api/server/restore/:filename` | Restore from a specified backup file | +| POST | `/panel/api/server/deleteBackup/:filename` | Delete a specified backup file | +| GET | `/panel/api/server/listBackups` | List all backups (filename, size, timestamp) | +| GET | `/panel/api/server/downloadBackup/:filename` | Download a specified backup file | +| POST | `/panel/api/server/backupConfig` | Update backup configuration | + +Response format: standard `Msg{Success, Msg, Obj}` JSON. + +## Backup File Format + +All backups use `.tar.gz` archives with a consistent internal structure regardless of database type. + +### Storage + +- Directory: `/etc/x-ui/backups/` +- Filename: `backup-YYYY-MM-DD-HHmmss.tar.gz` + +### Archive Contents + +``` +backup-2026-04-26-030000.tar.gz +├── metadata.json # {"dbType":"mariadb"|"sqlite","timestamp":"RFC3339","version":"v1.7.2.x"} +└── dump.sql # mysqldump output (MariaDB) or sqlite3 .dump output (SQLite) +``` + +### SQLite Backup Process +1. `database.Checkpoint()` to flush WAL +2. `exec.Command("sqlite3", dbPath, ".dump")` to generate SQL dump +3. Create metadata.json with `dbType: "sqlite"` +4. Bundle into tar.gz + +### MariaDB Backup Process +1. Build DSN from config +2. `exec.Command("mysqldump", "--single-transaction", "--routines", "--triggers", ...)` +3. Create metadata.json with `dbType: "mariadb"` +4. Bundle into tar.gz + +## Restore Logic + +1. Read `metadata.json` from the archive to extract `dbType` +2. Compare `dbType` with current panel database type +3. If mismatch → reject with error: "Backup type (X) does not match current database (Y)" +4. If match → create a safety backup of current database before proceeding +5. Stop panel → restore → restart panel +6. Restoration: + - SQLite: drop all tables, import `.dump` via sqlite3 + - MariaDB: drop database, recreate, import via `mysql` client + +### Restore Flow + +``` +Read metadata.json → validate dbType match + → create safety backup of current DB + → systemctl stop x-ui (or internal stop) + → execute restore (sqlite3 or mysql client) + → systemctl start x-ui (or internal restart) + → verify panel accessible +``` + +Safety backup filename pattern: `pre-restore-YYYY-MM-DD-HHmmss.tar.gz`. These are excluded from the retention count and auto-deleted after 7 days. + +## Retention Policy + +- Configurable via `max_backups` setting (default: 10, range: 1-100) +- After each backup creation, if total count exceeds `max_backups`, delete oldest files first +- Manual and scheduled backups share the same retention pool +- Space protection: refuse new backup if disk free space < 100 MB or backup directory total > 500 MB + +## Scheduled Snapshots + +Configuration fields in AllSetting: + +| Field | Type | Default | Description | +|-------|------|---------|-------------| +| `backupEnabled` | bool | false | Enable scheduled backups | +| `backupFrequency` | string | "daily" | "hourly", "every12h", "daily", "weekly" | +| `backupHour` | int | 3 | Hour of day (0-23), used by daily and weekly | +| `backupMaxCount` | int | 10 | Max backups to retain | + +### Schedule Runtime + +| Frequency | When it runs | +|-----------|-------------| +| `hourly` | Minute 0 of every hour | +| `every12h` | 00:00 and 12:00 | +| `daily` | `backupHour:00` each day | +| `weekly` | Sunday at `backupHour:00` | + +Scheduling logic in `web/job/backup_job.go`: +- Check `backupEnabled` on each tick +- Execute backup via `BackupService.CreateSnapshot()` +- Apply retention after each successful backup +- Log errors (do not crash the panel) + +## Panel UI Design + +### Location + +New "Backup" tab in the settings page (`settings.html`). + +### Layout + +**Scheduled Backup Section:** +- Enable toggle switch +- Frequency dropdown: Every Hour / Every 12 Hours / Every Day / Every Week +- Hour picker (visible when daily or weekly selected) +- Max backups input (1-100) +- Save button + +**Manual Operations Section:** +- "Create Backup Now" button + +**Backup List Section:** +- Table with columns: Filename, Timestamp, Size, Actions +- Action buttons per row: Download, Restore, Delete +- Restore shows a warning confirmation dialog +- Delete requires secondary confirmation +- Auto-refresh every 30 seconds + +## x-ui.sh CLI + +### New Subcommands + +| Command | Purpose | +|---------|---------| +| `x-ui backup` | Create an immediate manual backup | +| `x-ui restore ` | Restore from `/etc/x-ui/backups/` | +| `x-ui list-backups` | List all backups with size and timestamp | + +### db_menu Additions (menu 27) + +New items after existing 1-16: + +| # | Label | Action | +|---|-------|--------| +| 17 | Create database backup | Call `x-ui backup` | +| 18 | Restore from backup | Interactive file selection, then restore | +| 19 | List all backups | Show backup list | +| 20 | Configure auto-backup | Interactive config (frequency, retention) | + +### Implementation + +- `backup` and `restore` subcommands delegate to Go binary: `${xui_folder}/x-ui backup` / `${xui_folder}/x-ui restore --file=` +- `list-backups` uses `ls -lh /etc/x-ui/backups/` +- Restore flow in x-ui.sh: validate → confirm → `systemctl stop x-ui` → `${xui_folder}/x-ui restore` → `systemctl start x-ui` + +## Error Handling + +- `mysqldump` or `sqlite3` not found on system → clear error message with install instructions +- Disk full → reject backup, notify user +- Backup file corrupted (invalid tar.gz or missing metadata.json) → reject restore +- Database type mismatch → reject with clear message +- Panel not stopped cleanly before restore → timeout and manual recovery instructions + +## Testing + +- Unit tests for `BackupService` methods (mock exec.Command where possible) +- Integration test: create backup → verify contents → restore → verify data integrity +- Test both SQLite and MariaDB paths +- Test retention policy (create N+1 backups, verify oldest deleted) +- Test dbType mismatch rejection + +## Risks And Follow-Up + +- `mysqldump` may not be installed on all systems; x-ui.sh already has `install_mariadb_client` that can install it. Panel UI should detect and surface this. +- Large database backups may take significant time; UI should show progress or timeout gracefully. +- Restore requires panel downtime (stop/start). This is communicated to the user before confirmation. +- Existing `getDb`/`importDB` endpoints remain unchanged for backward compatibility.