Files
silverlinux-infra/docs/backups.md
T

6.3 KiB

Backups

Overview

This document describes the backup strategy for SilverLinux.

The goal is to ensure that all critical services can be restored after hardware failure, accidental deletion, corruption, or disaster recovery scenarios.

Current state:

Backup v2 operational

Backup System Status

Backup v1.4

Implemented:

  • HTML backup reports
  • Email notification after backup
  • Automatic archive creation
  • Backup logging
  • Restore validation

Backup v1.5

Implemented:

  • MSSQL backup fix
  • Runtime MSSQL data included correctly
  • Restore tests verified MSSQL data is present in backups
  • Improved backup structure

Backup v2

Implemented:

  • Backup metrics generation
  • Prometheus metrics export
  • Backup duration metric
  • Backup size metric
  • Backup success metric
  • Automatic low disk space handling
  • Automatic deletion of oldest backups when required
  • Improved logging

Status:

Operational

Backup Priorities

Critical

These items must always be backed up.

PostgreSQL

Contains:

  • OpenProject database
  • Gitea database
  • Future application databases

Importance:

Critical


Microsoft SQL Server

Data location:

/srv/docker/mssql/data

Required:

  • Native SQL Server database backups
  • MSSQL Compose configuration
  • MSSQL_SA_PASSWORD from the shared secrets backup
  • Runtime MSSQL data included by the v1.5+ backup structure

Importance:

Critical

Restore validation has verified that MSSQL data is present in current backup archives.


Gitea Data

Location:

/srv/docker/gitea/data

Contains:

  • Git repositories
  • Attachments
  • Configuration
  • User data

Importance:

Critical


OpenProject Assets

Location:

/srv/docker/openproject/assets

Contains:

  • Attachments
  • Uploaded files
  • User generated content

Importance:

Critical


Shared Secrets

Location:

/srv/secrets/company.env

Contains:

  • SMTP credentials
  • PostgreSQL passwords
  • MSSQL SA password
  • OpenProject secrets
  • Future DbGate password
  • Future OAuth secrets

Importance:

Critical

Without this file applications may not start correctly.


Nginx Proxy Manager

Locations:

/srv/docker/nginx-proxy-manager/data
/srv/docker/nginx-proxy-manager/letsencrypt

Contains:

  • Proxy configuration
  • SSL certificates
  • Domain routing

Importance:

Critical


Medium Priority

Portainer

Volume:

portainer_portainer_data

Contains:

  • Portainer users
  • Portainer configuration
  • Stack definitions

Importance:

Medium

Can be recreated if necessary.


Jitsi Configuration

Location:

/srv/docker/jitsi

Contains:

  • Jitsi configuration
  • Internal authentication configuration

Importance:

Medium


BaGet Data

Location:

/srv/docker/baget/data

Contains:

  • Hosted NuGet packages
  • SQLite database

Importance:

Medium


Gitea Actions Runner Configuration

Location:

/srv/docker/gitea-runner

Contains:

  • Runner deployment configuration
  • Runner state

The registration token is stored separately in /srv/secrets/company.env.

Importance:

Medium


DbGate Data

Location:

/srv/docker/dbgate/data

Contains:

  • Saved connections
  • DbGate application state

Also retain the Nginx Proxy Manager route and DbGate authentication credential.

Importance:

Medium


Xray Configuration

Location:

/srv/docker/xray

Contains:

  • Xray Docker Compose configuration
  • Xray runtime configuration
  • Endpoint credentials, UUIDs or keys if stored with the service

Importance:

Medium

Xray credentials and keys must be backed up securely and must never be committed to Git.


Backup Storage

Current backup capabilities:

  • Automatic archive generation
  • HTML backup report generation
  • Backup log generation
  • Email report delivery
  • Prometheus metrics export
  • Automatic cleanup when disk space is low
  • Automatic deletion of oldest backups when required

Current Location:

/srv/backups

Suggested Structure:

/srv/backups/
├── daily
├── weekly
└── monthly

Retention Policy

Current cleanup behavior:

  • Backup v2 monitors available disk space.
  • If disk space is low, the oldest backups are deleted automatically.
  • Cleanup is logged.

Daily

Keep:

7 days

Weekly

Keep:

4 weeks

Monthly

Keep:

12 months

Restore Priorities

Restore order:

  1. Shared secrets
  2. PostgreSQL
  3. Microsoft SQL Server
  4. Nginx Proxy Manager
  5. Gitea
  6. OpenProject
  7. Portainer
  8. Jitsi
  9. BaGet
  10. Gitea Actions Runner
  11. DbGate
  12. Xray

Restore Validation

Restore procedure has been validated.

Completed:

  • Dry-run restore
  • Full archive validation
  • Archive extraction

Verified:

  • PostgreSQL
  • Microsoft SQL Server
  • Gitea
  • BaGet
  • OpenProject assets
  • Docker configuration

Validation results:

  • MSSQL backup integrity verified
  • PostgreSQL backup verified
  • Restore archive extraction verified

Monitoring Integration

Backup v2 exports metrics for Prometheus.

Current metrics:

  • Backup success
  • Backup duration
  • Backup size

Export path:

Node Exporter Textfile Collector

Prometheus target status:

Validated

Automation

Current automation:

  • Daily backup execution
  • Root cron configuration
  • Backup wrapper command
  • Email report
  • HTML report
  • Automatic cleanup
  • Automatic archive generation

Status:

Operational

Email Notifications

Postfix is configured to relay through Gmail SMTP.

Implemented:

  • Gmail relay
  • App Password authentication
  • TLS encryption
  • Successful email delivery
  • Backup success emails
  • Backup report emails

Status:

Operational

Disaster Recovery Checklist

Step 1

Install:

  • Ubuntu Server
  • Docker
  • Docker Compose

Step 2

Restore:

/srv/secrets/company.env

Step 3

Restore:

/srv/docker

data directories.

Step 4

Restore PostgreSQL and Microsoft SQL Server databases.

Step 5

Deploy Docker stacks.

Step 6

Verify:

  • DNS
  • SSL certificates
  • Email delivery
  • Application access

  • docs/server.md
  • docs/security.md
  • docs/secrets.md
  • postgres/README.md
  • gitea/README.md
  • openproject/README.md
  • mssql/README.md
  • dbgate/README.md
  • docs/roadmap.md