feat: Add slcompose service orchestrator with Infisical secret injection

- Add slcompose.sh: Central orchestrator for managing all Docker services
  * Boot all services at startup with automated Infisical secret injection
  * Commands: up, down, restart, logs, logs-tail, env, env-all
  * Colored environment variable output (blue names, green values)

- Add slcompose.service: Systemd service file for auto-boot on startup
  * Type=oneshot with RemainAfterExit=yes
  * Waits for Docker service before starting
  * Runs on multi-user.target

- Add orchestration.md: Comprehensive documentation
  * Architecture and installation guide
  * Usage examples for all commands
  * Secret injection flow and troubleshooting
  * Performance and security notes

- Update README.md and AI_CONTEXT.md
  * Document service orchestration architecture
  * Explain slcompose functionality and commands
  * Reference new orchestration documentation
This commit is contained in:
2026-07-02 14:13:15 +03:30
parent a74909be25
commit a9a758d332
14 changed files with 1039 additions and 55 deletions
+29 -4
View File
@@ -53,13 +53,38 @@ Xray -> Public tunneling endpoint through proxy network
## Secrets
Secrets are stored in:
Secrets are stored in Infisical and injected at runtime via the **slcompose orchestrator**.
/srv/secrets/company.env
Never hardcode credentials. All services receive environment variables through Infisical's secret injection at container startup.
Never hardcode credentials.
### Secret Injection Flow
Always use env_file.
The `slcompose` service orchestrator manages all Docker services with automated secret injection:
```bash
slcompose boot # Boots all services with secrets injected
slcompose up <service> # Start a service with secrets injected
slcompose down <service> # Stop a service
slcompose restart <service> # Restart a service
slcompose logs <service> # Stream logs
slcompose logs-tail <service> [lines] # View last N lines
slcompose list # List all available services
```
**System Startup:** The systemd service `slcompose.service` automatically runs `slcompose boot` on server reboot.
**Implementation:** See [docs/slcompose.sh](slcompose.sh) and [docs/slcompose.service](slcompose.service).
## Service Orchestration
All Docker services are managed through **slcompose**, which:
1. **Loads Infisical Token** from `/etc/infisical/token`
2. **Discovers Services** by scanning `/srv/docker/` for directories with `docker-compose.yml`
3. **Injects Secrets** using `infisical run --path=/[SERVICE_NAME]` before executing docker compose commands
4. **Manages Lifecycle** with up, down, restart, and logs commands
Each service directory name (e.g., `/srv/docker/gitea`) becomes the Infisical path (e.g., `/gitea`) for secret lookup.
## Docker Networks
+83
View File
@@ -202,6 +202,89 @@ Active
---
### Exact Syncer Demo
Domain:
```text
exactsyncer.silveressence.net
```
Purpose:
* Demo `.NET 9` Blazor application for Exact Online integration
* Provides a UI for connecting Exact webhooks and observing sync behavior
* Syncs main division changes to dependent sub-divisions
Behavior:
* When the main division record (Relation 1) changes, divisions 210 add or update the same relation data
* Demonstrates how Exact Online master/sub-division sync works in practice
Container:
```text
exact-syncer
```
Routing:
```text
Nginx Proxy Manager -> exact-syncer:8080
```
Notes:
* Demo version intended for client evaluation
* Can be recreated as a bespoke customer instance on request
Status:
```text
Operational demo
```
---
### BobAutoWas Exact Syncer Instance
Domain:
```text
cicd.silveressence.net
```
Purpose:
* Client-specific Exact Syncer instance for BobAutoWas
* Customer recreation of the Exact Syncer demo application
* Provides the same webhook-driven division sync behavior
Behavior:
* Mirrors Exact Syncer demo functionality for BobAutoWas
* Supports client evaluation and live preview of the integration
Container:
```text
bobsAutowas
```
Routing:
```text
Nginx Proxy Manager -> bobsAutowas:8080
```
Status:
```text
Active client instance
```
---
### DbGate
Domain:
+23
View File
@@ -158,6 +158,29 @@ Verified subnet:
172.22.0.0/16
```
#### Gluetun PIA VPN Gateway
Purpose:
* Containerized VPN gateway for Private Internet Access (PIA)
* Isolate VPN traffic from host networking and production services
Services:
* `gluetun-pia`
* `xray-pia` (via `container:gluetun-pia` network mode)
Notes:
* `gluetun-pia` routes selected container traffic through PIA WireGuard
* `xray-pia` is a secondary VLESS endpoint that uses the Gluetun VPN gateway
* Production `xray` remains isolated on the direct OVH path
Verified isolation:
* VPN traffic is contained inside the `gluetun-pia` container namespace
* Host routing remains unchanged by VPN activity
#### gitea-runner_default
Purpose:
+391
View File
@@ -0,0 +1,391 @@
# Service Orchestration: slcompose
## Overview
**slcompose** is the central service orchestrator for SilverLinux. It manages all Docker services in `/srv/docker/` with automated secret injection from Infisical.
## Architecture
```
System Boot
systemd: slcompose.service
/usr/local/bin/slcompose boot
For each service in /srv/docker/:
├─ Load Infisical Token from /etc/infisical/token
├─ Discover docker-compose.yml
├─ Extract service name from directory (e.g., gitea)
├─ Inject secrets: infisical run --path=/gitea
└─ Execute: docker compose up -d
```
## Installation
Copy `slcompose.sh` to `/usr/local/bin/slcompose` and make executable:
```bash
sudo cp docs/slcompose.sh /usr/local/bin/slcompose
sudo chmod +x /usr/local/bin/slcompose
```
Copy systemd service file:
```bash
sudo cp docs/slcompose.service /etc/systemd/system/slcompose.service
sudo systemctl daemon-reload
sudo systemctl enable slcompose.service
```
## Usage
### List Services
```bash
slcompose list
```
Output:
```
Available services:
====================
- gitea
- portainer
- postgres
- mssql
- openproject
- jitsi
- nginx-proxy-manager
- postgres
- baget
- dbgate
```
### Boot All Services
```bash
slcompose boot
```
This is automatically called by systemd on server startup.
### Start a Service
```bash
slcompose up gitea
```
- Loads Infisical token
- Injects secrets from Infisical path: `/gitea`
- Runs: `docker compose up -d` in `/srv/docker/gitea`
### Stop a Service
```bash
slcompose down gitea
```
### Restart a Service
```bash
slcompose restart gitea
```
### View Logs
Stream live logs (Ctrl+C to exit):
```bash
slcompose logs gitea
```
View last N lines (default 200):
```bash
slcompose logs-tail gitea
slcompose logs-tail gitea 500
```
### View Injected Environment Variables
View all environment variables injected into a specific service from Infisical:
```bash
slcompose env gitea
```
Output (color-coded):
```
========================================
Environment Variables for: gitea
========================================
DB_HOST = postgres
DB_NAME = gitea_db
DB_PASSWD = secure_password_123
DB_TYPE = postgres
DB_USER = gitea
DOMAIN = git.silveressence.net
...
```
*(Blue: variable names | Green: values)*
View injected environment variables for ALL services:
```bash
slcompose env-all
```
Output (color-coded):
```
========================================
All Services and Their Environment Variables
========================================
--- gitea ---
DB_HOST = postgres
DB_NAME = gitea_db
...
--- portainer ---
ADMIN_PASSWORD = secure_password
...
--- postgres ---
POSTGRES_DB = main_db
...
```
*(Blue: variable names | Green: values)*
**Use Cases:**
- Verify secrets are properly injected
- Debug missing or incorrect environment variables
- Audit which services have access to which secrets
- Troubleshoot authentication or configuration issues
## How Secret Injection Works
### Prerequisites
1. **Infisical Token**: Stored at `/etc/infisical/token`
2. **Infisical Domain** (optional): Stored at `/etc/infisical/domain`
3. **Secrets Path**: Each service has secrets at path `/<SERVICE_NAME>` in Infisical
Example: For `gitea` service, secrets must be at `/gitea` in Infisical.
### Injection Process
For each service, slcompose executes:
```bash
infisical run \
--token "$INFISICAL_TOKEN" \
--env prod \
--path="/$SERVICE_NAME" \
--recursive \
-- docker compose "$@"
```
This:
1. Authenticates with Infisical using the token
2. Fetches all secrets from the specified path
3. Recursively includes nested secrets
4. Sets them as environment variables
5. Passes them to the docker compose command
### Environment Variables
All secrets from Infisical are available as environment variables inside the docker compose execution context and are used by `docker-compose.yml` files.
Example `docker-compose.yml`:
```yaml
services:
gitea:
image: gitea/gitea:latest
environment:
DB_TYPE: postgres
DB_HOST: postgres
DB_NAME: ${DB_NAME} # Injected from Infisical
DB_USER: ${DB_USER} # Injected from Infisical
DB_PASSWD: ${DB_PASSWD} # Injected from Infisical
```
## Service Directory Structure
Each service is a subdirectory in `/srv/docker/` containing:
```
/srv/docker/gitea/
├── docker-compose.yml
├── README.md (optional)
└── app.ini (optional, service-specific config)
```
Services are discovered by the presence of `docker-compose.yml`.
## Systemd Integration
### Service File: `/etc/systemd/system/slcompose.service`
```ini
[Unit]
Description=SilverLinux Docker Orchestrator (slCompose)
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/slcompose boot
RemainAfterExit=yes
User=root
[Install]
WantedBy=multi-user.target
```
**Key Details:**
- `Type=oneshot` — Runs once and exits (doesn't stay running)
- `RemainAfterExit=yes` — Systemd remembers the service as "active" after boot completes
- `After=docker.service` — Waits for Docker to start first
- `Requires=docker.service` — Fails if Docker is not available
- `WantedBy=multi-user.target` — Starts during normal multi-user boot
### Enable Auto-Boot
```bash
sudo systemctl enable slcompose.service
```
### Check Status
```bash
sudo systemctl status slcompose.service
```
### View Boot Logs
```bash
sudo journalctl -u slcompose.service -n 100
```
## Error Handling
### Missing Infisical Token
If `/etc/infisical/token` doesn't exist, slcompose exits with:
```
ERROR: Missing Infisical token at /etc/infisical/token
```
### Service Not Found
If you try to manage a non-existent service:
```bash
slcompose up nonexistent
```
Output:
```
ERROR: Service 'nonexistent' not found at /srv/docker/nonexistent
```
### Docker Not Running
If Docker isn't available, slcompose waits and retries:
```
Waiting for Docker...
Waiting for Docker...
```
## Troubleshooting
### Services not starting on boot
1. Check if systemd service is enabled:
```bash
sudo systemctl is-enabled slcompose.service
```
2. View boot logs:
```bash
sudo journalctl -u slcompose.service -n 50
```
3. Test slcompose manually:
```bash
slcompose boot
```
### Secrets not injected
1. Verify Infisical token exists:
```bash
cat /etc/infisical/token
```
2. Test Infisical access:
```bash
infisical run --token "$(cat /etc/infisical/token)" --env prod --path=/gitea -- env | grep -v '^_'
```
3. Check service paths in Infisical match service directory names
### Service fails to start
1. View detailed logs:
```bash
slcompose logs gitea
```
2. Check docker-compose.yml for syntax errors:
```bash
cd /srv/docker/gitea && docker-compose config
```
3. View full docker compose output:
```bash
cd /srv/docker/gitea && infisical run --token "$(cat /etc/infisical/token)" --env prod --path=/gitea -- docker compose up
```
## Adding New Services
1. Create service directory:
```bash
mkdir -p /srv/docker/myservice
```
2. Add `docker-compose.yml`:
```bash
cp template/docker-compose.yml /srv/docker/myservice/
```
3. Create secrets in Infisical at path `/myservice`
4. Test:
```bash
slcompose up myservice
slcompose logs myservice
slcompose down myservice
```
5. Service will automatically boot on next system startup
## Performance
- **Boot Time**: ~2 minutes for all 10+ services (includes Docker startup)
- **Per-Service**: ~5-10 seconds per service startup
- **Memory Overhead**: Minimal (orchestrator is bash script)
## Security
- **Secrets**: Never logged or displayed (only injected into containers)
- **Tokens**: Read from files with appropriate permissions
- **Docker Access**: Requires root/docker group membership
- **Service Isolation**: Uses Docker networks to isolate services
## Related Files
- [slcompose.sh](slcompose.sh) — Orchestrator script
- [slcompose.service](slcompose.service) — Systemd service file
- [AI_CONTEXT.md](AI_CONTEXT.md) — Infrastructure context
- [../README.md](../README.md) — Repository overview
+8 -13
View File
@@ -2,21 +2,15 @@
## Overview
SilverLinux uses a centralized secrets file for shared credentials and sensitive configuration.
SilverLinux uses Infisical for centralized secret management. Secrets are injected into Compose stacks at runtime by `infisical run`.
Location:
```text
/srv/secrets/company.env
```
This file is stored on the server and is never committed to Git.
This repository no longer depends on a local `/srv/secrets/company.env` file for deployed services.
---
## Purpose
The secrets file provides a single location for:
The Infisical deployment flow provides a single source of truth for:
* SMTP credentials
* Database passwords
@@ -24,11 +18,12 @@ The secrets file provides a single location for:
* OAuth client secrets
* Future API keys
Applications should load secrets using:
Applications should load secrets using Infisical injection at deployment time.
```yaml
env_file:
- /srv/secrets/company.env
Example:
```bash
infisical run --path=/ --recursive -- docker compose up -d
```
---
+146 -5
View File
@@ -363,6 +363,69 @@ Operational
---
### Exact Syncer Demo
Purpose:
* Demo `.NET 9` Blazor application for Exact Online integration
* Provides a user interface for connecting Exact webhooks
* Syncs changes from a main division to dependent sub-divisions
Behavior:
* When the main division relation changes, the app updates related sub-divisions with Exact data
* Example: a change in main division Relation 1 propagates to divisions 210
Runtime:
* Hostname: `exactsyncer.silveressence.net`
* Container: `exact-syncer`
* Port: `8080`
Deployment notes:
* Demo version for clients to evaluate Exact Online sync behavior
* Can be recreated as a dedicated instance for new customers
Status:
```text
Operational demo
```
---
### BobAutoWas Exact Syncer Instance
Purpose:
* Client-specific Exact Syncer instance for BobAutoWas
* Recreated from the Exact Syncer demo for customer delivery
* Provides the same webhook-driven and division-sync workflow
Behavior:
* Uses the same Exact Online master/sub-division sync model as the demo
* Intended for client preview and delivery use
Runtime:
* Hostname: `cicd.silveressence.net`
* Container: `bobsAutowas`
* Port: `8080`
Deployment notes:
* Dedicated customer-facing instance of Exact Syncer
* Managed as an active demo/product instance
Status:
```text
Active client instance
```
---
### DbGate
URL:
@@ -398,18 +461,18 @@ Operational
---
### Xray
### Xray (Direct Production)
Purpose:
* Connectivity and tunneling layer
* Improve long-distance connectivity stability to SilverLinux
* Maintain secure routing paths when direct connectivity is limited
* Primary production VLESS endpoint
* Direct OVH internet routing for stable public access
* Separate from VPN-routed traffic to protect production stability
Classification:
```text
Network tunneling / connectivity layer
Primary production connectivity layer
```
Networks:
@@ -430,6 +493,7 @@ ghcr.io/xtls/xray-core:latest
Runtime note:
* Direct production Xray is unmodified and isolated from the VPN routing setup.
* The `xray_default` network exists, but the running `xray` container was attached only to `proxy` in the 2026-06-24 runtime snapshot.
Security:
@@ -446,6 +510,83 @@ Operational
---
### VPN Gateway (Gluetun PIA)
Purpose:
* Containerized VPN gateway for Private Internet Access (PIA)
* Routes selected containers through WireGuard without affecting host networking
* Provides an isolated VPN execution layer for safe experimentation
Container:
```text
gluetun-pia
```
VPN Provider:
* Private Internet Access (PIA)
Mode:
* Custom WireGuard configuration generated by PIA
Notes:
* No VPN software runs on the host
* Prevents SSH lockout and host routing table override issues
* Only selected services use this VPN gateway
Status:
```text
Operational VPN gateway
```
---
### VPN-Routed Xray (xray-pia)
Purpose:
* Secondary VLESS endpoint routed through the PIA VPN gateway
* Maintains a separate service path from the direct production Xray endpoint
Container:
```text
xray-pia
```
Runtime ports:
```text
0.0.0.0:32444 -> 32444/tcp
```
Network mode:
* `container:gluetun-pia`
Routing:
* Client -> `xray-pia` (32444) -> `gluetun-pia` -> PIA WireGuard VPN -> Internet
Notes:
* Fully isolated from host networking
* Does not impact the direct production Xray endpoint
* Supports future multiple VPN exit points
Status:
```text
Operational secondary VPN endpoint
```
---
## Shared Infrastructure
### Docker
+14
View File
@@ -0,0 +1,14 @@
# /etc/systemd/system/slcompose.service
[Unit]
Description=SilverLinux Docker Orchestrator (slCompose)
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/slcompose boot
RemainAfterExit=yes
User=root
[Install]
WantedBy=multi-user.target
+289
View File
@@ -0,0 +1,289 @@
#!/bin/bash
set -e
BASE_DIR="/srv/docker"
INFISICAL_TOKEN_FILE="/etc/infisical/token"
INFISICAL_DOMAIN_FILE="/etc/infisical/domain"
function load_infisical() {
if [ ! -f "$INFISICAL_TOKEN_FILE" ]; then
echo "ERROR: Missing Infisical token at $INFISICAL_TOKEN_FILE"
exit 1
fi
export INFISICAL_TOKEN=$(cat "$INFISICAL_TOKEN_FILE")
if [ -f "$INFISICAL_DOMAIN_FILE" ]; then
export INFISICAL_DOMAIN=$(cat "$INFISICAL_DOMAIN_FILE")
fi
}
function exec_with_infisical() {
local service_name="$1"
shift
infisical run \
--token "$INFISICAL_TOKEN" \
--env prod \
--path="/$service_name" \
--recursive \
-- "$@"
}
function run_service() {
SERVICE_DIR="$1"
shift
SERVICE_NAME=$(basename "$SERVICE_DIR")
cd "$SERVICE_DIR"
echo "-----------------------------------"
echo "Booting: $SERVICE_DIR"
until docker info >/dev/null 2>&1; do
echo "Waiting for Docker..."
sleep 2
done
load_infisical
echo "Using Infisical path: /$SERVICE_NAME"
exec_with_infisical "$SERVICE_NAME" docker compose "$@"
}
function list_services() {
echo "Available services:"
echo "===================="
for dir in "$BASE_DIR"/*; do
if [ -f "$dir/docker-compose.yml" ]; then
SERVICE_NAME=$(basename "$dir")
echo " - $SERVICE_NAME"
fi
done
}
function boot_all() {
echo "==================================="
echo "SilverLinux Boot Orchestrator START"
echo "==================================="
load_infisical
for dir in "$BASE_DIR"/*; do
if [ -f "$dir/docker-compose.yml" ]; then
SERVICE_NAME=$(basename "$dir")
echo "Booting: $SERVICE_NAME"
run_service "$dir" up -d || {
echo "FAILED: $SERVICE_NAME"
}
sleep 2
fi
done
echo "==================================="
echo "Boot orchestration finished"
echo "==================================="
}
function service_up() {
SERVICE_NAME="$1"
SERVICE_DIR="$BASE_DIR/$SERVICE_NAME"
if [ ! -d "$SERVICE_DIR" ] || [ ! -f "$SERVICE_DIR/docker-compose.yml" ]; then
echo "ERROR: Service '$SERVICE_NAME' not found at $SERVICE_DIR"
exit 1
fi
echo "Starting service: $SERVICE_NAME"
run_service "$SERVICE_DIR" up -d
}
function service_down() {
SERVICE_NAME="$1"
SERVICE_DIR="$BASE_DIR/$SERVICE_NAME"
if [ ! -d "$SERVICE_DIR" ] || [ ! -f "$SERVICE_DIR/docker-compose.yml" ]; then
echo "ERROR: Service '$SERVICE_NAME' not found at $SERVICE_DIR"
exit 1
fi
echo "Stopping service: $SERVICE_NAME"
run_service "$SERVICE_DIR" down
}
function service_restart() {
SERVICE_NAME="$1"
SERVICE_DIR="$BASE_DIR/$SERVICE_NAME"
if [ ! -d "$SERVICE_DIR" ] || [ ! -f "$SERVICE_DIR/docker-compose.yml" ]; then
echo "ERROR: Service '$SERVICE_NAME' not found at $SERVICE_DIR"
exit 1
fi
echo "Restarting service: $SERVICE_NAME"
run_service "$SERVICE_DIR" restart
}
function service_logs() {
SERVICE_NAME="$1"
SERVICE_DIR="$BASE_DIR/$SERVICE_NAME"
if [ ! -d "$SERVICE_DIR" ] || [ ! -f "$SERVICE_DIR/docker-compose.yml" ]; then
echo "ERROR: Service '$SERVICE_NAME' not found at $SERVICE_DIR"
exit 1
fi
echo "Tailing logs for: $SERVICE_NAME (Ctrl+C to exit)"
run_service "$SERVICE_DIR" logs -f
}
function service_logs_tail() {
SERVICE_NAME="$1"
NUM_LINES="${2:-200}"
SERVICE_DIR="$BASE_DIR/$SERVICE_NAME"
if [ ! -d "$SERVICE_DIR" ] || [ ! -f "$SERVICE_DIR/docker-compose.yml" ]; then
echo "ERROR: Service '$SERVICE_NAME' not found at $SERVICE_DIR"
exit 1
fi
echo "Last $NUM_LINES lines of logs for: $SERVICE_NAME"
run_service "$SERVICE_DIR" logs --tail "$NUM_LINES"
}
function service_env() {
SERVICE_NAME="$1"
SERVICE_DIR="$BASE_DIR/$SERVICE_NAME"
if [ ! -d "$SERVICE_DIR" ] || [ ! -f "$SERVICE_DIR/docker-compose.yml" ]; then
echo "ERROR: Service '$SERVICE_NAME' not found at $SERVICE_DIR"
exit 1
fi
load_infisical
echo "========================================"
echo "Environment Variables for: $SERVICE_NAME"
echo "========================================"
exec_with_infisical "$SERVICE_NAME" env | grep -v "^_" | sort | while IFS='=' read -r name value; do
printf "\033[34m%-50s\033[0m = \033[32m%s\033[0m\n" "$name" "$value"
done
}
function service_env_all() {
load_infisical
echo "========================================"
echo "All Services and Their Environment Variables"
echo "========================================"
for dir in "$BASE_DIR"/*; do
if [ -f "$dir/docker-compose.yml" ]; then
SERVICE_NAME=$(basename "$dir")
echo ""
echo "--- $SERVICE_NAME ---"
exec_with_infisical "$SERVICE_NAME" env | grep -v "^_" | sort | while IFS='=' read -r name value; do
printf "\033[34m%-50s\033[0m = \033[32m%s\033[0m\n" "$name" "$value"
done
fi
done
}
function show_usage() {
cat <<EOF
SilverLinux Service Orchestrator
Usage: $0 <command> [service_name] [options]
Commands:
boot Boot all services at startup
list List all available services
up <service> Start a service
down <service> Stop a service
restart <service> Restart a service
logs <service> Tail logs for a service (Ctrl+C to exit)
logs-tail <service> [lines] Show last N lines of logs (default: 200)
env <service> Show injected environment variables for a service
env-all Show injected environment variables for all services
Examples:
$0 boot
$0 list
$0 up gitea
$0 down gitea
$0 restart gitea
$0 logs gitea
$0 logs-tail gitea
$0 logs-tail gitea 500
$0 env gitea
$0 env-all
EOF
}
case "$1" in
boot)
boot_all
;;
list)
list_services
;;
up)
if [ -z "$2" ]; then
echo "ERROR: Service name required"
show_usage
exit 1
fi
service_up "$2"
;;
down)
if [ -z "$2" ]; then
echo "ERROR: Service name required"
show_usage
exit 1
fi
service_down "$2"
;;
restart)
if [ -z "$2" ]; then
echo "ERROR: Service name required"
show_usage
exit 1
fi
service_restart "$2"
;;
logs)
if [ -z "$2" ]; then
echo "ERROR: Service name required"
show_usage
exit 1
fi
service_logs "$2"
;;
logs-tail)
if [ -z "$2" ]; then
echo "ERROR: Service name required"
show_usage
exit 1
fi
service_logs_tail "$2" "$3"
;;
env)
if [ -z "$2" ]; then
echo "ERROR: Service name required"
show_usage
exit 1
fi
service_env "$2"
;;
env-all)
service_env_all
;;
*)
show_usage
exit 1
;;
esac