Oclick021
a81aa80161
- Create docs/sentry.md: Complete Sentry self-hosted deployment guide * Architecture overview with distributed system components * Core services: Relay, Kafka, Snuba, ClickHouse, PostgreSQL, Redis, SeaweedFS * Data flow and storage requirements * Management via slcompose orchestrator * Backup requirements documentation - Update AI_CONTEXT.md: Add Sentry to core services * List Sentry as error tracking & observability platform * Add sentry.silveressence.net domain mapping - Update README.md: Add Sentry to running services * Include in comprehensive services list - Update roadmap.md: Mark Sentry as completed * Move from planned to completed section * Document all deployed components * List completed validation items * Reference deployment documentation - Update services.md: Full Sentry service documentation * Add running Sentry service section * Document architecture, networking, and management * Remove from Phase 2 planned deployments * Link to detailed deployment guide
232 lines
6.1 KiB
Markdown
232 lines
6.1 KiB
Markdown
# SilverLinux AI Context
|
|
|
|
## Purpose
|
|
|
|
SilverLinux is the primary self-hosted infrastructure server for Silver Solutions.
|
|
|
|
The goal is to host all development, collaboration, deployment and internal tooling.
|
|
|
|
## Server
|
|
|
|
Hostname: silverlinux
|
|
|
|
OS: Ubuntu Server 24.04 LTS
|
|
|
|
Provider: OVH
|
|
|
|
Public IP: 51.255.83.140
|
|
|
|
## Core Services
|
|
|
|
* PostgreSQL
|
|
* Microsoft SQL Server 2022 Express
|
|
* Gitea
|
|
* OpenProject
|
|
* Jitsi Meet
|
|
* Portainer
|
|
* Nginx Proxy Manager
|
|
* BaGet
|
|
* Gitea Actions Runner (`silverlinux-runner`)
|
|
* DbGate
|
|
* Xray
|
|
* Sentry (Error Tracking & Observability)
|
|
* Prometheus
|
|
* Grafana
|
|
* Node Exporter
|
|
* cAdvisor
|
|
* Alertmanager
|
|
|
|
## Domains
|
|
|
|
git.silveressence.net -> Gitea
|
|
|
|
team.silveressence.net -> OpenProject
|
|
|
|
meet.silveressence.net -> Jitsi
|
|
|
|
portainer.silveressence.net -> Portainer
|
|
|
|
nuget.silveressence.net -> BaGet (through Nginx Proxy Manager)
|
|
|
|
dbgate.silveressence.net -> DbGate (through Nginx Proxy Manager)
|
|
|
|
sentry.silveressence.net -> Sentry (through Nginx Proxy Manager)
|
|
|
|
Xray -> Public tunneling endpoint through proxy network
|
|
|
|
## Secrets
|
|
|
|
Secrets are stored in Infisical and injected at runtime via the **slcompose orchestrator**.
|
|
|
|
Never hardcode credentials. All services receive environment variables through Infisical's secret injection at container startup.
|
|
|
|
### Secret Injection Flow
|
|
|
|
The `slcompose` service orchestrator manages all Docker services with automated secret injection:
|
|
|
|
```bash
|
|
slcompose boot # Boots all services with secrets injected
|
|
slcompose up <service> # Start a service with secrets injected
|
|
slcompose down <service> # Stop a service
|
|
slcompose restart <service> # Restart a service
|
|
slcompose logs <service> # Stream logs
|
|
slcompose logs-tail <service> [lines] # View last N lines
|
|
slcompose list # List all available services
|
|
```
|
|
|
|
**System Startup:** The systemd service `slcompose.service` automatically runs `slcompose boot` on server reboot.
|
|
|
|
**Implementation:** See [docs/slcompose.sh](slcompose.sh) and [docs/slcompose.service](slcompose.service).
|
|
|
|
## Service Orchestration
|
|
|
|
All Docker services are managed through **slcompose**, which:
|
|
|
|
1. **Loads Infisical Token** from `/etc/infisical/token`
|
|
2. **Discovers Services** by scanning `/srv/docker/` for directories with `docker-compose.yml`
|
|
3. **Injects Secrets** using `infisical run --path=/[SERVICE_NAME]` before executing docker compose commands
|
|
4. **Manages Lifecycle** with up, down, restart, and logs commands
|
|
|
|
Each service directory name (e.g., `/srv/docker/gitea`) becomes the Infisical path (e.g., `/gitea`) for secret lookup.
|
|
|
|
## Docker Networks
|
|
|
|
proxy
|
|
internal
|
|
jitsi-docker-jitsi-meet-aa76415_meet.jitsi
|
|
xray_default
|
|
gitea-runner_default
|
|
|
|
MSSQL is attached only to `internal`. DbGate is attached to both `internal` and `proxy`.
|
|
|
|
Jitsi uses `jitsi-docker-jitsi-meet-aa76415_meet.jitsi` for its internal cluster and `proxy` for web access. Xray is currently attached to `proxy`; `xray_default` exists but had no running containers attached in the 2026-06-24 runtime snapshot.
|
|
|
|
## Authentication
|
|
|
|
Gitea:
|
|
|
|
* Local accounts
|
|
* OpenID disabled
|
|
* Google OAuth planned
|
|
* GitHub OAuth planned
|
|
|
|
OpenProject:
|
|
|
|
* Local accounts
|
|
|
|
## CI/CD
|
|
|
|
Gitea Actions is enabled globally.
|
|
|
|
The global self-hosted runner is operational at `/srv/docker/gitea-runner` with the name `silverlinux-runner`.
|
|
|
|
Silver 2.0 uses `.gitea/workflows/package.yml` to publish NuGet packages to BaGet when a push to `net-8-version` has a commit message containing `[Package]`.
|
|
|
|
The runner registration token is stored in `/srv/secrets/company.env`. The BaGet publishing key is stored as the repository Actions secret `BAGET_API_KEY`.
|
|
|
|
## Databases
|
|
|
|
PostgreSQL 17 and Microsoft SQL Server 2022 Express are operational.
|
|
|
|
MSSQL uses hostname `mssql` on the `internal` Docker network. Port 1433 is not published.
|
|
|
|
DbGate provides authenticated HTTPS administration for MSSQL and PostgreSQL at `https://dbgate.silveressence.net`.
|
|
|
|
Xray is a connectivity layer, not an application backend. It runs from `ghcr.io/xtls/xray-core:latest` and publishes `0.0.0.0:32443 -> 8080/tcp`. Its endpoint exposure and access policy should be reviewed during security checks.
|
|
|
|
## Monitoring
|
|
|
|
The monitoring stack is operational.
|
|
|
|
Components:
|
|
|
|
* Prometheus
|
|
* Grafana
|
|
* Node Exporter
|
|
* cAdvisor
|
|
* Alertmanager
|
|
|
|
Validated:
|
|
|
|
* Prometheus UI
|
|
* Prometheus targets
|
|
* Node Exporter
|
|
* cAdvisor
|
|
* Backup metrics
|
|
* Grafana datasource
|
|
* Alertmanager email routing
|
|
|
|
Grafana has custom backup, system and Docker monitoring dashboards.
|
|
|
|
Alertmanager implements incident intelligence with incident groups:
|
|
|
|
* `backup_failure`
|
|
* `system_pressure`
|
|
* `storage_pressure`
|
|
* `database_outage`
|
|
* `monitoring_failure`
|
|
* `container_health`
|
|
|
|
## Backup
|
|
|
|
Backup v5.3 is production.
|
|
|
|
Implemented:
|
|
|
|
* Daily cron backup
|
|
* HTML reports
|
|
* Email reports through Postfix and Gmail SMTP
|
|
* Automatic archive generation
|
|
* Automatic cleanup
|
|
* Prometheus metrics export
|
|
* Restore validation
|
|
* Lock file protection
|
|
* Disk space validation
|
|
* Central logging
|
|
* Structured failure handling
|
|
|
|
Verified:
|
|
|
|
* PostgreSQL
|
|
* MSSQL
|
|
* Gitea
|
|
* BaGet
|
|
* OpenProject assets
|
|
* Docker configuration
|
|
|
|
## Documentation
|
|
|
|
Read all files in:
|
|
|
|
docs/
|
|
|
|
and the README.md of each service folder before making infrastructure recommendations.
|
|
|
|
Roadmap:
|
|
|
|
* Phase 1 completed: Grafana, Prometheus, Node Exporter, cAdvisor, Alertmanager, backup metrics integration, incident intelligence
|
|
* Phase 1 remaining: Loki, Redis, Authelia
|
|
* Phase 2 completed: Backup v5.3 automation, restore validation
|
|
* Phase 2 remaining: CrowdSec, Sentry, Fail2Ban upgrade, Cloudflare rate limiting and WAF
|
|
* Phase 3: SonarQube, Harbor, Backstage, Nextcloud
|
|
|
|
See:
|
|
|
|
```text
|
|
docs/roadmap.md
|
|
```
|
|
|
|
## Important Decisions
|
|
|
|
* Plane.so removed
|
|
* OpenProject selected
|
|
* Centralized secrets implemented
|
|
* Nginx Proxy Manager selected
|
|
* PostgreSQL selected as shared database platform
|
|
* BaGet deployed at nuget.silveressence.net through Nginx Proxy Manager
|
|
* Gitea Actions and a global self-hosted runner selected for package publishing
|
|
* MSSQL Express deployed alongside PostgreSQL
|
|
* DbGate selected as the database administration portal
|
|
* Xray introduced as a tunneling and connectivity layer
|
|
* Multi-network segmentation formalized: internal, proxy and isolated infrastructure networks
|