Maintenance
Upgrade Procedures, Key Rotation, and Scheduled Maintenance
Document Version: 1.1.0
Last Updated: 2026-04-07
1. Maintenance Schedule
| Task |
Frequency |
Downtime Required |
Priority |
| Platform upgrade |
As released |
Yes (rolling for K8s) |
High |
| Database backup verification |
Weekly |
No |
High |
| TLS certificate renewal |
Before expiry |
No (reload) |
Critical |
| Secret cipher key rotation |
90 days |
No (background) |
High |
| JWT signing key rotation |
On demand |
No |
Medium |
| Master key rotation |
Annual |
Yes (planned) |
Critical |
| Database vacuum/analyze |
Weekly |
No |
Medium |
| Log rotation/cleanup |
Daily |
No |
Low |
| DR test |
Quarterly |
No (test env) |
High |
Pre-Upgrade Checklist
- [ ] Review release notes for breaking changes
- [ ] Create full database backup
- [ ] Verify backup integrity
- [ ] Document current version and configuration
- [ ] Schedule maintenance window (if required)
- [ ] Notify affected users
Kubernetes Upgrade
# Update Helm repo
helm repo update
# Diff changes
helm diff upgrade mazevault mazevault/mazevault \
-n mazevault -f values-production.yaml
# Create backup
kubectl exec -n mazevault deploy/mazevault-backend -- \
pg_dump -h $DB_HOST -U $DB_USER $DB_NAME | gzip > backup-pre-upgrade.sql.gz
# Upgrade
helm upgrade mazevault mazevault/mazevault \
-n mazevault -f values-production.yaml \
--wait --timeout 10m
# Verify
kubectl get pods -n mazevault
curl -sk https://vault.example.com/api/v1/health | jq .
On-Premise Upgrade
cd /opt/mazevault
# Backup
./scripts/mazevault-backup.sh
# Pull new images (connected) or load from offline package (air-gapped)
docker compose pull
# OR
docker load < images/mazevault-backend-1.0.17.tar
docker load < images/mazevault-frontend-1.0.17.tar
docker load < images/mazevault-ocsp-1.0.17.tar
# Update image tags in docker-compose.yml
# Apply upgrade
docker compose up -d
# Verify
docker compose ps
curl -sk https://localhost/api/v1/health | jq .
Post-Upgrade Verification
- [ ] All containers/pods running
- [ ] Health endpoint returns
healthy
- [ ] Database migrations applied successfully
- [ ] Secret access verified (read a test secret)
- [ ] Certificate operations verified
- [ ] Agent heartbeats resuming
- [ ] Audit log recording events
- [ ] License validation successful
Rollback Procedure
# Kubernetes
helm rollback mazevault -n mazevault
# On-Premise
docker compose down
# Restore previous image tags in docker-compose.yml
docker compose up -d
# Restore database if migration was applied
3. Database Maintenance
Vacuum and Analyze
# On-Premise
docker exec client-postgres psql -U mazevault -d mazevault -c "VACUUM ANALYZE;"
# Azure
# Automatic for Azure Database for PostgreSQL Flexible Server
Reindex (if needed)
docker exec client-postgres psql -U mazevault -d mazevault -c "REINDEX DATABASE mazevault;"
Monitor Database Size
docker exec client-postgres psql -U mazevault -d mazevault -c "
SELECT pg_size_pretty(pg_database_size('mazevault')) as db_size;
"
4. Secret Rotation Schedule
| Key Type |
Rotation |
How |
| Secret cipher keys |
Every 90 days |
Automatic (background re-encryption) |
| JWT signing keys |
On demand |
API call or admin interface |
| Master encryption key |
Annual |
Planned maintenance window |
| API keys |
Organization policy |
Manual via admin interface |
JWT Key Rotation
Rotate JWT signing keys via the admin interface:
Settings → Security → JWT Signing Keys → Rotate
The system performs seamless key rollover — existing tokens remain valid until expiry, and new tokens are signed with the new key.
5. Certificate Maintenance
CA Key Rotation
Root CA keys should be rotated according to your organization's certificate policy. Typical schedules:
| CA Type |
Recommended Lifetime |
| Root CA |
10-20 years |
| Intermediate CA |
5-10 years |
| Issuing CA |
2-5 years |
CRL Maintenance
CRLs are regenerated automatically. Verify CRL freshness:
curl -sk https://vault.example.com/api/v1/crl/pem | \
openssl crl -inform PEM -noout -lastupdate -nextupdate
6. Maintenance Window Communication
Template for maintenance notification:
Subject: MazeVault Scheduled Maintenance — [DATE]
Dear Team,
MazeVault will undergo scheduled maintenance on [DATE] from [TIME] to [TIME] (UTC).
During this window:
- The platform will be [briefly unavailable / in read-only mode]
- Affect: [describe impact]
Actions required:
- [Any pre-maintenance actions]
Post-maintenance:
- Service will be restored automatically
- Verify access at https://vault.example.com
For questions, contact: [support contact]
7. Automated System Jobs
MazeVault runs several background jobs automatically. These do not require manual intervention but are listed here for operational awareness, firewall planning, and troubleshooting.
Backend Jobs
Certificate Management
| # |
Job |
Interval |
Description |
| 1 |
Certificate lifecycle scheduler |
1h + 6h |
Marks expired certificates, checks auto-renewal eligibility, auto-archives expired items. Syncs status from external CAs every 6 hours |
| 2 |
Certificate renewal scheduler |
1h |
Detects certificates nearing expiry, enqueues renewal tasks, processes renewal queue (5 concurrent workers, PostgreSQL advisory lock) |
| 3 |
CRL regeneration |
168h full / 24h delta |
Regenerates Certificate Revocation Lists per CA. Defaults: full CRL weekly, delta CRL daily (configurable per CA) |
| 4 |
CA sync |
5 min |
Discovers and imports certificates from connected external CA accounts |
| 5 |
ACME server cleanup |
15 min / 30 min |
Cleans up expired ACME nonces (15 min) and expired certificate orders (30 min) |
Secrets & Rotation
| # |
Job |
Interval |
Description |
| 6 |
Rotation scheduler |
1 min |
Detects due secret and certificate rotations. Executes via 3-worker pool with database row-level locking and exponential backoff |
| 7 |
Sync rules engine |
1 min |
Evaluates secret synchronization rules and triggers sync to configured external targets |
| 8 |
Key rotation |
24h |
Rotates internal system keys (RSA signing, cipher, API keys) with configurable grace periods |
Notifications & Reporting
| # |
Job |
Interval |
Description |
| 9 |
Notification scheduler |
24h |
Sends expiry alerts for certificates expiring within 30 days and rotation failure alerts via configured channels (Teams, Slack, Email, JIRA) |
| 10 |
Weekly report |
1 min check |
Generates weekly compliance and audit summary report. Triggers every Monday at 7:00 AM (Europe/Prague) |
Azure Entra ID
| # |
Job |
Interval |
Description |
| 11 |
Entra ID sync |
1h |
Synchronizes Azure Entra ID app registrations and their credentials |
| 12 |
Entra credential expiry monitor |
1h |
Monitors Entra app credentials for upcoming expiry (30-day and 7-day thresholds), sends alerts |
Licensing
| # |
Job |
Interval |
Description |
| 13 |
License scheduler |
5 min / 1h / 24h |
Heartbeat to license server (5 min), usage metrics reporting (1h), full license validation and status renewal (24h) |
Audit & Logging
| # |
Job |
Interval |
Description |
| 14 |
Audit log retention |
24h |
Enforces configured audit log retention policy, removes expired entries |
| 15 |
Audit archive |
5 min |
Batches audit events and archives to configured long-term storage (S3, GCS, Azure Blob) |
| 16 |
Log stream flush |
5 sec |
Flushes buffered audit and event logs to configured SIEM endpoints (Splunk, Elasticsearch, Syslog) |
System & Infrastructure
| # |
Job |
Interval |
Description |
| 17 |
Archive cleanup |
1h |
Hard-deletes archived secrets and certificates after retention period expires |
| 18 |
TLS certificate reload |
30 sec |
Hot-reloads server TLS certificate from disk without restart. Configurable via MAZEVAULT_TLS_RELOAD_INTERVAL |
| 19 |
Gateway health monitor |
30 sec |
Monitors gateway heartbeats, detects stale gateways (3 consecutive missed heartbeats), triggers DR failover if enabled |
| 20 |
External change detector |
1h |
Orchestrator mode only — detects drift between local secret metadata and values stored in external secrets manager |
Agent Jobs
| Job |
Interval |
Description |
| Secret synchronization |
Configurable (default 5 min) |
Syncs secrets from backend to local store |
| Heartbeat |
60 seconds |
Reports agent status, metrics, and version |
| Certificate discovery |
1 hour |
Scans infrastructure for certificates (filesystem, JKS, Windows cert store) |
| Update check |
24 hours |
Checks for newer agent version availability |
Gateway Jobs
| Job |
Interval |
Description |
| Heartbeat |
60 seconds |
Reports gateway status and connectivity |
| Task polling |
10 seconds |
Polls backend for pending certificate operations |
| Queue reconciliation |
30 seconds |
Reconciles local task queue with backend state |
Configurable intervals
Most intervals are fixed by design for optimal performance. User-configurable intervals:
- Agent secret sync —
SYNC_INTERVAL_SECONDS environment variable
- TLS certificate reload —
MAZEVAULT_TLS_RELOAD_INTERVAL environment variable (default: 30s)
- CRL regeneration — configurable per CA in CA settings (full and delta intervals independently)