Files
basil/scripts
Paul R Kartchner fb18caa3c2
All checks were successful
Basil CI/CD Pipeline / Shared Package Tests (push) Successful in 1m10s
Basil CI/CD Pipeline / Code Linting (push) Successful in 1m18s
Basil CI/CD Pipeline / Web Tests (push) Successful in 1m29s
Basil CI/CD Pipeline / Security Scanning (push) Successful in 1m14s
Basil CI/CD Pipeline / API Tests (push) Successful in 1m45s
Basil CI/CD Pipeline / Trigger Deployment (push) Successful in 12s
Basil CI/CD Pipeline / Build All Packages (push) Successful in 1m31s
Basil CI/CD Pipeline / E2E Tests (push) Has been skipped
Basil CI/CD Pipeline / Build & Push Docker Images (push) Successful in 14m27s
feat: add comprehensive PostgreSQL backup and restore scripts
Added production-grade backup and restore scripts for PostgreSQL servers
that can backup all databases automatically with retention management.

New scripts:
- scripts/backup-all-postgres-databases.sh - Backs up all databases on a
  PostgreSQL server with automatic retention, compression, verification,
  and notification support
- scripts/restore-postgres-database.sh - Restores individual databases
  with safety backups and verification
- scripts/README-POSTGRES-BACKUP.md - Complete documentation with examples,
  best practices, and troubleshooting

Features:
- Automatic detection and backup of all user databases
- Excludes system databases (postgres, template0, template1)
- Backs up global objects (roles, tablespaces)
- Optional gzip compression (80-90% space savings)
- Automatic retention management (configurable days)
- Integrity verification (gzip -t for compressed files)
- Safety backups before restore operations
- Detailed logging with color-coded output
- Backup summary reporting
- Email/Slack notification support (optional)
- Interactive restore with confirmation prompts
- Force mode for automation
- Verbose debugging mode
- Comprehensive error handling

Backup directory structure:
  /var/backups/postgresql/YYYYMMDD/
    - globals_YYYYMMDD_HHMMSS.sql.gz
    - database1_YYYYMMDD_HHMMSS.sql.gz
    - database2_YYYYMMDD_HHMMSS.sql.gz

Usage examples:
  # Backup all databases with compression
  ./backup-all-postgres-databases.sh -c

  # Custom configuration
  ./backup-all-postgres-databases.sh -h db.server.com -U backup_user -d /mnt/backups -r 60 -c

  # Restore database
  ./restore-postgres-database.sh /var/backups/postgresql/20260120/mydb_20260120_020001.sql.gz

  # Force restore (skip confirmation)
  ./restore-postgres-database.sh backup.sql.gz -d mydb -f

Automation:
  # Add to crontab for daily backups at 2 AM
  0 2 * * * /path/to/backup-all-postgres-databases.sh -c >> /var/log/postgres-backup.log 2>&1

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 21:39:32 -07:00
..

Basil Deployment Scripts

This directory contains scripts for automated deployment of Basil.

Scripts Overview

deploy.sh

Main deployment script that handles the complete deployment process.

Features:

  • Pre-deployment backup creation
  • Docker image pulling from registry
  • Container restart with health checks
  • Automatic cleanup of old images
  • Comprehensive logging

Usage:

# With environment variables
DOCKER_USERNAME=myuser DOCKER_REGISTRY=docker.io IMAGE_TAG=latest ./deploy.sh

# Or source from .env.deploy
source ../.env.deploy && ./deploy.sh

Environment Variables:

  • DOCKER_USERNAME (required): Docker registry username
  • DOCKER_REGISTRY (optional, default: docker.io): Registry URL
  • IMAGE_TAG (optional, default: latest): Image tag to pull

manual-deploy.sh

Interactive wrapper around deploy.sh with user prompts.

Features:

  • Interactive prompts for configuration
  • Saves configuration to .env.deploy for future use
  • Confirmation before deployment

Usage:

./manual-deploy.sh

The script will prompt for:

  1. Docker Hub username
  2. Docker registry (default: docker.io)
  3. Image tag (default: latest)
  4. Confirmation to proceed

webhook-receiver.sh

Webhook server that listens for deployment triggers from CI/CD.

Features:

  • HTTP webhook endpoint
  • Secret-based authentication
  • Automatic deployment on webhook call
  • Systemd service support

Usage:

# Manual run (foreground)
WEBHOOK_PORT=9000 WEBHOOK_SECRET=mysecret ./webhook-receiver.sh

# Or install as systemd service (recommended)
sudo cp basil-webhook.service /etc/systemd/system/
sudo systemctl enable basil-webhook
sudo systemctl start basil-webhook

Environment Variables:

  • WEBHOOK_PORT (optional, default: 9000): Port to listen on
  • WEBHOOK_SECRET (optional, default: changeme): Authentication secret

Webhook Endpoint:

POST http://localhost:9000/hooks/basil-deploy
Header: X-Webhook-Secret: your-secret

basil-webhook.service

Systemd service file for running webhook receiver as a system service.

Installation:

# 1. Copy service file
sudo cp basil-webhook.service /etc/systemd/system/

# 2. Edit environment variables in the file
sudo nano /etc/systemd/system/basil-webhook.service

# 3. Reload systemd
sudo systemctl daemon-reload

# 4. Enable and start service
sudo systemctl enable basil-webhook
sudo systemctl start basil-webhook

# 5. Check status
sudo systemctl status basil-webhook

Service Management:

# Start service
sudo systemctl start basil-webhook

# Stop service
sudo systemctl stop basil-webhook

# Restart service
sudo systemctl restart basil-webhook

# View logs
sudo journalctl -u basil-webhook -f

# Check status
sudo systemctl status basil-webhook

Deployment Workflow

Automatic Deployment (CI/CD)

  1. Developer pushes to main branch
  2. Gitea Actions runs tests and builds images
  3. Images pushed to Docker registry
  4. Gitea Actions calls webhook endpoint
  5. Webhook server receives call and executes deploy.sh
  6. Production server pulls new images and restarts

Manual Deployment

  1. Run ./manual-deploy.sh
  2. Enter configuration when prompted
  3. Confirm deployment
  4. Script executes deployment process

Logs

All scripts log to the parent directory:

  • deploy.log - Deployment script logs
  • webhook.log - Webhook server logs
  • webhook-error.log - Webhook server errors

View logs:

# Deployment logs
tail -f ../deploy.log

# Webhook logs
tail -f ../webhook.log

# Webhook errors
tail -f ../webhook-error.log

# Systemd service logs
sudo journalctl -u basil-webhook -f

Configuration Files

.env.deploy (in parent directory)

Stores deployment configuration. Created from .env.deploy.example.

DOCKER_USERNAME=your-dockerhub-username
DOCKER_REGISTRY=docker.io
IMAGE_TAG=latest
WEBHOOK_PORT=9000
WEBHOOK_SECRET=your-random-secret

Important: This file is gitignored and should never be committed.

webhook-config.json (auto-generated)

Generated by webhook-receiver.sh. Configures the webhook endpoint.

Location: Created in scripts directory when webhook-receiver.sh runs.

Troubleshooting

Script Permission Denied

chmod +x *.sh

Docker Pull Fails

# Check credentials
docker login -u $DOCKER_USERNAME

# Check image exists
docker pull $DOCKER_REGISTRY/$DOCKER_USERNAME/basil-api:$IMAGE_TAG

Webhook Not Responding

# Check service status
sudo systemctl status basil-webhook

# Check if port is listening
sudo netstat -tlnp | grep 9000

# Check firewall
sudo ufw status

Health Check Fails

# Check API logs
docker-compose logs api

# Check API manually
curl http://localhost:3001/health

# Check database connection
docker-compose exec api npx prisma studio

Security Notes

  1. Never commit secrets: .env.deploy is gitignored
  2. Use strong webhook secret: 32+ character random string
  3. Firewall webhook port: Allow only from known IPs if possible
  4. Use HTTPS: Configure reverse proxy for webhook in production
  5. Backup before deploy: Script creates automatic backups

Directory Structure

scripts/
├── README.md                    # This file
├── deploy.sh                    # Main deployment script
├── manual-deploy.sh             # Interactive deployment
├── webhook-receiver.sh          # Webhook server
├── basil-webhook.service        # Systemd service file
└── webhook-config.json          # Generated webhook config (auto-created)

Parent directory:
├── .env.deploy                  # Deployment config (gitignored)
├── .env.deploy.example          # Example config
├── deploy.log                   # Deployment logs (gitignored)
├── webhook.log                  # Webhook logs (gitignored)
└── backups/                     # Automatic backups (gitignored)

References