basil/.gitea/workflows/README.md

# Basil CI/CD Pipeline Documentation

## Overview

The Basil CI/CD pipeline is a comprehensive workflow that automates testing, building, and deployment of the Basil recipe management application. It consolidates all quality checks, security scanning, and deployment automation into a single workflow file.

## Workflow Triggers

The pipeline is triggered on:

- **Push to main or develop branches**: Full pipeline with deployment (main only)
- **Pull requests to main or develop**: Tests and builds only, no deployment
- **Tagged releases** (v*): Full pipeline with semantic version tagging

## Pipeline Stages

### Stage 1: Parallel Quality Checks (~8 minutes)

All these jobs run in parallel for maximum efficiency:

- **lint**: ESLint code quality checks
- **test-api**: API unit tests with PostgreSQL service
- **test-web**: React web application unit tests
- **test-shared**: Shared package unit tests
- **security-scan**: NPM audit, secret scanning, dependency checks

### Stage 2: Build Verification (~7 minutes)

- **build**: Compiles all packages (depends on Stage 1 passing)

### Stage 3: E2E Testing (~15 minutes)

- **e2e-tests**: Playwright end-to-end tests (depends on build)

### Stage 4: Docker & Deployment (~11 minutes, main branch only)

- **docker-build-and-push**: Builds and pushes Docker images to Harbor
- **trigger-deployment**: Calls webhook to trigger server-side deployment

## Total Pipeline Duration

- **Pull Request**: ~30 minutes (Stages 1-3)
- **Main Branch Deploy**: ~41 minutes (All stages)

## Required Gitea Secrets

Configure these in your Gitea repository settings (Settings → Secrets):

| Secret Name | Description | Example |
|-------------|-------------|---------|
| `HARBOR_REGISTRY` | Harbor registry URL | `harbor.pkartchner.com` |
| `HARBOR_USERNAME` | Harbor robot account username | `robot$basil+basil-cicd` |
| `HARBOR_PASSWORD` | Harbor robot account token | `ErJh8ze6VvZDnviVwc97Jevf6CrdzRBu` |
| `HARBOR_PROJECT` | Harbor project name | `basil` |
| `WEBHOOK_URL` | Deployment webhook endpoint | `http://localhost:9000/hooks/basil-deploy` |
| `WEBHOOK_SECRET` | Webhook authentication secret | `4cd30192f203f5ea905...` |

## Image Naming Convention

### Tags

The workflow creates multiple tags for each image:

- `latest`: Latest build from main branch
- `main-{short-sha}`: Specific commit (e.g., `main-abc1234`)
- `v{version}`: Semantic version tags (for tagged releases)

### Image Names

```
harbor.pkartchner.com/basil/basil-api:latest
harbor.pkartchner.com/basil/basil-api:main-abc1234
harbor.pkartchner.com/basil/basil-web:latest
harbor.pkartchner.com/basil/basil-web:main-abc1234
```

## Deployment Process

### Automated Deployment (Main Branch)

1. Developer pushes to `main` branch
2. Pipeline runs all tests and builds
3. Docker images built and pushed to Harbor
4. Webhook triggered with deployment payload
5. Server receives webhook and runs deployment script
6. Script pulls images from Harbor
7. Docker Compose restarts containers with new images
8. Health checks verify successful deployment

### Manual Deployment

If you need to deploy manually or rollback:

```bash
cd /srv/docker-compose/basil

# Deploy latest
export IMAGE_TAG=latest
./scripts/deploy.sh

# Deploy specific version
export IMAGE_TAG=main-abc1234
./scripts/deploy.sh
```

## Security Features

### Security Gates

- **NPM Audit**: Checks for vulnerable dependencies (HIGH/CRITICAL)
- **Secret Scanning**: Detects hardcoded credentials in code
- **Trivy Image Scanning**: Scans Docker images for vulnerabilities
- **Dependency Checking**: Reports outdated packages

### Fail-Fast Behavior

- All tests must pass before Docker build starts
- Health checks must pass before deployment completes
- Any security scan failure stops the pipeline

## Caching Strategy

The workflow uses GitHub Actions cache to speed up builds:

- **NPM Dependencies**: Cached between runs
- **Docker Layers**: Cached using GitHub Actions cache backend
- **Playwright Browsers**: Cached for E2E tests

## Artifacts

The workflow uploads artifacts that are retained for 7-14 days:

- **Test Coverage**: Unit test coverage reports for all packages
- **Playwright Reports**: E2E test results and screenshots
- **Build Artifacts**: Compiled JavaScript/TypeScript output

## Monitoring

### View Workflow Runs

1. Go to your Gitea repository
2. Click the "Actions" tab
3. Select a workflow run to see detailed logs

### Check Deployment Status

```bash
# Webhook service logs
journalctl -u basil-webhook -f

# Deployment script logs
tail -f /srv/docker-compose/basil/deploy.log

# Container status
docker ps | grep basil

# Application health
curl https://basil.pkartchner.com/health
```

## Rollback Procedures

### Scenario 1: Bad Deployment

Deploy a previous working version:

```bash
cd /srv/docker-compose/basil
export IMAGE_TAG=main-abc1234  # Previous working SHA
./scripts/deploy.sh
```

### Scenario 2: Rollback Workflow Changes

Restore previous workflows:

```bash
cd /srv/docker-compose/basil
rm -rf .gitea/workflows/
mv .gitea/workflows-archive/ .gitea/workflows/
git add .gitea/workflows/
git commit -m "rollback: restore previous workflows"
git push origin main
```

### Scenario 3: Emergency Stop

Stop containers immediately:

```bash
cd /srv/docker-compose/basil
docker-compose down
```

## Troubleshooting

### Common Issues

**Issue: Workflow fails at Docker login**
- Solution: Verify Harbor secrets are configured correctly
- Check: Harbor service is running and accessible

**Issue: Image push fails**
- Solution: Verify robot account has push permissions
- Check: Harbor disk space is sufficient

**Issue: Webhook not triggered**
- Solution: Verify webhook URL and secret are correct
- Check: Webhook service is running (`systemctl status basil-webhook`)

**Issue: Deployment health check fails**
- Solution: Check container logs (`docker logs basil-api`)
- Check: Database migrations completed successfully
- Rollback: Previous containers remain running on health check failure

**Issue: Tests are flaky**
- Solution: Review test logs in artifacts
- Check: Database service health in workflow
- Consider: Increasing timeouts in playwright.config.ts

## Local Development

### Test Workflow Locally

You can test parts of the workflow locally:

```bash
# Run all tests
npm run test

# Run E2E tests
npm run test:e2e

# Run linting
npm run lint

# Build all packages
npm run build

# Build Docker images
docker-compose build

# Test Harbor login
echo "ErJh8ze6VvZDnviVwc97Jevf6CrdzRBu" | \
  docker login harbor.pkartchner.com \
  -u "robot\$basil+basil-cicd" \
  --password-stdin
```

## Maintenance

### Weekly Tasks

- Review security scan results in workflow logs
- Check Harbor UI for vulnerability scan results
- Monitor workflow execution times
- Review and clean up old image tags in Harbor

### Monthly Tasks

- Rotate Harbor robot account credentials
- Update base Docker images if needed
- Review and optimize caching strategy
- Update dependencies (npm update)

### Quarterly Tasks

- Review and update Playwright browser versions
- Audit and remove unused workflow artifacts
- Performance testing and optimization
- Documentation updates

## Performance Optimization

Current optimization techniques:

- **Parallel job execution**: Stage 1 jobs run concurrently
- **NPM caching**: Dependencies cached across runs
- **Docker layer caching**: Reuses unchanged layers
- **Selective deployment**: Only main branch triggers Docker build

Future optimization opportunities:

- Build matrix for multiple Node versions
- Split E2E tests into parallel shards
- Implement build artifact reuse
- Add conditional job skipping (skip tests if only docs changed)

## Support

For issues or questions:

- Check workflow logs in Gitea Actions tab
- Review deployment logs: `/srv/docker-compose/basil/deploy.log`
- Check this documentation
- Review archived workflows in `.gitea/workflows-archive/` for comparison

## Architecture Diagram

```
┌─────────────────────────────────────────────────────┐
│  Developer Push to main                              │
└─────────────────┬───────────────────────────────────┘
                  │
                  v
┌─────────────────────────────────────────────────────┐
│  Gitea Actions Workflow (main.yml)                   │
├─────────────────────────────────────────────────────┤
│  Stage 1 (Parallel):                                 │
│  ├─ lint                                             │
│  ├─ test-api                                         │
│  ├─ test-web                                         │
│  ├─ test-shared                                      │
│  └─ security-scan                                    │
│                                                       │
│  Stage 2: build                                      │
│                                                       │
│  Stage 3: e2e-tests                                  │
│                                                       │
│  Stage 4 (main only):                                │
│  ├─ docker-build-and-push → Harbor Registry          │
│  └─ trigger-deployment → Webhook                     │
└─────────────────┬───────────────────────────────────┘
                  │
                  v
┌─────────────────────────────────────────────────────┐
│  Server (localhost)                                  │
├─────────────────────────────────────────────────────┤
│  Webhook Service (port 9000)                         │
│         │                                             │
│         v                                             │
│  Deploy Script (/srv/.../scripts/deploy.sh)         │
│  ├─ Login to Harbor                                  │
│  ├─ Create pre-deployment backup                     │
│  ├─ Pull new images from Harbor                      │
│  ├─ Update docker-compose.override.yml               │
│  ├─ Restart containers                               │
│  ├─ Health checks                                    │
│  └─ Cleanup old images                               │
└─────────────────┬───────────────────────────────────┘
                  │
                  v
┌─────────────────────────────────────────────────────┐
│  Basil Application Running                           │
│  https://basil.pkartchner.com                        │
└─────────────────────────────────────────────────────┘
```

## Version History

- **v1.0** (2026-01-14): Initial consolidated workflow with Harbor integration
  - Merged 5 separate workflows into single main.yml
  - Added Harbor registry support
  - Implemented webhook-based deployment
  - Added comprehensive security scanning
  - Optimized with parallel job execution