basil/TESTING.md

# Testing Documentation

This document provides comprehensive guidance on testing in the Basil project, including unit tests, integration tests, E2E tests, and security testing.

## Table of Contents

1. [Overview](#overview)
2. [Testing Stack](#testing-stack)
3. [Running Tests](#running-tests)
4. [Unit Testing](#unit-testing)
5. [Integration Testing](#integration-testing)
6. [E2E Testing](#e2e-testing)
7. [Security Testing](#security-testing)
8. [CI/CD Integration](#cicd-integration)
9. [Writing Tests](#writing-tests)
10. [Best Practices](#best-practices)
11. [Troubleshooting](#troubleshooting)

## Overview

The Basil project uses a comprehensive testing strategy covering all layers:

- **Unit Tests**: Test individual functions and components in isolation
- **Integration Tests**: Test API endpoints and database interactions
- **E2E Tests**: Test complete user workflows across the full stack
- **Security Tests**: Scan for vulnerabilities and security issues

## Testing Stack

### Unit & Integration Tests
- **Vitest**: Fast unit testing framework with TypeScript support
- **@testing-library/react**: React component testing utilities
- **@testing-library/jest-dom**: Custom Jest matchers for DOM assertions
- **Supertest**: HTTP assertion library for API testing

### E2E Tests
- **Playwright**: Cross-browser end-to-end testing framework
- Supports Chrome, Firefox, Safari, and mobile browsers

### Security Testing
- **npm audit**: Built-in npm vulnerability scanner
- ESLint with security rules
- Docker image scanning

## Running Tests

### Run All Tests

```bash
# Run all unit tests across all packages
npm test

# Run with coverage
npm run test:coverage

# Run tests in watch mode (development)
npm run test:watch
```

### Package-Specific Tests

#### API Package

```bash
cd packages/api

# Run unit tests
npm test

# Run with coverage
npm run test:coverage

# Run in watch mode
npm run test:watch

# Run with UI
npm run test:ui
```

#### Web Package

```bash
cd packages/web

# Run component tests
npm test

# Run with coverage
npm run test:coverage

# Run in watch mode
npm run test:watch
```

#### Shared Package

```bash
cd packages/shared

# Run type validation tests
npm test
```

### E2E Tests

```bash
# Run all E2E tests (headless)
npm run test:e2e

# Run with UI mode (interactive)
npm run test:e2e:ui

# Run in headed mode (see browser)
npm run test:e2e:headed

# Run specific test file
npx playwright test e2e/recipes.spec.ts

# Run specific browser
npx playwright test --project=chromium

# Run mobile tests only
npx playwright test --project="Mobile Chrome"
```

## Unit Testing

### API Unit Tests

Located in `packages/api/src/**/*.test.ts`

**Example: Testing a Service**

```typescript
import { describe, it, expect, vi } from 'vitest';
import { StorageService } from './storage.service';

describe('StorageService', () => {
  it('should save file locally', async () => {
    const service = StorageService.getInstance();
    const mockFile = createMockFile();

    const result = await service.saveFile(mockFile, 'recipes');

    expect(result).toMatch(/^\/uploads\/recipes\//);
  });
});
```

**Running API Tests**

```bash
cd packages/api
npm test

# Run specific test file
npm test -- storage.service.test.ts

# Run with coverage
npm run test:coverage
```

### Web Unit Tests

Located in `packages/web/src/**/*.test.{ts,tsx}`

**Example: Testing a React Component**

```typescript
import { render, screen, waitFor } from '@testing-library/react';
import { describe, it, expect, vi } from 'vitest';
import RecipeList from './RecipeList';

describe('RecipeList', () => {
  it('should display recipes', async () => {
    render(<RecipeList />);

    await waitFor(() => {
      expect(screen.getByText('Recipe Title')).toBeInTheDocument();
    });
  });
});
```

## Integration Testing

Integration tests verify API endpoints and database interactions.

### API Integration Tests

Located in `packages/api/src/routes/**/*.test.ts`

**Example: Testing an API Endpoint**

```typescript
import request from 'supertest';
import { describe, it, expect } from 'vitest';
import app from '../index';

describe('GET /api/recipes', () => {
  it('should return paginated recipes', async () => {
    const response = await request(app)
      .get('/api/recipes')
      .expect(200);

    expect(response.body).toHaveProperty('data');
    expect(response.body).toHaveProperty('total');
  });
});
```

### Database Testing

Integration tests use a test database. Configure in `.env.test`:

```env
DATABASE_URL=postgresql://basil:basil@localhost:5432/basil_test
```

**Setup Test Database**

```bash
cd packages/api

# Run migrations on test database
DATABASE_URL="postgresql://basil:basil@localhost:5432/basil_test" npm run prisma:migrate

# Seed test data (if needed)
DATABASE_URL="postgresql://basil:basil@localhost:5432/basil_test" npm run prisma:seed
```

## E2E Testing

E2E tests validate complete user workflows using Playwright.

### Configuration

Playwright configuration: `playwright.config.ts`

```typescript
export default defineConfig({
  testDir: './e2e',
  baseURL: 'http://localhost:5173',
  webServer: {
    command: 'npm run dev',
    url: 'http://localhost:5173',
  },
});
```

### Writing E2E Tests

Located in `e2e/**/*.spec.ts`

**Example: Testing Recipe Workflow**

```typescript
import { test, expect } from '@playwright/test';

test('should create and view recipe', async ({ page }) => {
  // Navigate to app
  await page.goto('/');

  // Click create button
  await page.click('button:has-text("Create Recipe")');

  // Fill form
  await page.fill('input[name="title"]', 'Test Recipe');
  await page.fill('textarea[name="description"]', 'Test Description');

  // Submit
  await page.click('button[type="submit"]');

  // Verify recipe was created
  await expect(page.locator('h1')).toContainText('Test Recipe');
});
```

### E2E Test Best Practices

1. **Use data-testid attributes** for reliable selectors
2. **Mock external APIs** to avoid flaky tests
3. **Clean up test data** after each test
4. **Use page object pattern** for complex pages
5. **Test critical user paths** first

### Debugging E2E Tests

```bash
# Run with headed browser
npm run test:e2e:headed

# Run with UI mode (interactive debugging)
npm run test:e2e:ui

# Generate trace for failed tests
npx playwright test --trace on

# View trace
npx playwright show-trace trace.zip
```

## Security Testing

### NPM Audit

```bash
# Run security audit
npm audit

# Fix automatically fixable vulnerabilities
npm audit fix

# Audit specific package
cd packages/api && npm audit
```

### Dependency Scanning

Monitor for outdated and vulnerable dependencies:

```bash
# Check for outdated packages
npm outdated

# Update dependencies
npm update
```

### Code Scanning

ESLint is configured with security rules:

```bash
# Run linter
npm run lint

# Fix auto-fixable issues
npm run lint -- --fix
```

### Docker Security

Scan Docker images for vulnerabilities:

```bash
# Build images
docker-compose build

# Optional: Use Trivy for scanning
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
  aquasec/trivy image basil-api:latest
```

## CI/CD Integration

### Gitea Actions Workflows

The project includes comprehensive CI/CD workflows in `.gitea/workflows/`:

#### 1. CI Pipeline (`ci.yml`)

Runs on every push and PR:
- Linting
- Unit tests for all packages
- Build verification
- Coverage reporting

#### 2. E2E Tests (`e2e.yml`)

Runs on main branch and nightly:
- Full E2E test suite
- Mobile browser testing
- Screenshot and trace artifacts

#### 3. Security Scanning (`security.yml`)

Runs weekly and on main branch:
- NPM audit
- Dependency checking
- Code quality scanning
- Docker image security

#### 4. Docker Build & Deploy (`docker.yml`)

Runs on main branch and version tags:
- Build Docker images
- Push to registry
- Deploy to staging/production

### Setting Up Gitea Actions

1. **Enable Gitea Actions** in your Gitea instance settings
2. **Configure Runners** (act_runner or Gitea Actions runner)
3. **Set Repository Secrets**:
   ```
   DOCKER_REGISTRY=registry.example.com
   DOCKER_USERNAME=your-username
   DOCKER_PASSWORD=your-password
   ```

4. **Verify Workflows**: Push to trigger pipelines

### CI Environment Variables

Configure these in your Gitea repository settings:

```env
# Database (for CI)
DATABASE_URL=postgresql://basil:basil@localhost:5432/basil_test

# Docker Registry
DOCKER_REGISTRY=your-registry.com
DOCKER_USERNAME=your-username
DOCKER_PASSWORD=your-token

# Optional: External Services
SENTRY_DSN=your-sentry-dsn
ANALYTICS_KEY=your-analytics-key
```

## Writing Tests

### General Guidelines

1. **Follow AAA Pattern**: Arrange, Act, Assert
   ```typescript
   it('should do something', () => {
     // Arrange
     const input = setupTestData();

     // Act
     const result = functionUnderTest(input);

     // Assert
     expect(result).toBe(expected);
   });
   ```

2. **Use Descriptive Names**: Test names should describe behavior
   ```typescript
   // Good
   it('should return 404 when recipe not found')

   // Bad
   it('test recipe endpoint')
   ```

3. **Test One Thing**: Each test should verify one specific behavior

4. **Mock External Dependencies**: Don't rely on external services
   ```typescript
   vi.mock('axios');
   vi.mocked(axios.get).mockResolvedValue({ data: mockData });
   ```

5. **Clean Up**: Reset mocks and state after each test
   ```typescript
   afterEach(() => {
     vi.clearAllMocks();
   });
   ```

### Test Structure

```
packages/
├── api/
│   └── src/
│       ├── services/
│       │   ├── storage.service.ts
│       │   └── storage.service.test.ts
│       └── routes/
│           ├── recipes.routes.ts
│           └── recipes.routes.test.ts
├── web/
│   └── src/
│       ├── components/
│       │   ├── RecipeCard.tsx
│       │   └── RecipeCard.test.tsx
│       └── services/
│           ├── api.ts
│           └── api.test.ts
└── shared/
    └── src/
        ├── types.ts
        └── types.test.ts
```

## Best Practices

### Unit Tests

✅ **Do:**
- Test pure functions and logic
- Mock external dependencies
- Use factory functions for test data
- Test edge cases and error conditions
- Keep tests fast (< 50ms per test)

❌ **Don't:**
- Test implementation details
- Make network requests
- Access real databases
- Test framework code
- Write brittle tests tied to DOM structure

### Integration Tests

✅ **Do:**
- Test API endpoints end-to-end
- Use test database with migrations
- Test authentication and authorization
- Verify database state after operations
- Test error responses

❌ **Don't:**
- Test external APIs directly
- Share state between tests
- Skip cleanup after tests
- Hard-code test data IDs

### E2E Tests

✅ **Do:**
- Test critical user workflows
- Use stable selectors (data-testid)
- Mock external APIs
- Run in CI pipeline
- Take screenshots on failure

❌ **Don't:**
- Test every possible path (use unit tests)
- Rely on timing (use waitFor)
- Share test data between tests
- Test implementation details
- Make tests too granular

## Coverage Goals

Aim for the following coverage targets:

- **Unit Tests**: 80%+ coverage
- **Integration Tests**: Cover all API endpoints
- **E2E Tests**: Cover critical user paths

Check coverage:

```bash
# API Package
cd packages/api && npm run test:coverage

# Web Package
cd packages/web && npm run test:coverage

# View HTML coverage report
open packages/api/coverage/index.html
```

## Troubleshooting

### Common Issues

#### Tests Failing Locally

1. **Database connection errors**
   ```bash
   # Ensure PostgreSQL is running
   docker-compose up -d postgres

   # Run migrations
   cd packages/api && npm run prisma:migrate
   ```

2. **Module resolution errors**
   ```bash
   # Clear node_modules and reinstall
   rm -rf node_modules package-lock.json
   npm install
   ```

3. **Port already in use**
   ```bash
   # Kill process on port 3001
   lsof -ti:3001 | xargs kill -9
   ```

#### E2E Tests Timing Out

1. **Increase timeout** in playwright.config.ts
2. **Check webServer** is starting correctly
3. **Verify database** migrations ran successfully

#### CI Tests Failing

1. **Check logs** in Gitea Actions UI
2. **Verify secrets** are configured correctly
3. **Run tests locally** with same Node version
4. **Check database** service is healthy

### Getting Help

- Review test logs and error messages
- Check existing tests for examples
- Consult Vitest docs: https://vitest.dev
- Consult Playwright docs: https://playwright.dev

## Continuous Improvement

Testing is an ongoing process. Regularly:

1. **Review test coverage** and add tests for uncovered code
2. **Refactor flaky tests** to be more reliable
3. **Update tests** when requirements change
4. **Add tests** for bug fixes to prevent regressions
5. **Monitor CI pipeline** performance and optimize slow tests

---

**Last Updated**: 2025-10-24
**Maintained By**: Basil Development Team