basil/TESTING.md

Testing Documentation

This document provides comprehensive guidance on testing in the Basil project, including unit tests, integration tests, E2E tests, and security testing.

Table of Contents

1. Overview
2. Testing Stack
3. Running Tests
4. Unit Testing
5. Integration Testing
6. E2E Testing
7. Security Testing
8. CI/CD Integration
9. Writing Tests
10. Best Practices
11. Troubleshooting

Overview

The Basil project uses a comprehensive testing strategy covering all layers:

• Unit Tests: Test individual functions and components in isolation
• Integration Tests: Test API endpoints and database interactions
• E2E Tests: Test complete user workflows across the full stack
• Security Tests: Scan for vulnerabilities and security issues

Testing Stack

Unit & Integration Tests

• Vitest: Fast unit testing framework with TypeScript support
• @testing-library/react: React component testing utilities
• @testing-library/jest-dom: Custom Jest matchers for DOM assertions
• Supertest: HTTP assertion library for API testing

E2E Tests

• Playwright: Cross-browser end-to-end testing framework
• Supports Chrome, Firefox, Safari, and mobile browsers

Security Testing

• npm audit: Built-in npm vulnerability scanner
• ESLint with security rules
• Docker image scanning

Running Tests

Run All Tests

# Run all unit tests across all packages
npm test

# Run with coverage
npm run test:coverage

# Run tests in watch mode (development)
npm run test:watch

Package-Specific Tests

API Package

cd packages/api

# Run unit tests
npm test

# Run with coverage
npm run test:coverage

# Run in watch mode
npm run test:watch

# Run with UI
npm run test:ui

Web Package

cd packages/web

# Run component tests
npm test

# Run with coverage
npm run test:coverage

# Run in watch mode
npm run test:watch

Shared Package

cd packages/shared

# Run type validation tests
npm test

E2E Tests

# Run all E2E tests (headless)
npm run test:e2e

# Run with UI mode (interactive)
npm run test:e2e:ui

# Run in headed mode (see browser)
npm run test:e2e:headed

# Run specific test file
npx playwright test e2e/recipes.spec.ts

# Run specific browser
npx playwright test --project=chromium

# Run mobile tests only
npx playwright test --project="Mobile Chrome"

Unit Testing

API Unit Tests

Located in packages/api/src/**/*.test.ts

Example: Testing a Service

import { describe, it, expect, vi } from 'vitest';
import { StorageService } from './storage.service';

describe('StorageService', () => {
  it('should save file locally', async () => {
    const service = StorageService.getInstance();
    const mockFile = createMockFile();

    const result = await service.saveFile(mockFile, 'recipes');

    expect(result).toMatch(/^\/uploads\/recipes\//);
  });
});

Running API Tests

cd packages/api
npm test

# Run specific test file
npm test -- storage.service.test.ts

# Run with coverage
npm run test:coverage

Web Unit Tests

Located in packages/web/src/**/*.test.{ts,tsx}

Example: Testing a React Component

import { render, screen, waitFor } from '@testing-library/react';
import { describe, it, expect, vi } from 'vitest';
import RecipeList from './RecipeList';

describe('RecipeList', () => {
  it('should display recipes', async () => {
    render(<RecipeList />);

    await waitFor(() => {
      expect(screen.getByText('Recipe Title')).toBeInTheDocument();
    });
  });
});

Integration Testing

Integration tests verify API endpoints and database interactions.

API Integration Tests

Located in packages/api/src/routes/**/*.test.ts

Example: Testing an API Endpoint

import request from 'supertest';
import { describe, it, expect } from 'vitest';
import app from '../index';

describe('GET /api/recipes', () => {
  it('should return paginated recipes', async () => {
    const response = await request(app)
      .get('/api/recipes')
      .expect(200);

    expect(response.body).toHaveProperty('data');
    expect(response.body).toHaveProperty('total');
  });
});

Database Testing

Integration tests use a test database. Configure in .env.test:

DATABASE_URL=postgresql://basil:basil@localhost:5432/basil_test

Setup Test Database

cd packages/api

# Run migrations on test database
DATABASE_URL="postgresql://basil:basil@localhost:5432/basil_test" npm run prisma:migrate

# Seed test data (if needed)
DATABASE_URL="postgresql://basil:basil@localhost:5432/basil_test" npm run prisma:seed

E2E Testing

E2E tests validate complete user workflows using Playwright.

Configuration

Playwright configuration: playwright.config.ts

export default defineConfig({
  testDir: './e2e',
  baseURL: 'http://localhost:5173',
  webServer: {
    command: 'npm run dev',
    url: 'http://localhost:5173',
  },
});

Writing E2E Tests

Located in e2e/**/*.spec.ts

Example: Testing Recipe Workflow

import { test, expect } from '@playwright/test';

test('should create and view recipe', async ({ page }) => {
  // Navigate to app
  await page.goto('/');

  // Click create button
  await page.click('button:has-text("Create Recipe")');

  // Fill form
  await page.fill('input[name="title"]', 'Test Recipe');
  await page.fill('textarea[name="description"]', 'Test Description');

  // Submit
  await page.click('button[type="submit"]');

  // Verify recipe was created
  await expect(page.locator('h1')).toContainText('Test Recipe');
});

E2E Test Best Practices

1. Use data-testid attributes for reliable selectors
2. Mock external APIs to avoid flaky tests
3. Clean up test data after each test
4. Use page object pattern for complex pages
5. Test critical user paths first

Debugging E2E Tests

# Run with headed browser
npm run test:e2e:headed

# Run with UI mode (interactive debugging)
npm run test:e2e:ui

# Generate trace for failed tests
npx playwright test --trace on

# View trace
npx playwright show-trace trace.zip

Security Testing

NPM Audit

# Run security audit
npm audit

# Fix automatically fixable vulnerabilities
npm audit fix

# Audit specific package
cd packages/api && npm audit

Dependency Scanning

Monitor for outdated and vulnerable dependencies:

# Check for outdated packages
npm outdated

# Update dependencies
npm update

Code Scanning

ESLint is configured with security rules:

# Run linter
npm run lint

# Fix auto-fixable issues
npm run lint -- --fix

Docker Security

Scan Docker images for vulnerabilities:

# Build images
docker-compose build

# Optional: Use Trivy for scanning
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
  aquasec/trivy image basil-api:latest

CI/CD Integration

Gitea Actions Workflows

The project includes comprehensive CI/CD workflows in .gitea/workflows/:

1. CI Pipeline (ci.yml)

Runs on every push and PR:

• Linting
• Unit tests for all packages
• Build verification
• Coverage reporting

2. E2E Tests (e2e.yml)

Runs on main branch and nightly:

• Full E2E test suite
• Mobile browser testing
• Screenshot and trace artifacts

3. Security Scanning (security.yml)

Runs weekly and on main branch:

• NPM audit
• Dependency checking
• Code quality scanning
• Docker image security

4. Docker Build & Deploy (docker.yml)

Runs on main branch and version tags:

• Build Docker images
• Push to registry
• Deploy to staging/production

Setting Up Gitea Actions

1. Enable Gitea Actions in your Gitea instance settings

2. Configure Runners (act_runner or Gitea Actions runner)

3. Set Repository Secrets:

   DOCKER_REGISTRY=registry.example.com
   DOCKER_USERNAME=your-username
   DOCKER_PASSWORD=your-password
   

4. Verify Workflows: Push to trigger pipelines

CI Environment Variables

Configure these in your Gitea repository settings:

# Database (for CI)
DATABASE_URL=postgresql://basil:basil@localhost:5432/basil_test

# Docker Registry
DOCKER_REGISTRY=your-registry.com
DOCKER_USERNAME=your-username
DOCKER_PASSWORD=your-token

# Optional: External Services
SENTRY_DSN=your-sentry-dsn
ANALYTICS_KEY=your-analytics-key

Writing Tests

General Guidelines

1. Follow AAA Pattern: Arrange, Act, Assert

   it('should do something', () => {
     // Arrange
     const input = setupTestData();
   
     // Act
     const result = functionUnderTest(input);
   
     // Assert
     expect(result).toBe(expected);
   });
   

2. Use Descriptive Names: Test names should describe behavior

   // Good
   it('should return 404 when recipe not found')
   
   // Bad
   it('test recipe endpoint')
   

3. Test One Thing: Each test should verify one specific behavior

4. Mock External Dependencies: Don't rely on external services

   vi.mock('axios');
   vi.mocked(axios.get).mockResolvedValue({ data: mockData });
   

5. Clean Up: Reset mocks and state after each test

   afterEach(() => {
     vi.clearAllMocks();
   });
   

Test Structure

packages/
├── api/
│   └── src/
│       ├── services/
│       │   ├── storage.service.ts
│       │   └── storage.service.test.ts
│       └── routes/
│           ├── recipes.routes.ts
│           └── recipes.routes.test.ts
├── web/
│   └── src/
│       ├── components/
│       │   ├── RecipeCard.tsx
│       │   └── RecipeCard.test.tsx
│       └── services/
│           ├── api.ts
│           └── api.test.ts
└── shared/
    └── src/
        ├── types.ts
        └── types.test.ts

Best Practices

Unit Tests

✅ Do:

• Test pure functions and logic
• Mock external dependencies
• Use factory functions for test data
• Test edge cases and error conditions
• Keep tests fast (< 50ms per test)

❌ Don't:

• Test implementation details
• Make network requests
• Access real databases
• Test framework code
• Write brittle tests tied to DOM structure

Integration Tests

✅ Do:

• Test API endpoints end-to-end
• Use test database with migrations
• Test authentication and authorization
• Verify database state after operations
• Test error responses

❌ Don't:

• Test external APIs directly
• Share state between tests
• Skip cleanup after tests
• Hard-code test data IDs

E2E Tests

✅ Do:

• Test critical user workflows
• Use stable selectors (data-testid)
• Mock external APIs
• Run in CI pipeline
• Take screenshots on failure

❌ Don't:

• Test every possible path (use unit tests)
• Rely on timing (use waitFor)
• Share test data between tests
• Test implementation details
• Make tests too granular

Coverage Goals

Aim for the following coverage targets:

• Unit Tests: 80%+ coverage
• Integration Tests: Cover all API endpoints
• E2E Tests: Cover critical user paths

Check coverage:

# API Package
cd packages/api && npm run test:coverage

# Web Package
cd packages/web && npm run test:coverage

# View HTML coverage report
open packages/api/coverage/index.html

Troubleshooting

Common Issues

Tests Failing Locally

1. Database connection errors

   # Ensure PostgreSQL is running
   docker-compose up -d postgres
   
   # Run migrations
   cd packages/api && npm run prisma:migrate
   

2. Module resolution errors

   # Clear node_modules and reinstall
   rm -rf node_modules package-lock.json
   npm install
   

3. Port already in use

   # Kill process on port 3001
   lsof -ti:3001 | xargs kill -9
   

E2E Tests Timing Out

1. Increase timeout in playwright.config.ts
2. Check webServer is starting correctly
3. Verify database migrations ran successfully

CI Tests Failing

1. Check logs in Gitea Actions UI
2. Verify secrets are configured correctly
3. Run tests locally with same Node version
4. Check database service is healthy

Getting Help

• Review test logs and error messages
• Check existing tests for examples
• Consult Vitest docs: https://vitest.dev
• Consult Playwright docs: https://playwright.dev

Continuous Improvement

Testing is an ongoing process. Regularly:

1. Review test coverage and add tests for uncovered code
2. Refactor flaky tests to be more reliable
3. Update tests when requirements change
4. Add tests for bug fixes to prevent regressions
5. Monitor CI pipeline performance and optimize slow tests

---

Last Updated: 2025-10-24 Maintained By: Basil Development Team