Testing Guide for Ethoscope Project

This document provides comprehensive guidelines for testing the Ethoscope project, including both the device and node packages.

Overview
Test Structure
Running Tests
Writing Tests
Test Types
Coverage Reports
CI/CD Integration
Best Practices
Troubleshooting

Overview

The Ethoscope project uses pytest as the primary testing framework with comprehensive test coverage for both device and node packages. The testing infrastructure supports:

Unit tests for individual components
Integration tests for component interactions
Functional tests for end-to-end workflows
Coverage analysis and reporting
Automated test execution
Mock hardware and network components

Test Structure

Directory Layout

ethoscope/
├── src/
│   ├── ethoscope/                 # Device package
│   │   └── ethoscope/tests/
│   │       ├── conftest.py        # Test configuration and fixtures
│   │       ├── fixtures/          # Test utilities and mock objects
│   │       ├── unittests/         # Unit tests
│   │       ├── integration_api_tests/  # API integration tests
│   │       ├── integration_server_tests/  # Server integration tests
│   │       └── static_files/      # Test data (images, videos)
│   └── node/                      # Node package
│       └── tests/
│           ├── conftest.py        # Test configuration and fixtures
│           ├── fixtures/          # Test utilities and mock objects
│           ├── unit/              # Unit tests
│           ├── integration/       # Integration tests
│           └── functional/        # Functional tests
├── test-requirements.txt          # Test dependencies
├── run_tests.py                   # Main test runner
└── TESTING.md                     # This document

Test Files

conftest.py: Contains pytest fixtures and configuration
fixtures/: Mock objects, test utilities, and helper functions
test_*.py: Individual test files following pytest conventions

Running Tests

Quick Start

# Install test dependencies
pip install -r test-requirements.txt

# Run all tests
python run_tests.py

# Run specific package tests
python run_tests.py --package device
python run_tests.py --package node

# Run with coverage
python run_tests.py --coverage

Device Package Tests

# Using the device package Makefile
cd src/ethoscope/
make test                # Run all tests
make test-unit          # Run unit tests only
make test-integration   # Run integration tests only

# Using pytest directly
cd src/ethoscope/
python -m pytest ethoscope/tests/
python -m pytest ethoscope/tests/unittests/
python -m pytest ethoscope/tests/integration_api_tests/

# Using the shell script
cd src/ethoscope/ethoscope/tests/
./run_all_tests.sh

Node Package Tests

# Using the node package Makefile
cd src/node/
make test                # Run all tests
make test-unit          # Run unit tests only
make test-integration   # Run integration tests only
make test-functional    # Run functional tests only

# Using pytest directly
cd src/node/
python -m pytest tests/
python -m pytest tests/unit/
python -m pytest tests/integration/
python -m pytest tests/functional/

# Using the shell script
cd src/node/
./run_tests.sh

Test Options

Pytest Markers

# Run only unit tests
pytest -m "unit"

# Run only integration tests
pytest -m "integration"

# Skip slow tests
pytest -m "not slow"

# Run only slow tests
pytest -m "slow"

# Run only hardware tests (device package)
pytest -m "hardware"

Verbosity and Output

# Verbose output
pytest -v

# Very verbose output
pytest -vv

# Show local variables in tracebacks
pytest -l

# Stop on first failure
pytest -x

# Show N slowest tests
pytest --durations=10

Writing Tests

Test File Structure

"""
Test module for [component name].

This module contains tests for [brief description].
"""

import pytest
from unittest.mock import Mock, patch
from ethoscope.component import ComponentToTest


class TestComponentToTest:
    """Test class for ComponentToTest."""
    
    def test_basic_functionality(self):
        """Test basic functionality works correctly."""
        component = ComponentToTest()
        result = component.do_something()
        assert result == expected_value
    
    def test_edge_case(self):
        """Test edge case handling."""
        component = ComponentToTest()
        with pytest.raises(ValueError):
            component.do_something_invalid()
    
    @pytest.mark.slow
    def test_performance_intensive(self):
        """Test that takes a long time to run."""
        # Mark slow tests for optional exclusion
        pass
    
    @pytest.mark.integration
    def test_integration_with_other_component(self):
        """Test integration with other components."""
        pass

Using Fixtures

def test_with_mock_device(mock_ethoscope_device):
    """Test using a mock device fixture."""
    device = mock_ethoscope_device
    assert device.id == "test_device_001"
    assert device.status == "running"

def test_with_mock_database(mock_database):
    """Test using a mock database fixture."""
    db = mock_database
    db.connect()
    result = db.fetchone()
    assert result is not None

def test_with_temp_directory(temp_dir):
    """Test using a temporary directory."""
    test_file = temp_dir / "test.txt"
    test_file.write_text("test content")
    assert test_file.exists()

Mocking Guidelines

# Mock external dependencies
@patch('ethoscope.hardware.camera.PiCamera')
def test_camera_interface(mock_camera):
    mock_camera.return_value.resolution = (640, 480)
    # Test code here

# Mock network calls
@patch('requests.get')
def test_api_call(mock_get):
    mock_get.return_value.json.return_value = {"status": "ok"}
    # Test code here

# Use provided mock objects
def test_with_mock_hardware(mock_camera, mock_stimulator):
    # Test code using mock hardware
    pass

Test Types

Unit Tests

Test individual functions, methods, and classes in isolation.

Location: tests/unit/ or tests/unittests/

Characteristics:

Fast execution (< 1 second per test)
No external dependencies
High coverage of edge cases
Mock all dependencies

Example:

def test_calculate_distance():
    """Test distance calculation function."""
    point1 = (0, 0)
    point2 = (3, 4)
    distance = calculate_distance(point1, point2)
    assert distance == 5.0

Integration Tests

Test interactions between components and external systems.

Location: tests/integration/ or tests/integration_api_tests/

Characteristics:

Test component interactions
May use real databases or services
Moderate execution time
Test data flows

Example:

def test_device_scanner_integration():
    """Test device scanner with real network."""
    scanner = DeviceScanner()
    devices = scanner.scan_network()
    assert isinstance(devices, list)

Functional Tests

Test complete workflows and user scenarios.

Location: tests/functional/

Characteristics:

End-to-end testing
User-focused scenarios
Longer execution time
Real or realistic data

Example:

def test_complete_experiment_workflow():
    """Test complete experiment from start to finish."""
    # Setup experiment
    # Run tracking
    # Collect data
    # Verify results
    pass

Coverage Reports

Generating Coverage Reports

# HTML report (recommended for development)
pytest --cov=ethoscope --cov-report=html

# Terminal report
pytest --cov=ethoscope --cov-report=term-missing

# XML report (for CI/CD)
pytest --cov=ethoscope --cov-report=xml

# Combined reports
pytest --cov=ethoscope --cov-report=html --cov-report=xml --cov-report=term-missing

Coverage Thresholds

The project aims for:

Overall coverage: 70%
Unit tests: 85%
Integration tests: 60%
Critical components: 90%

Viewing Coverage Reports

# Open HTML report in browser
firefox htmlcov/index.html

# View terminal report
cat coverage-report.txt

CI/CD Integration

GitHub Actions Example

name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: 3.8
    - name: Install dependencies
      run: |
        pip install -r test-requirements.txt
        pip install -e src/ethoscope/
        pip install -e src/node/
    - name: Run tests
      run: python run_tests.py --coverage
    - name: Upload coverage
      uses: codecov/codecov-action@v1

Pre-commit Hooks

# Install pre-commit
pip install pre-commit

# Setup pre-commit hooks
pre-commit install

# Run manually
pre-commit run --all-files

Best Practices

Test Organization

One test file per module: test_module_name.py
Group related tests: Use test classes
Descriptive test names: Explain what is being tested
Arrange-Act-Assert: Structure tests clearly

Test Data

Use fixtures: For reusable test data
Small test data: Keep test data minimal
Realistic data: Use realistic values
Clean up: Remove test data after tests

Performance

Fast unit tests: Keep unit tests under 1 second
Mark slow tests: Use @pytest.mark.slow
Parallel execution: Use pytest-xdist for parallel runs
Profile tests: Use pytest --durations=10

Reliability

Independent tests: Tests should not depend on each other
Deterministic tests: Avoid random behavior
Clean state: Reset state between tests
Handle failures: Graceful failure handling

Documentation

Test docstrings: Explain what is being tested
Comment complex logic: Explain non-obvious test logic
Update documentation: Keep docs in sync with tests

Troubleshooting

Common Issues

Import Errors

# Solution: Install packages in development mode
pip install -e src/ethoscope/
pip install -e src/node/

# Or add to PYTHONPATH
export PYTHONPATH="${PYTHONPATH}:src/ethoscope:src/node"

Hardware Dependencies

# Mock hardware for testing
pytest -m "not hardware"

# Install hardware dependencies
pip install RPi.GPIO picamera

Database Issues

# Use in-memory database for tests
pytest --db-url=sqlite:///:memory:

# Clean test database
rm test_database.db

Slow Tests

# Skip slow tests
pytest -m "not slow"

# Run tests in parallel
pytest -n auto

Debug Mode

# Drop into debugger on failure
pytest --pdb

# Drop into debugger on first failure
pytest -x --pdb

# Show local variables
pytest -l

# Verbose output
pytest -vv

Performance Analysis

# Show slowest tests
pytest --durations=10

# Profile test execution
pytest --profile

# Memory usage analysis
pytest --memory-profile

Getting Help

Documentation: Check this file and inline docstrings
Issue Tracker: Report bugs and request features
pytest Documentation: https://docs.pytest.org/
Coverage.py Documentation: https://coverage.readthedocs.io/

Contributing

When adding new tests:

Follow the existing test structure
Use appropriate test types (unit/integration/functional)
Add proper documentation
Ensure tests are reliable and fast
Update this documentation if needed

Remember: Good tests are the foundation of maintainable code!

FilesExpand file tree

TESTING.md

Latest commit

History

TESTING.md

File metadata and controls

Testing Guide for Ethoscope Project

Table of Contents

Overview

Test Structure

Directory Layout

Test Files

Running Tests

Quick Start

Device Package Tests

Node Package Tests

Test Options

Pytest Markers

Verbosity and Output

Writing Tests

Test File Structure

Using Fixtures

Mocking Guidelines

Test Types

Unit Tests

Integration Tests

Functional Tests

Coverage Reports

Generating Coverage Reports

Coverage Thresholds

Viewing Coverage Reports

CI/CD Integration

GitHub Actions Example

Pre-commit Hooks

Best Practices

Test Organization

Test Data

Performance

Reliability

Documentation

Troubleshooting

Common Issues

Import Errors

Hardware Dependencies

Database Issues

Slow Tests

Debug Mode

Performance Analysis

Getting Help

Contributing