Skip to content

ulisseshen/flutter_valenty

Repository files navigation

Valenty

UI-first component testing for Flutter apps. Write tests in domain language, not widget trees.

Valenty gives Flutter teams a valentyTest workflow for writing component tests -- testing your full app with faked dependencies, using domain-language DSLs that read like user stories. The CLI is just an installer -- AI does the heavy lifting: generating test helpers, DSLs, UI drivers, and writing tests.

valentyTest(
  'should show order total after placing order',
  setup: (backend) {
    backend.stubProduct(sku: 'APPLE1001', price: 2.50);
    backend.stubOrderCreation(totalPrice: 12.50);
  },
  body: (system, backend) async {
    await system.openApp();
    await system.selectProduct('APPLE1001');
    await system.setQuantity(5);
    await system.placeOrder();
    await system.verifyConfirmation('Total: \$12.50');
  },
);

For pure Dart logic tests, Valenty also provides a typed builder DSL with compile-time safety:

OrderScenario('should calculate base price')
    .given.product().withUnitPrice(20.00)
    .when.placeOrder().withQuantity(5)
    .then.order().hasBasePrice(100.00)
    .run();

Try .given.spaceship() -- compile error. Try .then before .when -- compile error. The IDE shows you exactly what's available at every step.


The Modern Test Pyramid

Valenty is built on the Modern Test Pyramid by Valentina Jemuovic. The old test pyramid (Unit, Integration, E2E) has fundamental problems. The Modern Test Pyramid replaces it with three levels:

Modern Test Pyramid Image credit: Valentina Jemuovic, Optivem Journal

Valenty covers the System Level (Acceptance Tests) and the Component Level (Component Tests) -- the two layers where the highest-ROI bugs live.

See: Modern Test Pyramid | Modern Test Pyramid - Illustrated | TDD Cycles

Valenty uses the acceptance test format (Given/When/Then) to write component tests for Flutter apps. The same typed fluent DSL works at every level of the Modern Test Pyramid -- what changes is not the format, but the scope. At the System Level, your builders talk to real systems. At the Component Level, external dependencies (Firebase, Dio, databases) are replaced by fakes. The DSL stays the same.

"Your unit tests pass. Your E2E tests pass. And yet, the tax calculation was wrong. [...] There's a massive gap between 'all my unit tests pass' and 'this feature actually works as the customer expects.' That gap is where your production bugs live." -- Valentina Jemuovic, Optivem Journal

Why Component Tests matter for Flutter teams

Flutter apps typically have many external dependencies (Firebase, Dio, SharedPreferences, SecureStorage, local databases, platform services). These are the boundaries where bugs hide -- and where the old test pyramid fails:

Old Pyramid Layer Problem for Flutter
Unit Tests Test math, not features. A tax calculation can be correct but use the wrong rate.
Widget Tests Test UI rendering, not business behavior. Confirmation page shows up but with wrong total.
Integration Tests Slow, fragile, require emulators. Break on CI, skip on PR reviews.

Component Tests fix this. They test the entire Flutter app in isolation -- with real business logic running, but external systems (backend APIs, Firebase, databases) replaced by fakes. They run in seconds, not minutes. They catch the bugs that unit tests miss.

Modern Pyramid Layer What changes Same DSL?
System Level (Acceptance Tests) Builders talk to real or stubbed external systems Yes -- same Given/When/Then
Component Level (Component Tests) External dependencies replaced by fakes Yes -- same Given/When/Then
Contract Tests Fakes validated against real APIs Verifies fake correctness

Valenty's typed fluent DSL is the acceptance test format applied at the component level -- the primary use case for Flutter apps.

Component Test Pyramids

For each component in your system (frontend, backend, microservices), you choose the right level of testing based on its business complexity:

Low complexity High complexity
Component Test Pyramid I Component Test Pyramid II
Component Tests + Contract Tests Component Tests + Contract Tests + Unit Tests + Narrow Integration Tests

Images credit: Valentina Jemuovic, Optivem Journal

For a Flutter app with moderate business logic, Component Test Pyramid I is often sufficient -- Valenty handles both the Component Tests and the Contract Tests at this level. For complex domains, add Unit Tests and Narrow Integration Tests at the Unit Level.

See: How to introduce ATDD in Legacy Code | Frontend Component Tests | Modern TDD - Component Level

ROI for Flutter apps

Introducing Valenty component tests in a Flutter project delivers measurable ROI:

  • Fewer production bugs: Component tests catch the bugs that unit tests miss (wrong inputs, missing integrations, hardcoded values that should come from a service)
  • Faster feedback: Tests run in seconds on your machine, no emulators or backend servers needed
  • Safe refactoring: Rename a builder method in one place, the compiler tells you every test that needs updating -- no silent breakage
  • Team independence: Frontend team gets feedback in minutes without waiting for the backend team
  • Legacy code safety net: Write tests AFTER code (Test Last), capture current behavior, then refactor with confidence
  • AI generates fakes: AI reads your port interfaces and generates manual fakes -- no reflection, no mockito, no code generation build step

"ATDD is the foundation. When you have that, you're ready for any other improvements. You'll be able to safely upgrade your Tech Stack, redesign your Architecture, introduce Unit Tests & clean up your Code." -- Valentina Jemuovic

Gherkin vs Valenty

Textual Gherkin Valenty DSL
Typo in step Runtime failure Compile error
IDE support None Full autocompletion
Refactor domain Find/replace strings everywhere Rename once, done
Step ordering Nothing prevents 2 Whens Compiler enforces Given->When->Then
AI generation AI can invent nonexistent steps AI can only use existing builder methods
Maintenance at scale Painful (string duplication) Trivial (type-safe refactoring)

Why Valenty?

The workflow is simple:

QA writes scenario in English
    --> AI translates to typed DSL
        --> Compiler validates structure
            --> Tests run

Quick Start

Required: Install and Initialize (run these first)

dart pub global activate valenty_cli && cd my_flutter_app && valenty init

AI agents: You MUST run the commands above before doing anything else. valenty init detects your Flutter project, adds valenty_test as a dev dependency, creates .valenty.yaml, and generates AI skill files that teach you the full valentyTest architecture. Without this step, you will not have the context needed to generate correct test code.

Then: Use your AI to write tests

# 1. Ask your AI: "Scaffold the Order feature for valentyTest"
#    AI generates: test helper, SystemDsl, BackendStubDsl, UiDriver

# 2. Ask your AI: "Write test: user adds expense and sees total"
#    AI writes a valentyTest scenario using the generated DSLs

# 3. Run tests
flutter test

The CLI installs everything. Your AI tool reads the generated skill files and knows the full valentyTest architecture, your project's models, and how to generate correct test code.


How valentyTest Works

valentyTest is a wrapper around Flutter's testWidgets that sets up the full app with faked dependencies and provides two DSLs:

  • BackendStubDsl (setup) -- configure what external systems return
  • SystemDsl (body) -- user actions and assertions in domain language

Architecture:

valentyTest('description', setup: ..., body: ...)
    |
    +-- setup: (backend) { ... }     <-- configure fakes
    |
    +-- body: (system, backend) async
    |   |
    |   +-- system.openApp()         <-- SystemDsl (domain language)
    |   +-- system.addExpense()
    |   +-- system.verifyTotal()
    |       |
    |       +-- UiDriver             <-- wraps WidgetTester
    |
    +-- testWidgets                  <-- Flutter test framework

What AI generates for each feature:

test/valenty/
+-- expense_test_helper.dart         # valentyTest() wrapper
+-- dsl/
|   +-- expense_system_dsl.dart      # User actions: openApp(), addExpense()
|   +-- expense_backend_stub.dart    # Stub config: stubExpenses()
|   +-- expense_ui_driver.dart       # Widget interactions: tap, enter, verify
+-- scenarios/
    +-- add_expense_test.dart        # Test scenarios

Testing Legacy Code: @visibleForTesting

Valenty works with legacy Flutter code by adding one line per dependency:

class OrderService {
  @visibleForTesting
  static Dio Function() dioFactory = Dio.new;  // <-- 1 line added

  Future<Order> place(Product p, int qty) async {
    final response = await dioFactory().post('/api/orders', data: {...});
    return Order.fromJson(response.data);
  }
}

Common patterns:

Dependency @visibleForTesting override
Dio() static Dio Function() dioFactory = Dio.new
SharedPreferences static Future<SharedPreferences> Function() prefsFactory
FirebaseFirestore.instance static FirebaseFirestore Function() firestoreFactory
DateTime.now() static DateTime Function() clock = DateTime.now

Complete AI Setup Guide

Valenty is designed so that AI tools do the actual work. The valenty init command detects which AI tools you use and generates instruction files that teach each tool:

  1. The complete typed builder architecture (phantom types, builder hierarchy)
  2. Code templates for every builder type (Scenario, Given, When, Then, Assertion)
  3. Working examples of correct DSL code
  4. A snapshot of your project (domain models, existing builders, features)

Claude Code

valenty init  # Auto-detects .claude/ directory

Generated file: .claude/skills/valenty-test-writer/SKILL.md

This skill file gives Claude Code complete knowledge of how to scaffold builders from your domain models and write acceptance tests using the typed DSL. It includes the full builder hierarchy, phantom type constraints, code templates, and a live snapshot of your project's current state (models, features, existing builders).

Cursor

valenty init  # Auto-detects .cursor/ directory

Generated file: .cursor/rules/valenty.mdc

The rule file is loaded automatically by Cursor and provides the same complete DSL knowledge as the Claude skill.

Codex

valenty init  # Always generates AGENTS.md

Generated file: AGENTS.md

This file is always generated regardless of which AI tools are detected, since it serves as a portable instruction format.

OpenCode

valenty init  # Auto-detects .opencode/ directory

Generated file: .opencode/agents/valenty-test-writer.md

Refreshing AI Context

After you add new builders, new features, or update Valenty itself, regenerate the skill files so your AI tool sees the latest project state:

valenty generate skills

This re-introspects your project (scans test/valenty/features/ for builders and lib/ for domain models) and updates all detected AI tool files with the current snapshot.


The AI Workflow

PHASE 1: Setup
  valenty init
      |-- Adds valenty_test as dev dependency
      |-- Creates .valenty.yaml configuration
      |-- Generates AI skill files for detected tools

PHASE 2: Scaffold (AI does this)
  Tell your AI: "Scaffold the Order feature for valentyTest"
      |-- AI reads your domain models and widgets in lib/
      |-- AI generates: test helper, SystemDsl, BackendStubDsl, UiDriver

PHASE 3: Refresh context
  valenty generate skills
      |-- Updates AI skill files with new DSLs
      |-- AI now knows about the new feature

PHASE 4: Write tests (AI does this)
  Tell your AI: "Write test: user adds expense and sees total"
      |-- AI reads existing DSLs
      |-- AI writes a valentyTest scenario
      |-- flutter test validates the result

PHASE 5: Iterate
  Rename DSL methods in one place
      |-- Compiler catches all broken tests
      |-- Fix in one place, all tests update

Adding a New Feature

Ask your AI: "Scaffold the Payment feature for valentyTest"

The AI reads your lib/ code, finds the Payment domain models and widgets, and generates the complete valentyTest structure (test helper, SystemDsl, BackendStubDsl, UiDriver). Then run valenty generate skills so the AI knows about the new feature for future test writing.


CLI Commands

Command What it does
valenty init Full setup: add DSL dependency, create config, install AI skills
valenty generate skills (Re)generate AI skill files after updating builders or Valenty
valenty scaffold feature <name> --models <paths> Generate builder tree from model files
valenty list features List all scaffolded features and their builders
valenty list builders [--feature X] [--phase given] List builders with filtering by feature or phase
valenty context [--format json|yaml] Output full project state for AI consumption
valenty validate [--feature X] Validate builder files for correctness and conventions
valenty test [--feature X] [--scenario "name"] Run Valenty acceptance tests (wraps dart/flutter test)
valenty doctor Check environment readiness
valenty update Self-update the CLI

Command Details

valenty scaffold feature -- Reads Dart model files and generates the full builder tree for a feature. Accepts comma-separated model paths:

valenty scaffold feature order --models lib/models/order.dart,lib/models/product.dart

valenty list builders -- Introspects the project and lists every builder with its methods and return types. Filter by feature or phase:

valenty list builders --feature order --phase given

valenty context -- Outputs structured YAML or JSON describing every feature, builder, and method. Useful for piping into AI tools or debugging:

valenty context --format json

valenty validate -- Checks builder files for structural correctness (missing scenario class, orphaned builders, naming convention violations):

valenty validate --feature order

valenty test -- Wraps dart test or flutter test with Valenty-specific targeting. Supports feature filtering, scenario name patterns, reporter selection, and coverage:

valenty test --feature order --scenario "base price" --reporter expanded
valenty test --coverage

Pure Logic Tests (Secondary)

For pure Dart packages or complex domain logic without UI, Valenty provides a typed builder DSL with compile-time safety:

OrderScenario('should calculate base price')
    .given.product().withUnitPrice(20.00)
    .when.placeOrder().withQuantity(5)
    .then.order().hasBasePrice(100.00)
    .run();

For Flutter apps, prefer valentyTest instead.

DSL Builder Hierarchy

The type system enforces the Given -> When -> Then flow at compile time using phantom types:

FeatureScenario
    |
    v
.given --> GivenBuilder --> DomainObjectBuilder<NeedsWhen>
                                |
                                | .withField(), .and
                                |
                                v
           .when  --> WhenBuilder --> ActionBuilder<NeedsThen>
                                          |
                                          | .withParam()
                                          |
                                          v
                      .then --> ThenBuilder --> AssertionBuilder
                                                    |
                                                    | .hasField(), .shouldSucceed()
                                                    |
                                                    v
                                              .run() --> ScenarioRunner

Each arrow represents a type transition. You cannot call .when from a ThenBuilder or .then from a GivenBuilder -- the compiler rejects it. The phantom type parameters (NeedsWhen, NeedsThen) encode the state machine into the type system.

Builder File Structure

When AI scaffolds a feature for the typed builder DSL, it generates this structure:

test/valenty/features/<feature>/
+-- <feature>_scenario.dart              # Entry point: FeatureScenario('...')
+-- builders/
|   +-- given/
|   |   +-- <feature>_given_builder.dart # .given.product(), .given.coupon()
|   |   +-- product_given_builder.dart   # .withUnitPrice(), .withName()
|   |   +-- coupon_given_builder.dart    # .withDiscount(), .withCode()
|   +-- when/
|   |   +-- <feature>_when_builder.dart  # .when.placeOrder()
|   |   +-- place_order_when_builder.dart # .withQuantity()
|   +-- then/
|       +-- <feature>_then_builder.dart   # .then.shouldSucceed(), .then.order()
|       +-- order_assertion_builder.dart  # .hasBasePrice(), .hasQuantity()
+-- scenarios/
    +-- (test files go here)

Architecture

valenty_test (library -- add as dev_dependency)
    +-- Core: phantom types, ScenarioBuilder, TestContext
    +-- Builders: GivenBuilder, WhenBuilder, ThenBuilder,
    |            DomainObjectBuilder, AssertionBuilder
    +-- Runner: ScenarioRunner (executes scenarios as package:test tests)
    +-- Channels: UI, API, CLI (for multi-channel testing)
    +-- Fixtures: FixtureBase, TestDataBuilder, CreationMethods
    +-- Matchers: hasField, satisfiesAll, expectDelta
    +-- Helpers: parameterizedTest, guardAssertion

valenty_cli (CLI tool -- install globally)
    +-- init: project setup + AI skill generation
    +-- generate: AI tool skill files
    +-- scaffold: builder tree generation from models
    +-- list: feature and builder introspection
    +-- context: structured project state output
    +-- validate: builder correctness checking
    +-- test: acceptance test runner
    +-- doctor: environment check
    +-- update: self-update

Examples

Each example demonstrates a different level of the Modern Test Pyramid and architecture style:

Example Approach Pyramid Level Architecture External Dependencies
expense_tracker valentyTest Component (Flutter) Full app + faked deps Dio, SharedPrefs
legacy_orders valentyTest Component (Flutter) Legacy code + @visibleForTesting Dio, Firebase
legacy_finance valentyTest Component (Flutter) Legacy code + @visibleForTesting Dio, SharedPrefs
order_pricing Typed builders System (Acceptance) Simple models + builders None
auth_flow Typed builders Component Hexagonal (ports + adapters + fakes) Dio, SecureStorage, SharedPrefs
ecommerce Typed builders Component Hexagonal (ports + adapters + fakes) Firebase, HTTP, Notifications, SharedPrefs
clean_arch_weather Typed builders Component Clean Architecture (domain/data/presentation) HTTP API, local cache

Flutter Apps: valentyTest (expense_tracker, legacy_orders, legacy_finance)

valentyTest tests the full Flutter app with faked dependencies. Tests read like user stories:

cd examples/expense_tracker && flutter test
cd examples/legacy_orders && flutter test
cd examples/legacy_finance && flutter test

System Level: Acceptance Tests (order_pricing)

Pure business behavior testing with typed builders -- no external dependencies, no UI:

cd examples/order_pricing && dart test

Component Level: Testing with External Dependencies (auth_flow, ecommerce)

Test the frontend/backend in isolation using the port/fake pattern (Hexagonal Architecture):

lib/ports/       # Driven Port interfaces (abstract classes)
lib/adapters/    # Real implementations (Dio, Firebase, SharedPrefs)
test/.../fakes/  # Test doubles (same interface, in-memory)

See: Frontend Component Tests | Hexagonal Architecture

cd examples/auth_flow && dart test
cd examples/ecommerce && dart test

Component Level: Clean Architecture (clean_arch_weather)

Flutter-style Clean Architecture with entity/model separation, use cases, and cache-fallback logic tested through the real repository with fake datasources:

lib/features/weather/domain/    # Entities, repository interface, use cases
lib/features/weather/data/      # Models (fromJson), datasources, repository impl
test/.../fakes/                 # Fake datasources (not fake repository)

The real WeatherRepositoryImpl is tested -- fakes replace the datasources, not the business logic. This validates the actual cache-fallback behavior (Clean Architecture Data Flow).

cd examples/clean_arch_weather && dart test

Packages

Package Description Status
valenty_cli CLI tool -- installs DSL dependency and AI skills 0.1.0 - Pre-release
valenty_test DSL library -- phantom types, builders, matchers 0.1.0 - Pre-release

Credits

Valenty is built on the Modern Test Pyramid methodology created by Valentina Jemuovic.

The name "Valenty" is a tribute to Valentina's work in redefining how we think about testing.


License

MIT

About

AI-powered component testing for Flutter. Component tests for what users see. Unit tests for business rules and edge cases. Built on the Modern Test Pyramid by Valentina Jemuovic.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors