Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ package already implements. Profile version in use: `21.205.0` (see
| Status | Capabilities |
| --- | --- |
| **Supported** | Common Activity and Workout read/write via typed profile messages; `FitFileBuilder` encode path; **header CRC** (14-byte headers) and **file-level CRC** on load / stream exhaustion (`check_crc=True` default); developer fields for common declaration patterns; streaming iterators (`FitFile.iter_file` / `iter_stream`); CSV export (`to_csv` / `to_rows`); Course and similar message types when you construct them yourself; **chained multi-segment** FIT decode via `from_bytes` / `from_file` (all segments projected into `records`); **compressed timestamp** reconstruction into field 253; **subfield resolution** (ref-field match → type / scale / offset / units; multi-ref AND; first match wins); **component expansion** for all Profile main-field sources (generated registry) **and** components on the active subfield; nested expansion + accumulator rollover; **unknown field ids** on known messages as `UnknownField` (decoded values + `raw_bytes`); **encode modes** `EncodeMode.PRESERVE` (default; unedited bit-identical + post-edit dirty re-project) and `EncodeMode.CANONICAL` (full re-project, normalized sizes/CRCs; optional `strict=True` precheck) — see [Encode policies](#encode-policies) |
| **Partial** | Unknown global messages via `GenericMessage` (readable; unedited preserve keeps wire bytes; post-edit re-encodes dirty records only); composable validation API (`validate_fit_file` / `FitFile.validate`) with WIRE + PROFILE + Activity FILE_TYPE levels — **PROFILE validation is CORE today** (developer-field subset + **ambiguous subfield** ERROR); opt-in **PRESERVATION** level reports unknown-field `raw_bytes` loss after edits; architecture decision **O1** keeps bundled `Profile.xlsx` as the full metadata source of truth with future DOMAIN/FULL scopes (FULL opt-in, not default strict) — see [`docs/FIT_CONFORMANCE_DESIGN.md`](docs/FIT_CONFORMANCE_DESIGN.md) §3.1; Builder `strict=True` is a thin wrapper over the same checks (WIRE+PROFILE+FILE_TYPE only) |
| **Not supported / incomplete** | Full PROFILE semantics (native field requirements, enums, units beyond subfield scale/units); file-type rules for non-Activity types (FILE_TYPE level fails closed); intentional `repair()` API (strict path never silent-repairs); bit-identical rewrite of compressed-timestamp dirty records when field 253 is not on the definition (strict raises; non-strict keeps compressed header) |
| **Partial** | Unknown global messages via `GenericMessage` (readable; unedited preserve keeps wire bytes; post-edit re-encodes dirty records only); composable validation API (`validate_fit_file` / `FitFile.validate`) with WIRE + PROFILE + **Activity and Workout** FILE_TYPE levels — **PROFILE validation is CORE today** (developer-field subset + **ambiguous subfield** ERROR); opt-in **PRESERVATION** level reports unknown-field `raw_bytes` loss after edits; architecture decision **O1** keeps bundled `Profile.xlsx` as the full metadata source of truth with future DOMAIN/FULL scopes (FULL opt-in, not default strict) — see [`docs/FIT_CONFORMANCE_DESIGN.md`](docs/FIT_CONFORMANCE_DESIGN.md) §3.1; Builder `strict=True` is a thin wrapper over the same checks (WIRE+PROFILE+FILE_TYPE only) |
| **Not supported / incomplete** | Full PROFILE semantics (native field requirements, enums, units beyond subfield scale/units); file-type rules for types other than Activity/Workout (e.g. Course — FILE_TYPE fails closed); intentional `repair()` API (strict path never silent-repairs); bit-identical rewrite of compressed-timestamp dirty records when field 253 is not on the definition (strict raises; non-strict keeps compressed header) |

If you need a construct listed as incomplete, prefer an official Garmin SDK or
wait for the phased work in the design doc (including the **Remaining gaps**
Expand Down Expand Up @@ -221,7 +221,7 @@ Validation is a first-class API. Levels match the design doc
| --- | --- | --- |
| `ConformanceLevel.WIRE` | Local IDs, definition field layout/sizes, data records vs active definition | Implemented |
| `ConformanceLevel.PROFILE` | Developer field declarations (`developer_data_id` / `field_description`) and base-type consistency; **ambiguous native subfields** (more than one Profile match) as ERROR | **CORE scope today** (+ ambiguous-subfield ERROR). Roadmap: DOMAIN then FULL rules from Profile.xlsx; FULL is opt-in, not default `strict` — design doc §3.1 (O1). Subfield *resolution* for decode/encode is separate and supported |
| `ConformanceLevel.FILE_TYPE` | `file_id` first/unique + required fields; Activity required messages and fields | **Activity only**; other `file_id.type` values **fail closed** (intentional until more validators exist) |
| `ConformanceLevel.FILE_TYPE` | `file_id` first/unique + required fields; Activity and Workout required messages and fields | **Activity + Workout**; other `file_id.type` values (e.g. Course) **fail closed** (intentional until more validators exist) |
| `ConformanceLevel.PRESERVATION` | Post-edit rewrite loss (e.g. `UnknownField.raw_bytes` cleared by mutation) | **Opt-in** — not in default levels / Builder `strict=True` |

Call validation on any `FitFile` or record list — after decode or before encode:
Expand Down
23 changes: 12 additions & 11 deletions docs/FIT_CONFORMANCE_DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ Baseline after wire-layer work (#44 / #45 and follow-ups on `main`):
| **Ambiguous subfields** (more than one match) | PROFILE ERROR (decode still uses first match) | Same policy |
| **Preservation encode** (`to_bytes(mode=EncodeMode.PRESERVE)` / `preserve=True`, default) for buffer-decoded files: unedited path bit-identical; **post-edit** path re-encodes dirty records and copies `source_bytes` for the rest | Supported | Aligns with design §6.1 |
| **Canonical encode** (`to_bytes(mode=EncodeMode.CANONICAL)` / `preserve=False`); optional `strict=True` precheck | Supported | Compressed-header expansion only when field 253 is on the definition; otherwise keep compressed (non-strict) or raise (strict) |
| Unknown global messages (`GenericMessage`); composable validation (`validate_fit_file` / `FitFile.validate`) with WIRE + PROFILE (developer fields + ambiguous-subfield ERROR) + Activity FILE_TYPE; opt-in **PRESERVATION** level; Builder `strict=True` wraps default levels only | Partial | Full Profile field/enum/units rules; Workout/Course FILE_TYPE |
| Full PROFILE semantics (native required fields, enums, units beyond subfield scale/units); non-Activity FILE_TYPE rules | Not supported / incomplete | Phases 3–4 and remaining-gaps table below |
| Unknown global messages (`GenericMessage`); composable validation (`validate_fit_file` / `FitFile.validate`) with WIRE + PROFILE (developer fields + ambiguous-subfield ERROR) + Activity/Workout FILE_TYPE; opt-in **PRESERVATION** level; Builder `strict=True` wraps default levels only | Partial | Full Profile field/enum/units rules; Course FILE_TYPE |
| Full PROFILE semantics (native required fields, enums, units beyond subfield scale/units); Course and other non-Activity/non-Workout FILE_TYPE rules | Not supported / incomplete | Phases 3–4 and remaining-gaps table below |
| Unknown field ids on known messages (`UnknownField` + `raw_bytes` on decode; survive post-edit when not mutated) | Supported | Mutating an unknown field clears `raw_bytes` (PRESERVATION ERROR if that level is selected) |

Until §11 Definition of Done is met, do not describe the library as fully
Expand All @@ -46,8 +46,8 @@ stable keys, later letters are stage placeholders until children are created.
| Post-edit PRESERVATION (edited files, dirty records) | Phase 4 / PRESERVATION level | **F** SHA-18 | **Done**: per-record dirty + mixed encode; opt-in PRESERVATION findings |
| Encode policies (canonical vs preserve, strict vs repair) | Phase 4 §6 | **G** SHA-19 | **Done**: `EncodeMode` + policy matrix; no silent invalid clamp; `repair()` API still future |
| Full PROFILE validation from bundled Profile.xlsx `21.205.0` | Phase 3 PROFILE | **H** (SHA-12 stage 4; child TBD) | Slice by message family / rule kind |
| Workout FILE_TYPE rules | Phase 4 §7 | **I** (SHA-12 stage 4; child TBD) | Not in first batch |
| Course FILE_TYPE rules | Phase 4 §7 | **J** (SHA-12 stage 4; child TBD) | Not in first batch |
| Workout FILE_TYPE rules | Phase 4 §7 | **I** SHA-21 | **Done**: required messages/fields for Workout; SDK Workout*.fit fixtures pass FILE_TYPE |
| Course FILE_TYPE rules | Phase 4 §7 | **J** (SHA-12 stage 4; child TBD) | Not in first batch; still fail-closed |
| Public capability matrix / release-note pass at claim flip | Phase 5 | **L** (SHA-12 stage 5; child TBD) | Update README + this section together |

Letter **A** is this status-truth pass (Multica **SHA-13**). Do not claim gaps
Expand Down Expand Up @@ -539,7 +539,8 @@ report.raise_for_errors()

`FitFileBuilder(strict=True)` is a thin wrapper over the same API (all default
levels, raise on error). Wire-range and Definition Message checks still run on
every `add`. FILE_TYPE rules cover Activity and fail closed for other types.
every `add`. FILE_TYPE rules cover Activity and Workout; other `file_id.type`
values (e.g. Course) fail closed.

**Already on the wire / compatibility path** (see Current status): layered
decode (`fit_tool/wire`), chained multi-segment load, header + file CRC,
Expand All @@ -550,9 +551,9 @@ messages, and preservation encode via `to_bytes(preserve=True)` for unedited
re-project).

This still does **not** complete the conformance claim. Remaining work includes
full Profile validation scopes (DOMAIN/FULL) and Workout/Course FILE_TYPE
validators — see **Remaining gaps** and Phases 3–5. Encode modes (G) are on the
compatibility path; the long-term `FitDocument` encode surface remains future.
full Profile validation scopes (DOMAIN/FULL) and Course FILE_TYPE validators —
see **Remaining gaps** and Phases 3–5. Encode modes (G) are on the compatibility
path; the long-term `FitDocument` encode surface remains future.

## 7. File-type validators

Expand Down Expand Up @@ -736,9 +737,9 @@ Exit: all produced standard files pass the selected Garmin and repository
validators without repair.

**Progress (partial):** unedited and **post-edit** PRESERVE paths exist; explicit
`EncodeMode` / `strict` / policy matrix landed (G / SHA-19). Activity FILE_TYPE
(fail-closed other types) exists. Workout/Course validators remain Multica I–J.
`repair()` API is still future.
`EncodeMode` / `strict` / policy matrix landed (G / SHA-19). Activity and
Workout FILE_TYPE exist (I / SHA-21); other types (including Course / J) fail
closed. `repair()` API is still future.

### Phase 5: API migration and performance

Expand Down
82 changes: 79 additions & 3 deletions fit_tool/tests/test_validation.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@
from fit_tool.profile.messages.lap_message import LapMessage
from fit_tool.profile.messages.record_message import RecordMessage
from fit_tool.profile.messages.session_message import SessionMessage
from fit_tool.profile.messages.workout_message import WorkoutMessage
from fit_tool.profile.messages.workout_step_message import WorkoutStepMessage
from fit_tool.profile.profile_type import FileType, Manufacturer, Sport
from fit_tool.profile.profile_type import FileType, Manufacturer, Sport, WorkoutStepDuration, WorkoutStepTarget
from fit_tool.validation import (
ConformanceLevel,
FitFileValidator,
Expand Down Expand Up @@ -60,6 +61,29 @@ def add_minimal_activity_messages(builder, record_message=None):
builder.add(activity)


def add_minimal_workout_messages(builder, step_message=None):
file_id = FileIdMessage()
file_id.type = FileType.WORKOUT
file_id.manufacturer = Manufacturer.DEVELOPMENT.value
file_id.product = 0
file_id.serial_number = 1234
file_id.time_created = 1_700_000_000_000
builder.add(file_id)

workout = WorkoutMessage()
workout.num_valid_steps = 1
workout.sport = Sport.CYCLING
builder.add(workout)

step = step_message if step_message is not None else WorkoutStepMessage()
step.message_index = 0
step.duration_type = WorkoutStepDuration.TIME
step.duration_time = 600.0
step.target_type = WorkoutStepTarget.OPEN
step.target_value = 0
builder.add(step)


class TestFitValidation(unittest.TestCase):

def test_builder_rejects_local_id_outside_wire_range(self):
Expand Down Expand Up @@ -98,9 +122,10 @@ def test_strict_activity_accepts_required_message_structure(self):
self.assertGreater(len(encoded), 0)

def test_strict_validation_fails_closed_for_unsupported_file_type(self):
# Course (and other non-Activity/non-Workout types) still fail closed.
builder = FitFileBuilder(strict=True)
file_id = FileIdMessage()
file_id.type = FileType.WORKOUT
file_id.type = FileType.COURSE
file_id.manufacturer = Manufacturer.DEVELOPMENT.value
file_id.product = 0
file_id.serial_number = 1234
Expand All @@ -110,6 +135,27 @@ def test_strict_validation_fails_closed_for_unsupported_file_type(self):
with self.assertRaisesRegex(FitValidationError, 'not implemented'):
builder.build_bytes()

def test_strict_workout_accepts_required_message_structure(self):
builder = FitFileBuilder(strict=True, auto_define=True, min_string_size=50)
add_minimal_workout_messages(builder)

encoded = builder.build_bytes()

self.assertGreater(len(encoded), 0)

def test_strict_workout_requires_workout_and_steps(self):
builder = FitFileBuilder(strict=True)
file_id = FileIdMessage()
file_id.type = FileType.WORKOUT
file_id.manufacturer = Manufacturer.DEVELOPMENT.value
file_id.product = 0
file_id.serial_number = 1234
file_id.time_created = 1_700_000_000_000
builder.add(file_id)

with self.assertRaisesRegex(FitValidationError, 'workout'):
builder.build_bytes()

def test_strict_validation_rejects_undeclared_developer_field(self):
developer_field = DeveloperField(
developer_data_index=0,
Expand Down Expand Up @@ -218,7 +264,7 @@ def test_validate_fit_file_raise_mode_matches_strict_builder(self):
def test_validate_wire_only_skips_file_type_rules(self):
builder = FitFileBuilder()
file_id = FileIdMessage()
file_id.type = FileType.WORKOUT
file_id.type = FileType.COURSE
file_id.manufacturer = Manufacturer.DEVELOPMENT.value
file_id.product = 0
file_id.serial_number = 1234
Expand All @@ -235,6 +281,15 @@ def test_validate_wire_only_skips_file_type_rules(self):
any('not implemented' in finding.message for finding in full_report.errors)
)

def test_validate_workout_file_type_report(self):
builder = FitFileBuilder(auto_define=True, min_string_size=50)
add_minimal_workout_messages(builder)
fit_file = builder.build()

report = validate_fit_file(fit_file, levels={ConformanceLevel.FILE_TYPE})
self.assertFalse(report.has_errors)
self.assertEqual(report.findings, [])

def test_fit_file_validator_legacy_facade(self):
builder = FitFileBuilder()
add_minimal_activity_messages(builder)
Expand Down Expand Up @@ -662,6 +717,27 @@ def test_file_type_findings_structure_errors(self):
self.assertTrue(report5.has_errors)
self.assertTrue(any('missing required field' in f.message for f in report5.errors))

# Workout: missing required step fields
w_builder = FitFileBuilder(auto_define=True, min_string_size=20)
w_file_id = FileIdMessage()
w_file_id.type = FileType.WORKOUT
w_file_id.manufacturer = Manufacturer.DEVELOPMENT.value
w_file_id.product = 0
w_file_id.serial_number = 1
w_file_id.time_created = 1_700_000_000_000
w_builder.add(w_file_id)
w_msg = WorkoutMessage()
# num_valid_steps intentionally unset
w_builder.add(w_msg)
bare_step = WorkoutStepMessage()
bare_step.workout_step_name = 'x'
# message_index / duration_type / target_type unset
w_builder.add(bare_step)
w_fit = w_builder.build()
report6 = validate_fit_file(w_fit, levels={ConformanceLevel.FILE_TYPE})
self.assertTrue(report6.has_errors)
self.assertTrue(any('missing required field' in f.message for f in report6.errors))

def test_legacy_validator_helpers(self):
builder = FitFileBuilder()
add_minimal_activity_messages(builder)
Expand Down
Loading
Loading