Skip to content

Inconsistent Output Baseline Comparisons #212

@mseng10

Description

@mseng10

Description

After fixing a number of failing tests in the recent weeks, I came across some inconsistencies in the baseline test cases. Essentially, all of the baseline comparisons that compare an image without the name OutputBaseline, are incorrectly comparing results. An example can be found in the AddNoiseToBinaryImageTestBaselineComparison, which produces:

/Users/mseng_l/ITKExamples-build/bin/ImageCompareCommand "--test-image" 
"/Users/mseng_l/ITKExamples-build/src/Core/Common/AddNoiseToBinaryImage/AddNoiseToBinaryImage.png" "--baseline-image" 
"/Users/mseng_l/ITKExamples-build/src/Core/Common/AddNoiseToBinaryImage/AddNoiseToBinaryImage.png"

As shown above, the baseline comparison is not comparing the example's output to the baseline image, but rather the baseline image to itself, which will always yield a passing result.

This and many others will likely require one of the following changes:

  • Remove baseline comparisons for examples that do not produce a consistent result (i.e. AddNoiseToBinaryImage due to MersenneTwisterRandomVariateGenerator)
  • Rename baseline images to OutputBaseline for examples that consistently produce and save their output
  • Implement ImageFileWriter for examples that do not save their output. Subsequently update the baseline comparison for that example, upload the image to Girder and finally update the Documentation.rst

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions