Skip to content

Fix five serialization flaky tests#36

Open
pengyuejiang wants to merge 1 commit into
IBM:masterfrom
pengyuejiang:fix-flaky
Open

Fix five serialization flaky tests#36
pengyuejiang wants to merge 1 commit into
IBM:masterfrom
pengyuejiang:fix-flaky

Conversation

@pengyuejiang
Copy link
Copy Markdown

The Issue

By using the plugin NonDex, 5 tests

com.ibm.og.json.type.ChoiceConfigTypeAdapterFactoryTest.serialization
com.ibm.og.json.type.ContainerConfigTypeAdapterFactoryTest.serialization
com.ibm.og.json.type.FilesizeConfigTypeAdapterFactoryTest.serialization
com.ibm.og.json.type.OperationConfigTypeAdapterFactoryTest.serialization
com.ibm.og.json.type.SelectionConfigTypeAdapterFactoryTest.serialization

may fail in the future due to assuming a deterministic order of fields in serialization strings. The Java specification does not guarantee such an order, yet its current implementation assumes one. This plugin shuffles the ordering of fields, which cause these tests to fail when run without the plugin. The goal of the fix is to make sure that no false alarms were made for this reason.

Reproduce Result

Here are the steps to reproduce the result:

git clone https://github.com/IBM/og && cd og
export JAVA_HOME=$(/usr/libexec/java_home -v1.8)
mvn clean install -pl og-configuration -am -DskipTests
mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl og-configuration -Dtest=ChoiceConfigTypeAdapterFactoryTest#serialization -DnondexRuns=10
mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl og-configuration -Dtest=ContainerConfigTypeAdapterFactoryTest#serialization -DnondexRuns=10
mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl og-configuration -Dtest=FilesizeConfigTypeAdapterFactoryTest#serialization -DnondexRuns=10
mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl og-configuration -Dtest=OperationConfigTypeAdapterFactoryTest#serialization -DnondexRuns=10
mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl og-configuration -Dtest=SelectionConfigTypeAdapterFactoryTest#serialization -DnondexRuns=10

The Fix

The current fix implemented deserializes the string and then compare them. Since these fixes rely on dependencies also used by the project, they do not introduce any new potential issues that are particular to the project itself, and can be trusted for their functionalities. The test in SelectionConfigTypeAdapterFactoryTest with case insensitiveness is also addressed.

Please do take a look and comment on whether this is a viable approach, hope to hear back from you soon :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant