Fix flaky test testInstanceTemplatePrototypeInstanceByVolume #36
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Problem
The test
com.ibm.cloud.is.vpc.v1.model.InstanceTemplatePrototypeInstanceByVolumeTest.testInstanceTemplatePrototypeInstanceByVolumeis flaky as was discovered through the plug-in NonDex. The testcom.ibm.cloud.is.vpc.v1.model.InstanceTemplatePrototypeInstanceByVolumeTest.testInstanceTemplatePrototypeInstanceBnyVolumecan fail because it compares two JSON strings whose key order can differ. Here are the full steps to reproduce the issue:The build will fail.
The current implementation compares two serializations with each other. However, it would be safer to directly compare the objects themselves to avoid any issues where serialization strings are different but representing the same object. NonDex shuffles the ordering of fields, so the test will fail without having any real issues.
The Fix
It's a very simple fix, instead of comparing serializations,
JsonParserfrom GSON is used to compare their deserialized forms to make sure that the test will not fail because a field in the serialization string is in a different ordering from another that is constructed from the same object.There are a few other tests in the project that have similar issues, which can also be discovered through running NonDex for the whole project. Please do give some feedbacks on whether you think this way of fixing it is feasible. Thanks!