fix(tests): fix vpc model tests to use robust Json comparison #78
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem:
This PR resolves intermittent test failures observed across various VPC model tests, using the NonDex tool.
https://github.com/TestingResearchIllinois/NonDex
In the pull request #38, the contributor didn't explain the root cause of the problem.
Root Cause
The toString() method on the SDK models relies on Gson serialization to produce a JSON string.
@Override public String toString() { return GsonSingleton.getGson().toJson(this); }from https://github.com/IBM/java-sdk-core/blob/main/src/main/java/com/ibm/cloud/sdk/core/service/model/GenericModel.java.Gson uses Java Reflection and getDeclaredFields() to determine the fields in the object. However,
This issue was also raised in the gson repository: https://github.com/google/gson/issues/617
Hence, in some JVM environments, the reflection API returns object fields in a non-deterministic (random) order. Simple string assertEquals fails in this scenario, leading to intermittent test failures.
Solution: Canonical JSON Comparison
To fix this, I have introduced a robust utility method, TestUtilities.assertJsonEquals(), and applied it to all affected model tests.
This new utility implements a canonical comparison strategy:
Comparison with JsonParser.parseString:
The fix in this Pull Request uses TreeMap .equals() comparison whereas JsonParser uses JsonElement .equals() comparison. Both fixes ignores iteration order and compares content, ensuring test stability across all environments.