Skip to content

Conversation

@Fr4nk03
Copy link

@Fr4nk03 Fr4nk03 commented Nov 5, 2025

Problem:

This PR resolves intermittent test failures observed across various VPC model tests, using the NonDex tool.
https://github.com/TestingResearchIllinois/NonDex

In the pull request #38, the contributor didn't explain the root cause of the problem.

Root Cause

The toString() method on the SDK models relies on Gson serialization to produce a JSON string.
@Override public String toString() { return GsonSingleton.getGson().toJson(this); } from https://github.com/IBM/java-sdk-core/blob/main/src/main/java/com/ibm/cloud/sdk/core/service/model/GenericModel.java.

Gson uses Java Reflection and getDeclaredFields() to determine the fields in the object. However,

The elements in the returned array are not sorted and are not in any particular order.
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Class.html#getDeclaredFields()

This issue was also raised in the gson repository: https://github.com/google/gson/issues/617

Hence, in some JVM environments, the reflection API returns object fields in a non-deterministic (random) order. Simple string assertEquals fails in this scenario, leading to intermittent test failures.

Solution: Canonical JSON Comparison

To fix this, I have introduced a robust utility method, TestUtilities.assertJsonEquals(), and applied it to all affected model tests.

This new utility implements a canonical comparison strategy:

  • It takes two non-deterministic JSON strings as input.
  • It uses Gson's deserialization to convert both strings into java.util.TreeMap<String, Object> objects.
  • TreeMap is deterministic: Since TreeMap inherently enforces alphabetical key order on the top level, this conversion normalizes the ordering of the model's fields.
  • It performs the final comparison using assertEquals(expectedMap, actualMap), which ensures the test passes based on content equality, regardless of the original key order in the input strings.

Comparison with JsonParser.parseString:

The fix in this Pull Request uses TreeMap .equals() comparison whereas JsonParser uses JsonElement .equals() comparison. Both fixes ignores iteration order and compares content, ensuring test stability across all environments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant