Skip to content

Conversation

@jpobst
Copy link
Contributor

@jpobst jpobst commented Sep 8, 2021

Context: dotnet/android#6271
Context: https://developercommunity.visualstudio.com/t/XamarinAndroid-binding-compiling-proble/1477963

We are seeing some bizarre results from seemingly simple code where we generate JNI delegate names that should not be possible, like:

delegate IntPtr _JniMarshal_PP_Ljava/Lang/String; (IntPtr jnienv, IntPtr klass)

which should be:

delegate IntPtr _JniMarshal_PP_L (IntPtr jnienv, IntPtr klass)

The code in question seems like it should notice that Ljava/Lang/String; starts with L and return L instead of the full string:

var jni_name = symbol.JniName;

if (jni_name.StartsWith ("L") || jni_name.StartsWith ("["))
    return "L";

return symbol.JniName;

Perhaps the user's current culture causes this to fail somehow? Regardless, we should follow best practices and follow .NET's string globalization suggestions. As such, this enables the relevant code analysis rules (CA1307;CA1309;CA1310) and fixes violations of them.

ex:

if (jni_name.StartsWith ("L", StringComparison.Ordinal) || jni_name.StartsWith ("[", StringComparison.Ordinal))

The catch is that the net6.0 versions of these rules are stricter and require overloads that do not exist in .NET Framework to fix the violations. Turning them on in .editorconfig affects all TFMs, so instead we are going to use Directory.Build.props to only enable them for !net6.0 builds.

Additionally, add a new .yaml step that shuts down the cached dotnet MSBuild processes between invocations, to fix the error that has been happening on Windows - NET Core:

error MSB3027: Could not copy "obj\\Release\Java.Interop.BootstrapTasks.dll" to "D:\a\1\s\bin\BuildRelease\Java.Interop.BootstrapTasks.dll". Exceeded retry count of 10. Failed.  [D:\a\1\s\build-tools\Java.Interop.BootstrapTasks\Java.Interop.BootstrapTasks.csproj]
error MSB3021: Unable to copy file "obj\\Release\Java.Interop.BootstrapTasks.dll" to "D:\a\1\s\bin\BuildRelease\Java.Interop.BootstrapTasks.dll". The process cannot access the file 'D:\a\1\s\bin\BuildRelease\Java.Interop.BootstrapTasks.dll' because it is being used by another process. [D:\a\1\s\build-tools\Java.Interop.BootstrapTasks\Java.Interop.BootstrapTasks.csproj]

@jpobst jpobst force-pushed the globalization-warnings branch from a92882d to d24f074 Compare September 8, 2021 15:59
@jpobst jpobst marked this pull request as ready for review September 8, 2021 16:26

public static IEnumerable<JniTypeName> FromSignature (string signature)
{
if (signature.StartsWith ("(")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this instead use signature.StartsWith('('), which "mirrors" changes elsewhere which use char instead of string, StringComparison overloads? For example the signature.IndexOf(')') change on the following line?

Likewise many of the other changes that only deal with a single character…

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally yes, but those overloads are introduced in NET Core 2.0/NET Standard 2.1 so they are not available to us.

@jonpryor
Copy link
Contributor

jonpryor commented Sep 8, 2021

Is this a guess that a culture-aware comparison is failing, or is this known? If so, which culture?

@jonpryor
Copy link
Contributor

jonpryor commented Sep 8, 2021

Looks like this is a culture-related issue; behold!

using System;
using System.Globalization;

class App {
    public static void Main ()
    {
        var infos = CultureInfo.GetCultures(CultureTypes.AllCultures);
        foreach (var ci in infos) {
            if (!"Ljava/lang/String;".StartsWith ("L", ignoreCase:false, culture:ci)) {
                Console.WriteLine(ci.Name);
            }
        }
    }
}

The above app tries "Lfoo".StartsWith("L") for every culture in .NET framework, and if string.StartsWith() returns false, prints out the culture name.

The result?

bs
bs-Cyrl
bs-Cyrl-BA
bs-Latn
bs-Latn-BA
hr
hr-BA
hr-HR
sr
sr-Cyrl
sr-Cyrl-BA
sr-Cyrl-ME
sr-Cyrl-RS
sr-Cyrl-XK
sr-Latn
sr-Latn-BA
sr-Latn-ME
sr-Latn-RS
sr-Latn-XK

There are 19 cultures, across 3 languages, for which our "Lwhatever".StartsWith("L") check won't work as we expect: Bosnian, Croatian, Serbian.

@jpobst
Copy link
Contributor Author

jpobst commented Sep 8, 2021

Good sleuthing! This was a guess on my part, so I'm glad you confirmed that it is the culprit!

@jpobst
Copy link
Contributor Author

jpobst commented Sep 8, 2021

For completeness, the issue is specifically "lj" which is considered to be a single letter in these languages. This explains why other strings like Landroid/view/Views; succeed but Ljava/lang/String; does not.

@jonpryor
Copy link
Contributor

jonpryor commented Sep 8, 2021

Commit message for review:

Context: https://github.com/xamarin/xamarin-android/issues/6271
Context: https://developercommunity.visualstudio.com/t/XamarinAndroid-binding-compiling-proble/1477963
Context: https://www.learncroatian.eu/blog/the-croatian-letters

If you attempt to build an Android Binding project on Windows within
a Bosnian, Croatian, or Serbian locale, the build may break as the
delegate type names created by 56955d9a may not be valid C#, e.g.:

	delegate IntPtr _JniMarshal_PP_Ljava/Lang/String; (IntPtr jnienv, IntPtr klass)

It *should* be declaringn a `_JniMarshal_PP_L` type, *not*
`_JniMarshal_PP_Ljava/Lang/String;`, the latter of which results in
numerous C# errors:

	error CS1003: Syntax error, '(' expected
	error CS1001: Identifier expected
	error CS1001: Identifier expected
	error CS1003: Syntax error, ',' expected
	error CS1003: Syntax error, ',' expected
	error CS1001: Identifier expected
	error CS1026: ) expected

The problem is caused by the interplay of two factors:

 1. Commit 56955d9a uses the culture-sensitive
    [`string.StartsWith(string)`][0] method to determine if a JNI
    type name starts with `L`:

        if (jni_name.StartsWith ("L") || jni_name.StartsWith ("[")) return "L";

 2. In the `bs`, `hr`, and `sr` locales, the strings `Lj` and `lj`
    are treated as a single letter, *distinct from* `L` or `l`.
    In those locales, this expression is *false*, not true:
    
        "Ljava/lang/String;".StartsWith ("L") // false in bs, hr, sr; true everywhere else

Additionally, this issue only arises when Java package names
starting with `java` are used, e.g. `Ljava/lang/Object;`.  Java types
from packages that *don't* start with `java` don't encounter this bug.

Fix this issue by enabling the [CA1307][1] and [CA1309][2],
previously disabled in commit ac914ce36.  These code analysis rules
require the use of string methods which use the
[`StringComparison`][3] enumeration, for which we then specify
`StringComparison.Ordinal`, which is *not* culture-sensitive.

One complication with enabling these rules is that the .NET 6+
version of these rules are stricter and require overloads that do not
exist in .NET Framework or .NET Standard to fix the violations.
Enabling these rules in `.editorconfig` affects all
`$(TargetFrameworkMoniker)`s; we will instead use
`Directory.Build.props` to only enable them for non-.NET 6+ builds.

[0]: https://docs.microsoft.com/en-us/dotnet/api/system.string.startswith?view=net-5.0#System_String_StartsWith_System_String_
[1]: https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1307
[2]: https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1309
[3]: https://docs.microsoft.com/en-us/dotnet/api/system.stringcomparison?view=net-5.0

@jpobst jpobst force-pushed the globalization-warnings branch 5 times, most recently from 7b38e3b to b07d2a8 Compare September 9, 2021 18:14
@jpobst jpobst force-pushed the globalization-warnings branch from b07d2a8 to d9ded1d Compare September 9, 2021 18:24
@jonpryor jonpryor merged commit 7068f4b into main Sep 9, 2021
@jonpryor jonpryor deleted the globalization-warnings branch September 9, 2021 19:11
jpobst added a commit that referenced this pull request Sep 30, 2021
Context: dotnet/android#6271
Context: https://developercommunity.visualstudio.com/t/XamarinAndroid-binding-compiling-proble/1477963
Context: https://www.learncroatian.eu/blog/the-croatian-letters

If you attempt to build an Android Binding project on Windows within
a Bosnian, Croatian, or Serbian locale, the build may break as the
delegate type names created by 56955d9 may not be valid C#, e.g.:

	delegate IntPtr _JniMarshal_PP_Ljava/Lang/String; (IntPtr jnienv, IntPtr klass)

It *should* be declaringn a `_JniMarshal_PP_L` type, *not*
`_JniMarshal_PP_Ljava/Lang/String;`, the latter of which results in
numerous C# errors:

	error CS1003: Syntax error, '(' expected
	error CS1001: Identifier expected
	error CS1001: Identifier expected
	error CS1003: Syntax error, ',' expected
	error CS1003: Syntax error, ',' expected
	error CS1001: Identifier expected
	error CS1026: ) expected

The problem is caused by the interplay of two factors:

 1. Commit 56955d9 uses the culture-sensitive
    [`string.StartsWith(string)`][0] method to determine if a JNI
    type name starts with `L`:

        if (jni_name.StartsWith ("L") || jni_name.StartsWith ("["))
            return "L";

 2. In the `bs`, `hr`, and `sr` locales, the strings `Lj` and `lj`
    are treated as a single letter, *distinct from* `L` or `l`.
    In those locales, this expression is *false*, not true:
    
        "Ljava/lang/String;".StartsWith ("L") // false in bs, hr, sr; true everywhere else

Additionally, this issue only arises when Java package names
starting with `java` are used, e.g. `Ljava/lang/Object;`.  Java types
from packages that *don't* start with `java` don't encounter this bug.

Fix this issue by enabling the [CA1307][1] and [CA1309][2] rules,
previously disabled in commit ac914ce.  These code analysis rules
require the use of string methods which use the
[`StringComparison`][3] enumeration, for which we then specify
`StringComparison.Ordinal`, which is *not* culture-sensitive.

One complication with enabling these rules is that the .NET 6+
version of these rules are stricter and require overloads that do not
exist in .NET Framework or .NET Standard to fix the violations.
Enabling these rules in `.editorconfig` affects all
`$(TargetFrameworkMoniker)`s; we will instead use
`Directory.Build.props` to only enable them for non-.NET 6+ builds.

Finally, add a new `.yaml` step that shuts down the cached `dotnet`
MSBuild processes between invocations, to fix the error that has been
happening on Windows - NET Core:

	error MSB3027: Could not copy "obj\\Release\Java.Interop.BootstrapTasks.dll" to "D:\a\1\s\bin\BuildRelease\Java.Interop.BootstrapTasks.dll". Exceeded retry count of 10. Failed.  [D:\a\1\s\build-tools\Java.Interop.BootstrapTasks\Java.Interop.BootstrapTasks.csproj]
	error MSB3021: Unable to copy file "obj\\Release\Java.Interop.BootstrapTasks.dll" to "D:\a\1\s\bin\BuildRelease\Java.Interop.BootstrapTasks.dll". The process cannot access the file 'D:\a\1\s\bin\BuildRelease\Java.Interop.BootstrapTasks.dll' because it is being used by another process. [D:\a\1\s\build-tools\Java.Interop.BootstrapTasks\Java.Interop.BootstrapTasks.csproj]

[0]: https://docs.microsoft.com/en-us/dotnet/api/system.string.startswith?view=net-5.0#System_String_StartsWith_System_String_
[1]: https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1307
[2]: https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1309
[3]: https://docs.microsoft.com/en-us/dotnet/api/system.stringcomparison?view=net-5.0
@github-actions github-actions bot locked and limited conversation to collaborators Apr 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants