System.Text.Json deserializing of object[bool] does not produce boolean

[radek7210]

I understand that the System.Text.Json is still in development. However, I would like to point out that the new deserializer produces very different results than the previous one.

Scenario:

public class MyRequest
{
  public string ParamName { get; set; }
  public object ParamValue { get; set; }
}

Controller method:

[HttpPost]
public ActionResult<string> Post([FromBody] MyRequest myRequest)
{
  return Content($"Type of '{myRequest.ParamName}' is {myRequest.ParamValue.GetType()}; value is {myRequest.ParamValue}");
}

Posting this json:

{
  "ParamName": "Bool param",
  "ParamValue": false
}

In .net core 2.1, the false value deserializes as boolean:
Type of 'Bool param' is System.Boolean; value is False

However, in .net core 3.0 preview 6, the false value deserializes as System.Text.Json.JsonElement:
Type of 'Bool param' is System.Text.Json.JsonElement; value is False

Will there be any chance to make the new deserializer work the same as in 2.1?

Note: we declare the ParamValue as object, as in the real app the values are of several different types, and so far the deserializer handled all them for us without issues. In 3.0, all this functionality is broken.

Thanks.

[steveharter]

Current functionality treats any object parameter as JsonElement when deserializing. The reason is that we don't know what CLR type to create, and decided as part of the design that the deserializer shouldn't "guess".

For example, a JSON string could be a DateTime but the deserializer doesn't attempt to inspect. For a "True" or "False" in JSON that is fairly unambiguous to deserialize to a Boolean, but we don't since we don't want to special case String or Number, we don't want to make an exception for True or False.

With the upcoming preview 7, it is possible to write a custom converter for object that changes that behavior. Here's a sample converter:

public class ObjectBoolConverter : JsonConverter<object>
{
    public override object Read(ref Utf8JsonReader reader, Type type, JsonSerializerOptions options)
    {
        if (reader.TokenType == JsonTokenType.True)
        {
            return true;
        }

        if (reader.TokenType == JsonTokenType.False)
        {
            return false;
        }

        // Forward to the JsonElement converter
        var converter = options.GetConverter(typeof(JsonElement)) as JsonConverter<JsonElement>;
        if (converter != null)
        {
            return converter.Read(ref reader, type, options);
        }

        throw new JsonException();

        // or for best performance, copy-paste the code from that converter:
        //using (JsonDocument document = JsonDocument.ParseValue(ref reader))
        //{
        //    return document.RootElement.Clone();
        //}
    }

    public override void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options)
    {
        throw new InvalidOperationException("Directly writing object not supported");
    }
}

Used like

var options = new JsonSerializerOptions();
options.Converters.Add(new ObjectConverter());

object boolObj = JsonSerializer.Parse<object>("true", options);
bool b = (bool)boolObj;
Debug.Assert(b == true);

object elemObj = JsonSerializer.Parse<object>(@"{}", options);
Debug.Assert(elemObj is JsonElement);

[radek7210]

sounds good, I'll use the custom converter. Thanks.

[John0King]

@steveharter Json and Javascript don't have many type, but boolean is known one.
System.Text.Json should map json known'd type to clr type directly without custom converter

json	clr
boolean	bool
number	double
string	string
null	null
undefined	null

[steveharter]

@John0King I provided the rationale previously. Automatically mapping number to double won't work for large decimals.

What are the scenarios for using system.object instead of the strong type (double, string, bool)?

[Mike-E-angelo]

FWIW, using an object/dynamic property for message deserialization is a security consideration and should be handled with care:

https://www.alphabot.com/security/blog/2017/net/How-to-configure-Json.NET-to-create-a-vulnerable-web-API.html

[John0King]

@steveharter
you are right, In many scenarios , handle JsonElement is much easier than an unknow/dynamic object type . it's just different than Json.Net that will break many people

[GSPP]

Maybe it's better to throw instead of deserializing as JsonElement. That way the decision can still be made in the future on how to deserialize to type object. Now, we're locked in to JsonElement.

I have personally wanted to deserialize to object a few times in the past with JSON.NET. I always wanted the "obvious" deserialization to bool etc.

If I want to deserialize to JsonElement then I can make the property a JsonElement.

[nick2893]

@steveharter
So, regarding dictionary<string,object>, what is this supposed to look like? The following throws an exception.

        if (reader.TokenType == System.Text.Json.JsonTokenType.StartObject) 
        { 
            // https://github.com/dotnet/corefx/issues/39953 
            // https://github.com/dotnet/corefx/issues/38713 
            var conv = options.GetConverter(typeof(Dictionary<string, object>)) as System.Text.Json.Serialization.JsonConverter<Dictionary<string, object>>; 
            if (conv != null) 
            { 
                return conv.Read(ref reader, type, options); 
            } 
            throw new System.Text.Json.JsonException(); 
        } 

Regarding the design choices;
What can be used to represent an Object other than dictionary?
What can be used to represent an Array other than array?

I can appreciate abstraction, but somebody has the make the final implementation decision. The current design completely ignores people who happened to stumble upon a Json file as a job requirement, and most likely need to use Json and System.Text.Json once and never again. For those folks, something that reasonably easily generates dictionarys and arrays and strings (Json files are one big string, right?) with little effort (such as System.Web.Script.Serialization.JavaScriptSerializer), and some (any?) reasonable default deserialization decisions/assumptions would make life so much easier.

[gmurray81]

I don't really like this design. You are making the most common use cases for a parser an impenetrable mess to serve the general correctness. To serve the 2% edge cases, you are making this yet another unusable .NET Json API for the 98% use cases. You have an options bucket, use it to OPT into the generally correct, but unhelpful behavior, not have it be the default.

[steveharter]

@gmurray81 can you explain your scenario for using System.Object (instead of just a bool) and what other values it may hold other than a bool?

Since JsonElement would still be used for non-bool types (e.g. objects, arrays, numbers) how would having a special case for bool make the code easier to consume?

[scalablecory]

In most cases when I've seen this, it has been an anti-pattern. object is rarely what people want, and have seen far too many model.Foo.ToString() from people who didn't realize they shouldn't be using object.

[scalablecory]

The JSON->POCO auto-generator in VS creates object members when it sees null, so this behavior breaks anyone migrating from such a thing.

But in most cases, this is again people just letting the default give them a bad model when they should be fixing what it gives them.

[John0King]

suggest to add a new strategy ：PreferRawObject:true to use default object mapping

json	clr
boolean	bool
number	double
string	string
null	null
undefined	null
Date(string with TimezoneInfo)	DateTimeOffset
Date(string without TimezoneInfo )	DateTime

[steveharter]

suggest to add a new strategy ：PreferRawObject:true to use default object mapping

Yes some global option is doable.

However, it is actually fairly easy to create a custom converter to handle the cases you show above. Earlier I provided the sample for bool and here's a sample for bool, double, string, DateTimeOffset and DateTime. I will get the sample added to

runtime/src/libraries/System.Text.Json/tests/Serialization/CustomConverterTests.Object.cs

Line 267 in 3e4a06c

private class SystemObjectNewtonsoftCompatibleConverter : JsonConverter<object>

private class SystemObjectNewtonsoftCompatibleConverter : JsonConverter<object>
{
    public override object Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        if (reader.TokenType == JsonTokenType.True)
        {
            return true;
        }

        if (reader.TokenType == JsonTokenType.False)
        {
            return false;
        }

        if (reader.TokenType == JsonTokenType.Number)
        {
            if (reader.TryGetInt64(out long l))
            {
                return l;
            }

            return reader.GetDouble();
        }

        if (reader.TokenType == JsonTokenType.String)
        {
            if (reader.TryGetDateTime(out DateTime datetime))
            {
                return datetime;
            }

            return reader.GetString();
        }

        // Use JsonElement as fallback.
        // Newtonsoft uses JArray or JObject.
        using (JsonDocument document = JsonDocument.ParseValue(ref reader))
        {
            return document.RootElement.Clone();
        }
    }

    public override void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options)
    {
        throw new InvalidOperationException("Should not get here.");
    }
}

[steveharter]

See the SystemObjectNewtonsoftCompatibleConverter sample class in

runtime/src/libraries/System.Text.Json/tests/Serialization/CustomConverterTests.Object.cs

Line 267 in 7eea339

private class SystemObjectNewtonsoftCompatibleConverter : JsonConverter<object>

for semantics similar to Json.NET (except for objects and arrays).