Serializer errors for dictionaries that don't implement IDictionary

[WayneHiller]

Trying to serialize the results of this Dapper query.

var installations = (await Db.QueryAsync<object>( @"select InstallationID AS Value, Name AS Text
                                                   from [AIS].[Abs].Installation" )).ToList();
var inst = JsonSerializer.Serialize( installations );

The error I get is:

Message: 
    System.InvalidCastException : Unable to cast object of type '<GetEnumerator>d__9' to type 'System.Collections.IDictionaryEnumerator'.
  Stack Trace: 
    JsonSerializer.HandleDictionary(JsonClassInfo elementClassInfo, JsonSerializerOptions options, Utf8JsonWriter writer, WriteStack& state)
    JsonSerializer.Write(Utf8JsonWriter writer, Int32 originalWriterDepth, Int32 flushThreshold, JsonSerializerOptions options, WriteStack& state)
    JsonSerializer.WriteCore(Utf8JsonWriter writer, PooledByteBufferWriter output, Object value, Type type, JsonSerializerOptions options)
    JsonSerializer.WriteCore(PooledByteBufferWriter output, Object value, Type type, JsonSerializerOptions options)
    JsonSerializer.WriteCoreString(Object value, Type type, JsonSerializerOptions options)
    JsonSerializer.Serialize[TValue](TValue value, JsonSerializerOptions options)

Should this work? Works fine in Json.Net.

[ahsonkhan]

What is the type of installations? Is it List<object>?

What does the output from Json.Net look like when you serialize installations? Can you please provide a sample app that reproduces the problem that I can run? It will help with root-causing the issue.

Thanks!

[WayneHiller]

It looks like the Type of each object is Dapper.SqlMapper.DapperRow

https://github.com/StackExchange/Dapper/blob/master/Dapper/SqlMapper.DapperRow.cs

[ahsonkhan]

This looks like a bug in how we deal with types that implement IDictionary<TKey, TValue> where we expect such types to return an enumerator that implements IDictionaryEnumerator (and types like DapperRow don't implement that interface).

Therefore, we are failing at this cast, since the enumerator that DapperRow returns is IEnumerator<KeyValuePair<string, object>> and not IDictionaryEnumerator:
https://github.com/dotnet/corefx/blob/b832f6ccdebd18469e4b6d03c1c9b2c9c6c14374/src/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Write.HandleDictionary.cs#L64

System.Text.Json.Serialization.Tests.ValueTests.DapperTest [FAIL]
        System.InvalidCastException : Unable to cast object of type '<GetEnumerator>d__9' to type 'System.Collections.IDictionaryEnumerator'.
        Stack Trace:
          E:\GitHub\Fork\corefx\src\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Write.HandleDictionary.cs(64,0): at System.Text.Json.JsonSerializer.HandleDictionary(JsonClassInfo elementClassInfo, JsonSerializerOptions options, Utf8JsonWriter writer, WriteStack& state)
          E:\GitHub\Fork\corefx\src\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Write.cs(40,0): at System.Text.Json.JsonSerializer.Write(Utf8JsonWriter writer, Int32 originalWriterDepth, Int32 flushThreshold, JsonSerializerOptions options, WriteStack& state)
          E:\GitHub\Fork\corefx\src\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Write.Helpers.cs(131,0): at System.Text.Json.JsonSerializer.WriteCore(Utf8JsonWriter writer, Object value, Type type, JsonSerializerOptions options)
          E:\GitHub\Fork\corefx\src\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Write.Helpers.cs(107,0): at System.Text.Json.JsonSerializer.WriteCore(PooledByteBufferWriter output, Object value, Type type, JsonSerializerOptions options)
          E:\GitHub\Fork\corefx\src\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Write.Helpers.cs(86,0): at System.Text.Json.JsonSerializer.WriteCoreString(Object value, Type type, JsonSerializerOptions options)
          E:\GitHub\Fork\corefx\src\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Write.String.cs(21,0): at System.Text.Json.JsonSerializer.Serialize[TValue](TValue value, JsonSerializerOptions options)
          E:\GitHub\Fork\corefx\src\System.Text.Json\tests\Serialization\Value.WriteTests.cs(47,0): at System.Text.Json.Serialization.Tests.ValueTests.DapperTest()

It doesn't look like this issue exists for the built-in IDictionary<TKey, TValue> types that we support since their enumerators all implement IDictionaryEnumerator.

cc @layomia, @steveharter - can one of you take a look at this? Here's a repro of the issue:
https://github.com/ahsonkhan/corefx/blob/085ca53fd648694a1c6cd966c87032f8d5aed71b/src/System.Text.Json/tests/Serialization/Value.WriteTests.cs#L324-L332

[steveharter]

Dictionaries that don't implement IDictionary (and corresponding IDictionaryEnumerator GetEnumerator();) will throw. Ideally we should have thrown NotSupportedException in 3.0 instead of letting this get to InvalidCastException.

Due to performance reasons, for 3.1 we are considering not casting the enumerator to IDictionaryEnumerator (to avoid boxing). Doing that should address this issue as well.

[ahsonkhan]

If we decide to cast the enumerator, we should consider the following approach (rather than the hard cast we have atm):

1. Test if the enumerator type returned is an IEnumerator<KeyValuePair<string, object>>. If yes, cast and use that.
2. Else, test if the enumerator is an IDictionaryEnumerator. If yes, cast and use that.
3. Else, throw NSE.

[steveharter]

If we decide to cast the enumerator,

It's not really casting -- just calling GetEnumerator on one of the two interfaces:

• If the dictionary implements IDictionary<TKey, TValue> then GetEnumerator() returns IEnumerator<KeyValuePair<TKey, TValue>>
• If the dictionary implements IDictionary then GetEnumerator() returns IDictionaryEnumerator.
• If both interfaces are implemented, we'd likely want to use the enumerator from IDictionary<TKey, TValue> pending perf.

[ANahr]

We have also been hit by this problem as lots of internal types only implement IDictionary<TKey, TValue> but not IDictionary.

[steveharter]

Reopen for 3.1 port

[ahsonkhan]

@WayneHiller, @ANahr - can you please retry using the latest nightly build and see if your issue is fixed?
You can reference the latest System.Text.Json NuGet package from https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json

[WayneHiller]

The latest nightly build worked perfectly for me. :)