Fail to deserialize events containing exceptions

[lesscodetxm]

I've been doing most of my actor testing using EventStore, but on one test server we have to use SqlServer for event persistence. On events that contain exceptions we get:

[14:46:36 ERR] Persistence failure when replaying events for persistenceId [...]. Last known sequence number [3]
System.InvalidCastException: Unable to cast object of type 'Newtonsoft.Json.Linq.JObject' to type 'Newtonsoft.Json.Linq.JValue'.
   at Newtonsoft.Json.Serialization.JsonFormatterConverter.GetTokenValue[T](Object value) in /_/Src/Newtonsoft.Json/Serialization/JsonFormatterConverter.cs:line 55
   at System.Runtime.Serialization.SerializationInfo.GetInt32(String name)
   at System.Exception..ctor(SerializationInfo info, StreamingContext context)
   at lambda_method(Closure , Object[] )
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateISerializable(JsonReader reader, JsonISerializableContract contract, JsonProperty member, String id) in /_/Src/Newtonsoft.Json/Serialization/JsonSerializerInternalReader.cs:line 1756
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) in /_/Src/Newtonsoft.Json/Serialization/JsonSerializerInternalReader.cs:line 573
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType) in /_/Src/Newtonsoft.Json/Serialization/JsonSerializerInternalReader.cs:line 2203
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id) in /_/Src/Newtonsoft.Json/Serialization/JsonSerializerInternalReader.cs:line 1943
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) in /_/Src/Newtonsoft.Json/Serialization/JsonSerializerInternalReader.cs:line 476
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) in /_/Src/Newtonsoft.Json/Serialization/JsonSerializerInternalReader.cs:line 167
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) in /_/Src/Newtonsoft.Json/JsonSerializer.cs:line 907
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings) in /_/Src/Newtonsoft.Json/JsonConvert.cs:line 828
   at Akka.Serialization.NewtonSoftJsonSerializer.FromBinary(Byte[] bytes, Type type)
   at Akka.Serialization.Serialization.Deserialize(Byte[] bytes, Int32 serializerId, String manifest)
   at Akka.Persistence.Sql.Common.Journal.AbstractQueryExecutor.ReadEvent(DbDataReader reader)
   at Akka.Persistence.Sql.Common.Journal.AbstractQueryExecutor.SelectByPersistenceIdAsync(DbConnection connection, CancellationToken cancellationToken, String persistenceId, Int64 fromSequenceNr, Int64 toSequenceNr, Int64 max, Action`1 callback)
   at Akka.Persistence.Sql.Common.Journal.SqlJournal.ReplayMessagesAsync(IActorContext context, String persistenceId, Int64 fromSequenceNr, Int64 toSequenceNr, Int64 max, Action`1 recoveryCallback) 

With EventStore we've configured an adapter that can properly deserialize derived types (by configuring the Json serializer to write the type information). I assume we need to do something similar for Sql Server but I'm unsure how to configure it.

Any tips? Or is this actually a bug?

[Aaronontheweb]

@lesscodetxm this is a standard issue with exception serialization due to how Exception itself stores its state - we're working on some ideas around a generalized solution for Akka.Remote here: akkadotnet/akka.net#3903 - what we do for remoting might also work for persistence, but we'll see.

In the short run, might be better to copy an Exception into a more serialization-friendly object containing the stack trace et al