DefaultExceptionSerializationStrategy Throw System.NullReferenceException

[czd890]

Occasionally throw exceptions:
AWSXRayRecorder.Handlers.AspNetCore : 2.6.0

Microsoft.AspNetCore.Server.Kestrel ERROR 58  Connection id "0HLOVUBU82R61", Request id "0HLOVUBU82R61:00000048": An unhandled exception was thrown by the application. 
System.NullReferenceException: Object reference not set to an instance of an object.
   at Amazon.XRay.Recorder.Core.Strategies.DefaultExceptionSerializationStrategy.<>c__DisplayClass14_0.<DescribeException>b__3(ExceptionDescriptor d)
   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Func`2 predicate, Boolean& found)
   at Amazon.XRay.Recorder.Core.Strategies.DefaultExceptionSerializationStrategy.DescribeException(Exception e, IEnumerable`1 subsegments)
   at Amazon.XRay.Recorder.Core.Internal.Entities.Entity.AddException(Exception e)
   at Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl.AddException(Exception ex)
   at Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

[yogiraj07]

Hi @czd890 ,
As per the stack trace above the null reference exception is generated at this point of the code. However can you provide detail exception stack trace? If this is the exception being logged, a small repro example would be helpful. We haven't had this issue reported previously so would to need dive deep into this issue.

A basic idea of your solution stack, and when this is triggered will be helpful. Is it when you have specific behavior in the application workflow during which causes above exception (since its occasional)?

My initial guess is ASP.NET Core X-Ray middleware is recording an exception which causes above unexpected behavior.

Thanks,
Yogi

[czd890]

Local debugging cannot retry this problem.

Mybe d is null on existingExceptionDescriptors.FirstOrDefault(d => d.Exception.Equals(e));

aws-xray-sdk-dotnet/sdk/src/Core/Strategies/DefaultExceptionSerializationStrategy.cs

Line 192 in bdb7471

ExceptionDescriptor innerExceptionDescriptor = existingExceptionDescriptors != null ? existingExceptionDescriptors.FirstOrDefault(d => d.Exception.Equals(e)) : null;

or e is null on FirstOrDefault(descriptor => e.Equals(descriptor.Exception))

aws-xray-sdk-dotnet/sdk/src/Core/Strategies/DefaultExceptionSerializationStrategy.cs

Line 150 in bdb7471

existingDescriptor = existingExceptionDescriptors.FirstOrDefault(descriptor => e.Equals(descriptor.Exception));

[czd890]

Tracked to where the error was thrown.

// Change 
ExceptionDescriptor innerExceptionDescriptor = existingExceptionDescriptors != null ? existingExceptionDescriptors.FirstOrDefault(d => d.Exception.Equals(e)) : null;

// to

ExceptionDescriptor innerExceptionDescriptor = existingExceptionDescriptors != null ? existingExceptionDescriptors.FirstOrDefault(d =>
{
    if (d.Exception == null)
    {
       //add log to local file
      // tracked 'd.Exception' is null.
    }
    return d.Exception.Equals(e);
}) : null;

AWSXRayRecorder.Instance.ExceptionSerializationStrategy = new MyExceptionSerializationStrategy();

Usage custom ExceptionSerializationStrategy, The code Copy from DefaultExceptionSerializationStrategy.

[yogiraj07]

Hi @czd890,
Thank you for detail analysis. We welcome to post a PR and happy to review it.

Best,
Yogi

[czd890]

I think this should be a matter worthy of further investigation. What can I help you?

[yogiraj07]

I think, while processing InnerException, you are probably hitting this code. This reassigns curDescriptor instance. I think the null check in the above comment is valid and also we should reorder this code after reassigning the curDescriptor. Let me know what you think.

Best,
Yogi

[czd890]

Seen from DescribeException method, d.Exception should not have a null case.

[srprash]

Seen from DescribeException method, d.Exception should not have a null case.

Hi @czd890 , I did not fully understand this. Also, would it be possible for you to provide a sample code that reproduces this issue. It would be very helpful.

Thanks,
Prashant

[czd890]

Hi @srprash , Maybe related to no async task or context loss.

[bhautikpip]

Hi @czd890,

I was looking through the thread. I found it difficult to catch up. Would you mind explaining about this issue in detail? And also repro steps of this issue would be helpful.

[srprash]

Closing this one due to inactivity. Please reopen if you still face the issue. Thanks!

[DamirAinullin]

Sporadically get similar exception in AWS Lambda:

{
            "Depth": 0,
            "ClassName": "System.NullReferenceException",
            "Message": "Object reference not set to an instance of an object.",
            "Source": "AWSXRayRecorder.Core",
            "StackTrace": [
                "   at Amazon.XRay.Recorder.Core.Strategies.DefaultExceptionSerializationStrategy.<>c__DisplayClass14_0.<DescribeException>b__3(ExceptionDescriptor d)",
                "   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Func`2 predicate, Boolean& found)",
                "   at Amazon.XRay.Recorder.Core.Strategies.DefaultExceptionSerializationStrategy.DescribeException(Exception e, IEnumerable`1 subsegments)",
                "   at Amazon.XRay.Recorder.Core.Internal.Entities.Entity.AddException(Exception e)",
                "   at Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl.AddException(Exception ex)",
                "   at OS33.Serverless.Core.AwsLambda`2.LambdaHandler(TData data, ILambdaContext context)"
            ],
            "HResult": -2147467261,
            "HelpURL": null
        }

Lambda is called by Cloufwatch Event by timer every five minutes. Can you help fix this bug? Any workarounds?