Don't fail scatter-gather when remote instances omit required members (e.g. EndpointDetails.Host)

[speedmind]

Summary

Since 6.20.0, the primary instance can silently drop an entire audit instance's results from scatter-gather queries (for example messages2, used by ServicePulse's "All messages" view and its endpoint filter). The response then carries X-Particular-Incomplete-Results: <instance>:error, and the primary logs Failed to query remote instance ... missing required properties including: 'host'.

This may be what's behind Particular/ServicePulse#3102.

Root cause

• Enable nullable annotations in ServiceControl.Persistence #5774 (6.20.0) added nullable annotations and marked EndpointDetails.Name and EndpointDetails.Host as C# required.
• System.Text.Json enforces required members on deserialization.
• Audit instances serialize their API responses with DefaultIgnoreCondition = WhenWritingNull. The endpoint host is often null, for example when the NServiceBus.OriginatingMachine header is missing and the originating address has no @machine part, which is common on non-MSMQ transports. In that case host is omitted from the JSON.
• When the primary deserializes that remote response (ScatterGatherApi), one such message throws. The catch then treats the whole instance as failed, so all of its results for that page are discarded.

On 6.19.x these properties weren't required, so the same payload deserialized fine.

Fix

SerializerOptions.Default (primary instance) now adds a JsonTypeInfo modifier that sets IsRequired = false on every property when reading JSON. C# required still enforces correct construction in our own code. JSON that comes from other instances, possibly running other versions, is read the same way as before 6.20.

Tests

• Added RemoteEndpointDetailsDeserializationTests. It deserializes an EndpointDetails without host, and runs a scatter-gather against a remote audit response that omits host. Both tests fail on master without the fix: the first throws on the missing required property, and the second reports the remote instance as failed. Both pass with the fix.
• Full ServiceControl.UnitTests suite: 453/453 passing locally. I haven't run the acceptance tests locally.

The change applies cleanly to release-6.21 if a patch release is wanted.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}