The Compliance Harness

On 30 April 2026, a refactoring commit during the framework’s own extraction out of its first host application began shipping empty Affidavits: every proposed write carried fields tagged ProvenanceSource.Empty and no real values. The entire test suite at the time — 330 of 330 tests — stayed green, because those tests asserted the shape of an Affidavit (field names, field counts, Fields.Length > 0 and the like) rather than the substance of its provenance (whether a field claiming a value actually swore to where that value came from). The regression surfaced only when a real user typed a real message and the review card came back blank.

The lesson the framework now enforces mechanically: a test suite can be 100% green and 0% truthful if it asserts shape, not meaning. Affiant.Testing.ComplianceHarness exists so that a host’s own inference strategies — the code that declares which fields an Affidavit carries and what the model is asked to fill them with, per Affidavits & Provenance — can’t fall into the same trap. It ships as a normal test-project package (Affiant.Testing.ComplianceHarness, whose Affiant dependencies as of 1.0.0-beta.3 are Affiant.Core, Affiant.Docket and Affiant.Policies, beside JsonSchema.Net and two Microsoft.Extensions.* packages; see Installation and Packages). Its entry point is ComplianceHarness.Verify(IServiceCollection). The class carries six public static members in all — Verify plus five checks. AssertProvenanceIsSubstantive is the one Verify runs itself, on every case it gets as far as projecting; the other four are parity assertions a host calls directly: AssertFieldSetParity, AssertToolNameRegistryParity, AssertFabricKeyParity and AssertToolErrorCodeRegistryParity.

What Verify checks

Verify takes the IServiceCollection your test project has assembled, builds a service provider from it, and does two things in sequence.

Discoverability. It reads every AffiantToolDescriptor registered in IAffiantToolRegistry — the registry AddAffiantTool<TStrategy>() populates, covered in Authoring Write Tools — and filters to the ones that describe a write operation (Operation.Kind of "WriteCreate" or "WriteUpdate") with a non-null InferenceStrategy. For each one, it checks whether an ITaskInferenceComplianceFixture is registered whose Strategy property matches. Any write strategy without a paired fixture becomes a MissingFixture.

Case execution and substance gating. For every fixture that is paired to a registered write strategy, Verify runs each of the fixture’s InferenceFixtureCase entries: it drives the strategy through the same inference path a real tool call takes — TaskInferenceRunner over TaskInferenceStep, though none of the filters that surround them in a live turn run here — projects the result with the framework’s own default projection, runs AssertProvenanceIsSubstantive against the resulting Affidavit, and then — independently of what that gate found — records whether the fixture’s own hand-written Assertion passed.

The result is a ComplianceVerificationResult:

public sealed record ComplianceVerificationResult(
    bool Passed,
    IReadOnlyList<MissingFixture> MissingFixtures,
    IReadOnlyList<FixtureFailure> FixtureFailures,
    IReadOnlyList<SubstanceFailure> SubstanceFailures);

Passed is true only when all three lists are empty, and the three are deliberately orthogonal:

• MissingFixtures — a write strategy with no paired fixture at all (discoverability).
• FixtureFailures — a fixture’s own Assertion returned false or threw, or the case couldn’t run (missing test double, unregistered strategy, an update-shaped tool whose case names no entity).
• SubstanceFailures — the harness’s own gate found a hollow Affidavit, regardless of what the fixture author chose to assert. This is the direct, executable guard against the empty-Affidavit regression class.

A fixture author who writes a narrow assertion — affidavit.Fields.Length > 0, say, which would have passed against an all-Empty Affidavit just as easily as a real one — does not get to silently skip the substance check. SubstanceFailures runs whether or not the fixture’s own Assertion would have caught the same problem.

Writing a fixture

A fixture implements ITaskInferenceComplianceFixture, from Affiant.Abstractions.Interfaces:

public interface ITaskInferenceComplianceFixture
{
    Type Strategy { get; }
    IEnumerable<InferenceFixtureCase> Cases { get; }
}

Strategy names the ITaskInferenceStrategy implementation this fixture verifies — it’s how Verify pairs a fixture to a write strategy’s registered AffiantToolDescriptor.InferenceStrategy. Cases is a sequence of InferenceFixtureCase records:

public sealed record InferenceFixtureCase(
    string Name,
    IReadOnlyList<AffiantChatMessage> History,
    IReadOnlyDictionary<string, object?> Arguments,
    Func<Affidavit, bool> Assertion,
    string? EntityId = null);

History is the conversation the strategy will infer against, as a list of the framework’s own backend-neutral AffiantChatMessage records — the record’s primary constructor is (string Role, string Content) and inference reads only those two; its five further optional strings (AuthorName, ModelId, ToolCallId, FunctionName, ArgumentsJson) exist so a tool-call turn survives a round trip through an IChatSessionStore. It is not Semantic Kernel’s ChatHistory, so a fixture compiles without an SK reference. Arguments are the arguments the case replays as the model’s own tool call: the harness passes them straight into the InferenceCompletionRequest, exactly as InferenceTriggerFilter does in a live turn. The harness runs no filters, though, so ToolArgumentCaptureFilter — which in a live turn also upserts the arguments that carry a value into the Context Fabric, as the proposed values of an EntityRef — does not run here. Assertion is your own predicate over the produced Affidavit. EntityId, added in 1.0.0-beta.3, names the entity a case targets when the write tool’s descriptor declares an update operation — an update-shaped Affidavit names the entity it updates, so the harness cannot project one without it; a create-shaped case leaves it null, which is the default. Continuing the LeaveTaskInferenceStrategy from Quickstart — the strategy behind the request_leave tool, whose six fields are Employee, StartDate, EndDate, LeaveType, Days and Reason (all but Days required) — a fixture for it looks like this:

using Affiant.Abstractions.Interfaces;
using Affiant.Abstractions.Models;

public sealed class LeaveComplianceFixture : ITaskInferenceComplianceFixture
{
    public Type Strategy => typeof(LeaveTaskInferenceStrategy);

    public IEnumerable<InferenceFixtureCase> Cases
    {
        get
        {
            AffiantChatMessage[] history =
            [
                new("user",
                    "I need annual leave from 2026-08-03 to 2026-08-07, I'll be at a family event."),
            ];

            yield return new InferenceFixtureCase(
                Name: "happy_path_annual_leave",
                History: history,
                Arguments: new Dictionary<string, object?>
                {
                    ["startDate"] = "2026-08-03",
                    ["endDate"] = "2026-08-07",
                    ["leaveType"] = "Annual",
                    ["reason"] = "Family event",
                },
                Assertion: affidavit =>
                    affidavit.Fields.Single(f => f.Name == "LeaveType").Value as string == "Annual");
        }
    }
}

The Assertion above checks one domain fact your own test cares about. It says nothing about provenance — that’s not a gap you need to fill by hand; it’s exactly what AssertProvenanceIsSubstantive checks independently, below.

Verify needs a way to turn History and Arguments into structured-output JSON without calling a real LLM. Register a test double for IInferenceCompletionPort (Affiant.Abstractions.Interfaces) that returns canned JsonElement output matching the shape TaskInferenceStep expects — an object keyed by field name, each with a value and a confidence:

using System.Text.Json;
using Affiant.Abstractions.Interfaces;
using Affiant.Abstractions.Models;

public sealed class RecordedInferencePort(string json) : IInferenceCompletionPort
{
    public Task<JsonElement> CompleteStructuredAsync(
        InferenceCompletionRequest request,
        CancellationToken cancellationToken = default)
        => Task.FromResult(JsonDocument.Parse(json).RootElement.Clone());
}

const string HappyPathJson = """
    {
        "StartDate":  { "value": "2026-08-03", "confidence": 0.95 },
        "EndDate":    { "value": "2026-08-07", "confidence": 0.95 },
        "LeaveType":  { "value": "Annual",     "confidence": 0.90 },
        "Reason":     { "value": "Family event", "confidence": 0.85 }
    }
    """;

Running Verify

Wire the strategy, the fixture, and the recorded port into an IServiceCollection, then call ComplianceHarness.Verify. The assertion is xunit’s; any test framework’s equivalent does as well:

using Affiant.Abstractions.Interfaces;
using Affiant.Abstractions.Models;
using Affiant.Core.Extensions;
using Affiant.Testing.ComplianceHarness;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

var services = new ServiceCollection()
    .AddAffiantCore()
    .AddSingleton<IInferenceCompletionPort>(new RecordedInferencePort(HappyPathJson))
    .AddAffiantTool<LeaveTaskInferenceStrategy>(
        functionName: "request_leave",
        operation: Operation.WriteCreate,
        entityType: "LeaveRequest")
    .AddSingleton<ITaskInferenceComplianceFixture>(new LeaveComplianceFixture());

var result = ComplianceHarness.Verify(services);

Assert.True(result.Passed,
    $"Compliance check failed.\n" +
    $"Missing fixtures: {string.Join(", ", result.MissingFixtures)}\n" +
    $"Fixture failures: {string.Join(", ", result.FixtureFailures)}\n" +
    $"Substance failures: {string.Join(", ", result.SubstanceFailures)}");

AddAffiantCore() is required — it registers IAffiantToolRegistry and IObservabilityEventStream<AffidavitEmittedEvent>, both of which Verify resolves directly. Worth knowing precisely what Verify does not need from the container: for each case, it constructs its own ContextFabric, TaskInferenceStep, and TaskInferenceRunner directly — one fresh, isolated ContextFabric per case, so cases never leak state into each other — rather than resolving them from DI. It then builds a SchemaDrivenAffidavitProjection from your strategy instance, whatever IFieldResolver, IDeterministicFieldSource and IPreviousValueSource implementations you’ve registered, and the same IObservabilityEventStream<AffidavitEmittedEvent> AddAffiantCore() provides, and projects the Affidavit through it — the framework’s own default projection path, described in Affidavits & Provenance. Note what follows from builds: Verify never resolves a host-registered IAffidavitProjection. If you are continuing from the quickstart, that host registers LeaveAffidavitProjection — the projection that gives an update-shaped write its entity id and each field’s previous value — and it is not the projection the harness measures; the harness’s own takes the entity id from the case’s EntityId and previous values from whatever IPreviousValueSource implementations the container holds. What it does need beyond AddAffiantCore(): your ITaskInferenceStrategy registered (AddAffiantTool<TStrategy>, which registers both the strategy and its AffiantToolDescriptor atomically), an IInferenceCompletionPort test double, and each ITaskInferenceComplianceFixture.

Discoverability has no lenient mode

The missing-fixture check is unconditional — there’s no flag on Verify to allowlist a strategy or soften the check for a subset of tools. Every AffiantToolDescriptor your registry holds for a WriteCreate/WriteUpdate operation with a non-null InferenceStrategy is checked, every time. Pairing is keyed on the strategy type rather than on the tool: register ten write tools backed by ten distinct strategies, pair fixtures to nine, and Verify reports one MissingFixture naming the tenth strategy’s type and function name — it does not pass by default and let you opt in to strictness later. Where one strategy backs several tools — the quickstart’s LeaveTaskInferenceStrategy backs both request_leave (create) and amend_leave (update) — one fixture pairs every descriptor that names it, and an unpaired shared strategy yields one MissingFixture per function name instead. A paired fixture’s cases then run against the first descriptor that names the fixture’s strategy — and AffiantToolRegistry.All hands the descriptors back as its ConcurrentDictionary’s values, in an order the registry does not define and which need not be the order they were registered in. Which of a shared strategy’s descriptors supplies the FunctionName and Operation.Kind the harness projects with is therefore unspecified: cases written for request_leave can just as well be run against amend_leave and its WriteUpdate, and a case that then sets no EntityId stops with the update-shaped FixtureFailure below instead of projecting at all — intermittently, run to run, rather than every time. Back each write tool with its own strategy. Where one already backs several — as the quickstart’s LeaveTaskInferenceStrategy does across request_leave (create) and amend_leave (update) — that restructuring is the remedy.

One boundary worth knowing: the check only considers descriptors that carry a non-null InferenceStrategy. AddAffiantTool<TStrategy>() always sets one, so the ordinary registration path is fully covered. A descriptor constructed and registered directly against IAffiantToolRegistry — bypassing AddAffiantTool<TStrategy> — with a null InferenceStrategy on a write operation is invisible to this check, since there is no strategy type to pair a fixture to in the first place. So is a descriptor whose Operation.Kind is neither "WriteCreate" nor "WriteUpdate" exactly: AddAffiantTool<TStrategy>() refuses only Operation.ReadQuery, so Operation.WriteDelete — or a host’s own verb, a bare "update" included, which Operation.IsUpdateShaped recognises everywhere else — registers happily and is never discovered here. Give a write tool one of the two spellings the check matches, and let your own verb travel beside the shape.

FixtureFailures: when a case can’t run, or your assertion says no

A FixtureFailure (StrategyType, FixtureCaseName, Reason) is your fixture’s own concern. Verify records one when:

• No IInferenceCompletionPort is registered at all — the case can’t run, and this is reported as a failure rather than thrown as an exception.
• The fixture’s Strategy type doesn’t resolve from the container as an ITaskInferenceStrategy — usually a missing or mistyped AddAffiantTool<TStrategy>() call.
• The descriptor driving the case declares an update-shaped operation — Operation.IsUpdateShaped(descriptor.Operation.Kind), which of the two kinds discoverability admits means "WriteUpdate" — and the case sets no EntityId. New in 1.0.0-beta.3; the reason names the tool, the operation kind and the missing InferenceFixtureCase.EntityId, and the case stops there, before projection.
• An exception escapes the rest of the case — the projection throwing, say. The reason names the exception’s type and message. Cancellation is the exception to this: an OperationCanceledException is rethrown and leaves Verify altogether.
• The case’s own Assertion throws.
• The case’s own Assertion runs cleanly and returns false.

One thing is conspicuously absent from that list: an exception raised inside your IInferenceCompletionPort double, or anywhere else inside inference. TaskInferenceRunner.RunAsync catches every non-cancellation exception — JsonException included — logs a warning, records an inference.failed event on the current activity, and returns an empty result. Nothing merges, so the case projects a hollow Affidavit, and what you get back is a SubstanceFailure — the per-fixture "(all cases)" one, if every case of the fixture is hollow — plus whatever your own Assertion makes of an Affidavit whose fields are all Empty. The exception’s type and message appear nowhere in the result, so a test double that throws reads like a strategy that infers nothing.

The rest are the failures a fixture author would expect from ordinary test-writing — a missing test double, a typo’d strategy registration, a case that doesn’t say which entity it updates, an assertion that doesn’t hold. None of them says anything about provenance quality; that’s the next section.

The substance gate: AssertProvenanceIsSubstantive

AssertProvenanceIsSubstantive(ITaskInferenceStrategy strategy, string fixtureCaseName, Affidavit affidavit) is public, static, and runs automatically inside Verify for every case that reaches projection — the first four failures above stop the case before the gate runs, while the two Assertion outcomes are recorded after it, so one case can carry a FixtureFailure and a SubstanceFailure at once — but it’s also usable standalone, so the identical check can be reused by other tooling without duplicating the logic. It returns a list of SubstanceFailure records:

public sealed record SubstanceFailure(
    Type StrategyType,
    string FixtureCaseName,
    string FieldName,
    string Reason);

FieldName names the specific field a check failed on; for a violation that isn’t scoped to one field, it carries a marker like "(affidavit)" instead. The method runs four checks, in order, against a single produced Affidavit:

1. Affidavit.Fields is non-empty. An empty Fields array is the empty-Affidavit regression in its most literal form — no sworn fields were produced at all. This check short-circuits: if it fails, the method returns immediately with a single SubstanceFailure and skips the per-field checks below, since there are no fields to check.

2. Every field carries a provenance chain. For each AffidavitField, if Provenance is null or Provenance.Current is null, that’s a failure — a field emitted with no provenance at all is indistinguishable from “the framework forgot to track it,” which is exactly what Rule 7 forbids.

3. A populated value must be sworn. If a field carries a real value (non-null, and — for strings — non-empty and non-whitespace) but its current ProvenanceTag.Source is ProvenanceSource.Empty, that’s the hollow signature itself: a field asserting a value while swearing nothing about where it came from. This is the check that would have caught the empty-Affidavit regression directly.

4. Required fields project to mandatory fields. If the strategy declared a TaskInferenceField with Required = true, the corresponding AffidavitField.IsMandatory must be true. A strategy that silently drops a field’s required status on the way to the Affidavit fails here.

The per-fixture invariant: prove the strategy can produce substance

Checks 2–4 above run per field, per case; check 1 is evaluated once per Affidavit, before there is a field to look at. Verify adds one more check at the fixture level: across all of a fixture’s cases that produced an Affidavit at all, at least one must be substantive — meaning at least one field’s current provenance source is not ProvenanceSource.Empty. A fixture whose every case yields an all-Empty Affidavit fails with a SubstanceFailure naming the fixture’s strategy type, "(all cases)" as the case name, and "(affidavit)" as the field name, even if every individual case passed checks 1–4 above (an all-Empty Affidavit with Fields.Length > 0 and every field carrying an explicit Empty tag is, technically, fully “tagged” — just never demonstrating that the strategy can produce a real value). This check is only raised when at least one case actually produced an Affidavit; a fixture whose every case already failed outright (recorded in FixtureFailures) doesn’t also get this failure piled on.

What the gate deliberately does not check

Two things the substance gate stops short of, on purpose. It does not require a Required = true field to actually carry a populated value — an empty mandatory field, correctly tagged ProvenanceSource.Empty, is a legitimate outcome the gate lets through; deciding whether that’s acceptable to approve is a review-time concern, not a projection-truthfulness concern the harness should adjudicate. Downstream, two things do happen to such a field, and neither is the harness’s business. A Standing Order cannot auto-approve over it: StandingOrderGuardrails.Apply degrades the verdict to ReviewerConfirmation with the blocked reason mandatory-field-empty whenever a mandatory field’s tag in force is Empty (an optional Empty field holds nothing back by rule). A human reviewer still may approve it, and whether they see the two halves separately is their own reviewer surface’s doing rather than the framework’s. EvidenceCardRequest carries the Affidavit itself — so every field’s IsMandatory flag travels with it, as isMandatory on the wire — along with the populated-confidence number, the empty-field count and the strategy’s per-field presentation hints; what it does not carry is a rendered required label, or any per-field mark that a field is empty. The quickstart sample’s vendored <affiant-evidence-card> element is where that rendering lives — it flags any field whose tag in force is Empty or whose confidence is 0, mandatory or not, and appends a required label of its own to a field whose isMandatory is true — and a host with its own reviewer surface writes both for itself. And it does not forbid ProvenanceSource.Empty outright across a case: a case that legitimately has no conversational basis to infer anything is allowed to yield an all-Empty Affidavit and pass the per-field checks cleanly — the “a strategy must produce substance somewhere” rule is enforced once, at the fixture level, not by banning Empty from any single case.

Why Verify runs the gate by default

There’s no parameter on Verify to disable AssertProvenanceIsSubstantive — it isn’t opt-in, and a fixture author cannot satisfy ComplianceHarness.Verify by writing a lenient Assertion and skipping the substance question. That’s the whole point: the regression that motivated this package passed all 330 of the suite’s tests cleanly, because none of them asked whether an Affidavit’s provenance was real. The gate exists precisely so that question gets asked automatically, on every case the harness gets as far as projecting, for every registered write strategy, whether or not the person writing the fixture thought to ask it.

The conformance driver: proving the shipped packages themselves are compliant

Verify and AssertProvenanceIsSubstantive prove something about your write strategies. As of 1.0.0-beta.3 — the conformance release — the same Affiant.Testing.ComplianceHarness package also ships ConformanceSuite, a different kind of check. It is a sibling of ComplianceHarness — both are static classes directly in the Affiant.Testing.ComplianceHarness namespace, so the type is ConformanceSuite, not ComplianceHarness.ConformanceSuite. What it proves: it proves the shipped .NET packages themselves satisfy the Affiant protocol’s own cross-implementation rulebook, using the same code the framework’s own release notes are derived from.

ConformanceSuite.Run(protocolRoot, writeRunTo) runs the rulebook’s declarative fixture suite and canonical byte vectors against whatever packages your project references, and returns every fixture’s outcome, the failing ids, and a run document. The rulebook itself is not fetched at run time — it’s vendored by the caller and pinned to a git tag, so “which rules does this build satisfy” always has an exact, checkable answer rather than depending on whatever the rulebook happened to say on the day the suite ran. This is the same driver tests/Affiant.Conformance.Tests runs in the framework’s own CI, reading against the rulebook’s v0.1.2 tag: all 63 pass on the 1.0.0-beta.3 candidate — 56 declarative fixtures and 7 canonical byte vectors — and the parity manifest committed beside the run log declares an empty failing list. The manifest also carries the eleven rules the rulebook’s own coverage lint exempts from a declarative fixture at v0.1 — two schema-level, three runtime, two telemetry-registry, three whose fixtures arrive with the rulebook’s first adapter at v0.2, and one exempt by construction — each with the reason and with whatever the .NET implementation checks in its place, where anything does.

Where AssertProvenanceIsSubstantive asks “did this write strategy produce real provenance,” ConformanceSuite asks “does this package, as shipped, actually behave the way the cross-implementation rulebook says an Affiant implementation must” — measured against the framework binaries a host is bound to, with each fixture’s gate built fresh from that fixture’s own given.gate and read from the host’s own vendored, pinned rulebook. It does not exercise a host’s own store, transport, executor, policies, authorization, or startup wire-up — most of that is AffiantWireUpValidator‘s concern at startup, and the host’s own compliance fixtures’, rather than this suite’s; a host’s own IWriteExecutor is checked by none of them. A host runs it from its own test project with the identical code the framework’s own conformance result comes from, rather than a re-implementation of the rulebook living beside somebody’s test.

Where this fits

Affidavits & Provenance covers the Affidavit, ProvenanceTag, and ProvenanceChain shapes this page checks against, and the seven-source determinism hierarchy that ProvenanceSource.Empty sits at the bottom of. The Seven Normative Rules states Rule 7 — the invariant the substance gate exists to enforce — in full. Authoring Write Tools covers [AffiantWriteTool], ITaskInferenceStrategy, and AddAffiantTool<TStrategy>() in depth; a fixture only makes sense once a write tool and its strategy exist. Packages has the full dependency graph, including where Affiant.Testing.ComplianceHarness sits relative to Affiant.Core.