Sanitization: Secrets in Logs

A session id in a log line, an API key in an exception message, a bearer token in a query string — each of these is a real leak, and each of them survives in a log aggregator far longer than the secret itself stays valid. The types in ActualLab.Compliance exist to fix that one problem: keep a secret out of logs and error messages without keeping it out of the wire.

That last part is what makes the approach cheap to adopt. SanitizedString<TSanitizer> is wire-compatible with string in every serialization format ActualLab supports, so changing a member's type from string to SanitizedString<T> is not a wire change and needs no migration.

Required Package

Package	Purpose
ActualLab.Core	The whole sanitization framework (ActualLab.Compliance)

SanitizedString<TSanitizer>

Pick a sanitizer, use it as the type argument, and the value masks itself whenever it's rendered inside a sanitization scope — which is exactly what SanitizingLogger opens around every log call:

[DataContract, MemoryPackable(GenerateType.VersionTolerant)]
public sealed partial record ApiCredentials(
    [property: DataMember(Order = 0), MemoryPackOrder(0), Key(0)] string UserId,
    [property: DataMember(Order = 1), MemoryPackOrder(1), Key(1)]
    [property: MemoryPackAllowSerialize] // MemoryPack-only: it can't see the registered formatter
    SanitizedString<Sanitizers.PrefixAndLengthHint> ApiKey
);

using (Sanitization.Begin()) // What SanitizingLogger opens around every log call
    WriteLine($"Authenticating {credentials.UserId} with {credentials.ApiKey}");
WriteLine(credentials.ApiKey.Value); // .Value is the raw one, and it's the only way to get it

Authenticating alice with <<sk* [16-31]>>
sk-7f3a91b8c2e4d6f0

The masking lives entirely in ToString(). Everything else about the type is deliberately transparent:

Member	Behavior
ToString()	The masked form while sanitization is active, the raw string otherwise
Value	The raw string, always
implicit operator SanitizedString<T>(string?)	Assign a string and it just works
explicit operator string(...)	Explicit on purpose, so the raw value can't slip into an interpolation by accident
Equals / GetHashCode / ==	Compare the raw values, so two equal secrets stay equal regardless of masking
IsEmpty, Empty	null and "" are the same thing, and no sanitizer masks an empty value

Equality on the raw value matters more than it looks: two different secrets can mask to identical text, and a lookup keyed on the masked form would collide.

var a = new SanitizedString<Sanitizers.LengthHint>("hunter2-hunter2");
var b = new SanitizedString<Sanitizers.LengthHint>("abcdefghijklmno"); // Same length, same mask
using (Sanitization.Begin())
    WriteLine($"{a} == {b}: {a == b}");

<<* [8-15]>> == <<* [8-15]>>: False

Wire Compatibility

Serialization always carries the raw value. Masking is a rendering concern, and a masked value on the wire would be silent data loss — the secret would be gone, not hidden.

Every format ships a converter that reads and writes the raw string, byte for byte identical to what a plain string member produces:

Format	Converter
System.Text.Json	SanitizedStringJsonConverter
Newtonsoft.Json	SanitizedStringNewtonsoftJsonConverter
MemoryPack	SanitizedStringMemoryPackFormatter
MessagePack	SanitizedStringMessagePackFormatter<>
Nerdbank.MessagePack	SanitizedStringNerdbankConverter<>
TypeConverter (config binding, ASP.NET model binding, …)	SanitizedStringTypeConverter

So the JSON for the record above still contains the key:

WriteLine(SystemJsonSerializer.Default.Write(credentials));

{"userId":"alice","apiKey":"sk-7f3a91b8c2e4d6f0"}

…and the binary payload is bit-identical to the one produced by the same record declared with a plain string:

[DataContract, MemoryPackable(GenerateType.VersionTolerant)]
public sealed partial record PlainCredentials(
    [property: DataMember(Order = 0), MemoryPackOrder(0), Key(0)] string UserId,
    [property: DataMember(Order = 1), MemoryPackOrder(1), Key(1)] string ApiKey
);

var plain = new PlainCredentials("alice", ApiKey);
foreach (var serializer in (IByteSerializer[])[MemoryPackByteSerializer.Default, MessagePackByteSerializer.Default]) {
    using var plainBytes = serializer.Write(plain);
    using var sanitizedBytes = serializer.Write(credentials);
    var isSame = plainBytes.WrittenSpan.SequenceEqual(sanitizedBytes.WrittenSpan);
    WriteLine($"{serializer.GetType().Name}: {isSame}");
}

MemoryPackByteSerializer: True
MessagePackByteSerializer: True

This protects logs, not sinks that serialize

SanitizedString<T> stops a secret from reaching a log line, an exception message, or a ToString()-based diagnostic. It does nothing about a sink that serializes — an RPC call, a database column, a JSON audit trail, an outbound webhook all receive the raw value. That's by design; if a value must not reach a particular sink, don't send it there.

The one place the substitution isn't free is MemoryPack: its source generator only accepts member types it can see a [MemoryPackable] annotation on, and SanitizedString<T>'s formatter is registered at runtime instead. Add [MemoryPackAllowSerialize] to the member (as in the snippet above) and MemoryPack defers to the registered formatter. Nothing about the payload changes.

The Sanitizers

Each sanitizer is named after what it leaves visible — that's the only thing distinguishing them.

Sanitizer	Leaves visible	sk-7f3a91b8c2e4d6f0 renders as
Sanitizers.Hidden	Nothing	<<hidden>>
Sanitizers.LengthHint	A length bucket	<<* [16-31]>>
Sanitizers.PrefixAndLengthHint	First 2 chars + a length bucket	<<sk* [16-31]>>
Sanitizers.Fingerprint	Identity, not content	<<2e513fef>>
Sanitizers.SessionString	A 4-char prefix + a hash, in Session.ToString()'s format	Ab3fSomeLongSessionId → Ab3f:297ccba4
Sanitizers.UriQuery	A query string, parameter by parameter, per its policy	see below
Sanitizers.RpcRequestQuery	Same, with ActualLab's RPC endpoint policy baked in	see below

WriteLine(Sanitizer.Sanitize<Sanitizers.Hidden>(ApiKey));
WriteLine(Sanitizer.Sanitize<Sanitizers.LengthHint>(ApiKey));
WriteLine(Sanitizer.Sanitize<Sanitizers.PrefixAndLengthHint>(ApiKey));
WriteLine(Sanitizer.Sanitize<Sanitizers.Fingerprint>(ApiKey));
WriteLine(Sanitizer.Sanitize<Sanitizers.SessionString>("Ab3fSomeLongSessionId"));
WriteLine(Sanitizer.Sanitize<Sanitizers.RpcRequestQuery>("?s=Ab3fSomeLongSessionId&p=Zm9vYmFy&f=mempack6"));

<<hidden>>
<<* [16-31]>>
<<sk* [16-31]>>
<<2e513fef>>
Ab3f:297ccba4
?s=Ab3f:297ccba4&p=<<Zm* [8-15]>>&f=mempack6

PrefixAndLengthHint falls back to LengthHint for values of 3 characters or fewer — a 2-character prefix of a 3-character secret isn't a hint, it's the secret.

null and "" pass through unchanged

null stays null: there's nothing to reveal, and a <<hidden>> standing in for an absent value would misreport what the object actually holds. The filtering happens once, in the non-virtual Sanitizer.Apply, so no sanitizer has to remember to do it. An empty string is passed through by every built-in policy for the same reason.

WriteLine(Sanitizer.Sanitize<Sanitizers.Hidden>(null) is null);
WriteLine($"'{Sanitizer.Sanitize<Sanitizers.Hidden>("")}'");

True
''

Length hints are buckets, not lengths

[16-31] is a power-of-two bucket, not a range someone chose. An exact length is itself a distinguisher — it narrows a candidate set, and for short structured values it can identify the value outright — so the exact length never leaks.

Fingerprint is the odd one out

Every other sanitizer throws information away. Fingerprint throws away the content but keeps the identity: equal values produce equal output.

WriteLine(Sanitizer.Sanitize<Sanitizers.Fingerprint>("[email protected]"));
WriteLine(Sanitizer.Sanitize<Sanitizers.Fingerprint>("[email protected]")); // Same input, same output
WriteLine(Sanitizer.Sanitize<Sanitizers.Fingerprint>("[email protected]"));

<<05125fe6>>
<<05125fe6>>
<<925de7fc>>

That's exactly what you want when you need to follow one value across a log — "the same client id appears in all three of these failures" is a question no other sanitizer can answer.

Fingerprint is not a redaction

It's a short, non-cryptographic hash of the value. Anyone holding a list of candidate values can hash them and match. Use it for high-entropy identifiers — session ids, client ids, tokens — and never for anything a reader could plausibly enumerate: email addresses, phone numbers, account numbers, postcodes.

Query strings

UriQuery sanitizes a URL query parameter by parameter: its selector maps a parameter name to the Sanitizer for that parameter's value, or to null to leave it alone. The leading ? is optional, and a valueless parameter is passed through.

RpcRequestQuery is UriQuery with the policy for ActualLab's own RPC endpoints, and that policy is deny by default — a parameter is readable only if it's explicitly listed, so a query parameter added later can't start leaking a credential just because no one remembered to mask it:

Parameter	Rendered with
RpcRequestQuery.OpenParameterNames — f, serializationFormat, c	Nothing: logged verbatim
RpcRequestQuery.SessionParameterNames — s, session	SessionString
Anything else — including p, the reconnect proof	PrefixAndLengthHint

Both name lists are settable static properties, so an app can extend the policy.

Turning Sanitization On

Sanitization is inactive unless a scope turns it on, and Sanitization is the ambient switch:

Member	Meaning
Sanitization.IsActive	Whether masking is currently on — false by default
Sanitization.Begin()	A scope in which values render masked
Sanitization.Suspend()	A scope that turns masking back off — nestable inside Begin()
Sanitization.Scope	The readonly struct both return; disposing it restores the previous state

WriteLine(credentials.ApiKey.ToString()); // Inactive by default
using (Sanitization.Begin()) {
    WriteLine(credentials.ApiKey.ToString());
    using (Sanitization.Suspend())
        WriteLine(credentials.ApiKey.ToString());
}

sk-7f3a91b8c2e4d6f0
<<sk* [16-31]>>
sk-7f3a91b8c2e4d6f0

Why off by default

A masking member is usually written as get => Sanitizer.MaybeSanitize<T>(field) — and a serializer reads that same getter. Were masking on by default, the masked form would be what lands on the wire and in the database: silent data loss no compiler can catch. So masking is off until something asks for it, and the thing that asks is SanitizingLogger, which opens a Begin() scope around each log call — on the logging thread only.

A scope does not flow across await

The flag is [ThreadStatic], not AsyncLocal, because it's read on every ToString() of every sanitized value and an AsyncLocal read is far too expensive for that path. The consequence: don't wrap awaited work in a Begin() scope — the continuation may resume on another thread, where the scope was never set. Keep the scope around the synchronous rendering itself.

SanitizingLogger

SanitizingLogger wraps an ILogger and opens a Sanitization.Begin() scope around every Log(...) call, so ISanitized values mask themselves in the log and nowhere else. SanitizingLoggerFactory applies that wrapper to every logger a factory creates — cheaper than wrapping each ILoggerProvider, and it covers providers registered later.

Registration	Use it when
logging.AddSanitizingLoggerFactory()	Masking is always on
logging.AddSanitizingLoggerFactory(bool mustSanitize)	The decision is already made — e.g. from configuration
logging.AddSanitizingLoggerFactory(Func<IServiceProvider, bool> mustSanitizeResolver)	The decision needs DI — e.g. "only on production instances"
factory.Sanitizing(bool mustSanitize = true)	There's no ILoggingBuilder to hook, e.g. when assigning StaticLog.Factory

var services = new ServiceCollection();
services.AddLogging(logging => logging.AddSanitizingLoggerFactory(
    c => c.GetRequiredService<IHostEnvironment>().IsProduction()));

// No ILoggingBuilder to hook? Wrap the factory itself
StaticLog.Factory = StaticLog.Factory.Sanitizing();

Sanitizing() skips re-wrapping a factory that's already a SanitizingLoggerFactory, so calling it twice costs nothing; passing false leaves the factory exactly as it was.

Sanitize lazily, or not at all

The scope only covers what is rendered inside the log call. Log.LogInformation("{Query}", Sanitizer.MaybeSanitize<T>(query)) evaluates its argument first, outside the scope, so it logs the raw value. Pass something whose ToString() is deferred — a SanitizedString<T> or an ISanitized — or mask unconditionally with Sanitizer.Sanitize<T> when the value only ever reaches a log.

var deferred = new SanitizedString<Sanitizers.RpcRequestQuery>("?s=Ab3fSomeLongSessionId");
var eager = Sanitizer.MaybeSanitize<Sanitizers.RpcRequestQuery>("?s=Ab3fSomeLongSessionId");
using (Sanitization.Begin()) { // What SanitizingLogger opens around Log(...)
    WriteLine(deferred); // ToString() runs inside the scope
    WriteLine(eager); // Already a plain string by the time the scope opens
}

?s=Ab3f:297ccba4
?s=Ab3fSomeLongSessionId

Sanitizer: Masking Without Wrapping

Not every value wants to be a SanitizedString<T>. When a type already holds a plain string and only one computed member needs masking, use Sanitizer directly:

API	Applies the sanitizer…
sanitizer.Apply(value)	Always — except to null, which passes through
sanitizer.MaybeApply(value)	Only while Sanitization.IsActive
Sanitizer.Sanitize<T>(value)	Always — the static form of Apply
Sanitizer.MaybeSanitize<T>(value)	Only while Sanitization.IsActive
Sanitizer.Get<T>() / Sanitizer.Get(Type)	— returns the single shared instance

All four masking APIs are annotated [return: NotNullIfNotNull(nameof(value))], so a non-null argument gives a non-null result and the nullable analysis stays exact.

Get<T>() and Get(Type) hand out the same instance, so two call sites can't end up with different policy. Sanitizers must be stateless and thread-safe.

MaybeSanitize<T> is the one to reach for in a getter or a ToString() — it honors the ambient scope without any wrapper type:

public sealed class ConnectRequest(string path, string query)
{
    public string Path { get; } = path;
    public string Query { get; } = query;
    // Computed, never serialized - so masking right in the getter is safe
    public string LogTitle => Path + Sanitizer.MaybeSanitize<Sanitizers.RpcRequestQuery>(Query);
}

var request = new ConnectRequest("/rpc/ws", "?s=Ab3fSomeLongSessionId&f=mempack6");
WriteLine(request.LogTitle);
using (Sanitization.Begin())
    WriteLine(request.LogTitle); // MaybeSanitize honors the scope, Sanitize wouldn't

/rpc/ws?s=Ab3fSomeLongSessionId&f=mempack6
/rpc/ws?s=Ab3f:297ccba4&f=mempack6

Don't do this in a serialized member

The reason SanitizedString<T> exists at all is that the obvious shortcut is a data-loss bug:

[DataContract]
public sealed record BadCredentials
{
    private readonly string _apiKey = "";

    // The masked value is what gets serialized, stored, and sent - the secret is simply lost
    [DataMember(Order = 0)]
    public string ApiKey {
        get => Sanitizer.Sanitize<Sanitizers.Hidden>(_apiKey);
        init => _apiKey = value;
    }
}

var bad = new BadCredentials { ApiKey = ApiKey };
WriteLine(SystemJsonSerializer.Default.Write(bad));

{"apiKey":"\u003C\u003Chidden\u003E\u003E"}

That's {"apiKey":"<<hidden>>"} — System.Text.Json escapes < and > by default. The mask went on the wire and into the database; the key is gone. SanitizedString<T> makes this impossible — its converters read Value and never call ToString(). If a member is serialized, never apply a sanitizer in its getter.

Writing Your Own

Derive from Sanitizer and override protected string Sanitize(string value) — that's the single extension point. The public Apply is non-virtual and already filtered null out, so the override only ever sees a real string. Name the type after what it leaves visible, and keep it stateless — a single instance is shared process-wide. A sanitizer with a parameterless constructor can be a SanitizedString<> type argument:

public sealed class EmailDomain : Sanitizer
{
    // Null never reaches here - Sanitizer.Apply passes it through
    protected override string Sanitize(string value)
    {
        var atIndex = value.IndexOf('@', StringComparison.Ordinal);
        return atIndex < 0
            ? Sanitize<Sanitizers.Hidden>(value)
            : Sanitizers.HiddenValue + value[atIndex..];
    }
}

The protected override and the public static string? Sanitize<TSanitizer>(string?) helper share a name on purpose: inside a sanitizer, Sanitize<Sanitizers.Hidden>(value) reads as "sanitize this with that policy", and the two never collide because their signatures differ.

UriQuery carries its policy delegate as a constructor argument, so it has no parameterless constructor and can't be a SanitizedString<> type argument. Derive from it with a fixed policy — exactly what RpcRequestQuery does — and you get one that can:

public sealed class DownloadQuery() : Sanitizers.UriQuery(SelectSanitizer)
{
    private static Sanitizer? SelectSanitizer(string name)
        => string.Equals(name, "token", StringComparison.OrdinalIgnoreCase)
            ? Get<Sanitizers.Hidden>()
            : null;
}

WriteLine(Sanitizer.Sanitize<EmailDomain>("[email protected]"));
WriteLine(Sanitizer.Sanitize<DownloadQuery>("?id=17&token=Zm9vYmFy"));
using (Sanitization.Begin())
    WriteLine(new SanitizedString<EmailDomain>("[email protected]"));

<<hidden>>@example.com
?id=17&token=<<hidden>>
<<hidden>>@example.com

Tagging Interfaces

Two interfaces let infrastructure recognize sanitized values without knowing their sanitizer:

Interface	Meaning
ISanitized	This type's ToString() takes Sanitization.IsActive into account
ISanitizedString	ISanitized + Value and Sanitizer — the non-generic face of SanitizedString<T>, for serializers and log formatters

Session, User and SessionInfo implement ISanitized, so they mask themselves in every log line a SanitizingLogger produces.

Choosing a Sanitizer

If the value is…	Use
A secret you never need to recognize again	Hidden
A secret where "was it set? was it truncated?" matters	LengthHint
A prefixed credential (sk-…, ghp_…) whose kind is worth seeing	PrefixAndLengthHint
A high-entropy identifier you need to correlate across log records	Fingerprint
A Fusion session id	SessionString
A URL query string	UriQuery / RpcRequestQuery
Low-entropy PII (email, phone, account number)	Hidden — never Fingerprint