Contaminated data cleanup

When the same person uses your product under different identities (for example anonymous and logged-in, or multiple devices), they can be assigned to different experiment variants. That contamination skews confidence intervals and conversion attribution: one person is counted as several, and their behavior is split across variants. For background on why this matters, see the blog post Your A/B Tests Are Lying to You (and You Don't Even Know It).

Hyppot addresses this by excluding contaminated data from experiment statistics rather than re-assigning or locking variants per person.

How Hyppot handles contamination

Hyppot does not persist per-user variant assignments or re-pin a person to a variant after identity changes. Variant resolution remains a deterministic function of experiment ID and user ID (the same (experiment, userId) pair always resolves to the same variant while the experiment definition is unchanged).

Instead, Hyppot detects when a person was exposed to more than one variant of the same experiment, marks those impressions as contaminated, and excludes them from statistics. Conversion and metric calculations use only non-contaminated impressions.

What counts as contamination

An impression is contaminated when a person saw more than one variant of the same experiment. Hyppot detects two cases:

1. Same user ID, multiple variants — a single user ID has impressions for more than one variant of the same experiment (for example due to an experiment definition change, or the same ID being reused across people on a shared device).
2. Aliased identities, multiple variants — two or more linked user IDs (an alias group) together have impressions for more than one variant of the same experiment (for example anonymous ID saw Control and logged-in ID saw Variant B).

When contamination is detected, all impressions for that person (or alias group) in that experiment are marked contaminated and excluded. If the situation later resolves to a single variant, the contamination flag is cleared.

How detection works

Contamination marking is eventually consistent — there can be a short delay before impressions appear as excluded in statistics.

Hyppot evaluates contamination at three points:

1. At alias time — when you call AliasUsersAsync, Hyppot immediately checks whether any linked identities saw different variants and flags impressions accordingly.
2. Background job — a hosted service runs every 10 minutes, re-evaluating recent impressions for late-arriving data and for same-user (non-aliased) multi-variant exposure.
3. At statistics time — when building experiment statistics, Hyppot also detects alias groups that span multiple variants as a safeguard.

User aliasing

User aliasing links multiple user IDs (for example "anon-123" and "user-456") as belonging to the same person. Once linked, Hyppot treats them as one identity for contamination detection: if any of the aliased IDs saw different variants of the same experiment, all their impressions for that experiment are excluded from statistics.

Alias users when you know two identities belong to the same person — typically at login, when an anonymous session ID can be linked to an authenticated account ID.

REST API

User aliasing uses the backend API with your API key (X-API-KEY header). The default path prefix is hyppot; see Configuration if yours differs.

Create or extend an alias group

POST {pathPrefix}/userAlias

Request body (JSON):

{
  "userIds": ["user-1", "user-2", "user-3"],
  "keepAtLeastUntil": "2025-12-31T23:59:59Z"
}

• userIds — list of user IDs to link. At least two IDs are required; fewer than two is a no-op. Maximum 2000 IDs per request.
• keepAtLeastUntil — optional; the earliest date until which the alias should be kept. If omitted, Hyppot uses now + UserAliasRetentionPeriod from configuration.

All listed users end up in the same alias group. If some were already in other groups, those groups are merged.

Get aliases for a user

GET {pathPrefix}/userAlias/{userId}

Returns a JSON array of user IDs aliased with the given user (excluding the user itself). Returns an empty array if the user has no aliases.

There is no public API for removing aliases.

.NET

When using in-process Hyppot (Hyppot.AspNetCore), inject IUserAliasService and call it directly. When using Hyppot.Sdk, the same interface is implemented by the SDK and calls the REST API above.

public class MyService
{
    private readonly IUserAliasService _userAliasService;

    public MyService(IUserAliasService userAliasService)
    {
        _userAliasService = userAliasService;
    }

    public async Task LinkUserIdentities(string anonId, string loggedInId)
    {
        await _userAliasService.AliasUsersAsync(
            new[] { new UserId(anonId), new UserId(loggedInId) });
    }

    public async Task LinkUserIdentitiesUntil(string anonId, string loggedInId, DateTime keepUntil)
    {
        await _userAliasService.AliasUsersAsync(
            new[] { new UserId(anonId), new UserId(loggedInId) },
            keepAtLeastUntil: keepUntil);
    }

    public async Task<IList<UserId>> GetLinkedUsers(string userId)
    {
        return await _userAliasService.GetUserAliasesAsync(new UserId(userId));
    }
}

Use a stable UserId format (for example from your auth system) so the same alias group is used across requests.

JavaScript / Browser

The browser SDK does not expose user aliasing. Perform aliasing from your backend when you know two identities belong to the same person (for example after login): call the user alias API or use the .NET SDK's IUserAliasService from your server.

Retention

Each alias row stores a KeepAtLeastUntil timestamp:

• For new aliases: keepAtLeastUntil from the request, or now + UserAliasRetentionPeriod if omitted.
• For existing aliases in the request: Hyppot keeps the later of the current KeepAtLeastUntil and the requested/default value.

Configure the default retention via UserAliasRetentionPeriod (default: 30 days). Removal of aliases is not exposed via the public API.

Viewing excluded data in statistics

In the experiment Statistics tab:

• Conversion statistics show Contaminated users excluded: N — the number of contaminated alias groups whose impressions were left out because they (or another identity in their alias group) saw more than one variant of the experiment.
• Metric statistics report a separate contaminated count (contaminatedResultsCount) for metric aggregations.

Both conversion rates and metric aggregations use only non-contaminated impressions.

Best practices

• Alias early — link anonymous and authenticated IDs at login, before the user accumulates impressions under both identities.
• Monitor the excluded count — a high "Contaminated users excluded" number means a large share of your sample was dropped; improving identity linking reduces it.
• Set retention to cover the experiment — pass keepAtLeastUntil (or configure UserAliasRetentionPeriod) so aliases remain for the duration of your experiment.