Experiments (beta)

Get detailed experiment result data for legacy experiments.

Reset all experiment results by deleting all existing data for an experiment.

Get details about all experiments in an environment.

Filtering experiments

LaunchDarkly supports the filter query param for filtering, with the following fields:

• flagKey filters for only experiments that use the flag with the given key.
• metricKey filters for only experiments that use the metric with the given key.
• status filters for only experiments with an iteration with the given status. An iteration can have the status not_started, running or stopped.

For example, filter=flagKey:my-flag,status:running,metricKey:page-load-ms filters for experiments for the given flag key and the given metric key which have a currently running iteration.

Expanding the experiments response

LaunchDarkly supports four fields for expanding the "Get experiments" response. By default, these fields are not included in the response.

To expand the response, append the expand query parameter and add a comma-separated list with any of the following fields:

• previousIterations includes all iterations prior to the current iteration. By default only the current iteration is included in the response.
• draftIteration includes the iteration which has not been started yet, if any.
• secondaryMetrics includes secondary metrics. By default only the primary metric is included in the response.
• treatments includes all treatment and parameter details. By default treatment data is not included in the response.

For example, expand=draftIteration,treatments includes the draftIteration and treatments fields in the response. If fields that you request with the expand query parameter are empty, they are not included in the response.

Create an experiment.

To run this experiment, you'll need to create an iteration and then update the experiment with the startIteration instruction.

To learn more, read Creating experiments.

Get details about an experiment.

Expanding the experiment response

LaunchDarkly supports four fields for expanding the "Get experiment" response. By default, these fields are not included in the response.

To expand the response, append the expand query parameter and add a comma-separated list with any of the following fields:

• previousIterations includes all iterations prior to the current iteration. By default only the current iteration is included in the response.
• draftIteration includes the iteration which has not been started yet, if any.
• secondaryMetrics includes secondary metrics. By default only the primary metric is included in the response.
• treatments includes all treatment and parameter details. By default treatment data is not included in the response.

For example, expand=draftIteration,treatments includes the draftIteration and treatments fields in the response. If fields that you request with the expand query parameter are empty, they are not included in the response.

Update an experiment. Updating an experiment uses the semantic patch format.

To make a semantic patch request, you must append domain-model=launchdarkly.semanticpatch to your Content-Type header. To learn more, read Updates using semantic patch.

Instructions

Semantic patch requests support the following kind instructions for updating experiments.

updateName

Updates the experiment name.

Parameters

• value: The new name.

Here's an example:

{
  "instructions": [{
    "kind": "updateName",
    "value": "Example updated experiment name"
  }]
}

updateDescription

Updates the experiment description.

Parameters

• value: The new description.

Here's an example:

{
  "instructions": [{
    "kind": "updateDescription",
    "value": "Example updated description"
  }]
}

startIteration

Starts a new iteration for this experiment. You must create a new iteration before calling this instruction.

An iteration may not be started until it meets the following criteria:

• Its associated flag is toggled on and is not archived
• Its randomizationUnit is set
• At least one of its treatments has a non-zero allocationPercent

Parameters

• changeJustification: The reason for starting a new iteration. Required when you call startIteration on an already running experiment, otherwise optional.

Here's an example:

{
  "instructions": [{
    "kind": "startIteration",
    "changeJustification": "It's time to start a new iteration"
  }]
}

stopIteration

Stops the current iteration for this experiment.

Parameters

• winningTreatmentId: The ID of the winning treatment. Treatment IDs are returned as part of the Get experiment response. They are the _id of each element in the treatments array.
• winningReason: The reason for the winner

Here's an example:

{
  "instructions": [{
    "kind": "stopIteration",
    "winningTreatmentId": "3a548ec2-72ac-4e59-8518-5c24f5609ccf",
    "winningReason": "Example reason to stop the iteration"
  }]
}

archiveExperiment

Archives this experiment. Archived experiments are hidden by default in the LaunchDarkly user interface. You cannot start new iterations for archived experiments.

Here's an example:

{
  "instructions": [{ "kind": "archiveExperiment" }]
}

restoreExperiment

Restores an archived experiment. After restoring an experiment, you can start new iterations for it again.

Here's an example:

{
  "instructions": [{ "kind": "restoreExperiment" }]
}

Create an experiment iteration.

Experiment iterations let you record experiments in individual blocks of time. Initially, iterations are created with a status of not_started and appear in the draftIteration field of an experiment. To start or stop an iteration, update the experiment with the startIteration or stopIteration instruction.

To learn more, read Starting experiment iterations.