Alerts

Guardrail and Key Metric Alerts Now Shown in the Alerts Table

Previously, the Alerts table on the Monitoring tab displayed Metric alerts only. Guardrail Metric alerts and Key Metric alerts generated email notifications to feature flag owners, but were not persisted in the UI.

Now all three types of alerts are shown on each flag’s Monitoring tab for any team member to see. The table has been simplified to display the most valuable fields at a glance, reducing cognitive load. Details less critical for quick triage remain available under an info icon.

As a refresher, here is quick summary of the three alert types:

• Metric Alert
  • Triggered: when the relative or absolute impact threshold in an Alert policy is reached
  • Configured: at the metric level, by creating an Alert policy
  • Monitors: percentage rollouts for all flags which have the same traffic type as the metric
• Guardrail Metric Alert
  • Triggered: when any statistically significant impact is detected, either desirable or undesirable
  • Configured: at the metric level, by marking a metric’s category as “Guardrail”
  • Monitors: percentage rollouts for all flags which have the same traffic type as the metric
• Key Metric Alert
  • Triggered: when any statistically significant impact is detected, either desirable or undesirable
  • Configured: at the feature flag level, by marking a metric as a “Key metric” for that flag
  • Monitors: percentage rollouts only for flags where the metric is a Key metric

Better Together: Split + Harness

New Colors, Names for Organization and Workspace

Starting on September 30th, we began a progressive rollout to update the Split UI, bringing it closer to the look of Harness:

Beyond a change in color scheme, you will also see two changes to terminology:

• Workspaces will now be known as Projects in the UI
• Organizations will now be known as Accounts in the UI

Note: These terminology changes are being made only to labels in the UI at this time. To avoid introducing a breaking change, the Admin API will continue to use the strings ws, workspace, organizationId, and orgId until further notice.

Monitoring

Traffic Insights and Alerts

The Alerts tab has been renamed Monitoring and expanded to show real-time traffic insights over time and any alerts fired for the flag on a single page.

By default, traffic over the Last 48 hours is shown, but you may also select the Last 7 days, another Time range, or a specific Feature flag version:

Changes made to flags (i.e., new flag versions) are displayed as vertical bars for context:

For more information, have a look at the Monitoring tab docs.

Metrics

Recalculate Key Metrics Only

When requesting a manual recalculation of metrics, you can reduce the computation time by limiting the recalculation to Key metrics. To select this faster path to results, click the Recalculate button and then click Key metrics only. If you prefer to recalculate Key, Supporting, and Guardrail metrics, click All metrics instead.

Reminder: Key, Supporting, and Guardrail metrics are automatically calculated on a schedule, starting five minutes after a feature flag change and going to progressively longer intervals until the last automated calculation at 28 days. You do not need to press the Recalculate button unless you want to kick off a calculation before the next scheduled run or after the 28-day mark.

Learn more in our docs:

• Automated calculation frequency
• Manually recalculating metrics
  

Metrics

Introducing Supporting Metrics

We’re excited to announce the next step in improving metric categorization for your feature releases and experiments: Supporting metrics. This new metric category gives you greater control over metrics monitored per feature flag, helping you focus on what truly matters.

What’s changing?

On the Metrics impact page, we have replaced “Organizational metrics” with “Supporting metrics.” Now, you can easily select specific additional metrics to track for each release, reducing noise and making it easier to analyze your results.

How do I assign flag-specific metrics?

Key metrics and Supporting metrics are specific to individual feature flags, and can be managed on a flag’s Metrics impact tab. Under each category, click “Add metric” to initiate the process of assigning metrics to the category. Once metrics are added, you can wait until the next scheduled calculation or manually “Recalculate metrics” to see results.

How do I ensure my important metrics are still monitored for every flag?

In June, Split introduced Guardrail metrics to improve your ability to define which metrics should be monitored for every flag. Guardrail metrics include automated alerting, meaning flag owners will be notified as their releases or experiments impact these metrics. 

Since Organizational metrics will no longer be available, we recommend adding important metrics to the new Guardrail metrics category ASAP to ensure they continue to be protected and limit any disruption of analyzing metric results. Guardrail metrics can be assigned in the metric definition:

The combination of Key metrics, Guardrail metrics, and Supporting metrics will reduce noise and increase sensitivity while ensuring important metrics are monitored for every feature release or experiment. We welcome your feedback as we continue to improve our metric results!

Monitoring

Guardrail Metric Alerts

Flag owners will now automatically receive alerts on any guardrail metric without manual configuration.

Alerts are sent to flag owners whenever a guardrail metric moves significantly in the desired or undesired direction. This feature is designed to help you make safe and accurate release decisions in a timely manner.

As a reminder, Guardrail Metrics (released 2024-06-14) ensure that an organization’s most crucial metrics are protected and monitored throughout every feature release and experiment by making their calculation automatic and mandatory for all flags that use the same traffic type as the metric.

Metrics are set as guardrail metrics for your workspace on the Metric definition page. View the docs here

Monitoring

Guardrail Metrics

Users can now categorize their organization’s most important metrics as  “guardrails”. Guardrail metrics are those your organization wants to protect during a feature release or experiment. Metrics are set as guardrail metrics for your workspace on the Metric definition page.

SDK Enhancements

Semantic Versioning Targeting

Using the latest Split SDKs, users can more easily define targeting rules for new features based on app, OS, and other versions using attribute-based targeting. The SDK then automatically serves the appropriate treatment to users without needing additional code configurations. Split’s native Semantic Versioning Targeting removes the additional complexities and manual work that comes with targeting different application versions, allowing users to seamlessly deliver different experiences.

Usability Enhancements

Left Navigation Enhancements

Left-hand navigation within the application has been optimized. With this change, we’ve narrowed the navigation bar, migrated the search function to a modal dialog, and moved account settings to the bottom of the page. This makes frequently used actions (e.g. workspace switching) more readily accessible within the UI. 

SDK Enhancements

Split Suite, iOS SDK

Users can automatically capture event data in Split using their iOS SDK without needing additional agents, integrations, or track calls. This eliminates the manual process of sending events to Split, allowing users to quickly set up metrics and alert policies.

Monitoring

Out-of-the-Box Metrics

Split automatically creates metrics for any events being auto-captured by the Split Suite and RUM Agents (Web, Android, iOS). This reduces the manual effort of creating metrics and allows users to easily calculate their engineering and performance metrics.

Usability Enhancements

Switch Updates 

Users can now use Switch, Split’s in-app AI assistant, to easily summarize their experimentation results. Simply click on any metric card and hit the summarize button for a full analysis of your data. Please note, that this is a Generative AI feature that leverages end-user/customer data and will only be available to users who have specifically requested it to be enabled via Split’s Support team. 

Usability Enhancements 

Change Request Management 

To help teams easily coordinate and collaborate on flag updates, we added shareable direct links to change requests. This allows teams to quickly share feature flag updates with key stakeholders and get faster approvals when needed.

Usability Enhancements

Switch Updates

Users can cancel a response from Switch mid-flight. This allows users to move forward quickly in cases where they asked the wrong question or if the response they needed was already on the screen.

Users can now also copy code snippets directly from Switch, enabling faster time to value.

Usability Enhancements 

Filtering Improvements 

To help teams quickly find the information they are looking for, Split has improved the filtering experience in these lists: feature flags, segments, and metrics. Users’ most recent search filters will be preserved as users continue to navigate the product. 

Feature Experimentation

Sequential Testing Update

Users can now reduce the minimum sample size to 100 at the organizational level via the admin setting. This allows users to get statistically significant results, faster.

Usability Enhancements

Toast Notification Update 

Toast notifications will now appear in the lower right corner of Split’s UI. This increases the visibility of the notification and enables users to easily take additional action within Split.