We have updated our Data Processing Addendum, for more information – Click here.

Guide to Product Metrics

Sessions

Construct engagement metrics such as session start and end, entry and exit rates, and session length.

A web session is a way to organize interactions between a user and the application within a specific time frame. A single session may involve navigation across multiple pages or browser windows. It can allow users to close the site fully and then return as long as they come back within a specific activity window. It is that continuous activity that marks an active session, and a common standard is to persist a session for 30 minutes from the point of last activity by the user. If a user returns after that window, the prior session is ended at the point of the last activity, and a new session is started.

There are additional criteria outside of activity in some implementations that determine the start and end of sessions. To avoid counting short bounces to a page as a session, some applications define one or more interaction events that are required for the session to be counted. There can also be additional triggers for a session’s end, such as logging out of the application or returning to the site through a different campaign or search criteria. These advanced implementations require some additional logic in the tracking code but do not change the nature of session start and end events for measurement purposes.

Event Tracking

Effective tracking of sessions requires some mechanism for persisting data about the state of the open session, typically through browser cookies or local storage. This necessarily ties the session to the device originating it, and it is worth noting that if the client does not allow cookies or manually clear the stored data, this will wipe the session state. This state can contain a session identifier, the campaign criteria (if changes are used for ending sessions), and timestamps for the session start and the last activity time. The last active time should be updated regularly to ensure accurate measurement of session inactivity in session end events. It is also helpful to track timestamps of particular interest to the user, such as the time of their first visit, their registration time, or the time of first exposure to specific feature releases, to include the time since those events for retention analysis.

For an example of one way this logic can be implemented, refer to our sample client library.

Session Start

A session start can be triggered when a user interacts with the application and either no session state exists or the most recent activity for that session occurred outside of the defined session inactivity window. At that time, a new session start event can be recorded, and the session state should be updated to reflect the newly created session. By incorporating a property for the time between this session start, and the user’s very first session start, the return rate of users can be captured for a variety of time frames and use cases.

Session End

The activity-based session definition’s major disadvantage is that it requires the lack of activity to define the session end. If a user closes the application, that instance cannot determine if you will return to the page a minute later to resume the session.

While some explicit session end actions may be defined to trigger this event, most sessions will end due to inactivity, and a session end event will only be fired if and when the user returns outside of that inactivity window and triggers a new session start. When this is triggered, the session end event can be fired for the prior session, and properties can be set to define the session’s length and activity. As a user may not return, this event is not guaranteed to fire. So reliance on this session end event is dubious unless generated through after the fact analysis and sent as an event once the inactivity gap is confirmed.

Session Ping

The alternative to generating a concrete session end event is using a ping or heartbeat event that indicates the session is still active. A cadence can be defined at which the application will report still being active and which can report the time elapsed since the last ping event was sent. This allows for session length and activity to be tracked by aggregating across the ping events. It also provides a session-related event that will be available even in the typical case where the session started prior to the user’s exposure to a given feature release.

As a user may have multiple instances of the application open simultaneously, it helps track the time between ping events and how much of that time this instance was active and in focus. This active, focused time will provide a more accurate picture of the time on site.

In addition to triggering the session ping event based on elapsed time, it should also be fired when the page is unloaded from the browser to ensure any activity since the last ping will be captured. This provides the opportunity to include an additional property to indicate whether this session counts as a bounce.

A bounced visit occurs when the visitor starts a session but then exits the application without navigating or engaging with the site. If the visit was a bounce, we can store that fact in the session state and return a “bounce” property of 1 in the final session ping event. However, the user may return during the same session and then complete the necessary interaction, in which case the session state can be checked on page exit. The property can be assigned a value of -1 to negate the previously incorrect bounce record. This approach allows for accurate tracking of the number of bounced sessions by summing these values together across all exit ping events, as seen in the bounce rate metric.

Metrics

The metrics built from session data are key to understanding the retention and engagement of users. A sticky product retains its users across sessions, resulting in not just a high session count for those users but also users that are returning days, weeks, or months after their first visit.

By tracking the age of the user with the session start events, a return rate metric can be calculated for various time windows. As the first session will have a user age of zero, this can be used to identify new visitors to the application and measure the adoption of the product. It is worth noting that adoption is more often moved by external factors (market trends, advertising), so it is a challenging metric to move through feature changes. By tracking the landing page and referrer information for session starts, it is possible to track the entry rate of sessions by their page or source.

The session provides a way to standardize user behavior over extended periods by building conversion ratio metrics that standardize the number of interactions the user performs to the number of sessions they have. User engagement is measured both by this level of interaction and the amount of time the user is active with the site. Engaged users will often spend more time on site, both overall and with regards to average session length – although the average session length metric requires an accurate session end event to be properly calculated.

Session Count

Implementation of Session Count metric in Split

Session Entry Rate

Implementation of Session Entry Rate metric in Split

Adoption

Implementation of Adoption metric in Split

Return Rate

Implementation of Return Rate metric in Split

Bounce Rate

Implementation of Bounce Rate metric in Split

Time on Site

Implementation of Time on Site metric in Split

Average Session Length

Implementation of Average Session Length metric in Split

Want to Dive Deeper?

We have a lot to explore that can help you understand feature flags. Learn more about benefits, use cases, and real world applications that you can try.

Create Impact With Everything You Build

We’re excited to accompany you on your journey as you build faster, release safer, and launch impactful products.

Want to see how Split can measure impact and reduce release risk? 

Book a demo