Analyze cohort performance
Analyze cohort performance
1. Make a list of all the segmentation criteria you have available from your data.
Before actually creating a cohort analysis, you need to know which dimensions (criteria) you can use. Here are some common examples: Usage: clicked “recommend to friend” in your product Attribution: purchased your product in the last 7 days Geographic: users from Texas, USA Demographic: users whose company size is over 50 employees Psychographic: sentiment analysis where customer is positive Behavioral: analysis shows them as price-conscious.
2. Select the criteria you'll use to group your data, based on the list you previously created.
For example, the user’s signup date to your product. The selection you make will usually be based on two things: What type of learnings you want to achieve from the cohort? For example, do you want to know about a specific product feature or about marketing activities? How concrete you want the output to be? For example, will you use the results directly in product development or on a strategic level? Generally speaking, the more specific the criteria you choose, the more concrete and actionable your results will be. For example, if you choose “sentiment analysis where the customer answered positively,” you will most likely not have an easy time applying the outputs from the chart to any decisions you want to make.
3. Choose a timeframe for the cohort itself and not for the entire analysis.
The timeframe is usually smaller for more detailed analysis; common cohort charts use monthly or quarterly timeframes. If the timeframe is large, movements can be harder to detect of the actual retention against the metric. For example, selecting yearly timeframes will not show fluctuations within the year itself, which may be important for your business. Therefore, best practice is to start smaller and then work your way to larger timeframes if possible, keeping in mind that the date range of the analysis cannot be smaller than your timeframe. You cannot analyze yearly cohorts in a date range of 3 months.
4. Decide which core metric to use in the analysis, like users who logged in at least once, purchased at least once, or paying subscribers.
This is the most interesting part, as we decide what information we are trying to understand through this analysis. Ideally, start with one of the derived metrics for each dimension of your North Star Metric (core metric). These metrics will have their time period defined by the previous step: metric “logged in to the product at least once” + timeframe “1 month”.
5. Apply a date range to your cohort analysis where you can see at least 3-4 separate movements of your cohort groupings.
This directly correlates with the selection of the timeframe of each cohort – if you chose monthly cohorts, you cannot choose a date range of only a few days. Choose a date range of one year for quarterly or monthly timeframes.
6. Finalize your cohort analysis definition by writing it out fully and ensure it makes sense and is actionable.
For example, all users grouped by signup date to our product who recommended our product to a friend, grouped by month, for the last 12 months.
7. Create/extract your cohort framing by pulling the data into a spreadsheet.
Add your raw data to a spreadsheet. In the first column, add your criteria, one to each row. For example, if you chose plan type, you will have one row for each of your plans. Add a column for each timeframe in your date range. For example, if you chose a monthly timeframe and a date range of the last quarter, add 3 columns: one for each month in the quarter.
8. Fill your metrics in the data table and calculate the cohorts for each metric by plotting the data into the rows/columns.
Start from the second column; it will always be 100% of your metric, as it is the first measurement period of the cohort. Each cohort is done by row. For each consecutive period, moving from left to right in your columns, calculate your metric by row, for each cohort group (criteria). How to calculate my metric for each cohort for each time period correctly? Let’s use our example “All users grouped by signup date to our product who recommended our product to a friend, grouped by month, for the last 12 months.” The metric recommended our product to a friend will be 1 if they did and 0 if they didn’t; then the sum of all users who signed up in January 2017 (who did this action) divided by the total in the first month will be our second result for each row: =SUM(users who recommended our product) / total users in month 1 = 0.95 or 95%
9. Learn how to read and interpret your cohort table correctly.
Now that your cohort table is filled with data, it’s time to learn how to read your cohort table. Using our example of “All users grouped by signup date to our product who recommended our product to a friend, grouped by month, for the last 12 months”: In May 2017, say we had 100 users sign up. In month 2, only 95.60% of those users used the recommendation feature. In September 2017, we had 100 users sign up. In month 3, only 95.10% of those users used the recommendation feature.