In this recent Webinar with Scott Middleton, we discussed the scope of “onboarding”:
where it starts,
where it ends and
how it relates to “Activation” and the “Aha” moment.
This video snippet here shows the Pirate Metrics and where “onboarding” fits.
The primary message here is that each user who downloads your Mobile App or lands on your Web App is on a journey to their first JTBD.
Job to be done #1 is “Aha” moment
JTBD is one methodology for Product Teams to align their product with the job that a potential customer needs “to to be done”.
When a user evaluates a product the 1st JTBD is a race to verify the product’s usefulness (value) against the time the user invests. Its a critical period.
In the diagram below, the graph illustrates the race to the “aha” moment. The steeper the initial curve, the faster the user establishes your product does the “Job”.
You probably don’t miss Letterman much but in the spirit of Dave, here is a not-too funny countdown on the build-versus-buy decision Product teams need to make about adding an engagement layer (specifically tips, tours, flows, hotspots etc) to your App.
As a recap, the Contextual mission is decoupling “App design” from “App Engagement experiments“.
So, we think that Developers own the cadence of the “App design” layer. Conversely Product Teams need to iterate fast on Experiments for feature adoption, onboarding, feedback etc.
10. Bugs and technical debt – Build once and fix many times – you can get open source tips and popup libraries for your flows but once you cobble it together you “own” the fixes and changes that start as small tweaks and end as big jobs.
9. No roadmap disruption – your backlog has hundreds of feature requests to be triaged and added to sprints. Do you really want to add Engagement experiments to your tech roadmap? Therefore, your experiments may or may-not work but you delayed features to try it out.
8. If it works you will keep using it – A corollary of the prior 2 points: if your Engagement activity is working then you will double-down and use it more. So then you double-down roadmap disruption.
7. Manual targeting – you start out with a flow for everyone but soon you realise you want to treat different personas with a unique experience. Your engineers now start saying a whole engine needs to be implemented to do right-place-right-time. Thats a lot of code.
6. Real-time Audience Segmentation and Pesonalization – engagement needs to happen in sub-seconds, if you are relying on hooking data out of analytics packages or CSV uploads, you’ve already lost the engagement opportunity.
5. Analytics that make sense – your team has invested in Mixpanel or Amplitude that are terrific platforms, but to instrument these tools around every subtle engagement on your flows, tips, tours, flows, tooltips is a data science job. So that is a “Hammer to crack a Walnut”. 4. Analytics you can see – contextual reports are important to Product Teams, you don’t want to have to dig around in Jira, the CRM and paste it all into a spreadsheet. If you are running engagement, you want to quickly see daily or weekly with zero effort how each experiment is performing, make micro or macro changes and measure again. 3. Delayed experiment iteration – Product Managers and Growth Teams need to iterate fast. Getting ensnared in the roadmap cadence is death to growth. 2. Outsource confusion – so….you’ve sent this non-core coding offshore? But mixed interpretation, timezone problems get some real mixed results. You know what I’m talking about 1. Discincented engineers will go and get a better job – tips, tours, flows are valuable to Product Managers but sooooo not a challenge to your engineering talent. “What font do you want again?”. Nuff said.
In the Part 1, we covered the importance of establishing goals around your engagement experiments and flows.
We could call this Part 2: “Segments are nice, Segments are dumb”.
Segmentation will eventually be “individualization” – we cover the steps needed to get there. Thinking in “Goals” are an important step.
Since Contextual’s inception, we provide default segments useful for tracking and grouping users. Names like Newbies, Light Users, Power Users, Churning, Zombies have filters for capturing generic “buckets” of users.
In addition, you can choose a combination of filters based on your own Custom Segments. This example below is creating a segment of “Recently active Project Managers”.Contextual
Once having defined and saved the “Active Project Managers” segment, your Product Team can then:
target tips, tours, flows, popups at this group (in conjunction with other triggers)
track the size and membership of that segment.
This is a very good granular way at looking at your user base and targeting flows and content that is customized for their job role.
But…
User’s don’t care about your segments.
Ask a user what segment they are in. You’ll get a blank look.
Segments ignore the needs hopes and wishes of each individual user. A segment aggregates and abstracts them into a “label”.
But each user is on their own unique journey and within a segment you should be seeking to personalise and respond to individual needs at scale. How do you scale for each unique snowflake?
Scaling Individualization
If you have 25K Monthly Active Users, then having 6 segments is easy for you to manage but mediocre for users.
One solution would be to create more segments – the ultimate solution would be to create as many segments as there are Users (25,000 segments!). That would be:
ridiculous!
a huge amount of work for the product team
always out of date.
Still not what a user wants from your product.
Artificial Intelligence will eventually make this possible: what Netflix does for movie recommendations or Facebook does in your feed. More on this later.
Goals – a user’s needs
Better than structuring your users into segments – goals align the Product Team’s interests with the user’s interests.
Its not very different but an important way of thinking about your user’s needs.
Example:
Already you can see a business GOAL or event looks like a customer progression in their own journey, there are some mandatory steps in the business process that each user must be aware of and complete:
DETAILS_FILED = YES, NO
TERMS_AGREED = YES, NO
QUOTE_CREATED = YES, NO
SALE = YES, NO
Then joining these journey goals with Contextual’s seamless tracking of the user’s behaviour, e.g:
Install date and time
Usage dates and times
Screen Visits, Session count, length etc
Touch events
Delivers a rich pool of rule-based or training data that can tell you more about the user that enriches data-driven segment toward goals and “individualisation”.
Today, by manually working backwards from the population who have achieved goals you can determine the “Next Best” segments you should be targeting. Contextual allows you to “what-if” audience size my testing goal-completers with other data. You could export or dump this data to a datalake (redshift, bigquery/bigtable, snowflake, Azure DW) or data-mining system for better tools for PCA and to seperate causality from correlation. Then you can compare goal-completer’s rows vs not.
You should end up with some observations like:
“80% of users who completed the introduction tour” resulted in DETAILS_FILED=YES”
“90% of users who completed the introduction tour” within 24 hours of registration resulted in DETAILS_FILED=YES”
The goal at scale
The interesting thing about goals is that unlike the 25,000 potential segments, there is a small number of goals that matter in the sequence of a user journey – so scaling with the above method is naturally a more manageable.
But…let’s face it, you don’t want to click through all your users to uncover nuances submerged in the data that lead to greater personalisation and individual needs.
DETAILS_FILED = YES is an important business goal in this app – the business relationship is established. The Product Team can learn a lot from what attributes distinguish these users from the DETAILS_FILED = NO users. There are also other filters that are pre-cursors, for example, users who have churned will automatically DOCUMENT_UPLOADED = NO.
From the Contextual data we can learn that these 2 goal based segments can be broken into (we chose) approximately 10 interesting segments.
For example, we know that users who viewed the “Completing the Document” tip tour have a higher success rate of DOCUMENT_UPLOADED = YES.
So one logical conclusion would be to keep re-showing this tour to users until they complete it. Another action could be to trigger a feedback question to these users.
Some other attributes are surprising – for example Android uploads from newer devices is a predictor of success. How the hell could the Product Team manually discovered that? The action is the Product Manager can schedule an investigation by developers to find a root cause.
Individualization with Machine Learning
Each of the 25,000 user’s journeys is describable by the data (behavioural, segmentation, goal, external enrichment).
Instead of the manual iterations above, you will see AI in platforms like Contextual by training on the “goal data” (supervised) to learn the models, then automate interaction with new users as they move through the journey.
The challenge is that both platforms and Product Teams outside silicon valley are not quite up to the task at the moment. So, purchasing decisions for on-boarding/engagement products are made without this as even a consideration, so we need to user the rule method and engines like Contextual to get results today.
Keep an eye out for companies like https://www.clearbrain.com/ who are early but pitching causal based analytics to convert customers.
This multipart series covers best practice for activating user’s on their journey. “Activation” has a specific meaning, the way it was defined in McClure’s Pirate Metrics – if you are not familiar with this, take a scan of this post.
In this post, we cover UX goals you want YOUR user’s to achieve. The next post will compare with segments. Contextual allows you to add segments from the beginning which is great, but segments also have problems that we’ll dive into.
Setting meaningful user goals
All app owners want to get their users to the “AHA” moment as quickly as possible – this increases engagement and “AHA” is an ACTIVATION event.
The problem is that many App owners don’t know what that means in their App:
There is not enough granularity of data to know.
New Apps have little or scarce data.
The bigger problem is that the goals are too big (not granular enough) – “convert triallers to subscribers” is not a goal, its a wish!
Work Backwards from goals to design
Steven Covey (of 7 habits fame) says “Begin with the end in mind”.
Often Product Teams visualize how they want to guide the user in the App. Implicitly the goal is embedded in the design discussions but quickly gets buried with the design’s colours, shapes, wording, validation logic. Its just human nature.
Instead we suggest to always keep goals at the center of design.
At Contextual we use Atlassian’s Confluence for writing the requirements docs and in the sample templates they have “Goal” and “Metric”.
They are encouraging a Product Manager to not just define User Stories but also “what does success look like”.
Above is an example from a Contextual Requirements doc. In this Confluence template the Metric column is encouraging a QUANTITATIVE measure for a GOAL.
This is awesome because the whole team can focus on what is the important business result. A better “Metric” for
"Customers can integrate with reduced support ticket load"
could be
"Reduce Integration related support tickets by 20%".
Note that it has nothing to do with how the product looks and all about the cost to the business and happiness of customers.
Make goals granular
In FULL SCALE***, Richardson says:
“Determine goals, milestones and priorities. These three tasks make people more productive. Productivity makes better use of your time. Time is directly related to growth. Growth is why we’re here. Therefore, growth is goals, milestones and priorities.”
Your goals might be too big and too abstract. For example:
"Reduce Integration related support tickets by 20%".
or
Increase conversion to paid by 30%
are desirable business goals but don’t map to a user in their journey.
So you need to map and align the user goals. Break the business goals into multiple user behaviours in your App. This might simply be a “BUY” or “PAY” button in the App but the following Pandora example breaks “activation” into 3 user behaviours that you would define as goals.
Example User Goals
The first step is to understand what your successful user journeys are. A famous example from Facebook was “if a user gets 7 friends in 10 days they will be a lifetime user”.
Three other examples are:
1. Pandora: I recall a speaker from Pandora explaining that an “activated” user was someone who had:
Played songs
Invited a friend
Saved a playlist
You can think of these as more granular goals.
2. Checklists and gamification. LinkedIn mission needs great profile data. They did an incredible job of getting people to update.
3. Spotify – the lead data scientist at Spotify told me: their top-tier users, the most engaged people, actually treated the application like an operating system. The data science team would analyse their behaviour and then map journeys for other users that made it easier to become top-tier.
In Summary: you need to create goals that you are able to measure that are meaningful to each user’s journey.
About now, you might be looking liking like this.
This is where your personal knowledge of your App is essential.
With your team:
break down your big (business) goals into smaller events you’ve seen. For example user behaviours like:
a user bookmarks an item that shows they have commitment.
If they hit the payment page but didn’t complete thats significant.
In the next post, we will discuss data-mining as a possible method of surfacing goals. If your team or tools have that, this is ideal.
your analytics platform. You may not have a team of engineers, data scientists and PH.D’s that LinkedIn and Spotify have but it’s likely you are already using analytics tools. It may not have a magical answer but you can do some “what-ifs” around user behaviour.
On some Contextual plans you get click/touch tracking**. So looking back 30 days for a question like “did a user touch/click BUY button?” could become a very useful goal.
There is a very good chance that a “click buy button” is exactly the behaviour you want to measure and 30 days history can be measured before and after you run a goal-based experiment. Outcome
You want define a “happy user journey” with granularity like Pandora. Here is some examples:
“We know 60% of users who don’t churn and Visit X page and Search for Y getting a successful result and Save an item become subscribers.
Users with Saved items are 60% more likely to make a purchase.
NOW you have a list of more granular goals.
In Contextual you can select goals based on:
1. Business Goals (the Custom Tag option above). In the Pandora example this could be the “Invite a Friend” process or the “Save Playlist” event. In other App is could be a purchase, getting an anonymous user to register an account etc.
2. Behavioural (Button Clicks, Page Views, or other InApp metrics in “Contextual Tags” like Session Counts, Upgrading to the latest App etc)
In this example from Wikipedia, to set a goal to measure people who registered – you simply mouse over the “JOIN WIKIPEDIA” button and clicks/touches on that will be selected as your goal.
Iterate your process
Because this post focusses on goals and segments, I won’t revisit creating user onboarding activities. Sometimes this is code design and sometimes its using a tool like Contextual to create walkthroughs. To see some examples of what Contextual can do to help onboard users or help guide them to discover and use new features, check theseposts.
Create flows/walkthroughs/tips etc to test your hypothesis
Measure the results. For example did the “BUY” button clicks relatively increase? Go back to the Contextual “Metric History” analytics chart (an example above). Can you quantitatively measure uplift on user interaction with your “BUY” button?
Refine or discard your hypothesis.
Repeat
In addition to analytics of Metric History, Contextual’s experiment analytics allow A/B splits against goal to prove/disprove hypothesis. Two detailed posts cover thistopic.
Next Post: Segmentation
Now that we have Goals defined in a granular and quantitative way, the next post will cover the pros and cons of segmentation and individual user journeys. Where is personalisation practical? If you want to be notified signup on the side-bar
Notes
** Contextual doesn’t try to replace dedicated analytics products but touch/click behavioural analytics are designed for Product, Customer Success and Growth teams to see the impact of tips and tours.
I was listening to a podcast recently with the founder of Headspace a popular meditation App. I’d not used the App but many people I know have – what amazed me on the podcast was that it was mentioned their revenues exceed $100M – it appears that the sector is big business!
I downloaded a few apps and discovered there are several use-cases that need to be handled: Relaxation, Commuting, Sleeping, Quick Breaks, Focus etc.
Compare this to something like Uber where you just have one-job-to-be-done (JTBD) and that is get a ride. These meditation Apps have to connect with each user’s main reason for downloading the App and get them started on that – its a disparate set of uses.
Here is 2 examples:
This App (I think it was Calm – I downloaded a bunch!) makes this really simple and targeted on what they want to get users doing. The App points to all the important features to get me focussed on my job. The goal is to get me to the “aha” experience.
Brain.fm have a nice App and they take a carousel approach to introducing the user to the products main features. Their unique value is they use some magic underneath the music (presumably Binaural Beat” or “Isochronic tones”) to increase the impact on the meditator.
I like they way they hit that with “Music designed for your brain”. It would directly create continuity from the:
1. marketing phase, where the user decided to install the App
2. onboarding the phase, where the user decides to keep using the App.
Overall, the first App is more effective. Sure, its less pretty (the Brain.fm carousel looks great). But it has 2 downsides:
a) its not contextual
b) they force the user to signup.
These two factors create a barrier to get to “aha”!
Both Carousels and contextual Tips/Tours are available in the Contextual platform, so its really up to your team to choose which method to use.
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.OkRead more