CASE STUDY - 2020
To put it simply, an Insight is a pattern in data. For ex: If a metric shows growth consecutively for >3 months, we generate a growth streak insight.
This helps VC’s disocver companies that might have easily been missed by current analysis tools. Some other patterns that we use for generating insights are -
- Metric crossover: If a company surpasses it’s competitors. Example: Figma surpassed Invision in Desktop visits
- Outlier: We use machine learning to forecast a range of values within which a metric should fall during next update cycle. If the value is outside of the range, we generated an Outlier Insight
- App ranking: Twitter is out of top 10 in Overall category on Play store
- Employee count: If employee count of a company crosses a pre-defined threshold, an Insight was generated.
Insights were served to users via a Feed like algorithm. They were selected based on user behaviour and pushed everyday at a specified time. The underlying logic depeneded on the companies a user had interacted with in the past(Recently, Frequently browsed), whether a given company was in a list (Portfolio, Followed) or if it’s similar to companies user might be interested in.
The Insights pushed for a company further depeneded on the metrics for which they were available. For example: An Insight for App downloads was given more preference than Install penetration.
Consider these user requests:
- I want to view all >6 months growth streaks in Fintech sector.
- For companies in my Portfolio, show me if there are any metric crossovers
- Show me all fundings that are >5M in Healthcare.
- Show me all Insights for 'Instacart' and it's competitors.
- Show me all App ranking movements under 'top 10' in Social category
All of these use cases would require some sort of filtering on the feed, which was not possible in the current architecture. The initial assumption behind the algorithmic appraoch was that Insights will be consumed passively. Like social media, a VC would login everyday check the Insights generated for them. However, the data told a different story. Only 10% of the users clicked on an Insight (per month). Users were not even scrolling the feed.
Apart from the engagement data, we also conducted a survey to guage the sentiment around feed. 80% of users said that they needed fine controls to filter insights.
The goal of insights was to surface new companies to VC’s that they would have missed otherwise. Pushing insights to users was one way to achieve that. The other was to hand the controls to users. I added filters to let them fine tune insights. This would allow users to filter Insights based on Geography, Data source, Insight type and other parameters.
Adding filters had another added benefit. Till now we were in the dark about the type of Insight that were actually valuable to the users. Adding filters would give us valuable data that’ll help us determine the relative importance of each Insight type.
Previously, clicking on an Insight opened up a modal window to display the details. This was not the right design pattern to view details. Modal is needed when an action requires user input or if you want to highlight a piece of information. None of these use cases were applicable here. Also, it took ~2-3 seconds to load the content during which a user couldn’t perform any action, which was frustrating.
To remedy this, I added another column to show the details. This allows you to maintain the context while investigating details at the same time.
Initially I positioned the filters at the top of the list. Since we already had the dropdown component in our design system, this was easy to implement. However, there were a lot UX issues with this architecture.
- This pattern was not scalable. Adding another filter or decreasing the screen size would require multiple rows to show them.
- It’s difficult to remember the position of a specific filter in a horizontal stack.
- In case of multi-select options, it’s challenging to show the current selection in a drop-down pattern
Most importantly, the options were not visible. This shifted the onus of exploring the UI on the users and made the experience un-inviting.
To entice users and show them the possibilities upfront, I opened up the fitlers related to Insights. Although this was better, now we had filters running down the length and breadth of the screen.
Finally, to reduce the clutter and make the experience more seamless, I moved all the filters in the sidebar. Expanding all the filters by default was overwhelming & redundant. None of the workflows we identified during research needed all the filters at once. To solve this, I added an option to collapse the filters.
Data informed decision making
I used engagement data to highlight the dissonance between how we thought users consumed insights and the actual usage.
Jobs to be done & team alignment
Framing the problem in terms of JTBD helped align the team towards a common goal – Surface relevant companies via Insights.
Importance of Mental models
The hypothesis that insights will be used passively led to the creation of feed-like experience. In hinsight, this was an erroneous assumption which could have been avoided.