Value Measurement
Role
UX Designer
Company
Emarsys (SAP)
Team composition
UX Designers
UX Researchers
Product Manager
Product Developers
Data Scientists
Tasks
Close collaboration with research
Research synthesis
Ideation workshop
Close collaboration with research
Close collaboration with research
Project type
Redesign
Duration
4 months
Discovery about the existing product
The foundational research to find out the existing problems by gathering real data
Discovery research overview
The discovery research was led by the UX Researcher on the project. We chose to do individual user interviews, with Emarsys customers, as well as external people who fit our target audience.
Key insights
Confusing, hard to understand UI; not enough data to make educated decisions.
No customizability features: users cannot adjust the report to their specific use-cases - such as different KPIs for each business or different orders of KPIs.
Users also expressed a need to be able to change group sizes (control group and treatment group) based on their use-cases.
It was not obvious how exactly the product & the measurements work.
The new concept
An early UI, which we would test to align on the visual direction
The goal - keep users well informed with supporting metrics and lots of data
At the top of the report, 2 widgets provide high-level statistics, to make it easier to draw quick conclusions
The KPI widgets have one primary metric that is prominent, and supporting elements provide statistical background
The chart, one of the most visually heavy components on the screen, is also paired with supporting data, to give users additional confidence
An expandable table with detailed metrics provides more data for users, who want to dive deep into the numbers
Testing the new concept
We decided to conduct concept research, led by the UX Researcher on the project. We chose to do individual user interviews, with external people who fit our target audience.
Key insights
Participants understood that some elements are for more data-minded users, but ultimately, they didn't get the meaning of them, and they wouldn't feel comfortable sharing it with their stakeholders.
The expandable table, aimed at showing data in detail, was reported to contain information that is already on the UI, and some metrics were hard to understand.
Participants found the "Treatment Group Stats" and "Control Group Stats" widgets to be the most important for them, because they gave an overview about the programs' performance on a high level. However, they mentioned that drawing comparisons was difficult.
Challenges
The scope was set loosely, so the project kept becoming sidetracked
Updates from me weren't frequent enough, which resulted in rushed and extended meetings
Too many design variations; some weren't in line with what and how we want to show
Learnings
With the help of my mentor and my team lead, I managed to take away actionable learning items, that would help me manage the rest of this project, as well as projects further down the line in my career:
Always make sure to clearly define the scope and have it documented, so the team can reference it
Keep members of the project in the loop at all times, and structure updates towards them in a meaningful, constructive way
Keep designs simple and purposeful
The end result
A simplified, easy-to-interpret report
Focus on the results, and keep it simple
The top widget presents information that enables users to draw conclusions at a glance
Detailed data and calculation methods are moved to the documentation, and the UI is focused on the end result
The KPI widgets are organized in a way, that the most informative and easily digestible element - the chart - is on the left, and the takeaway and the measured results are on the right
Learnings
What were my takeaways?
This was my first full project at the company and in my career, and as such, it wasn’t easy. But with the kind support of my mentor and my team lead, I was able successfully complete it and learn valuable lessons along the way. Lessons, like making sure that the scope is set firmly so we don’t keep deviating from it and/or expanding it. Or that it is easy to get lost in the myriad of numbers and the data because we deal with the product every single day, but what’s important is the flow the user is in and their goals. We have to put ourselves into their shoes and make the product accordingly.
Possible next steps
How could the product continue to be improved?
Even though my work on the product is completed, there are improvement opportunities still. Some feedback from the previous research phases is still valid, like the need to make the measurement window and the group sizes flexible, or to give users the ability to customize the KPIs. Regarding the changes, research can go into gathering feedback from customers, opening up the opportunity for a design iteration round.