Concept testing for new feature idea:
Is this idea viable and needed for users?
@Karrot






Overview

My Role/ Responsibilities

I served as the sole UX Researcher (UX Research Intern) in a community service team called 'Neighborhood Life' at a hyper-local app called Karrot.

Context

The 'Neighborhood Life' team was looking for a solution that could provide users with a sense of community and encourage them to actively share their stories. As a UX Researcher, I was tasked with testing whether the hypothesized feature was viable and had product-market fit.

Impact

Based on UX research findings, I discovered that the hypothesized feature is not an ideal solution and was eventually able to decrease development time by 50%, saving time and money on unavoidable reworks.

Duration

~21 days (10/20/2023 - 11/10/2023)

Stakeholders

2 UX Designers, 1 PM from
"Neighborhood Life" team

Methods

Concept testing, User Interviews (Semi-Structured), Idea Screening Session, Survey

Tools

Zoom, Google Workspace, Height, NAVER CLOVA, Figjam




Β Domain Context



Karrot is a community-driven marketplace that has MAUΒ of 19 million and connects people locally for buying and selling second-hand stuff.


Since 2021, Karrot has been working towards developing a hyperlocal social networking service for neighborhoods, called "Neighborhood Life," which was the team I was part of.


Users can ask questions and share their stories with neighbors within their selected neighborhood.



Background



Data analysis revealed a clear pattern in how community users interact with the service, specifically showing that:


Users make their first visit (Entry), browse the feed (Lurking), and start writing comments (Commenting) when they see posts they relate to.

‍



72% of users begin with commenting before writing their first post
and they write an average of 9-10 comments before writing their first post.

This implies there is a potential barrier, which results in only 0.8% of users writing posts.



From the previous research findings, it has been discovered that there is a number of barriers for users to write posts in community feed.


U1: "I feel a sense of burden when sharing my story in a space where an unspecified number of people exist."

‍
U2: "There are too many diverse stories being shared, and I see a lot of posts that I am not interested in.



- "Beginners" cohort (consisting of 6.3% of Neighborhood Life community who have not written any posts)


🧠 Psychological Insight #1
Users are showing 'evaluation apprehension' by highlighting the discomfort and hesitation they feel when sharing their stories in environments where they are unsure of who might be listening or when they feel overwhelmed by the presence of unrelated or diverse content.



Problem Space



🚨 Due to the existence of varying age groups and types of people, users do not find common interests or a sense of community.


🚨 The community feed lacks sufficient content because users are not posting enough which eventually leads to low interaction time.
‍
‍




Research Goals



🟒 Find the best solution to the problems above and use it for concept testing.


🟒 Evaluate the validity of the concept/ hypothesized feature idea and identify user needs.


🟒 Provide useful data to back up the hypothesized feature idea and help the team to launch with confidence.
‍

‍
‍

Research Timeline
‍


This is a screenshot of a shared timeline from a Notion page within the 'Neighborhood Life' database. The
research timeline is adjusted to Karrot's agile environment, which features a fast-paced and streamlined process.





Research Question




RQ1: "What is the most effective solution for encouraging users to write more posts?"
‍
‍
RQ2:
"Do users need our new hypothesized feature, and is it viable?"
‍
‍




Research Methods



1. Idea Screening Session


‍
To answer the first research question, I led a 1-hour stakeholder discussion session involving idea generation and screening in order to:
‍
‍
1. Evaluate and prioritize ideas based on data and previous research findings, ensuring that the most strategically aligned concepts to the problem space are identified while minimizing risks during this hands-on session.

2. Make stakeholders to feel that they are all in the loop and also contributing to the research.


Using FigJam, I visually laid out brainstormed ideas on the left, and screened ideas via SWOT analysis on the right. Stakeholders discussed and voted for the most reasonable and viable solution.





What we found from this session is...
‍
‍
The team hypothesized that sharing demographic characteristics would enhance the sense of community among users, leading to a greater willingness to share their stories. This hypothesis is based on the following reasons:


1. In Korea, the largest online community services (ex. NAVER cafe) are based on demographic characteristics, including moms, students, business owners, etc. This trend indicates that people tend to gather based on their demographic characteristics, suggesting a product-market fit.

2. Local business owners group has the highest engagement in terms of extended interaction time and greater number of comments and posts in Neighborhood Life.

🧠 Psychological Insight #2
This demonstrates the 'similarity-attraction effect', where communities built on shared demographic traits thrive as users feel understood, leading to increased engagement and story-sharing.

To address the first research question:



‍
"What is the most effective solution for encouraging users to write more posts?"



🟒 I concluded that grouping local Business Owners, which is the largest user group in "Neighborhood Life" service, is the most effective solution as of now.





2. Remote User Interviews
‍
‍
‍
‍Study Design:

I chose remote user interviews as my research method to answer the second research question.

Since it is an exploratory phase that aims to discover how users interact with other local business owners and their potential needs and pain points, I concluded that this approach is the most suitable one.

I explained to the stakeholders that this method will allow for direct user interactions, where we can observe their tones and reactions and also probe when necessary with follow-up questions.

Karrot typically conducts interviews remotely via Zoom to minimize the time and costing of recruiting participants across the country.


Study demographic:


The participant criteria were strictly controlled to ensure that demographic characteristics were all considered across the participants.





Since this is a hyperlocal community service, we hypothesized that...



πŸ’‘Local business owners who spend longer period of time in their local areas tend to engage more with other business owners, indicating a greater demand for this new feature.


Therefore, we divided the participants into two groups to test whether there is a difference in the extent of the desire for the new feature:

(conducted a survey screening in advance to see the distribution of different business types.)

‍
‍

‍

Sample interview questions:
‍

I implemented a funnel technique to break down interview participants' deeper motivations and attitudes specific to the hypothesized feature.

Started the conversation with broad and open-ended questions, such as,

"Can you tell me about your experience as a business owner? What is your day-to-day operations?"

and then finally got to the core question:

"How important is it for you to communicate with other local business owners in your area?"





Research Findings



Remember we hypothesized that...


‍
Local business owners who spend longer period of time in their local areas tend to engage more with other business owners, indicating a greater demand for this new feature.


‍
πŸ”΄ Unexpected Outcomes:

‍
The amount of time spent in local area didΒ NOT impact the willingness to communicate with other business owners.

‍
The willingness to communicate with other business owners both in-person and online was significantly influenced by the following two factors:


1. They are in a business that is currently too competitive
‍
2. The presence of individuals with the same business type in the local area if they are in an online community


Pattern A
: in a business that is currently too competitive and do not want this feature
Pattern B: in a business that is not competitive and want this feature
Pattern C: in a business that is not competitive and do not want this feature.

(One participant was excluded from the data analysis due to being identified as an outlier.)


Users in Pattern A exhibited the strongest dislike for the new feature idea and had the highest distribution of business types in Neighborhood Life and S.Korea overall.

U1 (Pattern A): "I barely talk to someone who does the same business as mine in my town. It is so competitive here (sigh), and I am afraid if they are gonna steal my idea. I will never use the service like that."


U8 (Pattern C): "I might hesitate to share my stories too often there because I would be looking like I have too much free time and a total jobless (laugh)."
‍
🧠 Psychological Insight #3
The competitive business environment and the presence of similar local businesses heighten their sense of limited resourcesβ€”'resource scarcity'β€”making them more cautious and selective in their interactions. However, they were more willing to share their stories with other business owners when they could remain anonymous and the community was nationwide.






To address the second research question:




"Do users need our new hypothesized feature, and is it viable?"




🟒 6 out of 9 participants indicated that they did NOT want this feature, leading to the conclusion that this new feature idea is neither viable nor needed for the targeted users.





Research Impact



- Evaluated that the hypothesized feature is not viable nor needed to our targeted user group, contrary to our team’s expectation.

- Based on UX research findings, we were able to decrease development time by 50%, saving time and money on unavoidable reworks.

- Made sure that stakeholders were all in the loop and shifted their perspectives throughout the research process!




"We have gone through numerous unavoidable reworks in the past, but this research has certainly prevented such issues from arising. Thank you so much for saving our time and energy!"

‍
- Quote from the Team Lead of Neighborhood Life

‍
‍



Lessons & Reflection



1. Absorbed various ways to approach and converse with users, I felt like I became a Chameleon!
‍
‍
It was beyond fascinating to meet a wide range of users, all possessing different personalities and levels of participation, across the country. During my 6-month internship, I moderated interviews with over 50 users, which helped me learn how to be adaptable and flexible, and advanced my moderating skills!
‍

2. Receiving feedback from Senior UX Researchers from other teams helped me stay on track and grow as a UXR.

‍
During my 6 months of internship, I had full responsibility for designing and conducting research, thereby enhancing my skills as a solo UX Researcher within my team. To stay on track, I regularly scheduled 1-on-1 meetings with Senior UX Researchers to receive valuable feedback which I enjoyed a lot and significantly expanded my perspectives.
‍
‍
3. Last but not least, I am grateful for the opportunity to work with talented and considerate co-workers, rich resources, and owning the project!

‍
I learned how to navigate myself in a big corporate setting and collaborate effectively with individuals from diverse professional backgrounds, contributing to my overall growth and adaptability!


‍

‍

Let's collaborate ✨

I’m open for collaboration and available for work.