Background

Research on the apps revealed the search is the most reportedly visited part of the app. Our data aligns with the self reflections of our users. We have also learned that there is an interest from app users to receive content recommendations. Non-Editing Participation name user research reinforced an opportunity across platform to suggest content in search based on user interest. These insights has sparked a hypothesis of suggesting articles in the Wikipedia Search.

OKR Hypothesis

This work is apart of the 2024-2025 Annual Plan Wiki Experiences 3.1 work.

Hypothesis
If we enhance the search field in the Android app to recommend personalized content based on a user's interest and display better results, we will learn if this improves user engagement by observing whether it increases the impression and click-through rate (CTR) of search results by 5% in the experiment groups compared to the control group over a 30-day A/B/C test. This improvement could potentially lead to a 1% increase in the retention of logged out users.

How will we know we were successful

Validation

• Search Satisfaction rate of 65%
• 1% higher search retention rate from experiment group vs control during experiment period
• 5% of unique users click suggestion in search more than once in a 15 day period
• 5% increase in CTR of Search from experiment group compared to control group
• Personalized suggestion has 10% higher CTR than Generalized Suggestions

Guardrails:

• Experiment group doesn't have a higher abandonment rate than control
• No more than 2% of feedback includes reports of NSFW, Vandalism or Offensive recommendations
• Search doesn't worsen geographic bias

Curiosities:

• Required: Do we see a difference in metrics between logged in and logged out users?
• Does the preference for the type of content shown in search differ by platform and language?
• Would users like to see suggestions presented somewhere other than search?
• Do people return to the search just to click a suggestion?
• Should there be a filter for the type of content suggested (BLP, NSFW, Controversial Topics, etc.?)

Decision Matrix

TBD

User Stories

• As a member of The Beyhive that read an article about Beyonce, I want to know that her new line of hair care products, Cecred, has a Wikipedia article the next time I open my search, so that I can read it, realize there is a reference to Fenty Beauty on it, and join a discussion on the talk page of if it has relevance.
• As a football fan that stays up to date with current events, I want to be notified that the latest AFCON 2023 statistics are available on Wikipedia, so that I can see if Nigeria made it to the next round despite being at work completing a research paper.
• As a student learning about astronomy, I want to know there is an article about Titan after I read the Saturn article, so that I can check the references used for my research paper.
• As a Wikipedian in Kolkata, I want encouragement to check out The National Library of India article, so that it inspires me to visit in person and explore the 2.2 million books available and use it to add citations to articles I care about.

Must Haves

• Retain access to recent search results
• Show suggestions in search when accessing search from the main page / explore feed and article view of Wikipedia in the main namespace
• Fall back solution for latency issues or lack of content
• Use existing APIs
• Experiment must be an ABC test using Metrics Platform

Nice to Haves

• Show top most recommendation related current article in the search bar

Target Quant Regions and Languages

South Asia & Sub Saharan Africa

User Testing Languages

• English
• Hindi
• French
• Arabic

User Testing Considerations

• Impact for screenreaders
• Impact for RtL readers
• Preferences based on Age

Resources

Work in Progress Product Deck