Wikipedia:Teahouse/Project

This project page might not be updated in a timely manner; please visit its meta page for the current version.

This project plan is a working draft for a pilot project in the second quarter of 2012 to be led by WMF Community Fellows in partnership with the WMF Community Department. The original version of this document, including talk page, exists on Meta at Research:Teahouse.

Concept

Tea, 1879-1880, by Mary Cassatt

The Teahouse is a populated, user-friendly welcome center/help space that organizes experienced editors to actively reach out to new users in a many-to-many setting and provides on-wiki encouragement and peer support to promising new editors to promote increased engagement and retention.

Imagine an on-wiki “peer support space” as an incubator not for content creation but for editor development. The goal is to help new editors become accustomed to community culture, ask questions, develop community relationships, etc. – supporting each other on their journey to become experienced Wikipedians.

Although the project will welcome all good faith new users, women are a particular target population. By creating a social-learning experience that helps integrate women into the community and support them in getting past barriers to participation, we hope to impact the gender gap.

Why "Teahouse?"

The name Teahouse is meant to evoke the idea of a comfortable social space for meaningful personal interaction among peers. The name Teahouse is also a nod to the English Wikipedia essay a nice cup of tea and a sit down, which urges editors to acknowledge one another's good points, and is often used to nudge people towards being congenial when things get heated.

The idea of a cafe-like space for new editors is not unknown on Wikipedia. For example, the Portuguese Wikipedia has Café do novatos.

Goals

Project Goals

• Improve editor retention among promising new editors, especially those from groups that are currently underrepresented (e.g. women), through pro-active outreach and invitations to engage.
• Model a social approach to new user support, help and socialization into the community distinct from existing 1 on 1 support models (i.e. mentorships, adoption programs, OTRS) and self-support options (i.e. help pages, tutorials).
• Give new users a place where they can easily and comfortably ask questions and receive explanations without fear of being bitten, to make them more confident editors.
• Give new users a place to interact with one another and provide peer support as they learn to do things the Wikipedia way.

Design Goals

• Provide a space that is specifically designed for new users which is visually appealing, interactive, and communicates a clear sense of purpose. The space will feature warm colors, inviting pictorial and thematic elements, simple mechanisms for communicating, and a warm welcome from real people.

Pilot

The Teahouse project will be piloted as a WMF Community Fellowship project in the first half of 2012 on EN:WP at WP:Teahouse. This pilot is intended as an experiment or proof-of-concept, and the outcome should demonstrate whether or not such an approach adds value to Wikipedia and the new-user experience by improving retention of good-faith new editors. We welcome partnership with members of other language projects to take the concept beyond English Wikipedia.

Targets

• At least 20 trained hosts active throughout the pilot
• 75 new editors utilizing the Teahouse weekly
• 50 to 100 new editors invited to participate in the Teahouse daily
• measurable increase in retention of new editors should be apparent via an increase in logins/editing sessions of participants vs control group. See below for a full list of comparative metrics being tracked

Users

Hosts

The Teahouse will be actively staffed by experienced Wikipedians. The role of these volunteer Hosts is to invite and welcome new editors to the Teahouse, answer questions, offer editing advice or support, and facilitate a peer-support environment.

Learn more about Teahouse Hosts, and becoming one, at Wikipedia:Teahouse/Hosts

New editors

The target population for this outreach effort are good faith editors who have already completed several edits, but are still in their first week of editing. Promising new editors will be invited to the Teahouse through 2 types of active outreach:

1) Partnerships with existing off-wiki outreach programs, meetups, and edit-a-thons (Global Education, GLAM, Girl Geeks, Chapter events, other women's editing events, etc.)

2) Daily database queries for:

• users who submit feedback dashboard messages
• newly autoconfirmed users, or first day editors with >10 edits

Note: Queries will be scanned by humans to filter out vandals, as the intention is to invite good-faith new editors.

Features and User Scenarios

Features

These are the main elements and requirements of the Teahouse space.

Q&A Board

• user should see that people are asking questions when they first arrive at the Teahouse and is able to browse/post/reply to questions
• clear call to action to post a new question (big button to create new section ) - avoid "forcing" new users to have to search an archive first, like at help desk.
• easy to find most common newbie questions (near the top, clear headings, relevant topics)
• easy to reply to existing question
• edit screen should be free of as much code as possible.

Introductions Space

• ability to see people (highlighting featured participants, image+text)
• ability to introduce self (user sees big button call to action, adds text)

Personalized Outreach Materials

• personalizable invitation template (with strong brand ID, colorful and friendly)
• personalizable thank you template (to be used by hosts to thank new visitors for stopping by)

Guide to Preferences setup

• call to action link "how to setup your preferences", with small pic of email pref setup, and link to fuller guide pages
• purpose is to encourage new user to enable email for notification purposes

Hosts Project Coordination

• linked from main teahouse page
• project space for hosts to organize their work (e.g. track tasks like recruiting, welcoming and answering questions), communicate with one another, and train new hosts

List of potential invitees

• database report that refreshes daily
• query of "promising candidates" (i.e. new users with 10+ edits in 1st day, new users who submit moodbar feedback) who should be invited to participate in Teahouse

Automated Notifications

• bot informs questioner on talk page of response, using a template that contains a link back to the question thread

IRC channel

• in-browser, freenode, setup pre-launch, the idea is to link from the main page but will not necessarily do so for launch, until have need and ability to staff (see nice to have section below)

Other Nice to Have Features (not planned for launch)

• Space to request peer review
• Bulletin board for opportunities to get involved in other projects, join lists, etc.
• Freenode IRC chat space for hosted events, etc.
• User box (linking to teahouse)
• Barnstar (awarding hosts or active peer supporters)
• Resources (links to existing help pages/resources, etc.)

User Scenarios

Main article: Teahouse user scenarios on Meta-Wiki

We have developed a set of user scenarios and use cases that describe some of the users we expect to benefit most from the Teahouse, and some common situations in which the Teahouse could support them.

Scenarios are divided into the following categories:

• Negotiating People, Policy and Process
• Finding Collaborators and Ways to Contribute
• Requesting Technical Help
• Online Followup of Off-Wiki Outreach

Project participants

Tea on the verandah, 1900-1910

• Sarah Stierch
• Jonathan Morgan
• Siko Bouterse

Timeline

Phase 1 - Planning, Feedback, Research (Dec/Jan)

Deliverable: Project plan complete. Assemble the necessary info, team, and buy-in to run with the project. Define participants, targets, and metrics for success.

Deadline: January 15 2012

Phase 2 - Design and Build the Space (Jan/Feb)

Deliverable: First version of the Cafe functional on EN:WP

Deadline: February 3 2012

Phase 3 - Outreach and Set-up Hosts (Jan)

Deliverable: 20 Hosts trained and ready to help new user

Deadline: February 10 2012

Phase 4 - Launch (Feb, March, April)

Deliverable: Invite at least 75 promising new editors per week and turn them into engaged Wikipedians

Deadline: February 15 2012 (launch)

Phase 5 - Measurement & Reporting (May)

Deliverable: Report on short-term outcomes measured against targets, plan for next phase of project (iterate, scale) if successful.

Deadline: May 15 2012

Metrics

During the course of the pilot we’ll be evaluating success by collecting the following metrics:

Quantitative metrics

• invites sent per day/week
• visitors per day/week
• visits per week per visitor
• active conversations per day (measured via page views and unique edits)
• count/average of new editors interacted with per host per week

Qualitative metrics

• examples of direct mentorship/coaching
• participant feedback on the Teahouse experience via surveys and/or interviews

Comparative metrics

longitudinal assessments of new users who participate in Teahouse in comparison to a roughly equivalent control cohort of new users who do not participate will include:

• edit count (broken down by namespace)
• bytes added
• logins (or editing sessions, however this is stored) per day/week/month
• logins since day 7, 14, 21, 30, 60, 90, and at 6 months and 1 year
• user rights gained
• blocks
• reverted edits
• articles created
• created articles deleted
• instances of collaborations with peers (editing on talk pages, articles, etc.)
• warning templates posted per user page

Rationale and Supporting Research

Reaching out early to new editors

A large corpus of previous research underscores the importance of providing new Wikipedia editors with support opportunities and personal, positive interactions very early on in their editing experience.

• Negative reinforcement, reverts and rejection are deciding factors in new users wanting to continue editing. The percentage of critical or negative messages (e.g. warnings, deletion notifications) left on new user talk pages has increased over time. These warnings have a negative impact on editor retention. New editors are much more likely to have their work reverted than more experienced editors. Having work rejected (reverted) has a strong, negative effect on subsequent activity, especially for new editors who make a high volume of initial edits. This negative effect has been getting stronger in recent years.
• Positive and frequent interactions between new editors and experienced editors makes a more welcome environment. New editors of Wikipedia do not perceive the same strong sense of community within Wikipedia that more experienced editors do. It's important to foster that environment; the more responses new editors receive from other editors to their on-Wikipedia activities, the more they edit. New editors who receive welcome messages, assistance and constructive criticism from more experienced editors stick around longer and edit more frequently. Receiving personalized messages and responses also makes new editors edit more, while receiving standardized/templated responses makes them edit less (i.e. Twinkle, Huggle).
• New editors that make a higher number of edits in their first edit session are more likely to stick around and become regular editors.
• Many new editors are not aware of on-wiki Help resources, or where they can go to find help. Having accessible, easy to understand resources for help and assistance is a key tool in new user retention and experience.

Encouraging women's participation

Only 9% of Wikipedia contributors are women While researchers have put forward a variety of theories to explain the skewed proportion of male to female Wikipedians, the gender gap is probably the result of a combination of factors. The Teahouse is designed to address a number of factors that contribute to the gender gap.

• Inspiring confidence through peer support. Common reasons women give for not editing Wikipedia include lacking the self-confidence that is required to edit, being conflict-averse, and a preference for experiences on other sites that emphasize social relationships and a welcoming tone which may be missing from many new editors' experiences of Wikipedia. Providing a welcoming, social environment to help new editors build self-confidence and access peer support to get through conflicts might enable us to retain more female contributors.
• Increasing enthusiasm to increase retention. Enthusiasm can be difficult to maintain when the right support system (peers, family, etc.) isn't in place in any environment. This is especially the case in communities where women, as the minority, might not have the support that males do to contribute; such as in open source communities like Wikipedia. An active on-wiki support system could help maintain enthusiasm and foster retention of more female Wikipedians.
• Encouraging promising women editors to recruit other promising new editors. Proactive invitations to participate can be key to deepening engagement with female contributors. By actively recruiting good faith new contributors to participate in the Teahouse and providing them with a welcoming environment, we hope to encourage them to contribute more and also invite their own friends to participate in Wikipedia.

Boosting on-wiki support and relationships

• Provide an on-wiki space for followup of offline activities. Off-wiki outreach and editing events have demonstrated some success in bringing new editors to Wikipedia by providing a social experience, group support and active mentorship for learning how to edit Wikipedia. Examples include GLAM edit-a-thons, Campus Ambassador events, Geek Girl Meetups, and WikiWomen parties. However, on-wiki follow-up support to encourage these new editors to continue editing and deepen their involvement in the community after these events are over is lacking. As a result, we may be missing opportunities to retain these editors after off-wiki events end. An on-wiki space that attempts to recreate some aspects of the off-wiki support environment as a follow-up to offline outreach might boost longer term retention.
• Help new users build collaborative relationships. Wikipedia has many contributors who have close ties due to long time editing and collaborations, but new users lack these ties. A support system for new editors helps to break the isolation of being new in a landscape where personal interaction is not specifically encouraged. The Teahouse can serve as a welcoming space that is non-intimidating and removes the feeling of being solitary as a new user. This is particularly important for groups like women who are in the minority, but fostering community relationships early on should be helpful for many different groups of new users.

Building a receptive environment for communication and teaching

• Providing a comfortable environment for new editors to learn 'the ropes' of editing. Providing support for fellow editors and allowing user identities to be formed through social experiences can improve communication between editors. As new users feel more included in the community through the Teahouse, they may become more receptive to learning about and understanding community policies. New users' contributions may improve and multiply as they have positive experiences in the Teahouse, find ways to connect to the community in a format that may feel more familiar than traditional Wikipedian communication spaces, and as they explore the curated information resources found in the Teahouse.
• Creating opportunities for group learning. While 1-on-1 mentoring relationships are one valuable way of teaching new users the ropes and socializing them into the editing community, they require a lot of commitment on the part of the mentor, and require a new user to have enough confidence to reach out to an experienced Wikipedia on their own, and are not scalable to meet the needs of all incoming new user. We believe that Teahouse will complement existing methods and programs by providing group learning and peer support opportunities, encouraging groups of new users to perform editing work and solve problems together and help each other out, with access to direct guidance or participation of Wikipedia veterans.