research-article

Answer Quality Characteristics and Prediction on an Academic Q&A Site: A Case Study on ResearchGate

Authors:

Spencer Goodwin,

Chengzhi ZhangAuthors Info & Claims

WWW '15 Companion: Proceedings of the 24th International Conference on World Wide Web

Pages 1453 - 1458

https://doi.org/10.1145/2740908.2742129

Published: 18 May 2015 Publication History

Abstract

Despite various studies on examining and predicting answer quality on generic social Q&A sites such as Yahoo! Answers, little is known about why answers on academic Q&A sites are voted on by scholars who follow the discussion threads to be high quality answers. Using 1021 answers obtained from the Q&A part of an academic social network site ResearchGate (RG), we firstly explored whether various web-captured features and human-coded features can be the critical factors that influence the peer-judged answer quality. Then using the identified critical features, we constructed three classification models to predict the peer-judged rating. Our results identify four main findings. Firstly, responders' authority, shorter response time and greater answer length are the critical features that positively associate with the peer-judged answer quality. Secondly, answers containing social elements are very likely to harm the peer-judged answer quality. Thirdly, an optimized SVM algorithm has an overwhelming advantage over other models in terms of accuracy. Finally, the prediction based on web-captured features had better performance when comparing to prediction on human-coded features. We hope that these interesting insights on ResearchGate's answer quality can help the further design of academic Q&A sites.

References

[1]

Liu, Y., Bian, J., & Agichtein, E. (2008, July). Predicting information seeker satisfaction in community question answering. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval (pp. 483--490). ACM.DOI=10.1145/1390334.1390417

Digital Library

[2]

Harper, F. M., Raban, D., Rafaeli, S., & Konstan, J. A. (2008, April). Predictors of answer quality in online Q&A sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(pp. 865--874). ACM. DOI= 10.1145/1357054.1357191.

Digital Library

[3]

Shah, C., & Pomerantz, J. (2010, July). Evaluating and predicting answer quality in community QA. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval (pp. 411--418). ACM. DOI= 10.1145/1835449.1835518

Digital Library

[4]

Fichman, P. (2011). A comparative assessment of answer quality on four question answering sites. Journal of Information Science,37(5), 476--486.DOI=10.1177/0165551511415584

Digital Library

[5]

John, B. M., Chua, A. Y., & Goh, D. H. L. (2011). What makes a high-quality user-generated answer? Internet Computing, IEEE, 15(1), 66--71. DOI=10.1109/MIC.2011.23

Digital Library

[6]

Chua, A. Y., & Banerjee, S. (2013). So fast so good: An analysis of answer quality and answer speed in community Question?answering sites. Journal of the American Society for Information Science and Technology,64(10), 2058--2068. DOI=10.1002/asi.22902

[7]

Jeng, W., He, D., & Jiang, J. (2014), User participation in an academic social networking service: A survey of open group users on Mendeley. Journal of the Association for Information Science and Technology.

[8]

Nández, G., & Borrego, Á. (2013). Use of social networks for academic purposes: a case study. Electronic Library, The, 31(6), 781--791.

Digital Library

[9]

Goodwin, S., Jeng, W. & He, D. (2014). Changing Communication on ResearchGate Through Interface Updates. ASIS&T 2014 Annual Meeting.

[10]

Jordan, K. (2014). Academics and their online networks: Exploring the role of academic social networking sites. First Monday, 19(11).

[11]

Thelwall, M., & Kousha, K. (2014). Academia. edu: Social Network or Academic Network?. Journal of the Association for Information Science and Technology, 65(4), 721--731.

Digital Library

[12]

Thelwall, M., & Kousha, K. (2014). ResearchGate: Disseminating, communicating, and measuring Scholarship?. Journal of the Association for Information Science and Technology.

[13]

Yahoo Design Patterns. Vote to Promote. https://developer.yahoo.com/ypatterns/social/objects/feedback/votetopromote.html

[14]

Kim, Soojung, and Sanghee Oh. "Users' relevance criteria for evaluating answers in a social Q&A site." Journal of the American Society for Information Science and Technology 60, no. 4 (2009): 716--727.

Digital Library

[15]

Suryanto, M. A., Lim, E. P., Sun, A., & Chiang, R. H. (2009, February). Quality-aware collaborative question answering: methods and evaluation. In Proceedings of the second ACM international conference on web search and data mining (pp. 142--151). ACM. DOI=10.1145/1498759.1498820

Digital Library

[16]

Jurczyk, P., & Agichtein, E. (2007, November). Discovering authorities in question answer communities by using link analysis. In Proceedings of the sixteenth ACM conference on Conference on information and knowledge management (pp. 919--922). ACM. DOI=10.1145/1321440.1321575

Digital Library

[17]

Ong, C. S., Day, M. Y., & Hsu, W. L. (2009). The measurement of user satisfaction with question answering systems. Information & Management,46(7), 397--403. DOI=10.1016/j.im.2009.07.004.

Digital Library

[18]

About us. (n.d.). ResearchGate. Retrieved January 25, 2015, from https://www.researchgate.net/about

[19]

Henri, F. (1992). Computer conferencing and content analysis. In Collaborative learning through computer conferencing (pp. 117--136). Springer Berlin Heidelberg. DOI=10.1007/978--3--642--77684--7_8

[20]

Pena-Shaff, J. B., & Nicholls, C. (2004). Analyzing student interactions and meaning construction in computer bulletin board discussions. Computers & Education, 42(3), 243--265. DOI=10.1016/j.compedu.2003.08.003

Digital Library

[21]

Viera, A. J., & Garrett, J. M. (2005). Understanding interobserver agreement: the kappa statistic. Fam Med, 37(5), 360--363.

Cited By

Mazloomzadeh IUddin GKhomh FSami A(2024)Reputation Gaming in Crowd Technical Knowledge SharingACM Transactions on Software Engineering and Methodology10.1145/369162734:1(1-41)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1145/3691627
He CDeng YHe LGuo QZhang YLu ZLi B(2024)Engage Wider Audience or Facilitate Quality Answers? a Mixed-methods Analysis of Questioning Strategies for Research Sensemaking on a Community Q&A SiteProceedings of the ACM on Human-Computer Interaction10.1145/36373278:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637327
Desai MMehta RRana D(2023)Contextual analysis of scholarly communications to identify the source of disinformation on digital scholarly platformsKybernetes10.1108/K-07-2022-099853:4(1434-1449)Online publication date: 19-Jan-2023
https://doi.org/10.1108/K-07-2022-0998
Show More Cited By

Index Terms

Answer Quality Characteristics and Prediction on an Academic Q&A Site: A Case Study on ResearchGate
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking
    2. Retrieval tasks and goals
      1. Document filtering
      2. Information extraction

Recommendations

Predictors of answer quality in online Q&A sites
CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Question and answer (Q&A) sites such as Yahoo! Answers are places where users ask questions and others answer them. In this paper, we investigate predictors of answer quality through a comparative, controlled field study of responses provided across ...
Citations Gone #Social: Examining the Effect of Altmetrics on Citations and Readership in Communication Research

Altmetrics are a relatively new phenomenon in research. These metrics measure the attention that research articles receive from nontraditional venues such as social media and the Internet. This study examined how these metrics affect both the readership ...
Academic social networking: From individual acts to institutional, disciplinary, national, and social phenomena
ABSTRACT
Academic social networking sites (ASNSs) are widely being adopted by individual academics. There has been an increasing research interest in this new phenomenon. This panel brings a group of researchers to share and discuss their recent research ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '15 Companion: Proceedings of the 24th International Conference on World Wide Web

May 2015

1602 pages

ISBN:9781450334730

DOI:10.1145/2740908

General Chairs:
Aldo Gangemi
National Research Council, Italy & Paris 13 University-CNRS, France
,
Stefano Leonardi
Sapienza University of Rome, Italy
,
Alessandro Panconesi
Sapienza University of Rome, Italy

Copyright © 2015 Copyright is held by the International World Wide Web Conference Committee (IW3C2).

Sponsors

IW3C2: International World Wide Web Conference Committee

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 May 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Major Projects of National Social Science Fund
National Social Science Fund Project
National Natural Science Foundation of China
Project of the Education Ministry's Humanities and Social Science

Conference

WWW '15

Sponsor:

IW3C2

WWW '15: 24th International World Wide Web Conference

May 18 - 22, 2015

Florence, Italy

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
492
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)5

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mazloomzadeh IUddin GKhomh FSami A(2024)Reputation Gaming in Crowd Technical Knowledge SharingACM Transactions on Software Engineering and Methodology10.1145/369162734:1(1-41)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1145/3691627
He CDeng YHe LGuo QZhang YLu ZLi B(2024)Engage Wider Audience or Facilitate Quality Answers? a Mixed-methods Analysis of Questioning Strategies for Research Sensemaking on a Community Q&A SiteProceedings of the ACM on Human-Computer Interaction10.1145/36373278:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637327
Desai MMehta RRana D(2023)Contextual analysis of scholarly communications to identify the source of disinformation on digital scholarly platformsKybernetes10.1108/K-07-2022-099853:4(1434-1449)Online publication date: 19-Jan-2023
https://doi.org/10.1108/K-07-2022-0998
Desai MMehta RRana D(2023)A Model to Identify Redundancy and Relevancy in Question-Answer Systems of Digital Scholarly PlatformsProcedia Computer Science10.1016/j.procs.2023.01.213218:C(2383-2391)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.procs.2023.01.213
Mondal SUddin GRoy C(2023)Automatic prediction of rejected edits in Stack OverflowEmpirical Software Engineering10.1007/s10664-022-10242-228:1Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s10664-022-10242-2
Roy P(2023)Predicting answer acceptability for question-answering systemInternational Journal on Digital Libraries10.1007/s00799-023-00357-225:4(555-568)Online publication date: 5-May-2023
https://doi.org/10.1007/s00799-023-00357-2
Yang Y(2023)One-class graph moderating attention neural network in quality assessment of creative ideasNeural Computing and Applications10.1007/s00521-023-09256-836:7(3369-3388)Online publication date: 2-Dec-2023
https://doi.org/10.1007/s00521-023-09256-8
Linja AMamun TMueller S(2022)When Self-Driving Fails: Evaluating Social Media Posts Regarding Problems and Misconceptions about Tesla’s FSD ModeMultimodal Technologies and Interaction10.3390/mti61000866:10(86)Online publication date: 23-Sep-2022
https://doi.org/10.3390/mti6100086
Verdi MSami AAkhondali JKhomh FUddin GMotlagh A(2022)An Empirical Study of C++ Vulnerabilities in Crowd-Sourced Code ExamplesIEEE Transactions on Software Engineering10.1109/TSE.2020.302366448:5(1497-1514)Online publication date: 1-May-2022
https://doi.org/10.1109/TSE.2020.3023664
Yan WDeng WSun XWang Z(2022)Q&A participation and behavioral patterns on academic social networking sites: a comparative study of academic, corporate and government institutions on ResearchGateAslib Journal of Information Management10.1108/AJIM-05-2021-014174:3(495-521)Online publication date: 10-Jan-2022
https://doi.org/10.1108/AJIM-05-2021-0141
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents