Cited By
View all- Vilone GSovrano FLognoul M(2024)On the Explainability of Financial Robo-Advice SystemsExplainable Artificial Intelligence10.1007/978-3-031-63803-9_12(219-242)Online publication date: 10-Jul-2024
An Empirical Study on Compliance with Ranking Transparency in the Software Documentation of EU Online Platforms
Pages 46 - 56
Abstract
Compliance with the European Union's Platform-to-Business (P2B) Regulation helps fostering a fair, ethical and secure online environment. However, it is challenging for online platforms, and assessing their compliance can be difficult for public authorities. This is partly due to the lack of automated tools for assessing the information (e.g., software documentation) platforms provide concerning ranking transparency. Our study tackles this issue in two ways. First, we empirically evaluate the compliance of six major platforms (Amazon, Bing, Booking, Google, Tripadvisor, and Yahoo), revealing substantial differences in their documentation. Second, we introduce and test automated compliance assessment tools based on ChatGPT and information retrieval technology. These tools are evaluated against human judgments, showing promising results as reliable proxies for compliance assessments. Our findings could help enhance regulatory compliance and align with the United Nations Sustainable Development Goal 10.3, which seeks to reduce inequality, including business disparities, on these platforms.
Data and materials: https://doi.org/10.5281/zenodo.10478546.
References
[1]
Peter Achinstein. 2010. Evidence, explanation, and realism: Essays in philosophy of science. Oxford University Press.
[2]
Hussam Alkaissi and Samy I McFarlane. 2023. Artificial hallucinations in Chat-GPT: implications in scientific writing. Cureus 15, 2 (2023).
[3]
Razvan Azamfirei, Sapna R. Kudchadkar, and James Fackler. 2023. Large language models and the perils of their hallucinations. Critical Care 27, 1 (March 2023), 120.
[4]
Adrien Bibal, Michael Lognoul, Alexandre De Streel, and Benoît Frénay. 2021. Legal requirements on explainability in machine learning. Artificial Intelligence and Law 29 (2021), 149--169.
[5]
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17--21, 2015, Lluís Màrquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton (Eds.). The Association for Computational Linguistics, 632--642.
[6]
Erik Brynjolfsson and Brian Kahin. 2002. Understanding the digital economy: data, tools, and research. MIT press.
[7]
Erik Brynjolfsson and JooHee Oh. 2012. The Attention Economy: Measuring the Value of Free Digital Services on the Internet. In Proceedings of the International Conference on Information Systems, ICIS 2012, Orlando, Florida, USA, December 16--19, 2012. Association for Information Systems. http://aisel.aisnet.org/icis2012/proceedings/EconomicsValue/9
[8]
Christoph Busch. 2023. From Algorithmic Transparency to Algorithmic Choice: European Perspectives on Recommender Systems and Platform Regulation. In Recommender Systems: Legal and Ethical Issues. Springer International Publishing Cham, 31--54.
[9]
European Commission. 2023. Report from the Commission on the first preliminary review of the P2B Regulation. https://digital-strategy.ec.europa.eu/en/library/report-commission-first-preliminary-review-p2b-regulation Accessed on: 2023-10-06.
[10]
Content European Commission, Directorate-General for Communications Networks and Technology. 2020. Commission Notice Guidelines on ranking transparency pursuant to Regulation (EU) 2019/1150 of the European Parliament and of the Council 2020/C 424/01. EUR-Lex. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52020XC1208(01) Accessed on: 2023-10-06.
[11]
Content European Commission, Directorate-General for Communications Networks and Technology. 2021. Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:52021PC0206. Accessed on: 2023-10-06.
[12]
European Parliament and Council. 2019. Regulation (EU) 2019/1150 of the European Parliament and of the Council of 20 June 2019 on promoting fairness and transparency for business users of online intermediation services (Text with EEA relevance). https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32019R1150 PE/56/2019/REV/1, OJ L 186, 11.7.2019, p. 57--79.
[13]
Peer Eyal, Rothschild David, Gordon Andrew, Evernden Zak, and Damer Ekaterina. 2021. Data quality of platforms and panels for online behavioral research. Behavior Research Methods (2021), 1--20.
[14]
Jennifer Fereday and Eimear Muir-Cochrane. 2006. Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International journal of qualitative methods 5, 1 (2006), 80--92.
[15]
Pablo A. Gajardo and Ariel I. La Paz. 2019. Measuring the strategic business and IT alignment in a digitally revolutionized economy. In 25th Americas Conference on Information Systems, AMCIS 2019, Cancún, Mexico, August 15--17, 2019. Association for Information Systems. https://aisel.aisnet.org/amcis2019/org_transformation_is/org_transformation_is/19
[16]
Golara Garousi, Vahid Garousi, Mahmood Moussavi, Günther Ruhe, and Brian Smith. 2013. Evaluating usage and quality of technical software documentation: an empirical study. In 17th International Conference on Evaluation and Assessment in Software Engineering, EASE '13, Porto de Galinhas, Brazil, April 14--16, 2013, Fabio Q. B. da Silva, Natalia Juristo Juzgado, and Guilherme Horta Travassos (Eds.). ACM, 24--35.
[17]
Golara Garousi, Vahid Garousi-Yusifoglu, Günther Ruhe, Junji Zhi, Mahmood Moussavi, and Brian Smith. 2015. Usage and usefulness of technical software documentation: An industrial case study. Inf. Softw. Technol. 57 (2015), 664--682.
[18]
Philipp Hacker, Johann Cordes, and Janina Rochon. 2022. Regulating Gatekeeper AI and Data: Transparency, Access, and Fairness under the DMA, the GDPR, and beyond. arXiv preprint arXiv:2212.04997 (2022).
[19]
Philipp Hacker and Jan-Hendrik Passoth. 2020. Varieties of AI Explanations Under the Law. From the GDPR to the AIA, and Beyond. In International Workshop on Extending Explainable AI Beyond Deep Models and Classifiers. Springer, 343--373.
[20]
Leah Henrickson and Albert Meroño-Peñuela. 2023. Prompting meaning: a hermeneutic approach to optimising prompt engineering with ChatGPT. AI & SOCIETY (Sept. 2023).
[21]
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16--20, 2020, Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 6769--6781.
[22]
Jiazheng Li, Lin Gui, Yuxiang Zhou, David West, Cesare Aloisi, and Yulan He. 2023. Distilling ChatGPT for Explainable Automated Student Answer Assessment. CoRR abs/2305.12962 (2023). arXiv:2305.12962
[23]
Vesna Lukovic. 2021. INFORMATION ASYMMETRIES IN ALGORITHMS AT DIGITAL PLATFORMS: MOTIVATIONS TO PARTICIPATE AND EU REGULATORY APPROACH. EMAN 2021--Economics & Management: How to Cope with Disrupted Times (2021), 167.
[24]
Catarina Dutilh Novaes and Erich H. Reck. 2017. Carnapian explication, formalisms as cognitive tools, and the paradox of adequate formalization. Synth. 194, 1 (2017), 195--215.
[25]
Department of Economic and Social Affairs. 2023. Goal 10: Reduce inequality within and among countries. https://sdgs.un.org/goals/goal10 Accessed on: 2023-10-06.
[26]
Observatory on the Online Platform Economy. 2021. Study on "Support to the Observatory for the Online Platform Economy". https://platformobservatory.eu/app/uploads/2021/01/P2B-Regulation-monitoring-analysis-January-2021.pdf. Accessed on: 2023-10-06.
[27]
Thalia Prastitou-Merdi. 2021. The Notion of "Online Intermediation Services" Found in the New EU Platform Regulation: Who Is Caught After All? In EU Internet Law in the Digital Single Market. Springer, 543--559.
[28]
Haocong Rao, Cyril Leung, and Chunyan Miao. 2023. Can ChatGPT Assess Human Personalities? A General Evaluation Framework. CoRR abs/2303.01248 (2023). arXiv:2303.01248
[29]
Shaping Europe's digital future. 2023. EU Observatory on the Online Platform Economy. https://digital-strategy.ec.europa.eu/en/policies/eu-observatory-online-platform-economy. Accessed on: 2023-10-07.
[30]
Francesco Sovrano. 2023. Replication Package for "Automating Regulatory Compliance: An Empirical Study on Ranking Transparency in the Software Documentation of EU Online Platforms". https://github.com/Francesco-Sovrano/Automating-Regulatory-Compliance-An-Empirical-Study-on-Ranking-Transparency-of-EU-Online-Platforms.
[31]
Francesco Sovrano, Monica Palmirani, and Fabio Vitali. 2020. Legal Knowledge Extraction for Knowledge Graph Based Question-Answering. In Legal Knowledge and Information Systems - JURIX 2020: The Thirty-third Annual Conference, Brno, Czech Republic, December 9--11, 2020 (Frontiers in Artificial Intelligence and Applications, Vol. 334), Serena Villata, Jakub Harasta, and Petr Kremen (Eds.). IOS Press, 143--153.
[32]
Francesco Sovrano and Fabio Vitali. 2022. Explanatory artificial intelligence (YAI): human-centered explanations of explainable AI and complex data. Data Mining and Knowledge Discovery (2022).
[33]
Francesco Sovrano and Fabio Vitali. 2022. How to Quantify the Degree of Explainability: Experiments and Practical Implications. In 31th IEEE International Conference on Fuzzy Systems, FUZZ-IEEE 2022, Padova, July 18--23, 2022. IEEE, 1--9.
[34]
Francesco Sovrano and Fabio Vitali. 2023. An objective metric for Explainable AI: How and why to estimate the degree of explainability. Knowledge-Based Systems 278 (2023), 110866.
[35]
Timo Speith. 2022. A review of taxonomies of explainable artificial intelligence (XAI) methods. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 2239--2250.
Index Terms
- An Empirical Study on Compliance with Ranking Transparency in the Software Documentation of EU Online Platforms
Recommendations
Advertising versus brokerage model for online trading platforms
The two leading online consumer-to-consumer platforms use very different revenue models: eBay.com in the United States uses a brokerage model in which sellers pay eBay on a transaction basis, whereas Taobao.com in China uses an advertising model in ...
OM Forum—Innovative Online Platforms: Research Opportunities
Economic growth in many countries is increasingly driven by successful startups that operate as online platforms. These success stories have motivated us to define and classify various online platforms according to their business models. This study ...
API documentation and software community values: a survey of open-source API documentation
SIGDOC '13: Proceedings of the 31st ACM international conference on Design of communicationStudies of what software developers need from API documentation have reported consistent findings over the years; however, these studies all used similar methods--usually a form of observation or survey. Our study looks at API documentation as artifacts ...
Comments
Information & Contributors
Information
Published In
April 2024
210 pages
ISBN:9798400704994
DOI:10.1145/3639475
- Co-chairs:
- Ana Paiva,
- Rui Abreu,
- Bonita Sharif,
- Xin Xia,
- Program Co-chairs:
- Prof. Abhik Roychoudhury,
- Margaret Storey
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
In-Cooperation
- Faculty of Engineering of University of Porto
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 June 2024
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Funding Sources
- Swiss National Science Foundation
- European Union H2020 research and innovation program
Conference
ICSE-SEIS'24
Sponsor:
ICSE-SEIS'24: 46th International Conference on Software Engineering: Software Engineering in Society
April 14 - 20, 2024
Lisbon, Portugal
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 179Total Downloads
- Downloads (Last 12 months)179
- Downloads (Last 6 weeks)36
Reflects downloads up to 09 Jan 2025
Other Metrics
Citations
Cited By
View all- Vilone GSovrano FLognoul M(2024)On the Explainability of Financial Robo-Advice SystemsExplainable Artificial Intelligence10.1007/978-3-031-63803-9_12(219-242)Online publication date: 10-Jul-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in