Upstream projects: Difference between revisions

Content deleted Content added

Inline

Latest revision as of 02:10, 7 January 2025

MediaWiki developers and users^[1] rely on several upstream and/or 3rd party projects for their architecture or processes. At the same time, they develop software projects used by others, which are downstream to them. These links define our location in the free software map, our neighbor communities, and the flows of feedback, patches, and contributors.

Let's identify our upstream projects, our neighbor communities, and the contributors that connect us with them.

Motivation

This is a first step to improve our relations with other communities, to increase the contributions received, and our influence in the projects that matter to us.

Our mid-term goals include:

identify the projects where we want to see significant development, to the point of sending patches as well
identify the communities where Wikimedia should be regularly active and heard
identify the people in the Wikimedia and upstream communities that know each other and act as bridge
identify organizations and events we should get in touch and be part of
get involved in bigger development efforts regularly, become a regular FOSS player

Invented here

See Maintainers for all components, libraries, and extensions that we maintain and deploy with or around MediaWiki.

For a larger list of software projects that we publish see Wikimedia Open Source.

Other projects from the MediaWiki engineering community
Name & URL	Description	Language(s)	Repository	Bug tracker	Main contacts
translatewiki.net	Where we organise localisation of MediaWiki.	PHP, Puppet	phab:diffusion/GTWN/	#translatewiki.net in Phabricator, translatewiki:Support	Nikerabbit (founder), Abijeet Patro (co-maintainer), Siebrand (co-maintainer), Raymond (MediaWiki updates responsible)
Pywikibot	Python Wiki Robot Framework to automatize tasks	Python	Diffusion	Phabricator (under "Pywikibot")	xqt

More stuff (hundreds of projects) by fellow MediaWiki developers and Wikimedians is also listed at:

MediaWiki dependencies

Projects that we use server-side or client-side as part of MediaWiki core, or WMF-deployed extensions.

Upstream components
Name & URL	Description	Language(s)	Repository	Bug tracker	Main contacts
GNU diff and diff3	Powers recent changes, history and undo button (i.e. most editing features).
Elasticsearch	Distributed real-time search and analytics engine.	Java	GitHub	GitHub
jQuery	Lots of DOM traversing, and other DOM-related helpers, for JavaScript.	JavaScript	GitHub	GitHub	OpenJS Foundation, Timo Tijhof
jQuery UI	Simple widgets for client-side UIs. Deprecated within MediaWiki.	JavaScript	GitHub	bugs.jqueryui.com	OpenJS Foundation, Timo Tijhof
Vue.js	Vue is a progressive framework for building user interfaces.	JavaScript	GitHub	GitHub
Unihan Database	For Chinese language conversion	Used as database
Smart Common Input Method	For Chinese language conversion	Used as database
libtabe	For Chinese language conversion	Used as database
ImageMagick	general purpose image scaling	C	No repo ImageMagick.tar.gz	Forum
Lua	Scripting language	C
ACE	Web based code editor	JavaScript, HTML	GitHub	GitHub	User:TheDJ (has signed contributor license)
VIPS	Image scaling of large images (VipsScaler)		GitHub	GitHub	User:Bryan
libRsvg	PNG renderer for SVG images	C	GNOME git	GNOME Gitlab (Wikimedia tagged issues)
MathJax	Math rendering in HTML/SVG/MathML; used by Math extension	JavaScript	GitHub	GitHub
CodeMirror	a browser based text editor (Used by the CodeMirror extension to provide syntax highlighting in the 2010 wikitext editor.)	JavaScript	GitHub	GitHub; GitHub (for version 6)
FFmpeg	Video conversion and scaling	C	ffmpeg	ffmpeg
MWLib	mediawiki parser and converter	Python	GitHub	GitHub
Pygments.org	Syntax highlighting	Python	Bitbucket	Bitbucket
Unicode CLDR (Common Locale Data Repository)	All we manage to take: language names, plural rules, language-territory information, ...			[1]	Nemo (CLDR Survey Tool manager)
CLDRPluralRuleParser	Plural support based on CLDR	JavaScript	GitHub	GitHub	Santhosh Thottingal
Apertium	Free software, rule-based machine translation; used by Translate, ContentTranslation and others.		Soureforge	[2]	Francis Tyers, Firespeaker, Unhammer
Ceph	Distributed storage system	C++	GitHub	Ceph
GlusterFS	Distributed file system			[3]
Leaflet	Maps on Mobile	JavaScript		[4]
Lilypond	used by the Score extension			Gitlab.com
MariaDB	Relational database.			[5]
memcached			GitHub	[6]
multilightbox	used by Extension:MultimediaViewer			[7]
MySQL	Relational database.			[8]
PHP	Programming language that MediaWiki is written in			[9]
Rangy	JavaScript range and selection library, used by VisualEditor			[10]
Swift	Distributed storage system			[11]
lightncandy	HTML templating (mustache and handlebars) in PHP	PHP	GitHub	GitHub	EBernhardson, Kaldari
Composer	Dependency manager for PHP libraries - used for managing MediaWiki libraries and CI	PHP	getcomposer.org	GitHub	Legoktm, Bryan Davis, Reedy
Blazegraph	RDF-graph database supporting RDF and SPARQL - used by WDQS	Java	Blazegraph.com	Jira / GitHub
Zotero translators	Used by Citoid to generate citation data from a URL or other identifier	JavaScript	GitHub	GitHub
Doctrine DBAL	Used to generate engine-specific SQL DDL files from a single abstract schema	PHP	GitHub	GitHub
Video.js	Used by TimedMediaHandler as an audio/video player interface	JavaScript	GitHub	GitHub	Brooke Vibber and TheDJ

Development and operations

Projects that make up part of our development process to operate the production cluster itself.

Upstream tools
Name & URL	Description	Language(s)	Repository	Bug tracker	Main contacts
Jenkins	Continuous integration	Java	GitHub	Jira	(CD Foundation), Hashar, MarkTraceur
Jenkins Job Builder	Job creation system	Python	Git	Openstack Storyboard	(OpenStack), Hashar, MarkTraceur
Zuul	Gerrit/Jenkins bridge trigger	Python	Git	Openstack Storyboard	(OpenStack), Hashar, MarkTraceur
Gerrit	Code review tool	Java	Gerrit	Monorail	(Google), QChris
Etherpad Lite	Real-time collaborative document editing	JavaScript (Node.js)	GitHub	GitHub	MarkTraceur, Mutante
PHPUnit	Unit testing framework for PHP	PHP	GitHub	GitHub
Node.js	Programming language, e.g. used by ESLint, and Karma.	C++, JavaScript	GitHub	GitHub	OpenJS Foundation, Timo Tijhof
QUnit	Unit testing framework for JavaScript.	JavaScript	GitHub	GitHub	OpenJS Foundation, Timo Tijhof
Doxygen	Documentation generation	C++, PHP	GitHub	GitHub	Timo Tijhof
Karma	Automation for running unit tests in web browsers and capturing their results.	JavaScript	GitHub	GitHub	(Google), Timo Tijhof
Grunt	Task runner for build and test scripts.	JavaScript	GitHub	GitHub	(jQuery Foundation)
Vagrant			GitHub	GitHub	(HashiCorp)
Logstash	A tool that can be used to collect, process and forward events and log messages	JRuby	GitHub	Jira	(Elastic), bd808
Logstash plugins	Additional plugins		GitHub	GitHub	(Elastic), bd808
Kibana	A browser based analytics and search interface for Elasticsearch that was developed primarily to view Logstash event data	JavaScript	GitHub	GitHub	(Elastic), bd808
Graphite/Carbon	Real-time graphing system and time series	Python	GitHub	GitHub
Grafana	Metrics visualisation	Go, JavaScript	GitHub	GitHub
JSDuck	Static site generator for documentation.	Ruby	GitHub	GitHub
Mailman suite	Mailing list server / management (see also Mailing lists administration)	Python	GitLab	GitLab	Legoktm, Ladsgroup
git-review	Command line tool for using Gerrit		Git	[12]	(OpenStack),
Phabricator	Task tracker and software development tool used by Wikimedia, see https://phabricator.wikimedia.org	PHP	Phorge.it	#phabricator-upstream and we.phorge.it	Andre Klapper
PHP CodeSniffer	Continuous integration	PHP	GitHub	GitHub	Legoktm
Phan	Continuous integration	PHP	GitHub	GitHub
Selenium	Automated Browser Testing			[13]
Netbox			GitHub	GitHub	Wikimedia Site Reliability Engineering#Infrastructure Foundations
Debian	Free Operating System used on all production servers and cloud VPS instances.	various	[14]	[15]	multiple Debian devs in SRE, Legoktm
Let's Encrypt	non-profit certificate authority - used for TLS (SSL) certificates for many services		github/letsencrypt		SRE Traffic team
Apache Trafficserver (ATS)	Reverse proxy and forward proxy server	C++	GitHub	GitHub	SRE Traffic team
Varnish	Reverse proxy server			[16]
nginx	Reverse proxy / web server			[17]
Puppet	Configuration management system.	Ruby	GitHub	Jira	Wikimedia SRE
RT (Request Tracker)	Ticket system used by Wikimedia Operations team			[18]
Icinga	Network system monitoring (icinga.wikimedia.org)			[19]
Znuny	Ticket system used for answering queries, see https://ticket.wikimedia.org/			Znuny: [20]
CiviCRM	Used for WMF Fundraising (now upstreaming patches)			[21]
WordPress	https://blog.wikimedia.org	PHP		Trac	(mailing lists)
Rawdog	https://en.planet.wikimedia.org/ etc	Python	http://offog.org/git/rawdog/
Django	Python web framework	Python	https://github.com/django/django
OpenStack Nova	OpenStack compute service.	Python	OpenStack GitHub	Launchpad	Wikimedia Cloud Services team
OpenStack Neutron	OpenStack software-defined-networking service.	Python	OpenStack GitHub	Launchpad	Wikimedia Cloud Services team
OpenStack Glance	OpenStack image service.	Python	OpenStack GitHub	Launchpad	Wikimedia Cloud Services team
OpenStack Cinder	OpenStack image service.	Python	OpenStack GitHub	Launchpad	Wikimedia Cloud Services team
OpenStack Trove	OpenStack image service.	Python	OpenStack GitHub	storyboard	Wikimedia Cloud Services team
OpenStack Magnum	OpenStack image service.	Python	OpenStack GitHub	Launchpad	Wikimedia Cloud Services team
OpenStack Heat	OpenStack image service.	Python	OpenStack GitHub	storyboard	Wikimedia Cloud Services team
OpenStack Horizon	OpenStack image service.	Python	OpenStack GitHub	Launchpad	Wikimedia Cloud Services team
OpenStack Keystone	OpenStack identity service.	Python	OpenStack GitHub	Launchpad	Wikimedia Cloud Services team
OpenStack Swift	OpenStack object storage service.	Python	OpenStack GitHub	Launchpad	Wikimedia SRE
Kubernetes	Cloud platform for running containers	Go	Kubernetes GitHub	Kubernetes issues on GitHub	Wikimedia SRE and Wikimedia Cloud Services team
Linux Kernel	The core that runs all of our servers	C	Git source tree	TBD, probably better to use the Debian BTS.	Wikimedia SRE and Wikimedia Cloud Services team and others.
Netfilter	Linux packet filtering framework	C	Netfilter source repositories	Netfilter bugzilla	Wikimedia SRE and Wikimedia Cloud Services team and others.
AJV (Another JSON Schema Validator)	JSONSchema Validator that powers WMF's streaming Event Platform components like EventGate and schema repositories.	NodeJS	AJV GitHub	AJV issues on GitHub	Wikimedia Data Engineering and MediaWiki Platform Team
Apache Superset	Dashboarding and data exploration tool	Python	Superset GitHub		Wikimedia Data Engineering
Turnilo	Data exploration tool	JavaScript	Turnilo GitHub		Wikimedia Data Engineering
libtiff	tiffinfo; see e.g. phab:T269908	C	libtiff GitHub
(what else?)

Miscellaneous

Integrated on Wikimedia projects

Projects that are used in some way (embedding, linking) with Wikimedia projects, but operated by volunteers e.g. in Wikimedia Cloud or Toolforge.

Upstream tools
Name & URL	Description	Language(s)	Repository	Bug tracker	Main contacts
IIPImage	Viewing and zooming of ultra high-resolution images (linked by default on Wikimedia Commons)	C++	SourceForge	SourceForge
Tesseract OCR	Automated text recognition (OCR)	C++	GitHub	GitHub	Stefan Weil
Hundreds of other packages on Tools
(what else?)

Editing tools

Applications commonly used by developers or editors. For a full listing, see the lists m:Open Source Toolset, m:FLOSS-Exchange, d:Q6584911.

Upstream tools
Name & URL	Description	Repository	Bug tracker
Multimedia software and SVG tools	Software used to contribute/work on multimedia in MediaWiki wikis, like Inkscape and GIMP. May play well with MediaWiki or not.
LibreOffice (formerly OpenOffice)	Productivity suite, can e.g. produce charts and export/publish documents to wikitext (via libreoffice-wiki-publisher).		[22]
Browser spellcheckers	Aiding editors to prevent errors before they're saved.	Over 120 in Firefox, see respective descriptions.
KIPI uploader and other third party upload tools	Multimedia contribution software by third parties, usually made for integration in their software (e.g. digiKam, Gwenview).
(what else?)

Web browsers

Name & URL	Description	Bug tracker
Chrome	Browser by Google	chromium.org (Wikimedia issues are specifically tracked)
Firefox	Browser by Mozilla	mozilla.org
Opera	Browser by Opera Software	opera.com
Internet Explorer	Browser by Microsoft	microsoft.com
Safari	Browser by Apple	webkit.org
(what else?)

Notes

↑ For instance, whether volunteers or for work: maintainers and contributors of MediaWiki core and extensions and other developers; Wikimedia projects editors and sysadmins; sysadmins of MediaWiki wikis.