Date: 2022-02-23

Status

accepted

Context

Decisions on how the development, testing and delivery infrastructure for Wikibase REST API are going to be set up, seem to influence and impact each of those infrastructure elements near- and mid-term. While the Wikibase Product Platform Team claims a right to adjust, adapt, and change any decisions over time when it seems valuable, we’d also like to make an initial call on how to set those pieces of environment and infrastructure that will be the most sensible based on our current knowledge and understanding of constraints and opportunities.

Options

We considered five different options where and how to develop an MVP of the Wikibase REST API:

Directly within Wikibase.git
As a package within the Wikibase.git monorepo
As a separate MediaWiki extension in its own repository
As a dependency (library) of mediawiki/wikibase
Without binding to MediaWiki at all

Option 1: REST API within Wikibase.git

Option Description	Development happens in Wikibase.git master branch, and the REST API lives within the Wikibase Repo extension
VCS Repository	Wikibase.git
CI System	Comes “for free” via existing Wikibase CI and CI on backports to REL branches
Release Version Compatibility	Develop on master, ensuring the API is compatible with needed major versions of MediaWiki
Test System	Any, but Beta Wikidata might come for free
Remarks	Keeping the API in Wikibase.git makes it very clear which API version is compatible with which WB version Likely the most backporting (in commit volume) but similarly complex as if kept in a separate extension Many things come “for free” (boilerplate, CI, other infrastructure) but at the cost of flexibility Leaves some room for accidental mistakes regarding the "clean architecture" approach. Slow CI

Option 2: REST API as a package within the Wikibase.git monorepo

Option Description	Development happens in Wikibase.git master branch, but as a separate “Wikibase package” independent from “main” Wikibase
VCS Repository	Wikibase.git
CI System	Comes for free via existing Wikibase CI and CI on backports to REL branches
Release Version Compatibility	Develop on master, backport when needed
Test System	Any, but Beta Wikidata might come for free
Remarks	Very similar to 1, but: Enforces some additional separation from Wikibase legacy monolith, which could serve the "clean architecture" approach. The API code can likely not live within a package entirely, because certain things (API endpoint implementations, concrete service implementations, places where dependencies from Wikibase internals are injected) depend on Wikibase and/or MediaWiki Might result in business logic living within the package, and “details” such as concrete implementations and “infrastructure” outside of it, which may be confusing

Option 3: REST API in separate MediaWiki extension

Option Description	Development happens in a separate MediaWiki extension
VCS Repository	Either inside Wikibase.git or in a separate repository. Might benefit from Gerrit “Depends-On” features with Wikibase
CI System	(?) Run with Wikibase master on our master and use REL branch for compatibility test
Release Version Compatibility	Backport only required changes to Wikibase.git internals to REL branches (?) Develop against Wikibase.git master by default and have a REL branch for the API extension (might help with CI and reduces confusion)
Test System	Any
Remarks	Not very different from developing inside the Wikibase (repo) extension, but possibly nicer separation A little more freedom Possibly faster CI More boilerplate Free feature toggle Deploying on Wikidata would require an additional WMF security review

Option 4: REST API as a dependency (library) of MediaWiki/Wikibase

Option Description	Package the API as a somewhat isolated library that would be loaded as a composer library
VCS Repository	Anywhere
CI System	Any
Release Version Compatibility	Same as Option 2: Develop against master, backport when needed
Test System	Any
Remarks	Unclear, if an API can sensibly be built entirely as a library: eventually, some application code will be needed. Thus it is similar to Option 2, but with some additional constraints.

Option 5: REST API without binding to MediaWiki at all (separate process)

Option Description	Create the REST API as an entirely separate service that doesn’t live within MediaWiki/Wikibase.
VCS Repository	Anywhere
CI System	Any
Release Version Compatibility	Backport required changes to Wikibase or have a compatibility layer for the older versions within the API
Test System	Any
Remarks	Lots of freedom, but accessing data from Wikibase becomes tricky. Direct SQL access and/or using the existing action API both don’t sound appealing. Instead it would probably require an entirely new API/interface/protocol for accessing wikibase data, and building a new API on top of a new API doesn’t sound great either.

Decision

The team has decided to go with option 1 and develop the REST API inside the core Wikibase.git repository, using the subdirectory /repo/rest-api/.

Reasons:

The team came to a consensus, that the Wikibase REST API should be an integral part of the Wikibase Repo extension.
Different options of implementing this approach (option 1 to 4 above) seem to have only few differences between each other, mostly regarding restrictions in code organisation.
While we aim to have a clear architectural boundary between the REST API code and the Wikibase internals, delivering the code as a fully independent package/library (options 2 and 4) to Wikibase seems impractical. We still want to build concrete data access services on top of existing Wikibase and MediaWiki implementation, and benefit from MediaWiki infrastructure for service wiring, registering the REST API endpoints and extending API handler classes. In order to also keep the benefits of having the related code, tests, and configuration closely together, we will instead keep everything within a directory in Wikibase/repo/ and enforce the dependency rules by other means, such as architecture checks in the linter.
Previous experience with the "Clean Architecture" approach by the FUN team suggests not to be too strict with repository separation and rather take the pragmatic approach of staying within Wikibase.git.
For some of the options (e.g. option 5) it is not currently clear, how exactly they would end up looking in practice.

This decision for an MVP of the REST API does not mean, that alternative options are ruled out for all future. Changes in code organisation, git repos or the testing system will always remain possible.