Scap¶
___ ____
⎛ ⎛ ,----
\ //==--'
_//| .·//==--' ____________________________
_OO≣=- ︶ ᴹw ⎞_§ ______ ___\ ___\ ,\__ \/ __ \
(∞)_, ) ( | ______/__ \/ /__ / /_/ / /_/ /
¨--¨|| |- ( / _______\___/ \___/ \__^_/ .__/
««_/ «_/ jgs/bd808 /_/
Scap is the deployment script used by Wikimedia Foundation to publish code and configuration on production web servers.
Running tests¶
Automated tests can be executed with make test or tox -e test.
For a list of Scap dependencies during testing, see requirements.txt and test-requirements.txt. Some Debian packages are also required to set up the running environment, for instance to install Python or pip.
To ensure compatiblity with multiple Debian distributions, we use Docker images defined using Blubber (see .pipeline/blubber.yaml). The image build installs the runtime dependencies from Debian packages while tests dependencies defined in test-requirements.txt are installed via pip. This lets us use ensure we run with the same set of Python module provided by Debian while using more recent versions of testing utilities than the one frozen by Debian (ex: pytest) and ensure we use the same version of flake8 regardless of the Debian distribution version.
To generate the container images and run tests in each of them, we provide a Makefile. To run tests against the default Debian distribution (defined by DEFAULT_VARIANT’ in `Makefile’), use `make test. To run all tests against all supported Debian distributions use make test-all. To run tests against a specific supported Debian distribution, use make test-<distro>, for example make test-bullseye.
The test* targets build the images using the Blubberfile syntax <https://wikitech.wikimedia.org/wiki/Blubber/User_Guide#Blubberfiles> using Blubber buildkit <docker-registry.wikimedia.org/repos/releng/blubber/buildkit>. The blubberfile is configured to run tests using tox at image build time.
The Wikimedia CI builds those images in a similar way and runs the same tests.
To build all images without running tests, use make images, useful when amending the Blubber config or changing dependencies.
Reporting Issues¶
You can report issues to the #scap project on phabricator.wikimedia.org
Credits¶
ASCII art derived from original work by Joan Stark [1] and the speed figlet font [2].
License¶
Background¶
Scap is a tool that was originally designed to deploy MediaWiki code on Wikimedia Foundation servers. In Summer/Fall 2015 functionality was added to Scap to allow it to deploy any git-based repositories on our deployment server.
What does “Scap” mean?¶
In about 2004, in the course of moving the wiki document roots from NFS to the local hard drives of the Apache servers, a number of shell scripts were introduced, probably written by Brion, notably:
sync-common, which pulled the shared code into the local server
sync-common-all, which ran sync-common on all servers
Tim added a script which would push the MW source code in common/php out to all servers, in addition to running a pre-deployment lint check, in order to reduce the (then frequent) incidents of site downtime due to PHP fatal errors. The natural name for this script, following the previous convention, would have been sync-common-php-all, but this had two problems:
It was too long
Its acronym was not pronounceable
Tim noticed that if the last two components of the name were swapped, a short and catchy acronym could be contrived. Thus, Scap was born.