Scap¶
___ ____
⎛ ⎛ ,----
\ //==--'
_//| .·//==--' ____________________________
_OO≣=- ︶ ᴹw ⎞_§ ______ ___\ ___\ ,\__ \/ __ \
(∞)_, ) ( | ______/__ \/ /__ / /_/ / /_/ /
¨--¨|| |- ( / _______\___/ \___/ \__^_/ .__/
««_/ «_/ jgs/bd808 /_/
Scap is the deployment script used by Wikimedia Foundation to publish code and configuration on production web servers.
Running tests¶
Automated tests can be executed with make test.
Scap comes with an automated suite invoked via scripts/check. The script does not install any dependencies though.
For a list of Scap dependencies during testing, see requirements.txt and test-requirements.txt. Some Debian packages are also required to set up the running environment, for instance to install Python or pip.
To ensure compatiblity with multiple Debian distributions, we use Docker images defined using Blubber (see .pipeline/blubber.yaml). The image build installs the runtime dependencies from Debian packages while tests dependencies defined in test-requirements.txt are installed via pip. This lets us use ensure we run with the same set of Python module provided by Debian while using more recent versions of testing utilities than the one frozen by Debian (ex: pytest) and ensure we use the same version of flake8 regardless of the Debian distribution version.
To generate the container images and run tests in each of them, we provide a Makefile. To run all tests against all supported Debian distributions use make test.
The test target builds the images using the Blubberfile syntax <https://wikitech.wikimedia.org/wiki/Blubber/User_Guide#Blubberfiles> using Blubber buildkit <docker-registry.wikimedia.org/repos/releng/blubber/buildkit>. The target then invokes the defined entrypoint scripts/check which runs the tests in each images.
The Wikimedia CI builds those images in a similar way and run the same tests.
To build all images without running tests, use make images, useful when amending the Blubber config or changing dependencies.
For each Debian distribution defined at the top of the makefile, rules let you build and test against a single distribution: make test-bullseye. You will probably want to use it when developing instead of testing against each of the distribution. When your change is ready, run it against all distributions to catch potential compatibility issues (make test).
Reporting Issues¶
You can report issues to the #scap project on phabricator.wikimedia.org
Credits¶
ASCII art derived from original work by Joan Stark 1 and the speed figlet font 2.
License¶
Background¶
Scap is a tool that was originally designed to deploy MediaWiki code on Wikimedia Foundation servers. In Summer/Fall 2015 functionality was added to Scap to allow it to deploy any git-based repositories on our deployment server.
What does “Scap” mean?¶
In about 2004, in the course of moving the wiki document roots from NFS to the local hard drives of the Apache servers, a number of shell scripts were introduced, probably written by Brion, notably:
sync-common, which pulled the shared code into the local server
sync-common-all, which ran sync-common on all servers
Tim added a script which would push the MW source code in common/php out to all servers, in addition to running a pre-deployment lint check, in order to reduce the (then frequent) incidents of site downtime due to PHP fatal errors. The natural name for this script, following the previous convention, would have been sync-common-php-all, but this had two problems:
It was too long
Its acronym was not pronounceable
Tim noticed that if the last two components of the name were swapped, a short and catchy acronym could be contrived. Thus, Scap was born.