Change propagation or dispatching is the process of updating Client sites with Repo changes / edits. This allows clients to update pages quickly after information on the repository changed.
NOTE:
- Change propagation is only possible with direct database access between the wikis (that is, inside a “wiki farm”).
- Change propagation does not support federation (change propagation between repositories) nor does it support multi-repository setups on the client wiki.
Change propagation requires several components to work together.
On the Repo, we need:
- Subscription management, so the repository knows which client wiki is interested in changes to which entities.
- A buffer of the changes themselves.
- Access to each client's job queue, to push EntityChangeNotificationJobs to.
On each Client, there needs to be:
- Usage tracking (see md_docs_topics_usagetracking).
- Access to sitelinks stored in the repository.
- ChangeHandler for processing changes on the repo, triggered by EntityChangeNotificationJobs being executed.
- AffectedPagesFinder, a mechanism to determine which pages are affected by which change, based on usage tracking information (see md_docs_topics_usagetracking).
- WikiPageUpdater, for updating the client wiki's state.
On the Repo
The main work on the Repo side of things is done by the DispatchChangesJob, which is scheduled at the end of the RecentChangeSaveHookHandler.
Usage Tracking and Subscription Management
Usage tracking and subscription management are described in detail in md_docs_topics_usagetracking.
Change Buffer
The change buffer holds information about each change, stored in the wb_changes table, to be accessed by the repo wiki when processing the respective changes.
On Clients
SiteLinkLookup
A SiteLinkLookup allows the client wiki to determine which local pages are “connected” to a given Item on the repository. Each client wiki can access the repo's sitelink information via a SiteLinkLookup service returned by ClientStore::getSiteLinkLookup(). This information is stored in the wb_items_per_site table in the repo's database.
ChangeHandler
The ChangeHandler::handleChanges() method gets called with a list of changes provided by a EntityChangeNotificationJobs. A ChangeRunCoalescer is then used to merge consecutive changes by the same user to the same entity, reducing the number of logical events to be processed on the client, and to be presented to the user.
ChangeHandler will then for each change determine the affected pages using the AffectedPagesFinder, which uses information from the wbc_entity_usage table (see md_docs_topics_usagetracking). It then uses a WikiPageUpdater to update the client wiki's state: rows are injected into the recentchanges database table, pages using the affected entity's data are re-parsed, and the web cache for these pages is purged.
WikiPageUpdater
The WikiPageUpdater class defines three methods for updating the client wikis state according to a given change on the repository:
- WikiPageUpdater::scheduleRefreshLinks()
- Will re-parse each affected page, allowing the link tables to be updated appropriately. This is done asynchronously using RefreshLinksJobs. No batching is applied, since RefreshLinksJobs are slow and this benefit more from deduplication than from batching.
- WikiPageUpdater::purgeWebCache()
- Will update the web-cache for each affected page. This is done asynchronously in batches, using HTMLCacheUpdateJob. The batch size is controlled by the purgeCacheBatchSize setting.
- WikiPageUpdater::injectRCRecords()
- Will create a RecentChange entry for each affected page. This is done asynchronously in batches, using InjectRCRecordsJobs. The batch size is controlled by the recentChangesBatchSize setting.
