Class: PegTokenizer

wt2html/tokenizer~PegTokenizer(env, options)

new PegTokenizer(env, options)

...
Parameters:
Name Type Description
env MWParserEnvironment
options Object
Source:

Extends

Methods

process()

Process text.

Process text. The text is tokenized in chunks and control is yielded to the event loop after each top-level block is tokenized enabling the tokenized chunks to be processed at the earliest possible opportunity.

Source:

setPipelineId()

Debugging aid: Set pipeline id.

Debugging aid: Set pipeline id.

Source:

setSourceOffsets()

Set start and end offsets of the source that generated this DOM.

Set start and end offsets of the source that generated this DOM.

Source:

tokenizeAs()

Tokenizes a string as a rule, otherwise returns an Error

.

Tokenizes a string as a rule, otherwise returns an Error

Source:

tokenizeAsync()

The main worker.

The main worker. Sets up event emission ('chunk' and 'end' events). Consumers are supposed to register with PegTokenizer before calling process().

Source:

Tokenize an extlink.

Tokenize an extlink.

Source:

tokenizesAsURL() → {boolean}

Tokenize a URL.

Tokenize a URL.

Source:
Returns:
Type
boolean

tokenizeSync()

Tokenize via a rule passed in as an arg.

Tokenize via a rule passed in as an arg. The text is tokenized synchronously in one shot.

Source:

tokenizeTableCellAttributes()

Tokenize table cell attributes.

Tokenize table cell attributes.

Source: