Parsoid
A bidirectional parser between wikitext and HTML5
Loading...
Searching...
No Matches
Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher Class Reference

This class is an attempt to fixup the token stream to reparse strings as tokens that failed to parse in the tokenizer because of SOL or other constraints OR because tags were being constructed in pieces or whatever. More...

+ Inheritance diagram for Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher:
+ Collaboration diagram for Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher:

Public Member Functions

 __construct (TokenTransformManager $manager, array $options)
 
 resetState (array $parseOpts)
 Resets any internal state for this token handler.
 
 onNewline (NlTk $token)
 This handler is called for newline tokens only.
Parameters
NlTk$tokenNewline token to be processed
Returns
TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.

 
 onEnd (EOFTk $token)
 This handler is called for EOF tokens only.
Parameters
EOFTk$tokenEOF token to be processed
Returns
TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.

 
 onAny ( $token)
 This handler is called for all tokens in the token stream except if (a) The more specific handlers above modified the token (b) the more specific handlers (onTag, onEnd, onNewline) have set the skip flag in their return values.(c) this handlers 'active' flag is set to false (can be set by any of the handlers).
Parameters
Token | string$tokenToken to be processed
Returns
TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.

 
 onAnyInternal ( $token)
 
- Public Member Functions inherited from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler
 setPipelineId (int $id)
 
 isDisabled ()
 Is this transformer disabled?
 
 onTag (Token $token)
 This handler is called for tokens that are not EOFTk or NLTk tokens.
 
 process ( $tokens)
 Push an input array of tokens through the transformer and return the transformed tokens.
 

Additional Inherited Members

- Protected Attributes inherited from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler
 $env
 
 $manager
 
 $pipelineId
 
 $options
 
bool $disabled = false
 This is set if the token handler is disabled for the entire pipeline.
 
bool $onAnyEnabled = true
 This is set/reset by the token handlers at various points in the token stream based on what is encountered.
 
 $atTopLevel = false
 

Detailed Description

This class is an attempt to fixup the token stream to reparse strings as tokens that failed to parse in the tokenizer because of SOL or other constraints OR because tags were being constructed in pieces or whatever.

This is a pure hack to improve compatibility with the core parser given that we dont have a preprocessor. This will be a grab-bag of heuristics and tricks to handle different scenarios.

Constructor & Destructor Documentation

◆ __construct()

Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::__construct ( TokenTransformManager  $manager,
array  $options 
)
Parameters
TokenTransformManager$managerThe manager for this stage of the parse.
array$optionsAny options for the expander.

Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.

Member Function Documentation

◆ onAny()

Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::onAny (   $token)

This handler is called for all tokens in the token stream except if (a) The more specific handlers above modified the token (b) the more specific handlers (onTag, onEnd, onNewline) have set the skip flag in their return values.(c) this handlers 'active' flag is set to false (can be set by any of the handlers).

Parameters
Token | string$tokenToken to be processed
Returns
TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.

Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.

◆ onAnyInternal()

Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::onAnyInternal (   $token)
Parameters
mixed$token
Returns
?TokenHandlerResult

◆ onEnd()

Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::onEnd ( EOFTk  $token)

This handler is called for EOF tokens only.

Parameters
EOFTk$tokenEOF token to be processed
Returns
TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.

Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.

◆ onNewline()

Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::onNewline ( NlTk  $token)

This handler is called for newline tokens only.

Parameters
NlTk$tokenNewline token to be processed
Returns
TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.

Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.

◆ resetState()

Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::resetState ( array  $parseOpts)

Resets any internal state for this token handler.

Parameters
array$parseOpts

Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.


The documentation for this class was generated from the following file: