This class is an attempt to fixup the token stream to reparse strings as tokens that failed to parse in the tokenizer because of SOL or other constraints OR because tags were being constructed in pieces or whatever. More...
Inheritance diagram for Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher:
Collaboration diagram for Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher:
Public Member Functions | ||||
| __construct (TokenTransformManager $manager, array $options) | ||||
| resetState (array $parseOpts) | ||||
| Resets any internal state for this token handler. | ||||
| onNewline (NlTk $token) | ||||
This handler is called for newline tokens only.
| ||||
| onEnd (EOFTk $token) | ||||
This handler is called for EOF tokens only.
| ||||
| onAny ( $token) | ||||
This handler is called for all tokens in the token stream except if (a) The more specific handlers above modified the token (b) the more specific handlers (onTag, onEnd, onNewline) have set the skip flag in their return values.(c) this handlers 'active' flag is set to false (can be set by any of the handlers).
| ||||
| onAnyInternal ( $token) | ||||
| Public Member Functions inherited from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler | ||||
| setPipelineId (int $id) | ||||
| isDisabled () | ||||
| Is this transformer disabled? | ||||
| onTag (Token $token) | ||||
| This handler is called for tokens that are not EOFTk or NLTk tokens. | ||||
| process ( $tokens) | ||||
| Push an input array of tokens through the transformer and return the transformed tokens. | ||||
Additional Inherited Members | |
| Protected Attributes inherited from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler | |
| $env | |
| $manager | |
| $pipelineId | |
| $options | |
| bool | $disabled = false |
| This is set if the token handler is disabled for the entire pipeline. | |
| bool | $onAnyEnabled = true |
| This is set/reset by the token handlers at various points in the token stream based on what is encountered. | |
| $atTopLevel = false | |
Detailed Description
This class is an attempt to fixup the token stream to reparse strings as tokens that failed to parse in the tokenizer because of SOL or other constraints OR because tags were being constructed in pieces or whatever.
This is a pure hack to improve compatibility with the core parser given that we dont have a preprocessor. This will be a grab-bag of heuristics and tricks to handle different scenarios.
Constructor & Destructor Documentation
◆ __construct()
| Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::__construct | ( | TokenTransformManager | $manager, |
| array | $options ) |
- Parameters
-
TokenTransformManager $manager The manager for this stage of the parse. array $options Any options for the expander.
Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.
Member Function Documentation
◆ onAny()
| Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::onAny | ( | $token | ) |
This handler is called for all tokens in the token stream except if (a) The more specific handlers above modified the token (b) the more specific handlers (onTag, onEnd, onNewline) have set the skip flag in their return values.(c) this handlers 'active' flag is set to false (can be set by any of the handlers).
- Parameters
-
Token | string $token Token to be processed
- Returns
- TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.
Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.
◆ onAnyInternal()
| Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::onAnyInternal | ( | $token | ) |
- Parameters
-
mixed $token
- Returns
- ?TokenHandlerResult
◆ onEnd()
| Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::onEnd | ( | EOFTk | $token | ) |
This handler is called for EOF tokens only.
- Parameters
-
EOFTk $token EOF token to be processed
- Returns
- TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.
Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.
◆ onNewline()
| Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::onNewline | ( | NlTk | $token | ) |
This handler is called for newline tokens only.
- Parameters
-
NlTk $token Newline token to be processed
- Returns
- TokenHandlerResult|null A TokenHandlerResult, or null to efficiently indicate that the input token is unchanged.
Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.
◆ resetState()
| Wikimedia\Parsoid\Wt2Html\TT\TokenStreamPatcher::resetState | ( | array | $parseOpts | ) |
Resets any internal state for this token handler.
- Parameters
-
array $parseOpts
Reimplemented from Wikimedia\Parsoid\Wt2Html\TT\TokenHandler.
The documentation for this class was generated from the following file:
- src/Wt2Html/TT/TokenStreamPatcher.php
