UtfNormalTest2.php File Reference

Other tests for the unicode normalization module. More...

Go to the source code of this file.

Functions
	getRow ( $f)
	my_assert ( $file, $line, $code)
	normalize_form_c ($c)
	The following section will be used for testing different normalization methods. More...
	normalize_form_c_php ($c)
	This set of functions is only useful if youve added a param to the following functions to force pure PHP usage. More...
	normalize_form_d ($c)
	normalize_form_d_php ($c)
	normalize_form_kc ($c)
	normalize_form_kc_php ($c)
	normalize_form_kd ($c)
	normalize_form_kd_php ($c)
	unichr ($c)
	unistr ($c)

Variables
	$count = 0
	$f = fopen($file, "r")
if(PHP_SAPI !='cli')	$file = "NormalizationTest.txt"
	$lineNo = 0
const	COMMENT '#'
	if ($f !==false)
const	SEPARATOR ';'

Detailed Description

Other tests for the unicode normalization module.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. http://www.gnu.org/copyleft/gpl.html

Definition in file UtfNormalTest2.php.

Function Documentation

getRow()

getRow

(

$f

)

Definition at line 231 of file UtfNormalTest2.php.

References $f, $ret, array(), as, COMMENT, SEPARATOR, and unistr().

my_assert()

my_assert	(		$file,
			$line,
			$code
	)

Definition at line 91 of file UtfNormalTest2.php.

References $lineNo, and global.

normalize_form_c()

normalize_form_c

(

$c

)

The following section will be used for testing different normalization methods.

• Pure PHP ~ no assertion errors ~ 6.25 minutes
• php_utfnormal.so or intl extension: both are wrappers around libicu so we list the version of libicu when making the comparison
• libicu Ubuntu 3.8.1-3ubuntu1.1 php 5.2.6-3ubuntu4.5 ~ 2200 assertion errors ~ 5 seconds ~ output: http://paste2.org/p/921566
• libicu Ubuntu 4.2.1-3 php 5.3.2-1ubuntu4.2 ~ 1384 assertion errors ~ 15 seconds ~ output: http://paste2.org/p/921435
• libicu Debian 4.4.1-5 php 5.3.2-1ubuntu4.2 ~ no assertion errors ~ 13 seconds
• Tests comparing pure PHP output with libicu output were added later and slow down the runtime.

Definition at line 69 of file UtfNormalTest2.php.

References UtfNormal\toNFC().

normalize_form_c_php()

normalize_form_c_php

(

$c

)

This set of functions is only useful if youve added a param to the following functions to force pure PHP usage.

I decided not to commit that code since might produce a slowdown in the UTF normalization code just for the sake of these tests. – hexmode

Returns

string

Definition at line 81 of file UtfNormalTest2.php.

References UtfNormal\toNFC().

normalize_form_d()

normalize_form_d

(

$c

)

Definition at line 70 of file UtfNormalTest2.php.

References UtfNormal\toNFD().

normalize_form_d_php()

normalize_form_d_php

(

$c

)

Definition at line 82 of file UtfNormalTest2.php.

References UtfNormal\toNFD().

normalize_form_kc()

normalize_form_kc

(

$c

)

Definition at line 71 of file UtfNormalTest2.php.

References UtfNormal\toNFKC().

normalize_form_kc_php()

normalize_form_kc_php

(

$c

)

Definition at line 83 of file UtfNormalTest2.php.

References UtfNormal\toNFKC().

normalize_form_kd()

normalize_form_kd

(

$c

)

Definition at line 72 of file UtfNormalTest2.php.

References UtfNormal\toNFKD().

normalize_form_kd_php()

normalize_form_kd_php

(

$c

)

Definition at line 84 of file UtfNormalTest2.php.

References UtfNormal\toNFKD().

unichr()

unichr

(

$c

)

Definition at line 210 of file UtfNormalTest2.php.

unistr()

unistr

(

$c

)

Definition at line 227 of file UtfNormalTest2.php.

Referenced by getRow().

Variable Documentation

$count

$count = 0

Definition at line 96 of file UtfNormalTest2.php.

Referenced by RedisLockManager\__destruct(), ApiQueryORM\addResults(), Benchmarker\bench(), ImageBuilder\buildTable(), Profiler\calltreeCount(), CheckStorage\check(), UserDupes\checkDupes(), NamespaceConflictChecker\checkNamespace(), DumpMultiWriter\closeAndRename(), LanguageKsh\convertPlural(), Language\convertPlural(), SpecialWatchlist\countItems(), Sanitizer\decodeCharReferencesAndNormalize(), CategoryViewer\doCategoryQuery(), FixExtLinksProtocolRelative\doDBUpdates(), PopulateParentId\doDBUpdates(), DeleteArchivedFilesImplementation\doDelete(), DeleteArchivedRevisionsImplementation\doDelete(), DeleteOldRevisions\doDelete(), JobQueueDB\doGetAbandonedCount(), JobQueueDB\doGetAcquiredCount(), SpecialRecentChanges\doHeader(), PopulateRevisionLength\doLenUpdates(), PopulateRevisionSha1\doSha1LegacyUpdates(), PopulateRevisionSha1\doSha1Updates(), MovePageForm\doSubmit(), ApiQueryTags\doTag(), MysqlUpdater\doTemplatelinksUpdate(), HTMLCacheUpdate\doUpdate(), DatabaseBase\errorCount(), StorageTypeStats\execute(), CheckBadRedirects\execute(), ApiQueryExternalLinks\execute(), ApiQueryLangLinks\execute(), ApiQueryDeletedrevs\execute(), ApiQueryIWLinks\execute(), MigrateUserGroup\execute(), ApiQueryFilearchive\execute(), DeleteOrphanedRevisions\execute(), NukePage\execute(), RemoveUnusedAccounts\execute(), ApiFeedContributions\execute(), CleanupSpam\execute(), ApiQueryBlocks\execute(), ApiQueryContributions\execute(), ApiQueryLogEvents\execute(), ApiQueryPagePropNames\execute(), RefreshFileHeaders\execute(), RenameDbPrefix\execute(), ShowJobs\execute(), ApiQueryImageInfo\execute(), ApiQueryAllUsers\execute(), ApiQueryContributors\execute(), RunJobs\execute(), NukeNS\execute(), CopyFileBackend\execute(), UpdateCollation\execute(), ApiQueryRevisions\execute(), SpecialRunJobs\executeJobs(), NextJobDB\executeReadyPeriodicTasks(), JobQueueGroup\executeReadyPeriodicTasks(), PPFrame_DOM\expand(), lessc\expandParentSelectors(), TextPassDumper\finalOptionCheck(), ConfEditor\findDeletionRegion(), CdbWriterPHP\finish(), MostcategoriesPage\formatResult(), MostinterwikisPage\formatResult(), ActiveUsersPager\formatRow(), CategoryPager\formatRow(), CreditsAction\getContributors(), InfoAction\getContributors(), JobQueueFederated\getCrossPartitionSum(), User\getEditCount(), DumpMultiWriter\getFilenames(), DeleteLogFormatter\getMessageParameters(), SpecialEditWatchlist\getNormalForm(), BacklinkCache\getNumLinks(), WantedPagesPage\getQueryInfo(), RandomImageGenerator\getRandomWordPairs(), UIDGenerator\getSequentialPerNodeIDs(), Language\handleExplicitPluralForms(), SearchHighlighter\highlightText(), ImagePage\imageLinks(), LoginForm\incLoginThrottle(), StatCounter\incr(), DatabaseOracle\indexExists(), ImageBuilder\init(), User\initEditCount(), RemoveUnusedAccounts\isInactiveAccount(), ProfilerMwprof\logData(), Fallback\mb_substr(), SpecialMergeHistory\merge(), Title\moveSubpages(), moveToExternal(), lessc\multiplySelectors(), UIDGenerator\newSequentialPerNodeIDs(), ChangesList\numberofWatchingusers(), ImagePage\openShowImage(), SpecialSpecialpages\outputPageList(), CheckLanguageCLI\outputText(), PurgeChangedPages\pageableSortedRows(), CoreParserFunctions\pagesincategory(), lessc_parser\peek(), PPFuzzTester\pickEntryPoint(), User\pingLimiter(), RebuildTextIndex\populateSearchIndex(), Language\preConvertPlural(), Preprocessor_Hash\preprocessToObj(), ImageBuilder\progress(), PurgeRedundantText(), Maintenance\purgeRedundantText(), ApiQueryContinueTestBase\query(), AllmessagesTablePager\reallyDoQuery(), EnhancedChangesList\recentChangesBlockGroup(), JobQueueDB\recycleAndDeleteStaleJobs(), JobQueueRedis\recyclePruneAndUndelayJobs(), lessc_parser\removeComments(), DbTestPreviewer\report(), ApiQueryProtectedTitles\run(), ApiQueryRedirects\run(), ApiQueryIWBacklinks\run(), ApiQueryImages\run(), ApiQueryLangBacklinks\run(), ApiQueryWatchlistRaw\run(), ApiQueryExtLinksUsage\run(), ApiQueryDuplicateFiles\run(), ApiQueryCategories\run(), ApiQueryAllCategories\run(), ApiQueryCategoryMembers\run(), ApiQueryPagesWithProp\run(), ApiQueryWatchlist\run(), ApiQuerySearch\run(), ApiQueryQueryPage\run(), ApiQueryAllPages\run(), ApiQueryAllImages\run(), ApiQueryLinks\run(), ApiQueryRandom\run(), ApiQueryAllLinks\run(), ApiQueryRecentChanges\run(), ApiQueryBacklinks\run(), ApiQueryRandom\runQuery(), lessc_parser\seek(), StatCounter\sendDeltasMemc(), StatCounter\sendDeltasUDP(), Maintenance\shouldExecute(), UserrightsPage\showEditUserGroupsForm(), MovePageForm\showSubpages(), SpecialAllpages\showToplevel(), SpecialUpload\showViewDeletedLinks(), SearchHighlighter\splitAndAdd(), SevenZipStream\stream_read(), UploadSourceAdapter\stream_read(), ApiEditPageTest\testEditAppend(), ApiEditPageTest\testEditConflict(), ApiEditPageTest\testEditConflict_bug41990(), ApiEditPageTest\testEditConflict_redirect(), SiteTest\testSetAndRemovePath(), GenericArrayObjectTest\testUnset(), lessc_parser\throwError(), lessc_parser\unit(), UserOptions\USAGER(), Exif\validate(), lessc_parser\whitespace(), DumpMultiWriter\writeClosePage(), DumpMultiWriter\writeCloseStream(), DumpMultiWriter\writeOpenPage(), DumpMultiWriter\writeOpenStream(), and DumpMultiWriter\writeRevision().

$f

$f = fopen($file, "r")

Definition at line 38 of file UtfNormalTest2.php.

Referenced by MWMemcached\_flush_read_buffer(), LinkCache\addLinkObj(), WikiRevision\downloadSource(), ForeignAPIRepo\fileExistsBatch(), findAuxFile(), JSParser\FunctionDefinition(), HtmlFormatterTest\getHtmlData(), BmpHandler\getImageSize(), getRow(), Maintenance\getStdin(), XCFHandler\getXCFMetaData(), JSParser\parse(), ConverterRule\parseFlags(), ChangesList\recentChangesFlags(), lessc\reduce(), ImageGalleryBase\setShowBytes(), ImageGalleryBase\setShowFilename(), StoreBatchTest\storecohort(), FileRepoTest\testFileRepoConstructionWithRequiredOptions(), TextContentTest\testGetParserOutput(), IPTest\testInvalidIPs(), GlobalTest\testReadOnlySet(), IPTest\testValidIPs(), and SpecialWhatLinksHere\whatlinkshereForm().

$file

if (PHP_SAPI !='cli') $file = "NormalizationTest.txt"

Definition at line 30 of file UtfNormalTest2.php.

Referenced by ExtensionsTestSuite\__construct(), IncludableSpecialPage\__construct(), UnlistedSpecialPage\__construct(), DumpDBZip2Output\__construct(), SpecialPage\__construct(), DumpFileOutput\__construct(), DumpPipeOutput\__construct(), DumpGZipOutput\__construct(), DumpBZip2Output\__construct(), Dump7ZipOutput\__construct(), ImageBuilder\addMissingImage(), lessc\addParsedFile(), OutputPage\addScriptFile(), DatabaseSqlite\attachDatabase(), SkinTemplate\buildContentNavigationUrls(), OutputPage\buildCssLinksArray(), GIFHandler\canAnimateThumbnail(), ImageHandler\canRender(), TiffHandler\canRender(), AutoLoaderTest\checkAutoLoadConf(), Sqlite\checkSqlSyntax(), UploadBase\checkXMLEncodingMissmatch(), ImagePage\delete(), LocalFile\delete(), ApiDelete\deleteFile(), UploadFromUrlTest\deleteFile(), ApiTestCaseUpload\deleteFileByTitle(), UploadFromUrlTestSuite\deleteFiles(), FileBackendTest\deleteFiles(), UploadBase\detectScript(), UploadBase\detectVirus(), Installer\dirIsExecutable(), SpecialRedirect\dispatchFile(), PopulateImageSha1\doDBUpdates(), TestFileOpPerformance\doPerfTest(), RevDel_FileList\doPostCommitUpdates(), MovePageForm\doSubmit(), FileBackendTest\doTestDoQuickOperations(), FileBackendTest\doTestGetDirectoryList(), FileBackendTest\doTestGetFileList(), ResourceLoaderLESSFunctions\embed(), ResourceLoaderLESSFunctions\embeddable(), CSSMin\encodeImageAsDataURI(), ApiQueryStashImageInfo\execute(), CheckImages\execute(), CountMessages\execute(), ApiQueryFilearchive\execute(), MwSql\execute(), UpdateSpecialPages\execute(), PatchSql\execute(), UploadStashCleanup\execute(), EraseArchivedFile\execute(), GenerateNormalizerDataAr\execute(), RefreshFileHeaders\execute(), DeleteBatch\execute(), GenerateUtf8Case\execute(), ApiImageRotate\execute(), UpdateSearchIndex\execute(), SpecialVersion\execute(), MoveBatch\execute(), RefreshImageMetadata\execute(), UpdateMediaWiki\execute(), SpecialUndelete\execute(), FormatMetadata\fetchExtendedMetadata(), FileRepo\fileExists(), ForeignAPIRepo\fileExistsBatch(), FileRepo\fileExistsBatch(), CoreParserFunctions\filepath(), UploadDumper\filterItem(), findAuxFile(), LocalRepo\findBySha1s(), Installer\findExtensions(), RepoGroup\findFileFromKey(), findFiles(), LocalRepo\findFiles(), FileRepo\findFiles(), lessc\findImport(), ApiParse\formatCss(), SpecialUndelete\formatFileRow(), SvgHandler\formatMetadata(), ImageListPager\formatValue(), GenerateCollationData\generateFirstChars(), SvgHandler\getAvailableLanguages(), FindHooks\getBadHooksFromFile(), FindHooks\getBadHooksFromPath(), MWDebug\getCallerDescription(), ExifBitmapHandler\getCommonMetaArray(), SvgHandler\getCommonMetaArray(), PurgeChangedFiles\getDeletedPath(), UploadForm\getDescriptionSection(), ImageHandler\getDimensionsString(), SpecialUpload\getDupeWarning(), WikiFilePage\getDuplicates(), MediaHandler\getEntireText(), SpecialUpload\getExistsWarning(), SpecialVersion\getExtAuthorsFileName(), FormatMetadata\getExtendedMetadataFromFile(), FormatMetadata\getExtendedMetadataFromHook(), SpecialVersion\getExtLicenseFileName(), SpecialUndelete\getFileComment(), getFileCommentFromSourceWiki(), SpecialUndelete\getFileLink(), WebRequest\getFileName(), MWDebug\getFilesIncluded(), WebRequest\getFileTempname(), SpecialUndelete\getFileUser(), getFileUserFromSourceWiki(), WikiFilePage\getForeignCategories(), MediaHandler\getGeneralLongDesc(), MediaHandler\getGeneralShortDesc(), FindHooks\getHooksFromFile(), FindHooks\getHooksFromPath(), UploadBase\getImageInfo(), SvgHandler\getImageSize(), ApiQueryImageInfo\getInfo(), MessageCache\getLocalCache(), CSSMin\getLocalFileReferences(), MWExceptionHandler\getLogMessage(), ImageHandler\getLongDesc(), SvgHandler\getLongDesc(), MediaHandler\getLongDesc(), Language\getMessagesFileName(), CSSMin\getMimeType(), CountMessages\getNumMessages(), SpecialFilepath\getRedirect(), ExifBitmapHandler\getRotation(), ResourceLoaderFileModule\getScriptURLsForDebug(), ImageHandler\getShortDesc(), MediaHandler\getShortDesc(), Skin\getSkinNames(), SpecialPageAliasTest\getSpecialPageAliases(), ResourceLoaderFileModule\getStyleURLsForDebug(), ForeignAPIFile\getThumbnails(), LocalFile\getThumbnails(), DjVuHandler\getUnserializedMetadata(), WebRequest\getUploadError(), ImageBuilder\imageCallback(), ImagePage\imageDupes(), BackupReader\importFromFile(), BackupReader\importFromStdin(), WikiRevision\importUpload(), UploadStash\initFile(), MediaWiki\initializeArticle(), MWCryptRand\initialRandomState(), SvgHandler\isAnimatedImage(), Title\isValidMoveTarget(), ResourceLoaderLESSTest\lessProvider(), RunJobs\loadBackoffs(), BackupDumper\loadPlugin(), Installer\locateExecutable(), RecompressTracked\logToFile(), Linker\makeImageLink(), Linker\makeImageLink2(), Linker\makeMediaLinkFile(), Linker\makeThumbLink2(), Linker\makeThumbLinkObj(), Title\moveTo(), BitmapHandler\mustRender(), ImportStreamSource\newFromFile(), ForeignDBFile\newFromRow(), OldLocalFile\newFromRow(), LocalFile\newFromRow(), ArchivedFile\newFromRow(), ImportStreamSource\newFromURL(), ImageBuilder\oldimageCallback(), UploadDumper\outputItem(), SpecialUploadStash\outputLocalFile(), SpecialUploadStash\outputLocallyScaledThumb(), SpecialUploadStash\outputRemoteScaledThumb(), SpecialUploadStash\outputThumbFromStash(), SpecialUploadStash\parseKey(), ApiUpload\performUpload(), Linker\processResponsiveImages(), ResourcesTest\provideMediaStylesheets(), ResourcesTest\provideResourceFiles(), XMPTest\provideXMPParse(), PruneFileCache\prune_directory(), FileRepo\publishBatch(), PurgeChangedFiles\purgeFromArchiveTable(), PurgeChangedFiles\purgeFromLogType(), LocalFile\purgeOldThumbnails(), LocalFile\purgeThumbList(), ForeignAPIFile\purgeThumbnails(), LocalFile\purgeThumbnails(), ResourceLoaderFileModule\readStyleFiles(), CSSMin\remapOne(), CheckStorage\restoreText(), JpegHandler\rotate(), BitmapHandler\rotate(), ApiQueryDuplicateFiles\run(), ApiQueryAllImages\run(), MessageCache\saveToLocal(), WikiFilePage\setFile(), ImagePage\setFile(), OutputPage\setFileVersion(), Dump7ZipOutput\setup7zCommand(), SqliteInstaller\setupDatabase(), SpecialUndelete\showFileConfirmationForm(), SpecialUploadStash\showUploads(), UploadBase\stashFile(), FileRepo\storeBatch(), RunJobs\syncBackoffs(), FileBackendTest\tearDownFiles(), UploadStashTest\testBug29408(), GIFHandlerTest\testGetImageArea(), PNGHandlerTest\testGetImageArea(), SvgTest\testGetIndependentMetaArray(), JpegTest\testGetIndependentMetaArray(), GIFHandlerTest\testGetIndependentMetaArray(), PNGHandlerTest\testGetIndependentMetaArray(), GIFHandlerTest\testGetMetadata(), PNGHandlerTest\testGetMetadata(), DjVuTest\testGetPageDimensions(), DjVuTest\testGetPageText(), BitmapScalingTest\testImageArea(), FormatMetadataTest\testInvalidDate(), JpegTest\testInvalidFile(), GIFHandlerTest\testIsAnimanted(), PNGHandlerTest\testIsAnimanted(), JpegTest\testJpegMetadataExtraction(), ExifRotationTest\testMetadata(), ExifRotationTest\testMetadataNoAutoRotate(), BitmapScalingTest\testNormaliseParams(), DjVuTest\testPageCount(), ExifRotationTest\testRotationRendering(), ExifRotationTest\testRotationRenderingNoAutoRotate(), BitmapScalingTest\testTooBigImage(), BitmapScalingTest\testTooBigMustRenderImage(), JpegMetadataExtractorTest\testUtf8Comment(), LocalFileTest\testWfLocalFile(), Language\transformUsingPairFile(), RefreshFileHeaders\updateFileHeaders(), Title\validateFileMoveOperation(), XmlTypeCheck\validateFromFile(), wfExtractThumbParams(), and XmlDumpWriter\writeUpload().

$lineNo

$lineNo = 0

Definition at line 97 of file UtfNormalTest2.php.

Referenced by my_assert().

COMMENT

const COMMENT '#'

Definition at line 33 of file UtfNormalTest2.php.

Referenced by getRow().

if

if( $f !==false)

Definition at line 98 of file UtfNormalTest2.php.

SEPARATOR

const SEPARATOR ';'

Definition at line 36 of file UtfNormalTest2.php.

Referenced by getRow().