Skip to content

can token_* be made faster? #16656

@staabm

Description

@staabm

Description

The following code:

https://3v4l.org/WAaL7

<?php

$start=hrtime(true); 

$code = '<?php if (PHP_INT_SIZE != 4) die("skip this test is for 32bit platform only");';
$tokens = token_get_all($code);

foreach ($tokens as $token) {
    if (is_array($token)) {
        echo "Line {$token[2]}: ", token_name($token[0]), " ('{$token[1]}')", PHP_EOL;
    }
}

$end=hrtime(true);
$eta=$end-$start;

echo $eta/1e+6; //nanoseconds to milliseconds

Resulted in this output:

...
0.116826

I feel this tokenization of small code samples is a bit slow and wonder whether there is some low hanging fruits which can be optimized?

I am using such code to optimize the performance of the PHPT Testrunne in PHPUnit via https://github.com/staabm/side-effects-detector

PHP Version

8.3.13

Operating System

ubuntu 24

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions