require() with >2GB

[thomas-0816]

Description

The following code:

<?php

ini_set('display_errors', 1);
ini_set('memory_limit', '10G');

$fp = fopen('/tmp/test.php', 'w');
fwrite($fp, '<?php return "');
for ($i=0; $i < 2048; $i++) {
    fwrite($fp, str_repeat('#', 1024*1024));
}
fwrite($fp, '";');
fclose($fp);

$a = require '/tmp/test.php';
echo strlen($a);

Resulted in this output:

0

But I expected this output instead:

2147483664 (or similar)

PHP Version

PHP 8.4.11 (cli) (built: Aug  3 2025 08:42:03) (NTS)
Copyright (c) The PHP Group
Built by Debian
Zend Engine v4.4.11, Copyright (c) Zend Technologies
    with Zend OPcache v8.4.11, Copyright (c), by Zend Technologies

Operating System

Ubuntu 22.04.5 LTS

[iluuu1994]

Hi @thbley. Is this a real-world issue? It happens because of an unsigned -> signed overflow, but even fixing that will break again at 4GB. The real fix is simple (switching to 64-bit integers), but it's ABI-breaking and thus is not possible for <=8.4. Might be worth adjusting for 8.5 at least. /cc @php/release-managers-85

Details

diff --git a/Zend/zend_globals.h b/Zend/zend_globals.h
index 48b978b5350..a845a68dbc3 100644
--- a/Zend/zend_globals.h
+++ b/Zend/zend_globals.h
@@ -357,7 +357,7 @@ struct _zend_php_scanner_globals {
 	zend_file_handle *yy_in;
 	zend_file_handle *yy_out;
 
-	unsigned int yy_leng;
+	size_t yy_leng;
 	unsigned char *yy_start;
 	unsigned char *yy_text;
 	unsigned char *yy_cursor;
diff --git a/Zend/zend_language_scanner.h b/Zend/zend_language_scanner.h
index 612c8454792..393d5f127ba 100644
--- a/Zend/zend_language_scanner.h
+++ b/Zend/zend_language_scanner.h
@@ -23,7 +23,7 @@
 #include "zend_globals.h"
 
 typedef struct _zend_lex_state {
-	unsigned int yy_leng;
+	size_t yy_leng;
 	unsigned char *yy_start;
 	unsigned char *yy_text;
 	unsigned char *yy_cursor;
diff --git a/Zend/zend_language_scanner.l b/Zend/zend_language_scanner.l
index d298ae8b9ea..e6d67f91d3f 100644
--- a/Zend/zend_language_scanner.l
+++ b/Zend/zend_language_scanner.l
@@ -64,7 +64,7 @@
 #define yytext       ((char*)SCNG(yy_text))
 #define yyleng       SCNG(yy_leng)
 #define yyless(x)    do { YYCURSOR = (unsigned char*)yytext + x; \
-                          yyleng   = (unsigned int)x; } while(0)
+                          yyleng   = (size_t)x; } while(0)
 #define yymore()     goto yymore_restart
 
 /* perform sanity check. If this message is triggered you should
@@ -921,7 +921,7 @@ ZEND_API void zend_multibyte_yyinput_again(zend_encoding_filter old_input_filter
 		ZVAL_STRINGL(zendlval, yytext, yyleng); \
 	}
 
-static zend_result zend_scan_escape_string(zval *zendlval, char *str, int len, char quote_type)
+static zend_result zend_scan_escape_string(zval *zendlval, char *str, size_t len, char quote_type)
 {
 	char *s, *t;
 	char *end;

[thomas-0816]

yes, people seem to write code coverage information as serialized string to files with >2GB.

Later reading these files is done with $code .= fread() and eval($code) instead of require $file.

see

pestphp/pest@49bf000 (fread+eval at the end)
pestphp/pest#1442

[thomas-0816]

maybe other option would be new functions supporting >2GB input/output, binary file format and compression:

• serialize_to_file(string $file, mixed $var): void;
• unserialize_from_file(string $file): mixed;

[iluuu1994]

@thbley New functions won't help unless we also duplicate the tokenizer code, which is not something we want. I don't quite understand why the Pest workaround with eval() works, since it's using the same tokenizer. Regardless, a fix for Pest could be to only use eval() if the file exceeds a certain size.

[iluuu1994]

I created #19545, targeting 8.3. This will at least address fix the problem for <=4GB string literals. >=4GB can only be fixed on master.

[thomas-0816]

I created #19545, targeting 8.3. This will at least address fix the problem for <=4GB string literals. >=4GB can only be fixed on master.

Thanks, the PR was closed without merging?

[iluuu1994]

It was merged manually. See 0a12aaa. GitHubs merged indicator is unreliable.

[thomas-0816]

Thanks!