You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Revamp model internals, unify generation flow, and extend platform support (#89)
This PR delivers a major refactor focused on simplifying the generation
pipeline, improving cross-platform native support, extending model
capabilities, and enhancing overall maintainability.
### Highlights:
- Stopping Criteria: Added `MaxLength`, `MaxTime`, `EosToken` and
`Interruptable` criteria for better generation control.
- PretrainedConfig: Introduced base config to reduce boilerplate across
model files.
- Platform Package: Converted to platform-package with dynamic loading
support for Linux, macOS, and Windows (x86_64 & ARM64).
- Optimized FFI: Cleaned up FFI bindings for better reliability and
developer ergonomics.
- AutoModel Refactor: Unified resolution logic to prioritize generic
models and avoid accidental task-specific selection.
- New Model Support: Added Gemma, Qwen3, Phi, and other language models.
- Introduced token-per-second (TPS) benchmarking
- Improved logits processors and model session handling
- Enhanced Audio/Image Utilities: Upgraded constructors and added better
error handling.
- Refactored `BPEModel` to support new merge format
- PSR-3 Logging: Logging support throughout the codebase, now defaults
to NullLogger
- Added full inference and utility test coverage
- Bumped shared library versions -onnxruntime and rindowmatlib
- Updated documentation and examples
Resolves#88Resolves#29
No known breaking changes.
TransformersPHP is designed to be functionally equivalent to the Python library, while still maintaining the same level
17
-
of performance and ease of use. This library is built on top of the Hugging Face's Transformers library, which provides
18
-
thousands of pre-trained models in 100+ languages. It is designed to be a simple and easy-to-use library for PHP
19
-
developers using a similar API to the Python library. These models can be used for a variety of tasks, including text
20
-
generation, summarization, translation, and more.
17
+
TransformersPHP is designed to be functionally equivalent to the Python library, while still maintaining the same level of performance and ease of use. This library is built on top of the Hugging Face's Transformers library, which provides thousands of pre-trained models in 100+ languages. It is designed to be a simple and easy-to-use library for PHP developers using a similar API to the Python library. These models can be used for a variety of tasks, including text generation, summarization, translation, and more.
21
18
22
-
TransformersPHP uses [ONNX Runtime](https://onnxruntime.ai/) to run the models, which is a high-performance scoring
23
-
engine for Open Neural Network Exchange (ONNX) models. You can easily convert any PyTorch or TensorFlow model to ONNX
24
-
and use it with TransformersPHP using [🤗 Optimum](https://github.com/huggingface/optimum#onnx--onnx-runtime).
19
+
TransformersPHP uses [ONNX Runtime](https://onnxruntime.ai/) to run the models, which is a high-performance scoring engine for Open Neural Network Exchange (ONNX) models. You can easily convert any PyTorch or TensorFlow model to ONNX and use it with TransformersPHP using [🤗 Optimum](https://github.com/huggingface/optimum#onnx--onnx-runtime).
25
20
26
-
TO learn more about the library and how it works, head over to
TO learn more about the library and how it works, head over to our [extensive documentation](https://codewithkyrian.github.io/transformers-php/introduction).
28
22
29
23
## Quick tour
30
24
31
-
Because TransformersPHP is designed to be functionally equivalent to the Python library, it's super easy to learn from
32
-
existing Python or Javascript code. We provide the `pipeline` API, which is a high-level, easy-to-use API that groups
33
-
together a model with its necessary preprocessing and postprocessing steps.
25
+
Because TransformersPHP is designed to be functionally equivalent to the Python library, it's super easy to learn from existing Python or Javascript code. We provide the `pipeline` API, which is a high-level, easy-to-use API that groups together a model with its necessary preprocessing and postprocessing steps.
> The ONNX library is platform-specific, so it's important to run the composer require command on the target platform
108
-
> where the code will be executed. In most cases, this will be your development machine or a server where you deploy
109
-
> your application, but if you're using a Docker container, run the `composer require` command inside that container.
99
+
> The ONNX library is platform-specific, so it's important to run the composer require command on the target platform where the code will be executed. In most cases, this will be your development machine or a server where you deploy your application, but if you're using a Docker container, run the `composer require` command inside that container.
110
100
111
101
## PHP FFI Extension
112
102
113
-
TransformersPHP uses the PHP FFI extension to interact with the ONNX runtime. The FFI extension is included by default
114
-
in PHP 7.4 and later, but it may not be enabled by default. If the FFI extension is not enabled, you can enable it by
115
-
uncommenting(remove the `;` from the beginning of the line) the
103
+
TransformersPHP uses the PHP FFI extension to interact with the ONNX runtime. The FFI extension is included by default in PHP 7.4 and later, but it may not be enabled by default. If the FFI extension is not enabled, you can enable it by uncommenting(remove the `;` from the beginning of the line) the
116
104
following line in your `php.ini` file:
117
105
118
106
```ini
@@ -129,14 +117,11 @@ After making these changes, restart your web server or PHP-FPM service, and you
129
117
130
118
## Documentation
131
119
132
-
For more detailed information on how to use the library, check out the
For more detailed information on how to use the library, check out the documentation : [https://codewithkyrian.github.io/transformers-php](https://codewithkyrian.github.io/transformers-php)
134
121
135
122
## Usage
136
123
137
-
By default, TransformersPHP uses hosted pretrained ONNX models. For supported tasks, models that have been converted to
138
-
work with [Xenova's Transformers.js](https://huggingface.co/models?library=transformers.js) on HuggingFace should work
139
-
out of the box with TransformersPHP.
124
+
By default, TransformersPHP uses hosted pretrained ONNX models. For supported tasks, models that have been converted to work with [Xenova's Transformers.js](https://huggingface.co/models?library=transformers.js) on HuggingFace should work out of the box with TransformersPHP.
140
125
141
126
## Configuration
142
127
@@ -151,27 +136,25 @@ Transformers::setup()
151
136
->setRemotePathTemplate('...') // Set the remote path template for downloading models. Defaults to `{model}/resolve/{revision}/{file}`
152
137
->setAuthToken('...') // Set the auth token for downloading models. Defaults to `null`
153
138
->setUserAgent('...') // Set the user agent for downloading models. Defaults to `transformers-php/{version}`
154
-
->setImageDriver('...') // Set the image driver for processing images. Defaults to `IMAGICK'
155
-
->setLogger('...'); // Set the logger for TransformersPHP. Defaults to `null`
139
+
->setImageDriver('...') // Set the image driver for processing images. Defaults to `VIPS`
140
+
->setLogger($logger) // Set a PSR-3 compatible logger. Defaults to `NullLogger` if not set
141
+
->apply(); // Apply the configuration
156
142
```
157
143
158
-
You can call the `set` methods in any order, or leave any out entirely, in which case, it uses the default values. For
159
-
more information on the configuration options and what they mean, checkout
144
+
You can call the `set` methods in any order, or leave any out entirely, in which case, it uses the default values. For more information on the configuration options and what they mean, checkout
160
145
the [documentation](https://codewithkyrian.github.io/transformers-php/configuration).
161
146
162
147
## Convert your models to ONNX
163
148
164
-
TransformersPHP only works with ONNX models, therefore, you must convert your PyTorch, TensorFlow or JAX models to
165
-
ONNX. It is recommended to use [🤗 Optimum](https://huggingface.co/docs/optimum) to perform the conversion and
166
-
quantization of your model.
149
+
TransformersPHP only works with ONNX models, therefore, you must convert your PyTorch, TensorFlow or JAX models to ONNX. We recommend using the [conversion script](https://github.com/huggingface/transformers.js/blob/main/scripts/convert.py) from Transformers.js, which uses the [🤗 Optimum](https://huggingface.co/docs/optimum) behind the scenes to perform the conversion and quantization of your model.
By default, TransformersPHP automatically retrieves model weights (ONNX format) from the Hugging Face model hub when
171
-
you first use a pipeline or pretrained model. This can lead to a slight delay during the initial use. To improve the
172
-
user experience, it's recommended to pre-download the models you intend to use before running them in your PHP
173
-
application, especially for larger models. One way to do that is run the request once manually, but TransformersPHP
174
-
also comes with a command line tool to help you do just that:
157
+
By default, TransformersPHP automatically retrieves model weights (ONNX format) from the Hugging Face model hub when you first use a pipeline or pretrained model. This can lead to a slight delay during the initial use. To improve the user experience, it's recommended to pre-download the models you intend to use before running them in your PHP application, especially for larger models. One way to do that is run the request once manually, but TransformersPHP also comes with a command line tool to help you do just that:
0 commit comments