Skip to content

HackermanAI/tokenizers

Repository files navigation

Tokenizers for Hackerman Text

Build file for Cython(.pyx) files

from setuptools import setup, Extension
from Cython.Build import cythonize

setup(
    ext_modules=cythonize(
        [
            Extension("txt", ["txt.pyx"], extra_compile_args=["-O3", "-std=c11"]),
            Extension("hackerman", ["hackerman.pyx"], extra_compile_args=["-O3", "-std=c11"]),
            Extension("scrpd", ["scrpd.pyx"], extra_compile_args=["-O3", "-std=c11"]),
            Extension("todo", ["todo.pyx"], extra_compile_args=["-O3", "-std=c11"]),
            Extension("odin", ["odin.pyx"], extra_compile_args=["-O3", "-std=c11"]),
        ], 
        compiler_directives={ "language_level": "3" },
    )
)

.sh build script

#!/bin/bash

if [[ $(uname) == "Darwin" ]]; then
    export SDKROOT=$(xcrun --sdk macosx --show-sdk-path)
fi

.venv/bin/python build.py build_ext --inplace

About

Tokenizers for Hackerman Text

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published