performance for python code called many times from rust

[brentp]

Hi, currently I have code like this:

impl<'py> CompiledPython<'py> {
    pub fn new(py: Python<'py>, f_string_code: &str) -> PyResult<Self> {
        let code = CString::new(f_string_code)?;
        let module = PyModule::from_code(py, &code, c_str!("user_code"), c_str!("user_code"))?;
        let f = module.getattr("main")?.extract()?;
        Ok(CompiledPython {
            _code: f_string_code.to_string(),
            _module: module,
            f,
        })
    }

    pub fn eval(&self, report: PyReportFragment) -> PyResult<String> {
        let result = self.f.call1((report,))?;
        if let Ok(result) = result.downcast::<PyString>() {
            Ok(result.to_string())
        } else if let Ok(result) = result.extract::<String>() {
            Ok(result)
        } else {
            Err(PyTypeError::new_err("Result is not a string"))
        }
    }
}

where the user passes code like this: def main(report): return f'{report.a.chrom}\t{report.a.start}\t{report.a.stop}'

I have 3 related questions:

1. Is there anything I can do to make the eval() run as fast as possible? I'll be calling it potentially millions of times.
2. in python, I can compile and then eval the code for better performance, I tried a bit to do that with the lower-level ffi stuff, but didn't get far. Would that be a way to improve performance?
3. I'd like to have it so the user only specifies, e.g. f'{report.a.chrom}\t{report.a.start}\t{report.a.stop}' for the python code, is this possible (without compile+eval with ffi stuff)?

thanks in advance!

[birkenfeld]

1. Seems already pretty straightforward the way you wrote it. Depending on what you do with the result, you might want to skip conversion to a Rust String.
2. PyModule::from_code already compiles the code to bytecode, so no.
3. You'd still have to compile the code, and the respective C APIs are not wrapped in PyO3 as far as I can see. However, instead of calling FFI you can call their wrappers on the Python level, i.e. the compile() builtin function.