Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
autoapi_member_order = "groupwise"
suppress_warnings = ["autoapi.python_import_resolution"]
autoapi_python_class_content = "both"
autoapi_keep_files = False # set to True for debugging generated files


def autoapi_skip_member_fn(app, what, name, obj, skip, options) -> bool: # noqa: ARG001
Expand Down
12 changes: 6 additions & 6 deletions docs/source/user-guide/common-operations/udf-and-udfa.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Scalar Functions

When writing a user-defined function that can operate on a row by row basis, these are called Scalar
Functions. You can define your own scalar function by calling
:py:func:`~datafusion.udf.ScalarUDF.udf` .
:py:func:`~datafusion.user_defined.ScalarUDF.udf` .

The basic definition of a scalar UDF is a python function that takes one or more
`pyarrow <https://arrow.apache.org/docs/python/index.html>`_ arrays and returns a single array as
Expand Down Expand Up @@ -93,9 +93,9 @@ converting to Python objects to do the evaluation.
Aggregate Functions
-------------------

The :py:func:`~datafusion.udf.AggregateUDF.udaf` function allows you to define User-Defined
The :py:func:`~datafusion.user_defined.AggregateUDF.udaf` function allows you to define User-Defined
Aggregate Functions (UDAFs). To use this you must implement an
:py:class:`~datafusion.udf.Accumulator` that determines how the aggregation is performed.
:py:class:`~datafusion.user_defined.Accumulator` that determines how the aggregation is performed.

When defining a UDAF there are four methods you need to implement. The ``update`` function takes the
array(s) of input and updates the internal state of the accumulator. You should define this function
Expand Down Expand Up @@ -153,8 +153,8 @@ Window Functions
----------------

To implement a User-Defined Window Function (UDWF) you must call the
:py:func:`~datafusion.udf.WindowUDF.udwf` function using a class that implements the abstract
class :py:class:`~datafusion.udf.WindowEvaluator`.
:py:func:`~datafusion.user_defined.WindowUDF.udwf` function using a class that implements the abstract
class :py:class:`~datafusion.user_defined.WindowEvaluator`.

There are three methods of evaluation of UDWFs.

Expand Down Expand Up @@ -207,7 +207,7 @@ determine which evaluate functions are called.

import pyarrow as pa
from datafusion import udwf, col, SessionContext
from datafusion.udf import WindowEvaluator
from datafusion.user_defined import WindowEvaluator

class ExponentialSmooth(WindowEvaluator):
def __init__(self, alpha: float) -> None:
Expand Down
2 changes: 1 addition & 1 deletion examples/python-udwf.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from datafusion import col, lit, udwf
from datafusion import functions as f
from datafusion.expr import WindowFrame
from datafusion.udf import WindowEvaluator
from datafusion.user_defined import WindowEvaluator

# This example creates five different examples of user defined window functions in order
# to demonstrate the variety of ways a user may need to implement.
Expand Down
4 changes: 4 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,10 @@ exclude = [".github/**", "ci/**", ".asf.yaml"]
locked = true
features = ["substrait"]

[tool.pytest.ini_options]
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"

# Enable docstring linting using the google style guide
[tool.ruff.lint]
select = ["ALL" ]
Expand Down
10 changes: 9 additions & 1 deletion python/datafusion/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,15 @@
from .io import read_avro, read_csv, read_json, read_parquet
from .plan import ExecutionPlan, LogicalPlan
from .record_batch import RecordBatch, RecordBatchStream
from .udf import Accumulator, AggregateUDF, ScalarUDF, WindowUDF, udaf, udf, udwf
from .user_defined import (
Accumulator,
AggregateUDF,
ScalarUDF,
WindowUDF,
udaf,
udf,
udwf,
)

__version__ = importlib_metadata.version(__name__)

Expand Down
2 changes: 1 addition & 1 deletion python/datafusion/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
from datafusion.dataframe import DataFrame
from datafusion.expr import Expr, SortExpr, sort_list_to_raw_sort_list
from datafusion.record_batch import RecordBatchStream
from datafusion.udf import AggregateUDF, ScalarUDF, WindowUDF
from datafusion.user_defined import AggregateUDF, ScalarUDF, WindowUDF

from ._internal import RuntimeEnvBuilder as RuntimeEnvBuilderInternal
from ._internal import SessionConfig as SessionConfigInternal
Expand Down
Loading