API: io functions for v2

[MarcoGorelli]

Currently we're a bit inconsistent with IO functions:

• from_native
• from_dict
• from_arrow
• new_series
• from_numpy

I think we might be better off movings towards:

• from_native: may return Series, DataFrame, or LazyFrame, depending on the input. This one stays
• DataFrame.from_native: like from_native(..., eager_only=True, allow_series=False)
• LazyFrame.from_native: like from_native(...).lazy()
• Series.from_native: like from_native(..., series_only=True)
• DataFrame.from_dict, DataFrame.from_arrow, DataFrame.from_numpy. Remove the top-level ones
• Series.from_iterable, Series.from_numpy. Remove nw.new_series, that's an inconsistent name

All of this would only affect the main namespace and stable.v2 onwards, stable.v1 should remain unaffected

PRs

• feat: Add DataFrame.from_dict #2885
• feat: Add DataFrame.from_numpy #2887
• feat: Add DataFrame.from_arrow #2891
• feat: Add Series.from_numpy #2893
• Add Series.from_iterable #2925
• fix: Add missing v2 constructor overrides #3006

[dangotbanned]

@MarcoGorelli I had a similar thought yesterday - but slightly smaller in scope.

from_native could be split into:

• from_frame (optionally as a data/lazy pair)
• from_series

Those would actually be shorter than the current version:

Code block

from typing import TYPE_CHECKING

import polars as pl

import narwhals as nw

if TYPE_CHECKING:
    from narwhals.typing import IntoFrame
    from narwhals.typing import IntoSeries

data = {"a": [1, 2, 3]}

frame: IntoFrame = pl.DataFrame(data)
lazy: IntoFrame = pl.DataFrame(data).lazy()
series: IntoSeries = pl.Series(data["a"])

# v1
nw.from_native(frame)
nw.from_native(lazy, eager_only=False)
nw.from_native(series, allow_series=True)
nw.from_native(series, series_only=True)

# v2
nw.from_frame(frame)
nw.from_frame(frame, eager_only=False)
nw.from_series(series)

I hadn't thought about adding @classmethod(s) though - I do like that as well!

Note

If you did still want functions as an entry-point - they could be named into_*

[MarcoGorelli]

Thanks!

Yeah I think i'd prefer classmethods

For into_, I think that's what I'd expect a method to be called - e.g. df.into_frame(), rather than nw.into_frame(df). But we can't control what methods upstream libraries have, so we may be stuck with nw.from_ / nw.DataFrame.from_ etc...

[dangotbanned]

#2116 (comment)

Fully agree @MarcoGorelli - more .(from|to)_* sounds good to me 😊

Slightly off-topic

I found some others on the Compliant-side that would also benefit from this treatment (#2055 (comment)).
Just wanna make sure they're in-mind when making the public-facing changes 🤝

[dangotbanned]

Stumbled into a nice way we could type all of this internally while I was thinking about from_numpy.
Wanna make sure this doesn't get lost (#2149 (comment))

The same idea could apply for any .(to|from)_* pair - and allows having a wider type in the .from_()

from typing import Protocol
from typing_extensions import TypeVar

from narwhals.typing import _1DArray
from narwhals.typing import _2DArray

if TYPE_CHECKING:
    import numpy as np
    from typing_extensions import Self
    from typing_extensions import TypeAlias

    from narwhals.utils import _FullContext


_NumpyScalar: TypeAlias = np.generic[Any]

ToNumpyT_co = TypeVar("ToNumpyT_co", covariant=True)
FromNumpyT_contra = TypeVar("FromNumpyT_contra", contravariant=True, default=ToNumpyT_co)


class NumpyConvertible(Protocol[ToNumpyT_co, FromNumpyT_contra]):
    def to_numpy(self) -> ToNumpyT_co: ...
    @classmethod
    def from_numpy(cls, value: FromNumpyT_contra, *, context: _FullContext) -> Self: ...


class SeriesImpl(NumpyConvertible[_1DArray, _NumpyScalar | _1DArray]): ...

class DataFrameImpl(NumpyConvertible[_2DArray]): ...

Note

Check out that thread for more context on how I arrived there (#2149 (comment))

[dangotbanned]

Just updating this with stuff from a merged PR so it doesn't get lost:

• refactor: Add CompliantSeries.from_numpy #2196 (comment)
• refactor: Add CompliantSeries.from_numpy #2196 (comment)
• refactor: Add CompliantSeries.from_numpy #2196 (comment)