feat(loader): Add dataframe/huggingface specific classes#80
Conversation
soniacq
left a comment
There was a problem hiding this comment.
Great PR! The main improvement is that Hugging Face data can now be used directly in the pipelines without converting it to a CSV first—this removes redundancy, which is excellent.
Just one minor fix: the docs/copy_of_examples folder should be removed, as it seems to have been committed by mistake. Apart from that, everything looks ready to merge.
If @fabiofelix did change the examples he needs to push the copy_of_examples in docs otherwise the "notebooks" are not forwarded to the documentation; just in case ✅ Let me have a Quick Look at the PR I have 5" before merging guys please Cheers |
simonprovost
left a comment
There was a problem hiding this comment.
Amazing PR @fabiofelix ! Nothing of a big deal to change just to convince me this is the right way / expected, let me know!
Cheers
examples/3-Case-Studies/4-Paris-Remarkable-Trees/1-Paris_Remarquable_Trees_Pipeline.ipynb
Show resolved
Hide resolved
examples/3-Case-Studies/4-Paris-Remarkable-Trees/1-Paris_Remarquable_Trees_Pipeline.ipynb
Show resolved
Hide resolved
...es/3-Case-Studies/4-Paris-Remarkable-Trees/2-Paris_Remarquable_Trees_Advanced_Pipeline.ipynb
Show resolved
Hide resolved
...es/3-Case-Studies/4-Paris-Remarkable-Trees/2-Paris_Remarquable_Trees_Advanced_Pipeline.ipynb
Show resolved
Hide resolved
...es/3-Case-Studies/4-Paris-Remarkable-Trees/2-Paris_Remarquable_Trees_Advanced_Pipeline.ipynb
Show resolved
Hide resolved
...es/3-Case-Studies/4-Paris-Remarkable-Trees/2-Paris_Remarquable_Trees_Advanced_Pipeline.ipynb
Show resolved
Hide resolved
...of_examples/3-Case-Studies/4-Paris-Remarkable-Trees/1-Paris_Remarquable_Trees_Pipeline.ipynb
Show resolved
Hide resolved
b293097 to
8fdb3a9
Compare
8fdb3a9 to
1fe8146
Compare
49a277a to
a084d59
Compare
There was a problem hiding this comment.
Brilliant PR! Approving on my side of the table! Thanks for this great one @fabiofelix!
Next time, let's try to have a separate commit when you are changing things a little outside the scope of the PR, such as the viz. color palettes in the examples, so that this commit can have a description explaining why the change was made to avoid confusion, please.
I'll let @soniacq merge once she approves the PR 🫡
Cheers!
a084d59 to
311fdf5
Compare
soniacq
left a comment
There was a problem hiding this comment.
Thank you, Fabio, for addressing all the comments.
The following PR is about:
It unifies
LoaderBaseinterface by:load_data_from_filewithloadmethodFileLoaderclass that inheritsLoaderBase. Therefore,CSVLoader,ParquetLoader, andShapefileLoaderinherit this classDataFrameLoaderandHuggingFaceLoader, which inheritLoaderBase, and moving specific codes fromLoaderFactory.LoaderFactorycode, mainlybuildandloadmethods.Now, it is possible to use HuggingFace (or dataframe) in a
pipelinein the same way as file loaders, calling the factory methodbuild.Before, the user should've retrieved data from HuggingFace, saved it as a file, and used a file loader with the
pipeline.📚 Documentation preview 📚: https://UrbanMapper--80.org.readthedocs.build/en/80/