Skip to content

Commit 2892a0d

Browse files
sanjaychelliahsanjaychelliah
committed
readme
1 parent 4c79f82 commit 2892a0d

File tree

9 files changed

+204
-16
lines changed

9 files changed

+204
-16
lines changed

‎.github/workflows/publish.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# This workflow will upload a Python Package using Twine when a release is created
2+
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
3+
4+
name: Publish package
5+
6+
on:
7+
push:
8+
tags:
9+
- '[0-9]+.[0-9]+.[0-9a-zA-Z]+' # Matches 1.2.3, 1.2.3alpha1 etc.
10+
11+
jobs:
12+
publish-pypi:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- uses: actions/checkout@v4
16+
- name: Set up Python
17+
uses: actions/setup-python@v4
18+
with:
19+
python-version: '3.9'
20+
- name: Install dependencies
21+
run: |
22+
python -m pip install --upgrade pip
23+
pip install -r .github/workflows/requirements.txt
24+
- name: Build and publish
25+
env:
26+
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
27+
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
28+
run: |
29+
python setup.py sdist bdist_wheel
30+
twine upload --non-interactive dist/*
31+
publish-github-release:
32+
needs: publish-pypi
33+
name: Create Release
34+
runs-on: ubuntu-latest
35+
steps:
36+
- uses: actions/checkout@v4
37+
- name: Create Release
38+
uses: softprops/action-gh-release@v1
39+
with:
40+
name: Release ${{ github.ref_name }}
41+
draft: false
42+
prerelease: false

‎.github/workflows/requirements.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
setuptools==67.7.2
2+
wheel==0.40.0
3+
twine==4.0.2

‎README.md‎

Lines changed: 84 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,84 @@
1-
# clarifai-python-datautils
1+
![Clarifai logo](docs/logo.png)
2+
3+
# Clarifai Python Data Utils
4+
5+
6+
[![Discord](https://img.shields.io/discord/1145701543228735582)](https://discord.gg/M32V7a7a)
7+
[![codecov](https://img.shields.io/pypi/dm/clarifai)](https://pypi.org/project/clarifai)
8+
9+
This is a collection of utilities for handling various types of multimedia data. Enhance your experience by seamlessly integrating these utilities with the Clarifai Python SDK. This powerful combination empowers you to address both visual and textual use cases effortlessly through the capabilities of Artificial Intelligence. Unlock new possibilities and elevate your projects with the synergy of versatile data utilities and the robust features offered by the [Clarifai Python SDK](https://github.com/Clarifai/clarifai-python). Explore the fusion of these tools to amplify the intelligence in your applications! 🌐🚀
10+
11+
[Website](https://www.clarifai.com/) | [Schedule Demo](https://www.clarifai.com/company/schedule-demo) | [Signup for a Free Account](https://clarifai.com/signup) | [API Docs](https://docs.clarifai.com/) | [Clarifai Community](https://clarifai.com/explore) | [Python SDK Docs](https://docs.clarifai.com/python-sdk/api-reference) | [Examples](https://github.com/Clarifai/examples) | [Colab Notebooks](https://github.com/Clarifai/colab-notebooks) | [Discord](https://discord.gg/XAPE3Vtg)
12+
13+
---
14+
## Table Of Contents
15+
16+
* **[Installation](#installation)**
17+
* **[Getting Started](#getting-started)**
18+
* **[Features](#features)**
19+
* [Image Utils](#image-utils)
20+
* **[Usage](#usage)**
21+
* **[Examples](#more-examples)**
22+
23+
24+
## Installation
25+
26+
27+
Install from PyPi:
28+
29+
```bash
30+
pip install clarifai-utils
31+
```
32+
33+
Install from Source:
34+
35+
```bash
36+
git clone https://github.com/Clarifai/clarifai-python-datautils
37+
cd clarifai-python-datautils
38+
python3 -m venv env
39+
source env/bin/activate
40+
pip3 install -r requirements.txt
41+
```
42+
43+
44+
## Getting started
45+
46+
Quick intro to Image Annotation Conversion feature
47+
48+
```python
49+
from clarifai_utils import Image_Annotations
50+
51+
annotated_dataset = Image_Annotations.import_from(path= 'folder_path', format= 'annotation_format')
52+
```
53+
54+
## Features
55+
56+
### Image Utils
57+
- #### Annotation Conversion
58+
- Load various annotated image datasets and export to clarifai Platform
59+
- Convert from one annotation format to other supported annotation formats
60+
61+
62+
63+
## Usage
64+
### Image Annotation conversion
65+
```python
66+
from clarifai_utils import Image_Annotations
67+
#import from folder
68+
coco_dataset = Image_Annotations.import_from(path='folder_path',format= 'coco_detection')
69+
70+
#clarifai dataset loader object
71+
coco_dataset.clarifai_loader()
72+
73+
74+
#info about loaded dataset
75+
coco_dataset.get_info()
76+
77+
78+
#exporting to other formats
79+
coco_dataset.export_to('voc_detection')
80+
```
81+
82+
## More Examples
83+
84+
See many more code examples in this [repo](https://github.com/Clarifai/examples).

‎clarifai_utils/image/annotation_conversion/README.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Annotation conversion
2+
3+
A framework to load, export and analyze different annotated datasets.
4+
5+
6+
## Usage
7+
8+
### Features
9+
```python
10+
from clarifai_utils import Image_Annotations
11+
#import from folder
12+
coco_dataset = Image_Annotations.import_from(path='folder_path',format= 'coco_detection')
13+
14+
#clarifai dataset loader object
15+
coco_dataset.clarifai_loader()
16+
17+
18+
#info about loaded dataset
19+
coco_dataset.get_info()
20+
21+
22+
#exporting to other formats
23+
coco_dataset.export_to('voc_detection')
24+
```
25+
26+
27+
### With Clarifai Python SDK
28+
```python
29+
from clarifai_utils import Image_Annotations
30+
coco_dataset = Image_Annotations.import_from(path='folder_path',format= 'coco_detection')
31+
32+
#clarifai SDK
33+
#export CLARIFAI_PAT={your personal access token} # set PAT as env variable
34+
from clarifai.client.dataset import Dataset
35+
dataset = Dataset(user_id="user_id", app_id="app_id", dataset_id="dataset_id")
36+
dataset.upload_dataset(dataloader=coco_dataset.clarifai_loader())
37+
38+
```
39+
40+
41+
## Supported Formats
42+
43+
| Annotation format | Format | TASK |
44+
| ------------------------------------------------------------------------------------------------ | ------- | --------------- |
45+
| [ImageNet](http://image-net.org/) | imagenet | classification |
46+
| [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html) | cifar | classification |
47+
| [MNIST](http://yann.lecun.com/exdb/mnist/) | mnist | classification |
48+
| [VGGFace2](https://github.com/ox-vgg/vgg_face2) | vgg_face2 | classification |
49+
| [LFW](http://vis-www.cs.umass.edu/lfw/) | lfw | classification |
50+
| [PASCAL VOC](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/htmldoc/index.html) | voc_detection | detection |
51+
| [YOLO](https://github.com/AlexeyAB/darknet#how-to-train-pascal-voc-data) | yolo | detection |
52+
| [COCO](http://cocodataset.org/#format-data) | coco_detection | detection |
53+
| [CVAT](https://opencv.github.io/cvat/docs/manual/advanced/xml_format/) | cvat | detection |
54+
| [Kitti](http://www.cvlibs.net/datasets/kitti/index.php) | kitti | detection |
55+
| [LabelMe](http://labelme.csail.mit.edu/Release3.0) | label_me | detection |
56+
| [Open Images](https://storage.googleapis.com/openimages/web/download.html) | open_images | detection |
57+
| [COCO(segmentation)](http://cocodataset.org/#format-data) | coco_segmentation | segmentation |
58+
| [Cityscapes](https://www.cityscapes-dataset.com/) | cityscapes | segmentation |
59+
| [ADE](https://www.cityscapes-dataset.com/) | ade20k2017 | segmentation |
60+
61+
62+
63+
## Resources
64+
This tool makes use of the [Datumaro Framework](https://github.com/openvinotoolkit/datumaro)

‎clarifai_utils/image/annotation_conversion/annotations.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ def import_from(cls, path: str, format: str) -> Dataset:
4747
A dataset object.
4848
4949
Example:
50-
>>> from clarifai-utils import Image_Annotations
50+
>>> from clarifai_utils import Image_Annotations
5151
>>> format = Image_Annotations.import_from(path=folder_path, format = 'coco_detection')
5252
"""
5353
if format not in IMAGE_ANNOTATION_FORMATS:
@@ -71,7 +71,7 @@ def get_info(self,) -> Dict[str, Any]:
7171
A dictionary containing the information about the dataset.
7272
7373
Example:
74-
>>> from clarifai-utils import Image_Annotations
74+
>>> from clarifai_utils import Image_Annotations
7575
>>> format = Image_Annotations.import_from(path=folder_path, format = 'coco_detection')
7676
>>> info = format.get_info()
7777
"""
@@ -92,7 +92,7 @@ def export_to(self, path: str, format: str) -> None:
9292
format (str): The format of the dataset.
9393
9494
Example:
95-
>>> from clarifai-utils import Image_Annotations
95+
>>> from clarifai_utils import Image_Annotations
9696
>>> format = Image_Annotations.import_from(path=folder_path, format = 'coco_detection')
9797
>>> format.export_to(path=output_folder_path, format = 'voc_detection')
9898
"""
@@ -116,7 +116,7 @@ def detect_format(path: str) -> str:
116116
The format of the dataset.
117117
118118
Example:
119-
>>> from clarifai-utils import Image_Annotations
119+
>>> from clarifai_utils import Image_Annotations
120120
>>> format = Image_Annotations.detect_format(path=folder_path)
121121
"""
122122
try:
@@ -137,7 +137,7 @@ def clarifai_loader(self,) -> ClarifaiDataLoader:
137137
A ClarifaiDataloader object.
138138
139139
Example:
140-
>>> from clarifai-utils import Image_Annotations
140+
>>> from clarifai_utils import Image_Annotations
141141
>>> format = Image_Annotations.import_from(path=folder_path, format = 'coco_detection')
142142
>>> clarifai_dataset_loader = format.clarifai_loader()
143143
"""

‎docs/logo.png‎

4.69 KB
Loading

‎setup.py‎

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import setuptools
22

3-
# with open("README.md", "r") as fh:
4-
# long_description = fh.read()
3+
with open("README.md", "r") as fh:
4+
long_description = fh.read()
55

66
with open("VERSION", "r") as f:
77
version = f.read().strip()
@@ -16,24 +16,22 @@
1616
packages = setuptools.find_namespace_packages(include=["clarifai_datautils*"])
1717

1818
setuptools.setup(
19-
name="clarifai-datautils",
19+
name="clarifai-utils",
2020
version=f"{version}",
2121
author="Clarifai",
2222
author_email="[email protected]",
2323
description="Clarifai Data Utils",
24-
long_description='Data utils for Clarifai',
24+
long_description=long_description,
2525
long_description_content_type="text/markdown",
2626
url="https://github.com/Clarifai/clarifai-python-datautils",
2727
packages=packages,
2828
classifiers=[
2929
"Topic :: Scientific/Engineering :: Artificial Intelligence",
3030
"Programming Language :: Python :: 3",
3131
"Programming Language :: Python :: 3 :: Only",
32-
"Programming Language :: Python :: 3.8",
3332
"Programming Language :: Python :: 3.9",
3433
"Programming Language :: Python :: 3.10",
3534
"Programming Language :: Python :: 3.11",
36-
"Programming Language :: Python :: 3.12",
3735
"Programming Language :: Python :: Implementation :: CPython",
3836
"License :: OSI Approved :: Apache Software License",
3937
"Operating System :: OS Independent",

‎tests/annotations/test_clarifai_loader.py‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
from clarifai_utils import \
2-
Image_Annotations # change this to 'from clarifai-utils import Image_Annotations'
1+
from clarifai_utils import Image_Annotations
32
from tests.utils.annotations import get_asset_path
43

54
IMAGENET_PATH = get_asset_path('imagenet_dataset')

‎tests/annotations/test_import_formats.py‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
import pytest
22

3-
from clarifai_utils import \
4-
Image_Annotations # change this to 'from clarifai-utils import Image_Annotations'
3+
from clarifai_utils import Image_Annotations
54
from tests.utils.annotations import get_asset_path
65

76
IMAGENET_PATH = get_asset_path('imagenet_dataset')

0 commit comments

Comments
 (0)