Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions manager-dashboard/user_scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
## Description
This read me will guide you on how to create a COCO file using the utility script for Google Drive and DropBox
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This read me will guide you on how to create a COCO file using the utility script for Google Drive and DropBox
This readme will guide you on how to create a COCO file using the utility script for Google Drive and DropBox


## Google Drive
You can find the utility script for Google Drive here: [generate_coco_from_drive.js](./generate_coco_from_drive.js)

### Prerequisites
- You must have a Google account
- Your image files should be stored in a public Google Drive folder
- You have access to Google Apps Script via https://script.google.com

### Creation Steps
- Create a Google Apps script project
- Go to https://script.google.com
- Click on "New Project"
- Rename the project name to your desired project name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

your-project-name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- Paste the utility script
- Replace any existing default code the code from this utility file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- Replace Placeholder Values
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Replace Placeholder Values
- Replace placeholder values

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- Replace `your_coco_export.json` with your output filename
- Replace `your_public_folder_id` with the ID of your Google Drive folder
> The folder ID is the alphanumeric string that appears after "/folders/" in the URL.\
> Eg: drive.google.com/drive/folders/**1prcCevijN5mubTllB2kr5ki1gjh_IO4u**?usp=sharing
- Run the script
- Save the project to Drive using the floppy disk 💾 icon
- Press Run
- Accept the authorization prompts the first time you run the script
- View COCO JSON Output
- Go to View > Logs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Go to View > Logs
- Go to `View > Logs`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added bold **

- Copy the Google Drive URL where the coco file is created and generated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

either one

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed created

- Download the json file

## DropBox
You can find the utility script for DropBox here: [generate_coco_from_dropbox.py](./generate_coco_from_dropbox.py)

### Prerequisites
- Create account: https://www.dropbox.com/register
- Create new App: https://www.dropbox.com/developers/apps
- Choose an API: Scoped access
- Choose the type of access you need: Full Dropbox
- Name your app: `your-app-name`
- Update `Permission type`
- Go to the app settings
- Click **Scoped App**
- Tick the following
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

permissions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- files.metadata.read
- files.content.write
- files.content.read
- sharing.write
- sharing.read
- Submit
- Generate new access token:
- Go to the app settings
- Click **Generated access token**
- Install uv on your system: https://docs.astral.sh/uv/getting-started/installation/
- Download the [generate_coco_from_dropbox.py](./generate_coco_from_dropbox.py) script
- Create a DropBox folder and upload images
- Your image files should be stored in a public DropBox folder
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe not. please check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove line 58


### Creation Steps
- Copy the folder pathname in DropBox
- Copy the generated access token from DropBox
- Run the script
```bash
# Help
uv run generate_coco_dropbox.py --help

# Sample
uv run generate_coco_dropbox.py "DROPBOX_ACCESS_TOKEN" "FOLDER_PATHNAME_IN_DROPBOX" "DESTINATION_EXPORT_FILE_NAME_IN_DROPBOX"

# Example
uv run generate_coco_dropbox.py sl.yourAccessTokenHere “/COCO TEST” “coco_export.json”
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
uv run generate_coco_dropbox.py sl.yourAccessTokenHere /COCO TEST” “coco_export.json
uv run generate_coco_dropbox.py sl.yourAccessTokenHere "/COCO TEST" "coco_export.json"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

```
- Download the exported coco json from the link in terminal or your DropBox folder
124 changes: 90 additions & 34 deletions manager-dashboard/user_scripts/generate_coco_from_dropbox.py
Original file line number Diff line number Diff line change
@@ -1,48 +1,94 @@
# /// script
# requires-python = ">=3.13"
# dependencies = [
# "requests<3",
# "httpx~=0.28.1",
# "colorama",
# ]
# ///
from pathlib import Path
from argparse import ArgumentParser
import requests
from colorama import init, Fore

import argparse
import textwrap
import httpx
import json
import re

# Initialize colorama
init(autoreset=True)


DROPBOX_PERMISSION_MESSAGE = f"""
{Fore.YELLOW}
----------------------------------------------------
Make sure the dropbox App includes these permissions
- files.metadata.read
- files.content.write
- files.content.read
- sharing.write
- sharing.read
"""


def dropbox_request_error_handler(res: httpx.Response):
try:
res.raise_for_status()
except httpx.HTTPStatusError as http_err:
print(f"{Fore.RED}HTTP error occurred while requesting {res.url}: {http_err}")
print(f"{Fore.RED}Response content: {res.text}")
raise
except httpx.RequestError as req_err:
print(
f"{Fore.RED}An error occurred while making the request to {res.url}: {req_err}"
)
raise
except Exception as err:
print(f"{Fore.RED}An unexpected error occurred: {err}")
raise
finally:
print(DROPBOX_PERMISSION_MESSAGE)


def dropbox_request(endpoint: str, data: object, *, access_token: str):
url = f"https://api.dropboxapi.com/2/{endpoint}"
headers = {
"Authorization": f"Bearer {access_token}",
"Content-Type": "application/json",
}
res = requests.post(
res = httpx.post(
url,
headers=headers,
data=json.dumps(data),
)
res.raise_for_status()
dropbox_request_error_handler(res)
return res.json()

def dropbox_content_request(endpoint: str, path: str, data: object, *, access_token: str):

def dropbox_content_request(
endpoint: str, path: str, data: object, *, access_token: str
):
url = f"https://content.dropboxapi.com/2/{endpoint}"
headers = {
"Authorization": f"Bearer {access_token}",
"Content-Type": "application/octet-stream",
"Dropbox-API-Arg": json.dumps({
"path": path,
"mode": "overwrite", # overwrite if exists
"autorename": False,
"mute": False
})
"Dropbox-API-Arg": json.dumps(
{
"path": path,
"mode": "overwrite", # overwrite if exists
"autorename": False,
"mute": False,
}
),
}
res = requests.post(
res = httpx.post(
url,
headers=headers,
data=json.dumps(data).encode("utf-8"),
)
res.raise_for_status()
dropbox_request_error_handler(res)
return res.json()


def list_all_files(folder_path: str, *, access_token: str):
ALLOWED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".webp"}
files = []
Expand All @@ -65,11 +111,14 @@ def list_all_files(folder_path: str, *, access_token: str):
files = sorted(files, key=lambda file: file["name"].lower())
# Filter out only files (not folders) that are supported
files = [
file for file in files
if file[".tag"] == "file" and Path(file["name"]).suffix.lower() in ALLOWED_EXTENSIONS
file
for file in files
if file[".tag"] == "file"
and Path(file["name"]).suffix.lower() in ALLOWED_EXTENSIONS
]
return files


def share_file_and_get_links(files, *, access_token: str):
total = len(files)
images = []
Expand All @@ -88,34 +137,40 @@ def share_file_and_get_links(files, *, access_token: str):
if res.get("links"):
link = res["links"][0]["url"]
else:
data = {
"path": path,
"settings": {
"requested_visibility": "public"
}
}
data = {"path": path, "settings": {"requested_visibility": "public"}}
res_create = dropbox_request(
"sharing/create_shared_link_with_settings",
data,
access_token=access_token,
)
link = res_create["url"]

raw_url = re.sub(r'&dl=0\b', '', link) + '&raw=1'
raw_url = re.sub(r"&dl=0\b", "", link) + "&raw=1"

images.append({
"id": i + 1,
"file_name": actual_path,
"coco_url": raw_url,
})
images.append(
{
"id": i + 1,
"file_name": actual_path,
"coco_url": raw_url,
}
)
return images


def main():
parser = ArgumentParser(description="Generate COCO file from images folder.")
parser = argparse.ArgumentParser(
description="Generate COCO file from images folder.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=textwrap.dedent(DROPBOX_PERMISSION_MESSAGE),
)
parser.add_argument("access_token", help="Access token for authentication")
parser.add_argument("images_folder", help="Path to the images folder")
parser.add_argument("export_file_name", help="Name of the export COCO file")
parser.add_argument(
"images_folder", help='Path to the images folder in dropbox. eg: "/COCO TEST"'
)
parser.add_argument(
"export_file_name",
help="Name of the export COCO file to be created in dropbox under provided images_folder",
)

args = parser.parse_args()

Expand All @@ -141,17 +196,18 @@ def main():
dropbox_content_request(
"files/upload",
absolute_export_file_name,
{ "images": public_images },
{"images": public_images},
access_token=access_token,
)

# Get temporary link
res = dropbox_request(
"files/get_temporary_link",
{ "path": absolute_export_file_name },
{"path": absolute_export_file_name},
access_token=access_token,
)
print(f"COCO file available at {res["link"]}")
print(f"COCO file available at {res['link']}")


if __name__ == "__main__":
main()