DODA is a data synthesizer that can generate high-quality object detection data for new domains in agriculture, and help the detectors adapt to the new domains.
| Model | Dataset | Resolution | Training Iters | Downlad Link |
|---|---|---|---|---|
| DODA-L2I | COCO | 512x512 | 30K | Google drive |
| DODA-L2I | COCO | 256x256 | 100K | Google drive |
| VAE | GWHD2021 | 256x256 | 170K | Google drive |
| ldm | GWHD 2+ | 256x256 | 150k | Google drive |
| DODA | GWHD2021 | 256x256 | 80K | Google drive |
conda create -y -n DODA python=3.8.5
conda activate DODA
pip install torch==1.13.0+cu116 torchvision==0.14.0+cu116 --extra-index-url https://download.pytorch.org/whl/cu116
pip install -r requirements.txt
bash Download_dataset.sh # download the dataset you need
python prepare_coco.py
python prepare_wheat_trainset.py && python prepare_wheat_testset.py # If you only want to generate data for your domain, and don't need to train the model, please skip this line
Generate images according to the bounding boxes of the COCO 2017 validation set:
First download the pretrained DODA-L2I to /models folder, then run:
python generate_coco_testimg.py
Modify the target_domain, cfg_scale, strength in generate_data.py, then run:
python generate_data.py
- Modify the parameters in
generate_custom_layout.pybased on the shape and number of wheat heads in your data to make the generated layouts match your dataset. And rungenerate_custom_layout.py - Modify the
ref_img_path,cfg_scale,strengthingenerate_custom_data.py, then runpython generate_custom_data.py
You can try our method to generate images for wheat through the GUI:
python wheat_gradio_box2image.py
Please upload BOTH the reference image and layout image image respectively as shown:
PS: The demo reference image and layout image can be found in the
/figuresfolder. More images can be found in/datasetfolder after runprepare_wheat_trainset.py
Or you can simply draw it yourself through drawing software. Each item should have a distinguishable color (with maximized values of the R, G, B channels), for example, (0, 0, 255), (255, 0, 255), etc. Below are some examples of possible layout images:
DODA training is divided into three parts, from first to last: VAE, ldm and L2I. This repository reads the data set through a txt file, so first, please write the file names of all the images in your own dataset into a txt file.
Modify the config in train.py :
config = 'configs/autoencoder/wheat_autoencoder_kl_64x64x3.yaml'
Modify the txt_file and data_root in the config file to the path of the filenames txt file and the path to your own dataset.
then train the VAE by running:
python train.py
VAE is very robust, so we recommend skipping VAE training and using the pre-trained weight kl-f4-wheat.ckpt we provide.
Modify the config in train.py :
config = 'configs/latent-diffusion/wheat_ldm_kl_4.yaml'
Modify the ckpt_path in the config file wheat_ldm_kl_4.yaml to the weight path of your VAE or the VAE provided by us.
Modify the txt_file and data_root in the config file to the path of the filenames txt file and the path to your own dataset.
then train the ldm by running:
python train.py
Modify the config in train.py :
config = 'configs/DODA/DODA_wheat_ldm_kl_4_layout_clip.yaml'
Modify the resume_path in train.py to the weight path of your ldm or the ldm provided by us.
Modify the txt_file and data_root in the config file to the path of the filenames of conditioned .txt files (e.g., train_cldm.txt and val_cldm.txt) and the path to your own dataset.
then train the cldm by running:
python train.py
Diffusion model is data hungry, and using more data always gives better results, so we strongly recommend mixing your data with GWHD for training. Mixing data can be achieved by putting all the images in your own dataset and the GWHD into one folder and writing the filenames of all images to one txt file.
When fine-tuning a detector with generated data, use mosaic data augmentation and train for one epoch with a fixed learning rate.