Sementic Segment To Anime Illust

Demo

Run .exe

Download demo and please execute "ssti/ssti.exe"

Load the segment image to the left frmae, reference image to the middle frame; basic images are given in "resources" foloder. It will automatically generate the new illust.

Run .py

Download pretrained models and copy "weights" folder to the root folder.

pip install -r requirements.txt

python ssti.py

It was tested in Anaconda environment.

Dataset

It used Anime-Portrait-Dataset as inputs.

and generated the labels with Anime-Face-Segmentation.

Train

It used NVlabs SPADE. After copy the pretrained model from "ssti/weights/" folder, you can incrementally train new model with below command.

python train.py --name ssti --dataset_mode custom --label_dir datasets/anime/train_label/ --image_dir datasets/anime/train_img/ --no_instance --label_nc 7 --niter 8 --niter_decay 8 --batchSize 4 --display_freq 10000 --save_epoch_freq 1 --use_vae --continue_train

It requires 8GB gpu memory.

References

[1] NVlabs SPADE

[2] Deep Learning Project — Drawing Anime Face with Simple Segmentation Mask

[3] Understanding GauGAN