TextCenGen: Attention-Guided Text-Centric Background Adaptation for Text-to-Image Generation

Tianyi Liang^†, Jiangqi Liu^†, Yifei Huang, Shiqi Jiang, Jianshen Shi, Changbo Wang, Chenhui Li^*

TextCenGen introduces a dynamic adaptation of the blank region for text-friendly image generation, and enhances T2I model outcomes on arbitrary prompt, catering to varied text positions.

News

• [2025/06/07] Our article on Xiaohongshu is going viral! Thank you all for your attention. We will continue to update according to our roadmap. 查看文章
• [2025/05/07] 🔥 We release the code and dataset.
• [2025/05/01] 🔥 We achieve ICML 2025 Poster Paper.

Roadmap

• 📱 Support for arbitrary resolution for mobile devices.
• 🤗 Add Hugging Face Space integration.
• 🚀 Extend to mmDiT architecture models like Flux and other generative models.
• 📱 A quick start guide.

Demos

Logo with Adaptive Natural Background

Mobile Devices Wallpaper

Comparison with Previous Works

More Results

Installation

git clone https://github.com/yourusername/TextCenGen.git
cd TextCenGen
pip install -r requirements.txt

Download

You can download the model in python script:

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="runwayml/stable-diffusion-v1-5")

hf_hub_download(repo_id="stabilityai/stable-diffusion-2-base")

sdxl_path = hf_hub_download(repo_id="stabilityai/stable-diffusion-xl-base-1.0")

If you cannot access to Huggingface, you can use hf-mirror to download models.

export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download runwayml/stable-diffusion-v1-5 --resume-download

huggingface-cli download stabilityai/stable-diffusion-2-base --resume-download

huggingface-cli download stabilityai/stable-diffusion-xl-base-1.0 -resume-download

Quick Start

Evaluate using Stable Diffusion 1.5:

python generate.py --prompt "Photo of a fruit made of feathers with a bee on it." --position "left"

Evaluation

Evaluate using Stable Diffusion XL or Stable Diffusion 2:

python sdxl_replacement.py --path "output/prompt_llm.txt" --start 0 --end 1000

Dataset

Our evaluation dataset combines:

• Synthesized prompts generated by ChatGPT (dataset/prompt_llm.txt)
• 700 prompts from the Prompt2Prompt template (Hertz et al. 2022) (dataset/prompt_ptp.txt) designed for attention guidance, focusing on specific objects and their spatial relationships
• 1,000 DiffusionDB prompts (Wang et al. 2022) (dataset/prompt_ddb.txt), chosen for their real-world complexity
• Desigen (Weng et al. 2024, https://arxiv.org/pdf/2403.09093) constructed a design template dataset(dataset/prompt_des.txt) with 771 validation dataset with advertisement banners prompt and layout to verify our approach

Acknowledgments

We would like to thank the following repositories for their valuable contributions:

• Free-Guidance-Diffusion
• prompt-to-prompt

Citation

If you find our work useful, please consider citing our paper:

@inproceedings{liang2025textcengen,
  title     = {TextCenGen: Attention-Guided Text-Centric Background Adaptation for Text-to-Image Generation},
  author    = {Liang, Tianyi and Liu, Jiangqi and Huang, Yifei and Jiang, Shiqi and Shi, Jianshen and Wang, Changbo and Li, Chenhui},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2025}
}

For any question, please feel free to contact us via email: tyliang@stu.ecnu.edu.cn and chli@cs.ecnu.edu.cn