update doc about the components in "private evolution for images using simulators"

Author: fjxmlzn

doc/source/getting_started/details/api.rst

@@ -15,12 +15,15 @@ Currently, the following APIs are implemented:
 
 * Images
 
-    * :py:class:`pe.api.StableDiffusion`: The APIs of `Stable Diffusion`_.
-    * :py:class:`pe.api.ImprovedDiffusion`: The APIs of the `improved diffusion model`_.
+    * :py:class:`pe.api.StableDiffusion`: The APIs of `Stable Diffusion`_ (introduced in [#pe1]_).
+    * :py:class:`pe.api.ImprovedDiffusion`: The APIs of the `improved diffusion model`_ (introduced in [#pe1]_).
+    * :py:class:`pe.api.DrawText`: The APIs that render text on images (introduced in [#pe3]_).
+    * :py:class:`pe.api.Avatar`: The APIs that generate avatars using `the Python Avatars library <python_avatars_>`__ (introduced in [#pe3]_).
+    * :py:class:`pe.api.NearestImage`: The APIs that utilize a static image dataset (introduced in [#pe3]_).
 
 * Text
 
-    * :py:class:`pe.api.LLMAugPE`: The APIs for text generation using LLMs. When constructing the instance of this API, an LLM instance is required. The LLM instances follow the interface of :py:class:`pe.llm.LLM`. Currently, the following LLMs are implemented:
+    * :py:class:`pe.api.LLMAugPE`: The APIs for text generation using LLMs (introduced in [#pe2]_). When constructing the instance of this API, an LLM instance is required. The LLM instances follow the interface of :py:class:`pe.llm.LLM`. Currently, the following LLMs are implemented:
 
         * :py:class:`pe.llm.OpenAILLM`: The LLMs from OpenAI APIs.
         * :py:class:`pe.llm.AzureOpenAILLM`: The LLMs from Azure OpenAI APIs.
@@ -31,6 +34,19 @@ Adding Your Own APIs
 
 To add your own APIs, you need to create a class that inherits from :py:class:`pe.api.API` and implements the :py:meth:`pe.api.API.random_api` and :py:meth:`pe.api.API.variation_api` methods.
 
-    
+
+.. rubric:: Citations
+
+.. [#pe1] `Differentially Private Synthetic Data via Foundation Model APIs 1: Images (ICLR 2024) <pe1_paper_>`__.
+.. [#pe2] `Differentially Private Synthetic Data via Foundation Model APIs 2: Text (ICML 2024 Spotlight) <pe2_paper_>`__.
+.. [#pe3] `Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Models <pe3_paper_>`__.
+
+
 .. _improved diffusion model: https://github.com/openai/improved-diffusion
 .. _Stable Diffusion: https://huggingface.co/CompVis/stable-diffusion-v1-4
+
+.. _python_avatars: https://github.com/ibonn/python_avatars
+
+.. _pe1_paper: https://arxiv.org/abs/2305.15560
+.. _pe2_paper: https://arxiv.org/abs/2403.01749
+.. _pe3_paper: https://arxiv.org/abs/2502.05505

doc/source/getting_started/details/callback_and_logger.rst

@@ -14,11 +14,13 @@ Currently, the following callbacks are implemented:
 
     * :py:class:`pe.callback.ComputeFID`: Computes the FID between the synthetic samples and the private samples.
     * :py:class:`pe.callback.SaveCheckpoints`: Saves the checkpoint of current synthetic samples to files.
+    * :py:class:`pe.callback.ComputePrecisionRecall`: Computes the `precision and recall metrics`_ between the synthetic samples and the private samples.
 
 * Images
 
     * :py:class:`pe.callback.SampleImages`: Samples some images from each class.
     * :py:class:`pe.callback.SaveAllImages`: Saves all synthetic images to files.
+    * :py:class:`pe.callback.DPImageBenchClassifyImages`: Trains classifiers on the synthetic images and evaluates them on the private images, following the settings in `DPImageBench`_.
 
 * Text
 
@@ -34,3 +36,6 @@ Currently, the following loggers are implemented:
 * :py:class:`pe.logger.LogPrint`: Prints the float results to the console and/or files using the logging module.
 * :py:class:`pe.logger.ImageFile`: Saves the images to files.
 * :py:class:`pe.logger.MatplotlibPDF`: Saves the matplotlib plots to PDF files.
+
+.. _precision and recall metrics: https://arxiv.org/abs/1904.06991
+.. _DPImageBench: https://github.com/2019ChenGong/DPImageBench

doc/source/getting_started/details/data.rst

@@ -19,6 +19,9 @@ For convenience, some well-known datasets are already packaged as :py:class:`pe.
     * :py:class:`pe.data.Cifar10`: The `CIFAR10 dataset`_.
     * :py:class:`pe.data.Camelyon17`: The `Camelyon17 dataset`_.
     * :py:class:`pe.data.Cat`: The `Cat dataset`_.
+    * :py:class:`pe.data.MNIST`: The `MNIST dataset`_.
+    * :py:class:`pe.data.CelebA`: The `CelebA dataset`_.
+    * :py:class:`pe.data.DigiFace1M`: The `DigiFace1M dataset`_.
     * In addition, you can easily load a custom image dataset from a (nested) directory with the image files using :py:meth:`pe.data.load_image_folder`.
 
 * Text datasets
@@ -50,6 +53,9 @@ To apply **Private Evolution** to your own private dataset, you need to create a
 .. _Cat dataset: https://www.kaggle.com/datasets/fjxmlzn/cat-cookie-doudou
 .. _CIFAR10 dataset: https://www.cs.toronto.edu/~kriz/cifar.html
 .. _Camelyon17 dataset: https://camelyon17.grand-challenge.org/
+.. _MNIST dataset: https://pytorch.org/vision/main/generated/torchvision.datasets.MNIST.html
+.. _CelebA dataset: https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html
+.. _DigiFace1M dataset: https://github.com/microsoft/DigiFace1M
 .. _Yelp dataset: https://github.com/AI-secure/aug-pe/tree/main/data
 .. _OpenReview dataset: https://github.com/AI-secure/aug-pe/tree/main/data
 .. _PubMed dataset: https://github.com/AI-secure/aug-pe/tree/main/data

doc/source/getting_started/details/embedding.rst

@@ -16,10 +16,12 @@ Currently, the following embeddings are implemented:
 * Images
 
     * :py:class:`pe.embedding.Inception`: The embeddings computed using the Inception model.
+    * :py:class:`pe.embedding.FLDInception`: The embeddings computed using the Inception model following the procedure in the `FLD`_ library.
 
 * Text
 
     * :py:class:`pe.embedding.SentenceTransformer`: The embeddings computed using the `Sentence Transformers`_ library.
 
 
 .. _Sentence Transformers: https://sbert.net/
+.. _FLD: https://github.com/marcojira/fld

doc/source/getting_started/details/runner.rst

@@ -6,3 +6,4 @@ API reference: :doc:`/api/pe.runner`.
 :py:class:`pe.runner.PE` manages the main **Private Evolution** algorithm by calling the other components discussed before. It has the following key methods:
 
 * :py:meth:`pe.runner.PE.run`: Runs the **Private Evolution** algorithm.
+* :py:meth:`pe.runner.PE.evaluate`: Evaluates the synthetic samples using the :py:class:`pe.callback.Callback` modules.

doc/source/getting_started/examples.rst

@@ -16,7 +16,7 @@ Images
 
     * **MNIST dataset**: `This example <MNIST example_>`__ shows how to generate differentially private synthetic images for the `MNIST dataset`_ using a text render.
     * **CelebA dataset (simulator-generated data)**: `This example <CelebA DigiFace1M example_>`__ shows how to generate differentially private synthetic images for the `CelebA dataset`_ using `the generated data from a computer graphics-based renderer for face images <DigiFace1M_>`__.
-    * **CelebA dataset (weak simulator)**: `This example <CelebA avatar example_>`__ shows how to generate differentially private synthetic images for the `CelebA dataset`_ using `a rule-based avatar generator <python_avatar_>`__.
+    * **CelebA dataset (weak simulator)**: `This example <CelebA avatar example_>`__ shows how to generate differentially private synthetic images for the `CelebA dataset`_ using `a rule-based avatar generator <python_avatars_>`__.
 
 Text
 ----
@@ -43,7 +43,7 @@ These examples follow the experimental settings in the paper `Differentially Pri
 .. _Stable Diffusion: https://huggingface.co/CompVis/stable-diffusion-v1-4
 
 .. _DigiFace1M: https://github.com/microsoft/DigiFace1M
-.. _python_avatar: https://github.com/ibonn/python_avatars
+.. _python_avatars: https://github.com/ibonn/python_avatars
 
 .. _Cat dataset: https://www.kaggle.com/datasets/fjxmlzn/cat-cookie-doudou
 .. _CIFAR10 dataset: https://www.cs.toronto.edu/~kriz/cifar.html