Purge InternVideo2 (#1451)

* Remove Internvideo2

Signed-off-by: Ao Tang <aot@nvidia.com>

* more to remove

Signed-off-by: Ao Tang <aot@nvidia.com>

* fix writer

Signed-off-by: Ao Tang <aot@nvidia.com>

* Enhance Clip class to include cosmos_embed1_frames and cosmos_embed1_embedding in total size calculation

Signed-off-by: Ao Tang <aot@nvidia.com>

* remove iv2

Signed-off-by: Ao Tang <aot@nvidia.com>

---------

Signed-off-by: Ao Tang <aot@nvidia.com>
Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com>
Signed-off-by: NeMo Bot <nemo-bot@nvidia.com>

Author: 2 people

.cursor/rules/modality-structure.mdc

@@ -58,7 +58,7 @@ Common operations:
 - Clip extraction
 - GPU H.264 encoding/decoding
 - Motion and aesthetic filtering
-- Embeddings (InternVideo2, Cosmos-Embed1)
+- Embeddings (Cosmos-Embed1)
 
 Task type: `VideoTask`
 

.github/workflows/cicd-main.yml

@@ -144,26 +144,6 @@ jobs:
       - uses: actions/checkout@v4
         with:
           submodules: recursive
-
-      - name: Cache InternVideo
-        id: cache-internvideo
-        uses: actions/cache@v4
-        with:
-          path: InternVideo
-          key: internvideo-${{ hashFiles('external/intern_video2_multimodal.patch') }}-09d872e5093296c6f36b8b3a91fc511b76433bf7
-
-      - name: Checkout InternVideo
-        if: steps.cache-internvideo.outputs.cache-hit != 'true'
-        uses: actions/checkout@v4
-        with:
-          repository: OpenGVLab/InternVideo
-          path: InternVideo
-          ref: 09d872e5093296c6f36b8b3a91fc511b76433bf7
-
-      - name: Patch InternVideo
-        if: steps.cache-internvideo.outputs.cache-hit != 'true'
-        run: cd InternVideo && patch -p1 < ../external/intern_video2_multimodal.patch
-
       - name: Free up disk space on Ubuntu
         run: |
           sudo rm -rf /usr/share/dotnet
@@ -178,9 +158,7 @@ jobs:
       - name: Run tests ${{ matrix.folder }} (CPU)
         timeout-minutes: 40
         run: |
-          sed -i "/InternVideo/d" .gitignore
           uv sync --link-mode copy --locked --extra audio_cpu --extra text_cpu --extra video_cpu --group test
-          uv add InternVideo/InternVideo2/multi_modality
           FOLDER="${{ matrix.folder }}"
           FOLDER="${FOLDER/stages-/stages/}"
           uv run coverage run --branch --source=nemo_curator -m pytest -v "tests/$FOLDER" -m "not gpu"

.gitignore

@@ -155,6 +155,3 @@ data/
 
 # macOS Files
 .DS_Store
-
-# InternVideo2 dependency (cloned by installation script)
-InternVideo/

CHANGELOG.md

@@ -35,7 +35,7 @@ NeMo Curator now supports comprehensive [video data curation](https://docs.nvidi
 - **Video splitting**: [Fixed-stride](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/clipping.html) and [scene-change detection (TransNetV2)](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/clipping.html) for clip extraction
 - **Semantic deduplication**: [K-means clustering and pairwise similarity](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/dedup.html) for near-duplicate clip removal
 - **Content filtering**: [Motion-based filtering](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/filtering.html) and [aesthetic filtering](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/filtering.html) for quality improvement
-- **Embedding generation**: InternVideo2 and Cosmos-Embed1 models for clip-level embeddings
+- **Embedding generation**: Cosmos-Embed1 models for clip-level embeddings
 - **Ray-based distributed architecture**: Scalable video processing with autoscaling support
 
 #### Audio

README.md

@@ -74,7 +74,7 @@ Process large-scale video corpora with distributed, GPU-accelerated pipelines fo
 | **Data Loading** | Local paths • S3-compatible storage • HTTP(S) URLs | [Load Data](https://docs.nvidia.com/nemo/curator/latest/curate-video/load-data/index.html) |
 | **Clipping** | Fixed-stride splitting • Scene-change detection (TransNetV2) | [Clipping](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/clipping.html) |
 | **Processing** | GPU H.264 encoding • Frame extraction • Motion filtering • Aesthetic filtering | [Processing](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/filtering.html) |
-| **Embeddings** | InternVideo2 and Cosmos-Embed1 for clip-level embeddings | [Embeddings](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/embeddings.html) |
+| **Embeddings** | Cosmos-Embed1 for clip-level embeddings | [Embeddings](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/embeddings.html) |
 | **Deduplication** | K-means clustering • Pairwise similarity for near-duplicates | [Deduplication](https://docs.nvidia.com/nemo/curator/latest/curate-video/process-data/dedup.html) |
 
 ---

docker/Dockerfile

@@ -62,16 +62,6 @@ COPY docker/common/install_ffmpeg.sh .
 RUN bash install_ffmpeg.sh && \
     rm install_ffmpeg.sh
 
-
-ARG INTERN_VIDEO_COMMIT=09d872e5093296c6f36b8b3a91fc511b76433bf7
-COPY external/intern_video2_multimodal.patch .
-# Clone InternVideo (Video curation dependency)
-RUN git clone https://github.com/OpenGVLab/InternVideo.git && \
-    cd InternVideo && \
-    git checkout ${INTERN_VIDEO_COMMIT} && \
-    patch -p1 < /opt/intern_video2_multimodal.patch && \
-    rm /opt/intern_video2_multimodal.patch
-
 FROM nemo_curator_dep AS nemo_curator
 
 WORKDIR /opt/Curator
@@ -81,8 +71,7 @@ COPY pyproject.toml uv.lock /opt/Curator/
 COPY nemo_curator/__init__.py nemo_curator/package_info.py /opt/Curator/nemo_curator/
 
 # Install Curator
-RUN uv sync --link-mode copy --locked --extra all --all-groups --no-cache && \
-  uv add /opt/InternVideo/InternVideo2/multi_modality
+RUN uv sync --link-mode copy --locked --extra all --all-groups --no-cache
 
 COPY . /opt/Curator
 

docs/about/concepts/video/abstractions.md

@@ -38,7 +38,7 @@ A stage represents a single step in your data curation workflow. Video stages ar
 - **Input/Output**: Read video files and write processed outputs to storage ([Save & Export Documentation](video-save-export))
 - **Video Clipping**: Split videos into clips using fixed stride or scene-change detection ([Video Clipping Documentation](video-process-clipping))
 - **Frame Extraction**: Extract frames from videos or clips for analysis and embeddings ([Frame Extraction Documentation](video-process-frame-extraction))
-- **Embedding Generation**: Generate clip-level embeddings using InternVideo2 or Cosmos-Embed1 models ([Embeddings Documentation](video-process-embeddings))
+- **Embedding Generation**: Generate clip-level embeddings using Cosmos-Embed1 models ([Embeddings Documentation](video-process-embeddings))
 - **Filtering**: Filter clips based on motion analysis and aesthetic quality scores ([Filtering Documentation](video-process-filtering))
 - **Caption and Preview**: Generate captions and preview images from video clips ([Captions & Preview Documentation](video-process-captions-preview))
 - **Deduplication**: Remove near-duplicate clips using embedding-based clustering ([Duplicate Removal Documentation](video-process-dedup))

docs/about/concepts/video/data-flow.md

@@ -29,8 +29,6 @@ Writer stages produce the following directories under the configured output path
 - `filtered_clips/`: MP4 files for filtered clips
 - `previews/`: WebP preview images for windows
 - `metas/v0/`: Per-clip JSON metadata files
-- `iv2_embd/`: Per-clip embeddings (pickle) for InternVideo2
-- `iv2_embd_parquet/`: Aggregated per-video embeddings (parquet) for InternVideo2
 - `ce1_embd/`: Per-clip embeddings (pickle) for Cosmos-Embed1
 - `ce1_embd_parquet/`: Aggregated per-video embeddings (parquet) for Cosmos-Embed1
 - `processed_videos/`: Per-video JSON metadata files

docs/about/release-notes/index.md

@@ -64,7 +64,7 @@ NeMo Curator now supports comprehensive [video data curation](../../curate-video
 - **Video splitting**: [Fixed-stride](../../curate-video/process-data/clipping.md) and [scene-change detection (TransNetV2)](../../curate-video/process-data/clipping.md) for clip extraction
 - **Semantic deduplication**: [K-means clustering and pairwise similarity](../../curate-video/process-data/dedup.md) for near-duplicate clip removal
 - **Content filtering**: [Motion-based filtering](../../curate-video/process-data/filtering.md) and [aesthetic filtering](../../curate-video/process-data/filtering.md) for quality improvement
-- **Embedding generation**: InternVideo2 and Cosmos-Embed1 models for clip-level embeddings
+- **Embedding generation**: Cosmos-Embed1 models for clip-level embeddings
 - **Enhanced captioning**: [VL-based caption generation with optional LLM-based rewriting](../../curate-video/process-data/captions-preview.md) (Qwen-VL and Qwen-LM supported) for detailed video descriptions
 - **Ray-based distributed architecture**: Scalable video processing with [autoscaling support](../concepts/video/architecture.md)
 

docs/admin/installation.md

@@ -87,13 +87,6 @@ curl -LsSf https://astral.sh/uv/install.sh | sh
 uv sync --all-extras --all-groups
 ```
 
-Optional InternVideo2 installation steps:
-
-```bash
-bash external/intern_video2_installation.sh
-uv add InternVideo/InternVideo2/multi_modality
-```
-
 :::
 
 :::{tab-item} Container Installation
@@ -164,40 +157,6 @@ If encoders are missing, reinstall `FFmpeg` with the required options or use the
 :::
 ::::
 
-### InternVideo2 Support (Optional for Video)
-
-Video processing includes optional support for InternVideo2. To install InternVideo2, run these commands before installing NeMo Curator based on whether you install via PyPI or from source:
-
-::::{tab-set}
-
-:::{tab-item} PyPI Installation
-```bash
-# Clone and set up InternVideo2
-git clone https://github.com/OpenGVLab/InternVideo.git
-cd InternVideo
-git checkout 09d872e5093296c6f36b8b3a91fc511b76433bf7
-
-# Download and apply NeMo Curator patch
-curl -fsSL https://raw.githubusercontent.com/NVIDIA/NeMo-Curator/main/external/intern_video2_multimodal.patch -o intern_video2_multimodal.patch
-patch -p1 < intern_video2_multimodal.patch
-cd ..
-
-# Add InternVideo2 to the environment
-uv add InternVideo/InternVideo2/multi_modality
-```
-
-:::
-
-:::{tab-item} Source Installation
-```bash
-# Inside the NeMo Curator folder
-bash external/intern_video2_installation.sh
-uv add InternVideo/InternVideo2/multi_modality
-```
-
-:::
-::::
-
 ---
 
 ## Package Extras