✨ Set up comprehensive Copilot instructions with complete API design architecture and framework overview for Python 3.12 and CUDA 12 (#1031)

* Initial plan

* Add comprehensive GitHub Copilot instructions for NVIDIA NeMo Curator

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Enhance Copilot instructions with explicit Python 3.12 and CUDA 12 focus

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Address review feedback: add PyTorch, remove unsupported fallback patterns, update API examples

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Add comprehensive API design principles and core components from api-design.md

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Expand API design section with comprehensive background and context from api-design.md

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Remove Dask references and background sections per review feedback

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Update task-centric architecture description to mention Ray

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Add comprehensive framework overview with key components, execution flow, and development guidelines

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Addressing PR comments

Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>

* Update .github/copilot-instructions.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Vibhu Jawa <vibhujawa@gmail.com>

---------

Signed-off-by: Vibhu Jawa <vibhujawa@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: VibhuJawa <4837571+VibhuJawa@users.noreply.github.com>
Co-authored-by: Vibhu Jawa <vjawa@nvidia.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: NeMo Bot <nemo-bot@nvidia.com>

Author: 3 people