Onnxruntime memory(RAM) usage for CUDAExecutionProvider seems higher than CPUExecutionProvider

[durgaivelselvan-mn-17532]

If I create the InferenceSession for my ONNX model in CPUExecutionProvider, the memory usage of the program seems to be minimal, but if I choose CUDAExecutionProvider as the provider, it uses more RAM than CPUExecutionProvider. Why is it so? Any particular reasons.
Here is the code I've used to profile memory usage,

import torch
import psutil
import gc
import time
import sys
import os
from onnxruntime import InferenceSession

class TestModel(torch.nn.Module):
    def __init__(self) -> None:
        super(TestModel, self).__init__()
        self.ip_l = torch.nn.Linear(1, 512)
        self.hidden = torch.nn.Sequential(*([torch.nn.Linear(512, 512)] * 1000))
        self.op_l = torch.nn.Linear(512, 1)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        self.op_l(self.hidden(self.ip_l(x)))
        return x

if __name__ == "__main__":
    ovr_ram = psutil.virtual_memory().total / (1024 * 1024)
    torch.onnx.export(
        TestModel(),
        torch.rand((1, 1)),
        "./test.onnx",
        verbose=True,
        input_names=["input"],
        output_names=["output"],
    )

    proc = psutil.Process()
    time.sleep(1)
    print(f"Before model init, \n\tRSS(mb) - {(proc.memory_percent() / 100) * ovr_ram}")

    model = InferenceSession("./test.onnx", providers=["CUDAExecutionProvider"], provider_options={"device_id": "0"})
    time.sleep(1)
    print(f"Model {id(model)} of size {getsize(model) / 1024} kb at {model.get_providers()}, \n\tRSS(mb) - {(proc.memory_percent() / 100) * ovr_ram}")

So, that means, for CUDAExecutionProvider, will there be a copy of CUDA nodes in RAM, along with the nodes assigned to CPUExecutionProvider?
Executed on Ubuntu 22.07 x86_64 with onnxruntime-gpu:1.13.1 and torch:2.0.0+cu117