docling with granite-docling-258M on AWS Sagemaker

[kkondaka]

Hi,

I have a question about using docling.
I am able to write a simple python script that uses docling to extract a small pdf file (on my EC2 - CPU only system) and chunk it using HybridChunker and it works fine.
Now I want to try using docling with granite-docling-258M model. It is my understanding that you need to use this model if I want to continuously ingest documents(pdf, word etc) of different sizes (500+ pages). Is my understanding correct?
Also, what is the maximum size of the document for docling running on CPU only system to extract? Are there any guidelines on when to use docling running on CPU vs docling running on GPU?
I am trying to deploy this model to AWS SageMaker endpoint. Is that something that can be done?

Thanks in advance
Krishna