Optimize vector store pagination for large datasets

[coderabbitai]

Description

The current implementation in ovos_persona_server/vector_stores.py (around lines 225-229) fetches all vector stores into memory before applying pagination logic. This approach is inefficient for large datasets.

Suggested Improvement

Implement database-level pagination using one of the following approaches:

• LIMIT/OFFSET pagination
• Cursor-based pagination

This will ensure only the required subset of records is fetched directly from the database, improving performance and memory efficiency.

References

• Pull Request: feat: embeddings #11
• Original Comment: feat: embeddings #11 (comment)
• Requested by: @JarbasAl