Skip to content

Commit 85ae339

Browse files
committed
Prevent single file storage from performing unnecessary N^2 loop
For single file storage we wrap the logic of get_multi with the at_once context manager so that `self.list()` (which is called by `self.get()`) actually caches the items rather than re-computing them at every call. This should largely mitigate the performance issue describe at #818 . The issue discussion also contains more background about this patch.
1 parent 54a90aa commit 85ae339

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

vdirsyncer/storage/singlefile.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,15 @@
44
import glob
55
import logging
66
import os
7+
from typing import Iterable
78

89
from atomicwrites import atomic_write
910

1011
from .. import exceptions
1112
from ..utils import checkfile
1213
from ..utils import expand_path
1314
from ..utils import get_etag_from_file
15+
from ..utils import uniq
1416
from ..vobject import Item
1517
from ..vobject import join_collection
1618
from ..vobject import split_collection
@@ -132,6 +134,12 @@ async def get(self, href):
132134
except KeyError:
133135
raise exceptions.NotFoundError(href)
134136

137+
async def get_multi(self, hrefs: Iterable[str]):
138+
async with self.at_once():
139+
for href in uniq(hrefs):
140+
item, etag = await self.get(href)
141+
yield href, item, etag
142+
135143
@_writing_op
136144
async def upload(self, item):
137145
href = item.ident

0 commit comments

Comments
 (0)