Skip to content

Commit c873858

Browse files
committed
Take pagination into account
Instead of requesting the whole list of servers, use pagination so that we can iter over a large number of servers without reaching any limit. Close #35
1 parent 6b99fdb commit c873858

File tree

1 file changed

+13
-1
lines changed

1 file changed

+13
-1
lines changed

caso/extract/nova.py

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,19 @@ def extract_for_project(self, project, lastrun, extract_to):
7474

7575
flavors = {flavor.id: flavor for flavor in nova.flavors.list()}
7676

77-
servers = nova.servers.list(search_opts={"changes-since": lastrun})
77+
servers = []
78+
limit = 200
79+
marker = None
80+
# Iter over results until we do not have more to get
81+
while True:
82+
aux = nova.servers.list(search_opts={"changes-since": lastrun},
83+
limit=limit,
84+
marker=marker)
85+
servers.extend(aux)
86+
87+
if len(aux) < limit:
88+
break
89+
marker = aux[-1].id
7890

7991
servers = sorted(servers, key=operator.attrgetter("created"))
8092

0 commit comments

Comments
 (0)