You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello I have a question,
I have a pytorch geometric dataset and I need at some point to iterate over all my processed paths like this :
`
class myclass(InMemoryDataset):
def init(self):
# Do the usual init
self.stage="true"
self.pre_transform_hash="non"
self.stage="oui"
self.cloud_ids=[x for x in range(1000000)]
def my_test(self):
start=time.time()
l_path=self.processed_file_names
for i in range(len(l_path)):
print("the whole loading took",time.time()-start)
start=time.time()
l_path[i]
print("accessing path i took",time.time()-start)
start=time.time()
self.processed_file_names[i]
print("accessing with processed_files_names took",time.time()-start)
@property
def processed_file_names(self):
return [
osp.join(self.stage, self.pre_transform_hash, f"{w}.h5")
for w in self.cloud_ids
]
`
Accessing the value in assigned_process paths took around 1e-5 second however if I don't assign a variable:
each iteration took over 2 seconds, why? Is processed_paths called every time in the for loop and never cached?
I know my question is quite dumb and not exactly pytorch geometric related
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello I have a question,
I have a pytorch geometric dataset and I need at some point to iterate over all my processed paths like this :
`
class myclass(InMemoryDataset):
def init(self):
# Do the usual init
self.stage="true"
self.pre_transform_hash="non"
self.stage="oui"
`
Accessing the value in assigned_process paths took around 1e-5 second however if I don't assign a variable:
each iteration took over 2 seconds, why? Is processed_paths called every time in the for loop and never cached?
I know my question is quite dumb and not exactly pytorch geometric related
Beta Was this translation helpful? Give feedback.
All reactions