-
Notifications
You must be signed in to change notification settings - Fork 32
Open
Labels
BugSomething isn't workingSomething isn't working
Description
Description
After merging the #128, shape inference will be disallowed and with it, passes that produce shapeless tensors.
In the current Deeploy codebase, I have stumbled upon only one such pass, and that is the MemPoolSplitMHSAPass which gets triggered on the Tests/ICCT_ITA_8 test.
The aforementioned PR disables that test until the Pass gets fixed and this issue follows that task.
To Reproduce
Run the following command from DeeployTest:
python testRunner_mempool.py -t Tests/ICCT_ITA_8
and Deeploy will shout at you.
To confirm that it is MemPoolSplitMHSAPass that creates shapeless tensors, I have created the following TopologyOptimizer:
class MemPoolTopologyOptimizer(TopologyOptimizer):
def optimize(self, graph: gs.Graph) -> Tuple[gs.Graph]:
shapeless = set(t.name for t in graph.tensors().values() if t.shape is None)
if len(shapeless) > 0:
print(f"Graph already has some shapeless tensors: {shapeless}")
for i, _pass in enumerate(self.passes):
prevShapeless = shapeless
graph = _pass.apply(graph)
graph.cleanup().toposort()
shapeless = set(t.name for t in graph.tensors().values() if t.shape is None)
newShapeless = shapeless - prevShapeless
if len(newShapeless) > 0:
print(f"The {i}th pass {type(_pass).__name__} produced tensors without shape. Tensors: {newShapeless}")
breakpoint()
return graphand used it instead of the normal optimizer for the MemPoolOptimizer in Platform.py.
Expected behavior
Deeploy should not shout at you anymore.
Metadata
Metadata
Assignees
Labels
BugSomething isn't workingSomething isn't working