Skip to content

[BUG] MemPoolSplitMHSAPass produces shapeless tensors #129

@lukamac

Description

@lukamac

Description

After merging the #128, shape inference will be disallowed and with it, passes that produce shapeless tensors.
In the current Deeploy codebase, I have stumbled upon only one such pass, and that is the MemPoolSplitMHSAPass which gets triggered on the Tests/ICCT_ITA_8 test.
The aforementioned PR disables that test until the Pass gets fixed and this issue follows that task.

To Reproduce

Run the following command from DeeployTest:

python testRunner_mempool.py -t Tests/ICCT_ITA_8

and Deeploy will shout at you.

To confirm that it is MemPoolSplitMHSAPass that creates shapeless tensors, I have created the following TopologyOptimizer:

class MemPoolTopologyOptimizer(TopologyOptimizer):

    def optimize(self, graph: gs.Graph) -> Tuple[gs.Graph]:
        shapeless = set(t.name for t in graph.tensors().values() if t.shape is None)
        if len(shapeless) > 0:
            print(f"Graph already has some shapeless tensors: {shapeless}")
        for i, _pass in enumerate(self.passes):
            prevShapeless = shapeless
            graph = _pass.apply(graph)
            graph.cleanup().toposort()
            shapeless = set(t.name for t in graph.tensors().values() if t.shape is None)
            newShapeless = shapeless - prevShapeless
            if len(newShapeless) > 0:
                print(f"The {i}th pass {type(_pass).__name__} produced tensors without shape. Tensors: {newShapeless}")
        breakpoint()
        return graph

and used it instead of the normal optimizer for the MemPoolOptimizer in Platform.py.

Expected behavior

Deeploy should not shout at you anymore.

Metadata

Metadata

Assignees

Labels

BugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions