graph_trainer debug log

[githubsgi]

Is there a graph_trainer log , like the following for dynamo, inductor , etc ?

TORCH_LOGS="+dynamo,+inductor,graph_breaks,recompiles,guards,output_code" TORCH_COMPILE_DEBUG=1

[yiming0416]

@githubsgi
The graph_trainer experiment has two compilation mode.

• By default it uses --compile.mode aot, you can find the dumped graphs under ./output/compile/ after running a training job under the AOT mode.

• You can also use --compile.mode jit, which uses torch.compile(fullgraph=True), under this mode, I think you can just use TORCH_LOGS as usual under the JIT mode.

We will be adding more logging and debugging utils so stay tuned!

[githubsgi]

@yiming0416, thanks. will check those out. Couple more quick questions.

1. Where do I find complete list of options available for both modes ?
2. How complete are the unit tests and integration tests at this point ?