feat: Compile model(s) using torch.compile
Using torch
2.0, we should compile our code.
Currently, the compilation crashes.
Correction: Compilation works, but recompiles on every step, e.g.:
Torchdynamo Profiler Report
These subgraphs were recompiled more than once due to guard failures.Guard failures indicate some condition assumed to be static by the tracer changed, making it unsafe to reuse the compiled
program.Function, Num Recompiles, Recompile Reasons
'forward' (classifier.py:58), 4,
tensor 'input_ids' strides mismatch at index 0. expected 51, actual 33
tensor 'input_ids' strides mismatch at index 0. expected 51, actual 41
tensor 'input_ids' strides mismatch at index 0. expected 51, actual 46
tensor 'input_ids' strides mismatch at index 0. expected 51, actual 35
Edited by Niklas Bauer