Warning: find_unused_parameters=True was specified in DDP constructor……

项目场景

使用 Pytorch Lightning 训练模型。

问题描述

Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters. This flag results in an extra traversal of the autograd graph every iteration, which can adversely affect performance.

解决方案

Trainer(strategy="ddp_find_unused_parameters_false")

引用参考

https://github.com/PyTorchLightning/pytorch-lightning/discussions/6761