Weight decay penalizes large weights by shrinking them slightly at each update, which is equivalent to L2 regularization. It limits the complexity of the network and overfitting. It is set in the optimizer; with Adam, the AdamW variant properly decouples it from the gradient.
Example
torch.optim.AdamW(modele.parameters(), lr=1e-3, weight_decay=1e-2)