basecls.solver.optimizer.lars#
LARS optimizer
References: rwightman/pytorch-image-models
- class basecls.solver.optimizer.lars.LARS(params, lr, momentum=0.0, nesterov=False, weight_decay=0.0, always_adapt=False)[源代码]#
基类:
Optimizer
Implements LARS algorithm.
LARS is proposed in “Large Batch Optimization for Deep Learning: Training BERT in 76 minutes”.
- 参数
params (
Union
[Iterable
[Parameter
],dict
]) – iterable of parameters to optimize or dicts defining parameter groups.lr (
float
) – learning rate.momentum (
float
) – momentum factor. Default:0.0
nesterov (
bool
) – enables Nesterov momentum. Default:False
weight_decay (
float
) – weight decay (L2 penalty). Default:0.0
always_adapt (
bool
) – apply adaptive lr to0.0
weight decay parameter. Default:False