ViperEkura
/
AstrAI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(tools): 修正训练脚本中的嵌入层参数分组判断条件

This commit is contained in:
ViperEkura 2025-11-19 17:47:33 +08:00
parent 3c7ed84516
commit 3bf2468905
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
tools/train.py
View File

@ -80,8 +80,8 @@ def train(
) )
param_groups = [ param_groups = [
{"params": [p for n, p in model.named_parameters() if "embedding" in n], "lr": max_lr * embdeding_lr_rate}, {"params": [p for n, p in model.named_parameters() if "embed" in n], "lr": max_lr * embdeding_lr_rate},
{"params": [p for n, p in model.named_parameters() if "embedding" not in n], "lr": max_lr} {"params": [p for n, p in model.named_parameters() if "embed" not in n], "lr": max_lr}
] ]
optim = AdamW( optim = AdamW(