From 8aaa0a6f4e504ecd1a973c85b09730467e72cdd2 Mon Sep 17 00:00:00 2001 From: rohitanshu <85547195+iamrohitanshu@users.noreply.github.com> Date: Tue, 21 May 2024 23:22:22 +0530 Subject: [PATCH] Fixed minor typo in docs - Training Tab.md (#6038) --- docs/05 - Training Tab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/05 - Training Tab.md b/docs/05 - Training Tab.md index 590117a3..eba62cb3 100644 --- a/docs/05 - Training Tab.md +++ b/docs/05 - Training Tab.md @@ -124,7 +124,7 @@ When you're running training, the WebUI's console window will log reports that i "Loss" in the world of AI training theoretically means "how close is the model to perfect", with `0` meaning "absolutely perfect". This is calculated by measuring the difference between the model outputting exactly the text you're training it to output, and what it actually outputs. -In practice, a good LLM should have a very complex variable range of ideas running in its artificial head, so a loss of `0` would indicate that the model has broken and forgotten to how think about anything other than what you trained it. +In practice, a good LLM should have a very complex variable range of ideas running in its artificial head, so a loss of `0` would indicate that the model has broken and forgotten how to think about anything other than what you trained it on. So, in effect, Loss is a balancing game: you want to get it low enough that it understands your data, but high enough that it isn't forgetting everything else. Generally, if it goes below `1.0`, it's going to start forgetting its prior memories, and you should stop training. In some cases you may prefer to take it as low as `0.5` (if you want it to be very very predictable). Different goals have different needs, so don't be afraid to experiment and see what works best for you.