Refining the generated output
In the vast ocean of language, every word is a potential catch, waiting to be brought to the surface. However, just as a fisher must break down their net to catch the right fish, language models must break down human language into smaller, manageable pieces—tokens, as we’ve seen earlier. But that isn’t enough; the goal is to gauge how creative or focused we want our language model to be. Just as a fisher might adjust their technique depending on whether they want a wide variety of fish or just the most prized catch, we can adapt the creativity of our model’s output using the temperature parameter. Lowering the temperature makes the model more focused and predictable, like aiming for a specific species of fish, while raising it adds creativity and variety, casting a wider net to see what comes in. Parameters such as top_p and top_k further refine this process. top_p acts like a selective net that only keeps the most significant...
