How one word relates to another and in which context it can be used. If this can be calculated precisely, then would that be the same as knowing exactly what to say and when?

How can a sequence of words be generated?

The 3 general approaches language models use according to the report (Eovito & Danilevsky, 2021, p.8) are:

1. Linguistic - by following a set of rules (grammatical structure)

2. Probabilistic/statistical - by calculating the probability of words occurring together

3. Embeddings - by representing words as vectors in a large dimensional space

I like learning languages. And it would be interesting to read more about how machines "learn" languages.

Reference

Eovito, A., Danilevsky, M. (2021). Language Models in Plain English: Humans Understanding How Machines Understand Language. O’Reilly Media Inc. https://learning.oreilly.com/library/view/language-models-in/9781098109073/