The monstrous server farm shown in the video is Elon Musk's latest toy, built to train large language models (LLMs) such as ChatGPT.
It is considered the most powerful server farm of its kind in the world, and it is designed to produce the most advanced language model to date by the end of this year.
---
**How is a language model trained?**
At the core of a language model are "parameters" — rules and reasoning patterns that guide how it generates text.
The training stages (broadly divided) include:
1. **Data Preparation**
Both the quantity and the quality and reliability of the data matter enormously. The collected data is processed and split into short segments.
2. **Forward Pass**
The model reads the text and attempts to predict the next words in a sentence based on what it has seen.
3. **Loss Calculation**
At this stage the model still makes many errors in predicting the next word. It learns from those mistakes and uses them to guide further training.
4. **Backward Pass**
By comparing its incorrect predictions against the correct data, the model determines how to update its parameters so that future predictions will be more accurate.
5. **Parameter Update**
The model applies the parameter updates planned in the previous stage.
6. **Cycles**
These five steps repeat over and over, with the model refining its parameters and becoming more accurate with each iteration.
---
The servers shown in the image are called **GPU clusters** — arrays of graphics processing units.
Even though this is not a graphics application, GPUs are used instead of conventional CPUs.
The reason is that GPUs contain a massive number of relatively simple processing cores, in contrast to conventional CPUs, which have only a handful of highly capable cores.
Training a language model does not require especially complex computation so much as it requires that computation to run simultaneously across as many processing cores as possible — even weaker ones — which is exactly why GPUs are better suited to the task.
---
And one more interesting fact to close with — the tremendous noise these servers produce has nothing to do with the computation itself.
The computation runs in complete silence; the noise is generated entirely by the fans working at full speed to cool the processors.