The record-breaking demand for Gemini has led to an interesting development - a new Google chip with Gemini's brain etched into it. But what does that actually mean?
-
The AI monster requires more and more resources and cloud services, and supply is far from sufficient. According to various reports, Google was forced to sign an agreement to pay $920 million a month to use xAI's servers, meaning it is directly helping to fund one of its competitors.
To solve the problem, Google announced that it will develop new chips for Gemini, with the computational processes embedded right at the silicon level. The expectation is that these chips will deliver 6 to 10 times more tokens compared to standard chips, thereby allowing server capacity to increase significantly while saving on energy and costs.
-
Standard processors, like those used in our computers and phones, consist of flexible logic gates capable of executing any computational task given to them via software code. This capability allows them to run millions of different applications effortlessly.
The problem is that this flexibility comes at a price.
It requires storing the code in RAM, allocating processing power to manage the computational processes within the processor, and transferring data back and forth between the processor and the RAM. This extra work translates into increased power consumption and heat generation.
ASIC processors are chips designed to perform a specific operation, such as video encoding or performing calculations for Bitcoin mining. If you have an Apple device, for example, its processor contains a dedicated chip for encoding video in a specific format, allowing it to perform the task highly efficiently and much faster compared to a standard processor.
Such processors do not rely on software. Their transistors are pre-wired to execute the same sequence of operations over and over again without the need for code-based process management. Data enters one side and exits the other after being processed according to a predetermined pattern.
-
Google will not etch the entirety of Gemini into silicon, but only its architecture, so that these chips can also serve future models without needing to be replaced.
The part etched into the silicon is the Transformer - an architecture that Google itself developed, which currently serves as the beating heart of the artificial intelligence industry.
Until a few years ago, language models would examine input token by token and try to find connections. In a Transformer, the model receives the entire input at once and analyzes the connections between all of its parts simultaneously. This method allows for much faster and more efficient input analysis, but its downside is that processing resources scale quadratically with input length. This limits the maximum context window, which currently stands at about a million tokens in most leading models.
Although there are already more efficient alternatives in development, Transformer is still the leading solution in the market, and it seems Google is betting billions on it for the long term as well.
Pictured: TPU 8i - Google's newest TPU chip.
--
👋 Hi, I'm Shlomo Strauss, and my posts are not written by artificial intelligence.
Feel free to follow me for more interesting science and technology content, or search for 'Shlomo Strauss's Blog' on Google.