GPT-4 Shatters Records: AI Giant Emerges with 1.8 Trillion Parameters
April 28, 2024GPT-4 is a new AI model with an unprecedented 1.8 trillion parameters, dwarfing its predecessor GPT-3 by tenfold.
The model incorporates 16 specialized experts, each with a substantial 111 billion parameters focused on multi-layer perceptrons (MLPs).
GPT-4's training involved 13 trillion tokens, spanning both text and code, and cost an astounding $63 million.
Operating GPT-4 requires considerable resources, including 128 GPUs and advanced parallelism techniques, and it's three times costlier to run than its predecessor.
The model features a breakthrough vision encoder, enhancing its ability to interpret visual data from web pages, images, and videos.
Looking ahead, GPT-5 promises to exponentially increase complexity, suggesting ongoing, rapid advancement in AI capabilities.
Summary based on 1 source
Get a daily email with more Tech stories
Source
DEV Community • Apr 27, 2024
DO YOU KNOW HOW BIG IS GPT-4?