Shamraiz | December 28, 2024
In a major breakthrough for the AI world, Chinese AI company DeepSeek has launched the DeepSeek V3, which is considered one of the most powerful “open” AI models ever released. The model, issued under a permissive license, can be downloaded by developers free of charge to modify and use for different purposes, including commercial applications.
DeepSeek V3 is an incredibly versatile model across text-based tasks. From generating code and translating languages to writing essays and crafting emails, the model delivers high-quality outputs with great ease. According to internal benchmarks, DeepSeek V3 outperforms not only its open-source competitors but also certain “closed” AI models that are only accessible via APIs.
In programming competitions hosted on Codeforces, DeepSeek V3 surpassed prominent models such as Meta’s Llama 3.1 405B, OpenAI’s GPT-4o, and Alibaba’s Qwen 2.5 72B. It also showcased dominance on Aider Polyglot, a specialized test that evaluates a model’s ability to write new code that integrates seamlessly with existing codebases.
A Technological Marvel – Performance and Specifications
DeepSeek V3’s technical specifications are impressive:
Speed: Processes 60 tokens per second—three times faster than its predecessor.
Scalability: Fully compatible with API integrations and open-source platforms.
Parameters: Boasts 671 billion MoE (Mixture of Experts) parameters, with 37 billion activated at any given time.
Training Data: Trained on a dataset containing 14.8 trillion high-quality tokens.
These advancements make DeepSeek V3 nearly 1.6 times the size of Meta’s Llama 3.1 and position it as a formidable force in the AI landscape.
Cost-Effective Training Amid Restrictions
One of the more astonishing aspects of the development of DeepSeek V3 is perhaps its cost efficiency. Functioning under U.S. restrictions on the purchase of high-end GPUs, DeepSeek was able to train this model on Nvidia H800 GPUs in just two months, spending only $5.5 million—what similar models require.
This took place in a data center significantly smaller than what other industries would use for something of this caliber.
While DeepSeek V3 is a technological advancement, it also represents the difficulties of operating under China’s regulatory environment. For example, the model does not discuss politically sensitive topics such as the Tiananmen Square incident, which aligns with the Chinese government’s guidelines to “embody core socialist values.” This limitation limits the model’s ability to provide fully impartial information.
High-Flyer Capital – Driving AI Innovation
DeepSeek is headquartered by Chinese quantitative hedge fund High-Flyer Capital Management. High-Flyer’s reputation lies in employing AI on trading strategies; its significant infrastructure is based on server clusters that count 10,000 Nvidia A100 GPUs – a testament to the ability to gain breakthroughs through AI.
High-Flyer’s founder, Liang Wenfeng, is a pioneer who sees the day where open-source AI models catch up with, if not surpass, closed-source systems such as OpenAI. “Closed systems are only a temporary advantage,” Wenfeng said, underlining the inevitability of technological parity.
