FedData collaborated with Intel to perform independent benchmark testing and analysis on the Gaudi 3 accelerator in our High Performance Computing Demo Lab and compared our results to similar tests Intel performed and against competing products for Enterprise and US Government AI deployments.
Problem: Agencies and enterprises want to scale AI but face obstacles—legacy infrastructure, budget limits, high GPU costs, energy constraints, and the need for open and auditable systems .
Solution: Gaudi 3 accelerators paired with Intel Xeon CPUs proved to perform as an equal or lower cost-effective, energy-efficient, and viable solution for AI deployments.
Testing: FedData benchmarked Gaudi 3 in realistic government-style workloads (LLMs, RAG chatbots, concurrent inference) using models like Llama 3 and Mistral to measure performance, throughput, latency, and power efficiency .
📊 Key Findings
- Gaudi 3 is up to 6× faster than Gaudi 2 on queries per second and token generation .
- Under high concurrency, Gaudi 3 achieves ~95% of H100’s raw throughput but with significantly better price-performance (up to 2.5×) and lower power draw than other alternatives.
- Efficient scaling across 70B-parameter LLMs shows Gaudi 3 can handle very large models without huge power/cooling penalties .
- Linear power draw and strong thermal stability make it ideal for constrained federal data centers .
- Built-in Intel security features (e.g., SGX, TME) and open frameworks (PyTorch, Hugging Face integration) align with federal AI governance needs .
🏛️ Significance
- For Government: Gaudi 3 offers an alternative to competing GPUs that costs less, is more sustainable, and is compliant with federal mandates. This lowers both CapEx and OpEx while meeting mission-critical performance demands.
- For Industry: Provides a way to scale AI infrastructure without vendor lock-in, enabling open software stacks and interoperability.
- Strategic: Strengthens Intel’s positioning in the U.S. government AI modernization market, with FedData acting as the trusted integrator for secure deployments .
To download a copy of the White Paper click here.