Each colour maps a published value to the exact source passage used to support it. Only bounded excerpts are public; administrators can inspect the complete captured source.
Title
Domyn builds Colosseum 355B, a sovereign AI foundation model, using NVIDIA DGX Cloud
…Join Technical Blog Subscribe Related Resources Data Center / Cloud English 中文 Continued Pretraining of State-of-the-Art LLMs for Sovereign AI and Regulated Industries with Domyn and NVIDIA DGX Cloud Jan 16, 2025 By Martin Cimmino , Paolo Albano , Meriem Bendris , Ziv Ilan , Ser…
Description
Domyn (formerly iGenius), an Italian AI company serving highly regulated sectors such as financial services and public administration, used NVIDIA DGX Cloud with over 3,000 NVIDIA H100 GPUs to continue-pretrain Colosseum 355B, a 355-billion-parameter foundation LLM. Within one week Domyn had access to the dedicated infrastructure, and within two months completed continued pretraining, achieving 82.04% accuracy on the MMLU benchmark. The model powers Domyn's business intelligence agent, Crystal, a sovereign AI solution deployed on private infrastructure.
…and inference optimization. Within one week of signing up to NVIDIA DGX Cloud, Domyn had private access to an environment with over 3K NVIDIA H100 GPUs, all with the following resources: A dedicated, high-bandwidth, RDMA-based netw…
…collaborate with NVIDIA to accelerate their LLM development for Colosseum 355B. In less than one week, Domyn had access to dedicated large-scale infrastructure tuned for AI workloads with over 3K GPUs and within two months, Domyn had completed continued pretraining for their largest LLM, Colosseum 355B. The work included the following: Increasing the number of parameters Increasing…
…ith general human knowledge and reasoning capabilities. At the end of training, Domyn was able to achieve 82.04% accuracy with Colosseum 355B in a 5-shot setting. LLM alignment When an LLM has been trained, it has a general understanding of t…
…Colosseum 355B’s capabilities As a key use case in the context of agentic AI , Domyn develops LLMs to power their business intelligence agent, Crystal , a sovereign AI solution. By building an end-to-end stack, Domyn provide a secure experience without rely…
Company
Domyn
…ust AI platform (software and hardware stack), and advanced AI expertise. Domyn Domyn, previously called iGenius, is an Italian technology company specializing in artificial intelligence for enterprises operating in highly regulated sectors, such as financial services and public administration. Domyn operates between Europe and the United States to put AI at the service of…
Country
Italy
…ust AI platform (software and hardware stack), and advanced AI expertise. Domyn Domyn, previously called iGenius, is an Italian technology company specializing in artificial intelligence for enterprises operating in highly regulated sectors, such as financial services and public administration. Domyn operates between Europe and the United States to put AI at the service of…
Region
Europe
…highly regulated sectors, such as financial services and public administration. Domyn operates between Europe and the United States to put AI at the service of people and businesses. It was founded in 2016 with the mission to humanize data and democratize busine…
Industry
Technology & Software
…ust AI platform (software and hardware stack), and advanced AI expertise. Domyn Domyn, previously called iGenius, is an Italian technology company specializing in artificial intelligence for enterprises operating in highly regulated sectors, such as financial services and public administration. Domyn operates between Europe and the United States to put AI at the service of…
Problem
Domyn aimed to develop a state-of-the-art foundational LLM within a tight timeline but faced challenges accessing large-scale GPU clusters (thousands of GPUs) and securing support for highly scalable training frameworks.
…d in 2016 with the mission to humanize data and democratize business knowledge. Domyn, an NVIDIA Inception partner, aimed to develop a state-of-the-art foundational LLM within a tight timeline but faced challenges in accessing large-scale GPU clusters (thousands of GPUs) and securing support for highly scalable training frameworks. During this engagement, Domyn developed the Colosseum 355B LLM , designed and d…
Solution
Domyn collaborated with NVIDIA to build Colosseum 355B using NVIDIA DGX Cloud infrastructure with over 3,000 H100 GPUs on a dedicated, high-bandwidth RDMA-based network with 500 TB of Lustre-based high-performance storage and the NVIDIA NeMo Framework. Domyn used continued pretraining (CPT) in FP8 precision via NeMo Framework's Transformer Engine, followed by supervised fine-tuning and Direct Preference Optimization (DPO) using NVIDIA NeMo Aligner. The model powers Domyn's business intelligence agent, Crystal, a sovereign AI solution built as an end-to-end stack including database integration, AI-assisted configuration, LLM-powered orchestration for tool usage, query execution and generation, and private deployment infrastructure.
…an environment with over 3K NVIDIA H100 GPUs, all with the following resources: A dedicated, high-bandwidth, RDMA-based network to facilitate model training communications for Colosseum 355B 500 TB of Lustre-based high-performance storage Access to the latest NVIDIA NeM…
…MA-based network to facilitate model training communications for Colosseum 355B 500 TB of Lustre-based high-performance storage Access to the latest NVIDIA NeMo Framework containers Domyn dataset highlights…
…of pretraining to accelerate training and reduce the model’s memory footprint. NeMo Framework integrates FP8 training out-of-the-box with the Transformer Engine library . Enabling FP8 can be done by adding the following parameters to the training c…
…next phase of learning. There are many techniques available to model builders. Domyn focused on supervised fine-tuning and human preferences alignment using Direct Preference Optimization (DPO). Supervised fine-tuning Supervised fine-tuning (SFT) is a foundational step in a…
…zed for chat interactions enabling models to answer in conversational settings. Domyn used NVIDIA NeMo aligner for Colosseum 355B’s chat instruction fine-tuning. The syntax for the chat data template uses the following structure outline: { "…
…to power their business intelligence agent, Crystal , a sovereign AI solution. By building an end-to-end stack, Domyn provide a secure experience without relying on centralized models: Database integration AI-assisted configuration LLM-powered orchestration for tool usage, query execution, and generation Private deployment infrastructure This approach enables Crystal to function as an isolated AI operating system, u…
Business value
In less than one week, Domyn had access to dedicated large-scale infrastructure with over 3K GPUs, and within two months had completed continued pretraining for Colosseum 355B, achieving 82.04% accuracy on the MMLU benchmark in a 5-shot setting. By optimizing training configuration, Domyn improved Model FLOP/s Utilization (MFU) from an initial 25% to 40% in BF16, and accelerated the overall training step 1.15x by moving to FP8.
…collaborate with NVIDIA to accelerate their LLM development for Colosseum 355B. In less than one week, Domyn had access to dedicated large-scale infrastructure tuned for AI workloads with over 3K GPUs and within two months, Domyn had completed continued pretraining for their largest LLM, Colosseum 355B. The work included the following: Increasing the number of parameters Increasing…
…ith general human knowledge and reasoning capabilities. At the end of training, Domyn was able to achieve 82.04% accuracy with Colosseum 355B in a 5-shot setting. LLM alignment When an LLM has been trained, it has a general understanding of t…
…: True deterministic_mode: False Using these parameters and model distribution, Domyn achieved an MFU of 40%, a significant improvement from the initial 25%. This marked improvement has a direct financial implication, enabling Domyn to c…
…ining in FP8, resulting in an increased MFU from 33% with BF16 to 37% with FP8. In addition, the overall training step accelerated 1.15x with FP8. This speedup was obtained by just enabling FP8 and can be increased if you take…
AI capabilities
Large Language Models, AI Model Development & MLOps, Agentic AI
…odel’s capabilities for domain-specific expertise Colosseum 355B’s capabilities As a key use case in the context of agentic AI , Domyn develops LLMs to power their business intelligence agent, Crystal , a sovereign AI solution. By building an end-to-end stack, Domyn provide a secure experience without rely…
Technology
NVIDIA DGX Cloud, NVIDIA H100 GPUs, NVIDIA NeMo Framework, NVIDIA NeMo Aligner, Colosseum 355B
…and inference optimization. Within one week of signing up to NVIDIA DGX Cloud, Domyn had private access to an environment with over 3K NVIDIA H100 GPUs, all with the following resources: A dedicated, high-bandwidth, RDMA-based netw…
…of pretraining to accelerate training and reduce the model’s memory footprint. NeMo Framework integrates FP8 training out-of-the-box with the Transformer Engine library . Enabling FP8 can be done by adding the following parameters to the training c…
…zed for chat interactions enabling models to answer in conversational settings. Domyn used NVIDIA NeMo aligner for Colosseum 355B’s chat instruction fine-tuning. The syntax for the chat data template uses the following structure outline: { "…
Deployment options
cloud
…and inference optimization. Within one week of signing up to NVIDIA DGX Cloud, Domyn had private access to an environment with over 3K NVIDIA H100 GPUs, all with the following resources: A dedicated, high-bandwidth, RDMA-based netw…
…ation LLM-powered orchestration for tool usage, query execution, and generation Private deployment infrastructure This approach enables Crystal to function as an isolated AI operating system, u…
