From Data Centers to AI Factories: Architecture, Economics and Investment Choices

An AI data center is not successful because it contains a large number of accelerators. It is successful when it can deliver useful AI output — training runs completed, inference requests served or simulations finished — at a cost, speed, reliability and power level that makes business sense.

That is the central idea behind the shift from a conventional data center to an AI factory. The first post in this series introduces that shift; the second examines the companies and infrastructure providers involved; and the third explains the architecture inside the rack. This article brings those perspectives together for infrastructure providers, silicon suppliers, enterprise IT leaders and investors.

1. Why “AI factory” is a useful operating model

A traditional data-centre discussion often starts with floor space, racks and available power. An AI-factory discussion starts with the product: how many useful tokens, model runs, predictions or simulations can the system produce?

That changes what operators need to manage:

• • Training workloads need accelerators, memory and networks to work together across nodes and racks.
  • Inference workloads may be more sensitive to latency, model size, memory capacity, batching and serving cost.
  • Data and storage pipelines must keep compute supplied with training data, retrieval content and checkpoints.
  • Power and cooling need to meet the actual load of dense accelerator systems—not just a facility’s nameplate capacity.
  • Scheduling and software determine whether hardware is productively used or left waiting.

The first post P1 frames the shift from data centers to AI factories. The energy outlook reinforces why it matters: the IEA estimates that global data-centre electricity use was about 415 TWh in 2024 and could reach around 945 TWh in 2030. LBNL estimates U.S. data-centre electricity use at 176 TWh in 2023, with a possible range of 325–580 TWh by 2028. These estimates cover data centres overall, not only AI, and should be treated as scenarios rather than precise predictions. [1][2]

Operational implication: an announced megawatt is not the same as a megawatt that can be energised on the date the workload needs it. Grid connection, substations, power distribution, cooling, water and equipment delivery all belong in the capacity plan.

2. Inside the AI Factory: board, rack and facility

The technical architecture post P3 describes the components that determine whether accelerator capacity becomes productive capacity.

The main building blocks

Memory is a system choice, not just a capacity number

For the NVIDIA H100 SXM platform, the product specification lists 80 GB of HBM and up to 3.35 TB/s of memory bandwidth. AMD lists 192 GB of HBM3 and up to 5.3 TB/s for its MI300X. These specifications show different possible trade-offs in memory capacity and bandwidth; they are not substitutes for workload-specific testing. [5][6]

HBM is part of the accelerator’s memory architecture. DDR5 and SOCAMM2-class host-memory options serve a different role. Treating host memory as an interchangeable substitute for HBM can lead to a flawed capacity plan. HBM4 should likewise be evaluated as part of a particular accelerator and package roadmap, rather than assumed to be a later field upgrade.

The architecture question is not only “How much memory is on the board?” It is also:

• • Can the intended model fit without excessive sharding or offload?
  • Does the model’s access pattern make bandwidth a constraint?
  • Is the platform supported by the required software stack?
  • Can the supply plan deliver the qualified boards, memory and systems together?

Rack-scale systems change facility assumptions

NVIDIA describes the GB200 NVL72 as a liquid-cooled rack-scale system combining 72 Blackwell GPUs and 36 Grace CPUs. The wider point is that high-end AI infrastructure is increasingly designed and operated as a rack-scale system, rather than as a set of independent, interchangeable servers. [4]

A simplified rack example helps illustrate the density:

The 10 kW node figure is an illustrative sizing assumption, not a specification for every eight-GPU server. In a real design, network and storage loads must also be included within the total IT power envelope.

At high rack densities, liquid cooling may be part of the base design. It also brings operational requirements: coolant distribution, leak detection, maintenance access, service procedures and clear accountability between the facility and IT teams. ASHRAE’s datacom guidance is a useful reference for thermal design considerations. [11]

3. Measure useful output, not just infrastructure activity

PUE is useful, but it does not tell an operator whether a cluster is completing useful work. Uptime Institute’s 2024 survey reports a mean PUE of about 1.56. For a new AI-oriented facility, a design target in the 1.15–1.30 range can be a useful planning ambition, but it is a site-specific target—not a guarantee or a universal benchmark. [3]

A useful operating dashboard should include measures such as:

A common analytical mistake is to equate a high GPU utilisation reading with high economic productivity. A system can appear busy yet deliver poor throughput for the specific workload. Conversely, a service may intentionally leave headroom to meet latency or availability commitments.

4. A transparent 1 Megawatt cost illustration

The following is a sensitivity model, not a current vendor quote or market price forecast. It is intended to show how utilization and power affect the economics.

Assuming:

• • 1 MW of IT load;
  • approximately 100 eight-GPU nodes, at 10 kW per node;
  • approximately 800 accelerators;
  • PUE of 1.2;
  • electricity price of $0.08/kWh;
  • a constant 1 MW IT load for the power calculation.

We arrive at the following top-level estimates:

The electricity price is an assumption for sensitivity analysis; actual prices vary by location, contract, demand charges and other factors. EIA’s monthly electricity data can help establish a regional baseline. [8]

Utilization makes a large difference

Now assume, purely for illustration, $400,000 per 8-GPU node. One hundred nodes would cost $40 million in server equipment before facility costs, cluster-wide networking, storage, software, support and spares. Assume four years of economic use and a 10% cost of capital; this annualizes to roughly $12.62M / year, before residual value.

*Assumes the facility continues to draw 1 MW of IT power. Actual partial-load power varies by system and workload; use the measured power curve in a real business case.

At 65% occupancy, the illustrative server capital and electricity together are about $2.96 per billable accelerator-hour — before facility capital, cluster networking, staff, maintenance, software, financing costs not captured by the assumption, taxes or operating reserves. This does not imply a market GPU-hour price. It shows why a business case must compare like with like: the same accelerator generation, memory, network, software, capacity commitment and service level.

For facility costs, don’t use a universal cost-per-megawatt figure as a substitute for a site estimate. As a feasibility sensitivity only, a U.S. project team might test a building-and-MEP allowance of $10–15 million per MW, excluding land, IT equipment and major utility-side grid upgrades. Replace this assumption with regional market data, an EPC estimate and a utility proposal. JLL’s outlook provides market context, but a local project cannot be priced from a global average. [10]

The cost model should ultimately compare:

All-in Cost per Productive GPU-hour = annualized infrastructure and IT capital + facility and operating costs + power and cooling + support and software, divided by billable, productive accelerator-hours.

5. Which infrastructure option fits which buyer?

The players post P2 examines the ecosystem. The following matrix translates that landscape into a buyer and operator decision.

Recommendation for IT leaders: start with the workload and its cost of delay. Use cloud or GPUaaS to test demand and model behavior while requirements are changing. Consider dedicated or owned capacity when workloads are persistent, predictable and sufficiently valuable to justify the capital, operating expertise and hardware refresh. Hybrid is often the sensible default: steady or sensitive workloads on dedicated capacity; experiments and bursts in the cloud.

A starting utilization threshold—such as 60–70% sustained demand—can be used in a buy-versus-rent sensitivity analysis. It is not a universal rule: the answer depends on the cloud price, ownership cost, workload variability, data movement, staffing, availability requirements and time horizon.

Budget and Delivery Planning

Indicative planning envelopes — not vendor commitments—might look like this:

For each option, ask for time to usable capacity, not just time to a signed contract or a completed building.

6. Where are the industry opportunities?

The largest near-term infrastructure requirements are most visible among hyperscalers, frontier-model developers and GPU-first cloud providers. That does not establish a reliable ranking of end-user industry spending: public disclosures do not provide a consistent comparison of AI infrastructure budgets by sector.

For enterprise opportunity, prioritize the workload rather than assuming every sector needs its own AI data center.

Low-friction starting projects often include document processing, internal knowledge retrieval, contact-center assistance, software development support, visual quality inspection and demand forecasting. Their infrastructure requirements vary widely; many can be evaluated on managed services before a company commits to owned hardware.

McKinsey’s estimate of $2.6–$4.4 trillion in potential annual economic value from generative AI is useful context for potential business impact, but it is not an estimate of AI infrastructure spending. [9]

7. What should infrastructure and silicon providers co-design?

The opportunity for AI silicon suppliers extends beyond selling accelerators. Infrastructure providers, colocation operators, hyperscalers and neoclouds can provide workload and operating data that helps suppliers design better systems.

For network suppliers, the key question is whether a fabric supports the intended workload and topology—not simply its advertised port speed. Current high-performance designs include 400- and 800-Gb/s-class networking options, but system performance needs to be measured on the target workload. [7]

For colocation and neocloud providers, collaboration with silicon suppliers can help qualify rack power, cooling and cabling before equipment arrives. For hyperscalers, colocation partnerships can help extend capacity or connectivity; for customers, the important issue is whether the resulting service offers a clear operational and commercial advantage over a hyperscaler’s own capacity.

8. Investor perspective: distinguish assets, operators and services

The SWOT below is a strategic framing, not a rating of individual companies. The players post, P2 provides the series’ market-participant perspective.

AI Practitioner’s view for Investors:

1. 1. Hyperscalers may be the strongest operators, but they offer less direct exposure to a single AI-facility thesis.

   2. Power-secured, pre-leased colocation capacity can be attractive where customers, power delivery and facility specifications are credible.

   3. Neoclouds can offer high growth potential, but should be underwritten on contracted utilisation and customer credit—not GPU orders or announced capacity alone.

   4. GPUaaS needs closer examination: a software-led marketplace and a balance-sheet-heavy GPU owner have very different risks.

9. Financing instruments should match the asset and the contract

An AI facility contains assets with different lives and risks. Financing a long-lived building and electrical system in the same way as rapidly refreshed accelerators can create a mismatch.

A sound project should separately test power delivery, contracted capacity, customer credit, utilisation, refresh costs and sensitivity to lower prices. A neocloud with a technically excellent cluster but weak contracts is not automatically a financeable project.

Closing view

The AI infrastructure race will not be decided by GPUs alone. It will be decided by organisations that bring together power, accelerators, memory, networking, cooling, software and contracted demand—and measure success in useful output rather than installed capacity.

For enterprise buyers, prove the workload before building the facility. For silicon suppliers, co-design with rack operators and data-centre teams. For infrastructure providers, sell dependable, usable capacity. For investors, ask whether the megawatts can be delivered and whether paying workloads will use them.

---

References

From this series

• • [P1] “The AI Infrastructure Revolution — From Data Centers to AI Factories.” https://www.anirvanbasu.com/the-ai-infrastructure-revolution/
  • [P2] “The Principal Players — Who are Building the AI Infrastructure Empire?” https://www.anirvanbasu.com/the-ai-infrastructure-revolution/the-key-players/
  • [P3] “The Technical Architecture: Inside AI Datacenters – Racks, Accelerators, Memory, and Networking.” https://www.anirvanbasu.com/the-ai-infrastructure-revolution/the-technical-architecture/

External sources

1. 1. International Energy Agency, Energy and AI. https://www.iea.org/reports/energy-and-ai
   2. Lawrence Berkeley National Laboratory, 2024 United States Data Center Energy Usage Report. https://eta-publications.lbl.gov/sites/default/files/2024-12/lbnl-2024-united-states-data-center-energy-usage-report.pdf
   3. Uptime Institute, Global Data Center Survey 2024. https://uptimeinstitute.com/resources/research-and-reports/global-data-center-survey-2024
   4. NVIDIA, GB200 NVL72. https://www.nvidia.com/en-us/data-center/gb200-nvl72/
   5. NVIDIA, H100 Tensor Core GPU. https://www.nvidia.com/en-us/data-center/h100/
   6. AMD, Instinct MI300X. https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html
   7. NVIDIA, Spectrum-X Ethernet. https://www.nvidia.com/en-us/networking/ethernet/spectrum-x/
   8. U.S. Energy Information Administration, Electric Power Monthly. https://www.eia.gov/electricity/monthly/
   9. McKinsey, The economic potential of generative AI: The next productivity frontier. https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier
   10. JLL, Global Data Center Outlook. https://www.jll.com/en-us/insights/market-dynamics/global-data-center-outlook
   11. ASHRAE, Datacom Series. https://www.ashrae.org/technical-resources/bookstore/datacom-series

For exact section- or paragraph-level references to the three WordPress posts, their page text or exported HTML would be needed.