But treating it as a single category obscures something important: AI workloads place demands on networks that are structurally different from traditional data traffic, and those differences have real implications for how capacity is planned, priced, and delivered.

It starts with the workload type

There are two distinct categories of AI network traffic, and they behave very differently.

Training traffic is generated when a model is being built or updated. It is bulk, scheduled, and highly parallelised, large volumes of data moving between thousands of GPUs, often within a single data centre or between a small number of connected facilities.

Latency tolerance is relatively high, but the sheer volume and the east-west nature of the traffic (moving between servers rather than between a server and an end user) puts pressure on internal network fabric and interconnection in ways that traditional north-south enterprise traffic does not.

Inference traffic is what happens when a trained model is actually used, every time someone sends a prompt, runs an image through a recognition model, or triggers an AI-powered process. Inference is real-time, unpredictable in volume, and highly latency-sensitive. Unlike training, it needs to happen close to the user.

Why it matters for wholesale networks

Traditional internet traffic, video streaming, web browsing, file transfers, follows broadly predictable patterns. It peaks in the evening, ebbs overnight, and scales gradually with population growth. Network planners have decades of experience modelling it.

AI inference traffic breaks those patterns. Demand can spike sharply when a new model launches or a product goes viral. The geographic distribution of traffic shifts as inference moves to edge locations. And the traffic profile itself, many small, low-latency requests rather than sustained bulk flows, requires different handling at the network layer.

Training workloads, meanwhile, are driving a new class of data centre interconnect requirement. Clusters of GPU infrastructure increasingly need 400G and 800G links between facilities, with the kind of consistency and low jitter usually associated with financial trading networks.

The capacity question

The honest answer is that nobody yet knows exactly how AI traffic will scale. Model efficiency is improving, inference is getting cheaper per query, but demand is growing faster than efficiency gains. The net effect on network capacity requirements remains genuinely uncertain.

What is clear is that the operators best positioned for what comes next are those investing now in high-capacity, low-latency interconnect, flexible edge infrastructure, and the ability to turn up capacity quickly when and where demand appears. The old model of forecasting traffic three years out and building to that number is increasingly difficult to defend.

AI traffic is not simply a new type of data, but a new pattern of demand and for wholesale networks, the distinction matters.

RELATED STORIES

AI now lies, denies, and plots: OpenAI’s o1 model caught attempting self-replication

AI agents are starting to talk: Are the networks ready?