AI hardware11 min read

Could Your Next AI Server Fit the Air-Cooled Rack You Already Own?

Axelera's Titania targets large-model inference in an air-cooled 4U server. Its simulated performance makes power, memory and deployment costs the questions to ask next.

K
Ken Jo

The next AI server may need a new electrical circuit, a different cooling arrangement and a software migration before it answers its first question. For a team bringing inference into its own building, those requirements can matter as much as the accelerator's price. A faster chip only becomes useful capacity when the surrounding system can keep it running.

On November 12, 2026, Dutch accelerator company Axelera AI announced the architecture behind Titania, its third generation of AI acceleration technology. Its proposed AX8 reference server puts eight accelerator packages and 1 TB of accelerator memory in a 4U, air-cooled system. Axelera specifies typical system power of 5–6 kW and targets the power and cooling environments of existing data centers. Those are announced design figures for a future platform, rather than measurements from a server customers can buy today. Axelera's November 12 announcement

The interesting question is whether that design could make large-model inference practical in facilities that already exist. Answering it means looking at memory capacity, the response each user receives, installation work and the software a team must maintain. Titania offers a useful case study because its product page also exposes the conditions behind its performance comparisons.

Titania takes Axelera from the edge toward larger inference systems

Axelera already makes Metis and Europa accelerators for edge and enterprise applications. Titania extends that product direction to larger inference deployments, combining two chiplets in each system-in-package, or SiP. The design uses the company's Digital In-Memory Computing architecture, programmable RISC-V compute and LPDDR6 memory. The announcement presents a portfolio spanning PCIe cards and OAM accelerator modules, with both air-cooled and liquid-cooled configurations planned through system partners.

That mix matters to a buyer because the accelerator package and the complete server are different purchasing decisions. A PCIe form factor does not establish that an existing chassis has the required slots, power delivery, airflow or software support. Likewise, the availability of an air-cooled reference design does not certify every room containing a rack. The appropriate comparison is between supported system configurations that a supplier will actually deliver and service.

An original deployment budget groups the server, power and cooling, software migration and ongoing operation.

Original explanatory graphic. A deployment budget includes the infrastructure and software needed to turn a server into a maintained inference service. The AX8 size and power references come from Axelera's November 12 announcement.

Axelera's product materials include a manufacturer illustration of the Titania chiplet architecture. It shows the intended design and its memory hierarchy, rather than a customer installation or a benchmark demonstration. The same page provides the configuration notes that make its performance charts interpretable.

One terabyte opens a capacity question before it answers a speed question

Axelera specifies 128 GB of LPDDR6 per Titania package. Its eight-package AX8 design therefore reaches the advertised 1 TB of accelerator memory, alongside a stated aggregate 19.6 TB/s of memory bandwidth and 20 PFLOPS of FP8 compute. The company also advertises support for FP4, MXFP4, FP8, MXFP8 and BF16. Precision and the arrangement of a model across packages are essential parts of interpreting those numbers. Announced architecture and reference-server specifications

For a large mixture-of-experts model, total parameters determine how much weight data must be stored, while only part of that model may be active for a particular token. Capacity can therefore remain a constraint even when the active computation looks manageable. Axelera uses that observation to make its case for high-capacity LPDDR6 and efficient data movement. It is pursuing a different balance of memory technology, compute and packaging from an accelerator built around HBM.

The announcement says one AX8 server is designed to hold a 1.6-trillion-parameter model at FP4, with room for the key-value cache. That is a vendor capacity claim under a particular precision. An application still needs to establish whether its exact model representation, context lengths and simultaneous requests fit the supported configuration. The working memory needed by a service grows beyond the stored weights as it serves more or longer conversations.

Keeping a model resident can remove one obstacle to a useful service, but users ultimately experience waiting time and answer quality. A model that fits may still need tuning to meet an application's response target. Conversely, a smaller model that already answers the task accurately may make extra capacity unnecessary. Choose the model and its acceptable precision from the job, then evaluate the hardware that can serve it.

Interactive speed and total throughput reward different designs

Imagine an internal document assistant used by employees during meetings. An answer that arrives quickly enough to keep a discussion moving has a different value from hundreds of answers completed overnight. For that assistant, the time before the first useful text and the pace of the continuing response matter alongside total tokens served. This is an illustrative workload, not a Titania trial.

A second team might process a backlog of documents without anyone watching individual responses. It can tolerate more waiting if batching increases the amount of work completed per hour. Large shared services may need both behaviors at different times of day. A comparison that fixes the speed per user can therefore produce a different ranking from one that maximizes aggregate throughput.

Axelera explicitly frames Titania around responsive inference and acknowledges on its product page that rack-scale GPUs can lead at very large batch sizes. Its charts include different model and rack configurations, rather than a universal result for every application. That is the useful buying question: which configuration meets the service's latency requirement at the expected concurrency and cost? Titania performance conditions

Workload questionEvidence to requestWhat a single peak number leaves unresolved
Can employees use a live document assistant?First-response time and generation speed at the intended concurrencyQueueing and response tails during busy periods
Can a batch service clear its overnight backlog?Completed jobs per hour with representative documentsThe effect of input length and batching on completion time
Can one server combine video and language tasks?End-to-end results with decoding, inference and application logic activeHost load, competing workloads and dropped or delayed work

Read the simulation labels before the performance ratios

The Titania product page identifies its performance figures as internal simulations run through Axelera's production compiler. One server comparison uses an eight-card Titania system simulated in August 2026 against competitor data from InferenceX dated September 2026. The stated conditions include FP4 and an 8,000-token input with a 1,000-token output. Other charts use October rack-level simulations and their own system configurations.

Those conditions explain what the graphs are trying to compare. They also limit the conclusions a buyer can draw from them. A simulation incorporating a compiler is useful engineering information, but it is not an independent run on production Titania hardware. Existing Metis or Europa deployments give context for Axelera's technology; they do not turn Titania projections into shipping-system measurements.

The announcement's claims of up to seven times better throughput per watt and up to twenty times better total cost of ownership belong to that vendor comparison framework. Neither ratio is a price quote or a forecast for an individual site's bill. The product page describes power comparisons as TDP against TDP and identifies costs as Axelera's TCO model. TDP is a design power figure, while a deployed service's electricity use depends on the complete system and its workload.

An original comparison guide separates model and request conditions from service latency, throughput and measured system power.

Original explanatory graphic. Use the model, precision, request lengths and concurrency together when interpreting a performance comparison. Source conditions: Axelera's Titania product page.

Ask for the configuration behind a ratio before using it in a procurement spreadsheet. Keep the model version, precision, quality checks and request distribution with the result. Then ask which power boundary was measured: accelerator, complete server or facility. Otherwise a comparison can reward a smaller measurement boundary instead of a better service.

Air cooling still needs a power and heat budget

The phrase “air-cooled” describes how the server moves heat into the surrounding air. That heat must still leave the room through a suitable facility cooling system. Axelera's proposed 5–6 kW typical AX8 consumption is substantial for one 4U machine. Facilities staff need the quoted configuration, installation requirements and expected workload before deciding whether a rack can accept it.

A simple planning scenario makes the scale tangible. A machine averaging 5–6 kW continuously for 24 hours would use 120–144 kWh at the server boundary that day. This is arithmetic using an assumed continuous average, not a Titania energy measurement; actual workload and idle behavior could change it. Cooling, power-conversion losses and the rest of the facility would add their own consumption.

Axelera's attraction is the possibility of using an existing data-center environment without rebuilding it around a high-density liquid-cooled rack. The saving depends on what that environment can already support. A site with spare electrical and thermal capacity faces a different project from one whose circuits and room cooling are already near their limits. Neither a 4U chassis nor a PCIe card makes the installation equivalent to upgrading a desktop PC.

Software continuity can matter as much as hardware capacity

Axelera says the Voyager SDK and AxeleraScript foundation extend across Metis, Europa and Titania. The announcement describes planned support for tensor, expert and pipeline parallelism, speculative decoding and KV-cache offloading or disaggregation. It also presents the Titania PCIe card as a future replacement path for the Europa-based Axelera Server 250p. Those promises are relevant to teams already evaluating the company's software ecosystem.

For an existing Europa user, the immediate question is how much of an application can carry over: supported operations, model conversion, integration code, monitoring and deployment packaging. For a team starting elsewhere, moving to Voyager is itself engineering work. A common SDK can reduce work across one vendor's products, while the application still needs validation on each target system. Treat portability as something to demonstrate with the actual model and surrounding pipeline.

Consider a video service that generates descriptions for recorded events. Titania's announced on-chip 8K decoding and 4K encoding for H.265 and AV1 could change the division of work between accelerator and host. The practical trial should include the supported codec settings, frame preparation, language model, storage and application response. A faster language model alone cannot establish that the complete service handles the required video load.

For a private document assistant, the corresponding work includes retrieval and access controls. Bringing inference on premises changes where the model runs, but document authorization, logs, backups and service credentials still need their own design. A hardware quote should therefore sit beside a software ownership plan: who updates the runtime, checks model changes and restores service after a failure?

A useful evaluation starts with six decisions

Titania's current status makes a staged evaluation more sensible than a purchase assumption. Start with the application and the baseline available today, then request evidence as the supported systems become available. A vendor demonstration can inform the next step if its configuration and limits are recorded. It should not be treated as a replacement for the eventual acceptance test at the intended site.

  1. Fix the job and service target. Select representative document, agent or video requests. Define acceptable answer quality, response time and busy-period concurrency before choosing the largest model.
  2. Record the current baseline. Use the existing system or hosted service to establish workload volume, latency and operating cost. Preserve model and request versions so later comparisons remain meaningful.
  3. Request the supported configuration. Obtain the planned model support, memory arrangement, host specification, software versions and system-partner responsibilities. Separate announced features from functions demonstrated in that configuration.
  4. Have the facility checked. Give the responsible team the supplier's electrical, airflow and installation requirements. Include neighboring equipment and available cooling capacity in its assessment.
  5. Test the complete service when hardware is available. Include request arrival patterns, retrieval, codecs and monitoring. Measure complete-server power over the same interval as completed work and response-time statistics.
  6. Price the operating commitment. Include integration time, support, maintenance and a practical fallback. Compare ownership costs against the workload actually expected, including quiet periods.

These steps also make a decision to keep existing equipment useful. If a smaller model already meets the target with little operational effort, avoiding an unnecessary migration preserves money and staff time. If memory capacity or installation work is the binding constraint, Titania's proposed design gives the next vendor conversation a concrete focus.

Production timing is part of the hardware specification

Axelera's November announcement targets production in the third quarter of 2027 and describes customer deployment commitments extending into early 2028. The product page uses the broader second-half-of-2027 production window and shows a 2028 upgrade path. These statements describe a roadmap, not general retail availability. Manufacturing start, delivery of a supported server and installation in a particular country are separate milestones.

For planning in November 2026, regional pricing, delivery dates and service coverage need a system partner's confirmation. A commitment to buy future hardware is not the same evidence as a running customer installation. An organization with an immediate inference requirement still needs an available solution and a supportable operating plan today. Titania can enter its future evaluation without becoming a dependency for a near-term launch.

The most valuable result would be useful inference capacity that a team can install, maintain and afford in its own facility. Titania makes that proposition explicit by linking memory, interactive performance and existing infrastructure. Its next test is to turn those design choices and simulations into supported systems measured on the work customers need done.

Sources


Keep the workload definition, supplier questions and eventual test results together so the next comparison starts from the same assumptions. Start your first note with Telli.sh.


← Back to Blog