Can old DDR4 give an AI server more breathing room?
Netlist and Marvell are bringing DDR4-based CXL designs to SC26. See where extra host memory could help and what to measure before buying it.
An internal search service feels quick until the morning rush. It has to fetch passages from thousands of documents before sending an answer request to a GPU. The frequently used passages fit in memory, but repeated searches through the rest of the collection go back to storage. People wait longer even though there is spare capacity on the accelerator. For the team running the service, a bigger cache on the server could help more than another GPU.
Two previews ahead of SC26 point to an additional way to build that cache. In a November 10 announcement, the CXL Consortium said that memory specialist Netlist planned to show a proof of concept called NVvault-D4, which makes existing DDR4 available as expanded host memory over Compute Express Link, or CXL. On November 13, chipmaker Marvell announced partner displays featuring its Structera X memory expansion controllers with DDR4 and DDR5. The announcements describe hardware intended for exhibition. They leave the prices, delivery dates, and results of the planned demonstrations open. SC26 runs November 15–20 in Chicago, with regular exhibits November 17–19 local time. CXL Consortium’s Netlist preview · Marvell’s SC26 announcement · SC26 official dates
The question for a buyer is whether this extra capacity can keep enough of a useful workload in memory without making its slowest requests unacceptably slow. For a team holding spare DDR4, reuse could improve the economics if the modules qualify and the system can put the right data there. The value depends on what is waiting today and where that waiting happens.
Where the extra memory goes
A CPU can reach its locally attached DRAM directly through its memory controller. With a compatible server and CXL memory expansion device, the CPU can also access memory behind that device over a CXL link. The expansion adds a place to hold data on the host side. Its path, bandwidth and latency differ from those of the local DRAM channels. The firmware and operating system have to expose the capacity, and the application or operating system needs a page placement policy that suits the workload.
This matters when someone proposes using stockpiled DDR4 in a newer server. The server can keep its local DDR5 and reach the DDR4 through a supported expansion device. The device controller, module qualification and host compatibility still matter. A proof of concept using existing DDR4 gives a reason to investigate reuse. It leaves open which modules a supported configuration could accept and which systems can run them reliably.
Local DRAM can hold frequently accessed pages while an expansion tier holds larger CPU-side caches. The accelerator’s high-bandwidth memory has its own capacity and bandwidth limits.
In the search example, the team could keep frequently read index data in local DDR5 and move a larger, less frequently read cache of passages or search results to the CXL tier. Once the service finds the right passages, its inference backend passes them to a model running on the GPU. Model weights and the data an accelerator kernel needs still have to fit wherever the inference engine expects to access them. The CXL expansion tier in these announcements addresses host memory capacity. GPU high-bandwidth memory, commonly called HBM, remains a separate resource.
Start with a trace of where requests spend time. If the CPU repeatedly waits for data that was evicted from its cache, more host memory may help keep that work out of storage. If the model spends most of its time reading weights from GPU memory and the host rarely stalls, the same purchase may have little effect on answer time. A faster data retrieval step can matter even when the model itself runs at the same speed. Measure both before attributing changes in the overall service to either one.
Netlist’s three prototypes tackle different jobs
Netlist’s NVvault-D4 is the one to follow for DDR4 reuse. The consortium names virtual machines, in-memory databases, high-performance computing and enterprise workloads as targets. A virtualization host offers a concrete example. Its busiest guests could stay on local memory while a team tests whether less active guest pages can live in the expansion tier. A database team might try the same idea with portions of a read cache. The database would need an allocation policy that keeps pages used by latency-sensitive queries close to the CPU. These are possible pilots for the capacity design. The preview gives no workload-level results. CXL Consortium’s Netlist preview
The two other Netlist designs illustrate why the part names matter. NVvault-HM uses DRAM caching in front of NAND-backed capacity, and the announcement discusses data movement and telemetry alongside use cases such as AI key-value caches and analytics. NVvault-PM is a separate CXL Type 3 design described with DRAM, NAND backup and an embedded energy source for power-loss handling. The preview positions PM for cases such as key-value cache retention, checkpoints and recovery. Teams evaluating backup or persistence would need details on the behavior of that specific design under their failure conditions. The D4 announcement gives no basis for assigning either design’s storage or recovery behavior to D4. CXL Consortium’s Netlist preview
A language model’s key-value cache offers a separate, illustrative AI test. The cache stores attention data from earlier tokens so the model can reuse it. An inference engine with host offload might move inactive sessions’ cache pages out of GPU memory and reload them when those sessions return. A team can test whether adding host capacity lets it retain more sessions and how much the transfers add to the time for a resumed request. Placement and copying depend on the engine and server. This example is a possible experiment for a host-tiering setup. Netlist’s explicit KV-cache positioning in this preview belongs to HM and PM.
Marvell’s announcement gives buyers a second implementation track to watch. Its partners plan to show systems built around the Structera X controller family using DDR4 and DDR5. The program and the media kit give context on the exhibition and the family. Buyers will still need to check which server configurations a supplier qualifies, what memory they accept, and how suppliers support the full system. The chip images in the kit are product-family material for reference. CXL Consortium’s SC26 pavilion program · Marvell’s SC26 announcement · Marvell Structera media kit
Measure the slow requests first
An average can hide the requests that make an application feel sluggish. For a search service, popular documents can produce quick hits all day while less common documents send the same workers to the expansion tier repeatedly. An increase in the slowest requests may be acceptable for a nightly indexing batch. A customer waiting for a search result may notice it immediately. The memory design should earn its place against the actual service target at peak load.
Independent researchers behind the MELODY study made this point concrete in 2025. They studied 265 workloads using four real CXL memory devices, five CPU platforms and seven latency levels. They examined sensitivity within a 140–410 nanosecond submicrosecond range and found that workload, tail latency and CPU behavior mattered for how systems handled CXL memory. The paper labels the tested expanders CXL-A through CXL-D. Its results offer no performance estimate for the announced SC26 systems. Its value for the buyer is in showing why one average latency figure or one favorable benchmark cannot settle how a given server will perform. MELODY study from ASPLOS 2025 · Full paper
Plan for three comparisons if the platform allows them. First reproduce the current service on its existing local DRAM. Then test more capacity with a tiered host. Finally, use an all-local configuration with the same total capacity to give the tiered result a meaningful reference. As a purely illustrative sizing exercise, compare 256 GiB of local DRAM plus 256 GiB of usable expansion with 512 GiB of local DRAM. Keep the application, dataset, software settings and offered load comparable, and record any platform or local-channel differences the configurations force on you. If the operational alternative is storage spill or sharding across another host, time and price that option too.
Compare the same workload and traffic in both configurations. Check the slowest requests and calculate what each usable unit of capacity costs.
- Replay traffic that fills the cache. Use a realistic request mix, including repeated and rarely accessed documents. Run from a cold start and again after caches warm. Watch median, p95 and p99 request latency, error rate and throughput at the same concurrency. For an AI search application, track retrieval and inference timings separately so a faster cache cannot mask slower model service or vice versa.
- Check which data actually moved. Inspect memory allocation by tier and CPU NUMA node, page migration and reclaim where available, and link or device counters the platform provides. An apparently good run could have kept almost every request in local DRAM. An apparently bad run could have put frequently read index pages on the remote tier by default. Record the placement policy next to each set of measurements.
- Test a stressful afternoon. Run a second job while the interactive workload is busy, warm the cache after a restart, and observe what happens when the service approaches its memory limits. Confirm how the operating system reports device or memory errors, how operators identify the affected host and how the service returns to its previous allocation policy. Recovery belongs in the pilot for any configuration the team would have to support.
- Set the pass criteria before looking at the charts. Use the team’s current p99 and error targets as the starting point. Write down how much additional capacity must be usable at that target and which cases can tolerate slower recovery. If a system meets targets only with a generous cache hit rate, use traces from especially busy days to check whether that hit rate survives production traffic.
Count the whole cost of reuse
Existing DIMMs can make the memory line in a proposal look good. A purchase still needs the expansion device, a supported host or upgrade, qualified modules and staff time for rollout and support. A device also needs a suitable slot or platform attachment. Check the planned allocation of PCIe lanes against storage, networking and accelerators. A memory upgrade makes less sense if it removes the resources the same workload uses to move data.
Calculate the cost per usable GiB at the required service level. Include any capacity reserved by the platform or given up to keep enough pages local, plus power and cooling, fleet qualification, spare parts and support over the intended service life. Check who supplies firmware updates and how the system exposes health and memory errors. If the tier allows a host to handle the same work that would otherwise spill to storage or require an additional machine, measure that saved work and account for the overhead of both options. A simple experiment can make the comparison concrete. Run the service at its peak request rate on both architectures and compare the cost of keeping the response time within target.
After SC26, watch for qualified configurations, documented module support, software compatibility and real workload traces. A released system could give teams with a large secondary working set another way to grow an existing server. The strongest example would be a service that holds more document data in the host, avoids repeated storage reads and stays within its old p99 limit at a lower cost. A team that can point to that result has a purchasing case it can repeat on the next server.
Sources
- CXL Consortium’s Netlist SC26 preview, November 10, 2026
- Marvell’s SC26 partner and connectivity announcement, November 13, 2026
- CXL Consortium’s SC26 pavilion program and event graphics
- Marvell Structera product-family press kit
- SC26 official program and exhibit dates
- MELODY paper and publication details, ASPLOS 2025 · Full paper
To keep pilot notes, workload traces, and purchasing decisions together, create a Telli.sh account.