How AI-Based Digital Post-Distortion and Hardware-in-the-Loop Testing Are Bridging 5G-Advanced and 6G Architectures

As the telecommunications industry continues to push the boundaries of 5G and 5G-Advanced networks to maximize uplink efficiency and modulation performance, wireless engineers are fundamentally rethinking where power amplifier linearization should occur and how it must be validated. Today, cutting-edge hardware-in-the-loop testing methodologies involving artificial intelligence-based digital post-distortion are successfully bridging the gap between current 5G RF measurements and the visionary, compute-heavy receiver architectures being designed for 6G.

Power amplifier linearity remains one of the most stubborn and difficult tradeoffs engineers face when designing modern wireless communication systems. A user equipment device—such as a smartphone, wearable, or industrial IoT sensor—can operate its power amplifier safely within its linear region by utilizing power backoff to preserve signal quality. However, doing so inherently reduces overall efficiency and severely drains battery life. Alternatively, the device can drive the power amplifier much closer to saturation, dramatically improving power efficiency while simultaneously introducing problematic in-band distortion and out-of-band spectral regrowth. In standard 5G deployments, and increasingly in bandwidth-constrained 5G-Advanced scenarios, that compromise becomes glaringly obvious as uplink performance demands continue to escalate.

This enduring engineering challenge is the primary catalyst attracting renewed attention to digital post-distortion. Unlike conventional digital pre-distortion techniques, which attempt to linearize the power amplifier directly at the transmitter, digital post-distortion shifts a vital portion of the compensation burden over to the receiver. In the context of the cellular uplink, this paradigm means that the base station actively attempts to recover and reconstruct a signal that was originally transmitted by a much more efficient, highly nonlinear user device. The concept carries profound significance for battery-powered equipment operating near the cell edge, where uplink power efficiency is paramount to maintaining connection integrity and network coverage.

Although digital post-distortion is frequently discussed in forward-looking academic and industrial forums as a cornerstone of 6G systems, its practical, rigorous evaluation is firmly rooted in today’s 5G testing environments. Modern engineering teams are confronting critical questions that demand immediate, measurable answers: Exactly how much uplink signal distortion can a next-generation receiver tolerate? Can artificial intelligence-based receivers successfully recover signals far beyond the limits of conventional, legacy algorithms? Furthermore, what specific testbed configurations and instrumentation setups are required to accurately compare these competing approaches under thoroughly realistic operating conditions?

Why the Uplink Remains Difficult

The fundamental physics governing orthogonal frequency-division multiplexing signals in the 5G uplink create a high peak-to-average power ratio, which inevitably leads to nonlinear distortions whenever user equipment power amplifiers are operated close to saturation. As cellular standards incorporate increasingly dense modulation orders to boost data rates, the operational margin for signal impairments shrinks dramatically. While higher-order constellations successfully improve spectral efficiency, they are also inherently more vulnerable to severe error-vector-magnitude degradation triggered directly by power amplifier compression and phase distortion.

Traditionally, this complex problem has been managed locally at the user equipment level via transmitter linearization, calibration, and conservative power amplifier operation. Yet, this traditional approach imposes severe penalties in terms of hardware complexity, overall power consumption, and thermal management overhead. For modern consumer smartphones, compact wearables, and remote Internet of Things nodes, those operational penalties are far from trivial.

This is precisely why receiver-side compensation strategies are being revisited by the wireless community, particularly as 5G-Advanced sharpens the industry’s focus on uplink performance and 6G research groups contemplate aggressive new targets for spectral efficiency. Digital post-distortion does not magically eliminate the strict regulatory need for spectral compliance, nor does it render power amplifier nonlinearity entirely harmless. Instead, it offers an entirely different system architecture partitioning strategy: a portion of the heavy burden associated with recovering a distorted uplink transmission is intelligently shifted away from the constrained mobile device and onto the robust network infrastructure receiver.

Why AI-Based Digital Post-Distortion Matters

Classical post-distortion methods have historically relied on rigidly predefined analytical models of nonlinear device behavior. While those traditional methods can perform adequately in highly controlled, idealized scenarios, real-world uplink signals are dynamically shaped by a multitude of simultaneous physical effects, including power amplifier nonlinearity, multipath fading, additive noise, synchronization errors, and hardware implementation nonidealities. This chaotic combination makes a purely model-based compensation approach increasingly impractical and brittle, especially when looking beyond current commercial 5G deployments toward 5G-Advanced and future 6G networks.

This is where artificial intelligence-based receivers offer a compelling alternative, as they possess the unique capability to learn complex patterns from representative empirical data rather than relying entirely on a fixed analytical model. A prominent example of this architectural shift is Nokia Bell Labs’ HybridDeepRx, a sophisticated neural-network-based receiver specifically engineered to handle complex orthogonal frequency-division multiplexing waveforms plagued by severe channel impairments and transmitter nonlinearity. In advanced test setups, this neural receiver effectively replaces a significant portion of the conventional base station receiver chain.

The primary role of such an AI receiver extends far beyond simply cleaning up or denoising a compromised signal. The underlying neural model simultaneously addresses wireless channel-related effects and severe nonlinear distortion, subsequently performing precise demapping to generate vital soft information for downstream channel decoding. A defining architectural characteristic of this approach is its ability to seamlessly alternate between frequency-domain and time-domain processing stages.

While the frequency-domain stages efficiently mitigate channel propagation effects in a manner familiar to conventional wireless engineers, the time-domain stages are exceptionally well-suited to handling the complex, rapid transients of power-amplifier-induced distortion. This hybrid structure establishes the receiver as an ideal candidate for rigorously testing whether artificial intelligence-driven digital post-distortion can consistently outperform conventional uplink receiver processing when transmitters are intentionally pushed into highly nonlinear operating regions.

Hardware-in-the-Loop Testbed

To evaluate these advanced concepts credibly, engineering teams from Nokia Bell Labs and Rohde & Schwarz developed a sophisticated hardware-in-the-loop testbed centered around standard-compliant 5G signal generation and wideband signal analysis. The core objective of this collaboration was to directly compare traditional, conventional receiver processing against cutting-edge AI-based reception under controlled yet thoroughly realistic uplink impairment conditions.

The experimental testbed relies on a Rohde & Schwarz SMW200A vector signal generator to synthesize the standard-compliant 5G uplink waveform, apply an accurate power amplifier model, and subsequently introduce realistic wireless channel emulation. This capability is fundamentally important because it allows researchers to inject controlled amounts of nonlinear power amplifier distortion without physically swapping out hardware components between test runs. Engineers can thus vary the amplifier operating point and distortion levels in a completely repeatable manner, which is an absolute necessity when attempting to draw valid performance comparisons between conventional and AI-driven receiver behaviors.

On the receiving end of the test loop, a high-performance FSWX signal and spectrum analyzer captures the impaired uplink waveform, providing the necessary RF bandwidth and dynamic range required for advanced multi-domain analysis. The captured signal data is then routed into the Rohde & Schwarz vector signal explorer software, which acts as the central software analysis environment for the entire platform.

Within this environment, the software performs standard RF demodulation steps, supports critical key performance indicator extraction—including block error rate, bit error rate, system throughput, and adjacent channel leakage ratio—and, crucially, hosts user-defined artificial intelligence models packaged in the ONNX format. Nokia Bell Labs’ HybridDeepRx receiver is directly imported into this measurement flow, executing real-time, GPU-accelerated inference. This creates a direct, seamless bridge between advanced AI model development and physical RF test instrumentation, allowing engineers to process the exact same captured waveform through either a legacy algorithm chain or an advanced neural-network-based receiver.

Placing the AI model directly inside a realistic, instrument-backed signal analysis flow yields immense practical value. Instead of relying on abstract, offline software simulations, engineers can evaluate competing receiver strategies using identical waveforms, identical impairment parameters, and standardized key performance indicators. Furthermore, because the vector signal generator can independently apply wireless channel fading and precise power amplifier compression, the testbed can explore a vast spectrum of realistic uplink scenarios, ranging from mildly affected links to heavily compressed transmissions.

From 5G Measurement to 6G Architecture

Ultimately, AI-based digital post-distortion serves as a clear illustration of how rigorous 5G testing methodologies are actively shaping the future of wireless architecture. While the underlying waveforms, performance metrics, and measurement disciplines are firmly anchored in today’s 5G and 5G-Advanced engineering efforts, the overarching architectural questions being answered point directly toward 6G. Specifically, the industry is exploring whether a meaningful share of the uplink linearization burden can be permanently moved away from power-constrained user devices and absorbed by compute-rich network infrastructure.

This architectural shift holds immense promise because the cellular uplink is historically the domain where device-level constraints are least forgiving. If future generations of base stations can reliably recover significantly more distorted signals, mobile user equipment can operate using simpler, highly power-efficient transmitters even under exceptionally difficult link conditions. For users situated at the extreme edge of a cell, this transition could translate into vastly improved energy efficiency without suffering a corresponding penalty in network coverage or data throughput.

Although this technology paradigm remains in its relative infancy and substantial technical hurdles—such as ensuring robust generalization across diverse devices, dynamic channels, and varied deployment conditions—still lie ahead, the availability of comprehensive hardware-in-the-loop testbeds provides the industry with the practical means to study these challenges today. By combining realistic waveform sources, precise hardware impairment emulation, wideband signal capture, and integrated toolchain AI inference, engineers are turning theoretical post-distortion concepts into measurable, verifiable engineering realities.