Can gradient bandit sampling improve blind beamforming for intelligent reflecting surfaces?

Why blind beamforming avoids channel estimation

Intelligent reflecting surfaces coordinate the phase shifts of many reflective elements so that reflected signals add up constructively at a target receiver. The standard way to set those phase shifts is model-based: estimate the channel of every propagation path, then optimize. The paper argues this route is rarely taken in practice because each reflected channel is easily overwhelmed by the much stronger direct channel, and because existing IRS channel-estimation methods are incompatible with current networking protocols and hardware.

Blind beamforming sidesteps this by tuning phase shifts from measured SNR feedback alone, with no channel state information. The paper treats this as a reinforcement learning problem: each candidate phase-shift vector is an arm, and the received SNR under that vector is the reward. The existing conditional sample mean (CSM) method, which tries random phase-shift vectors and then picks each element's phase by its highest average SNR, becomes the reference point the new method is measured against.

CSM is an epsilon-greedy bandit in disguise

The paper's first analytical move is to show that the existing CSM blind beamforming method is equivalent to an epsilon-greedy algorithm for the multi-armed bandit problem. In each block, with probability epsilon the controller explores by drawing every phase shift uniformly from the discrete set; with probability 1 minus epsilon it exploits by choosing the phase shift with the highest estimated conditional sample mean so far. The original CSM method is the special case that spends the first T0 blocks exploring and then exploits for the remaining T minus T0 blocks.

This reframing matters because it turns a heuristic into a parameterized decision problem. The quantity to minimize is cumulative regret, defined against the phase-shift solution that would be obtained if the conditional sample means were known exactly. The paper notes that regret is nonnegative and equals zero only when every element's chosen phase matches that ideal solution.

An upper bound turns epsilon into a formula

Optimizing epsilon directly is hard because the regret objective has no convenient closed form in epsilon. The paper instead derives an upper approximation of the expected cumulative regret over T blocks, combining a term that grows with epsilon and a term that shrinks with it. Minimizing that bound yields a closed-form choice of the exploration probability.

The resulting values are concrete: for T of 1000, 1500, 2000, 2500 and 3000 blocks, the optimized epsilon is 0.498, 0.431, 0.388, 0.359 and 0.336, and the corresponding T0 for the CSM and random-max-sampling baselines is 498, 646, 776, 897 and 1008 blocks. The pattern is that more blocks justify less exploration. In field tests, epsilon of 0.1 and 0.9 both produced the highest cumulative regret, while the optimized epsilon performed much better, which the authors present as evidence that neither excessive exploration nor excessive exploitation is desirable.

Gradient sampling replaces uniform exploration

The second contribution changes how exploration is done rather than how much. Instead of drawing phase shifts uniformly, the proposed gradient sampling scheme keeps a numerical preference variable for every element and phase option, converts those preferences into a softmax probability, and samples each phase shift from that distribution. After each block, the preference of the chosen phase is increased when the observed SNR exceeds the running average reward and decreased otherwise, scaled by a learning rate.

Because sampling is soft rather than uniform, the algorithm no longer separates exploration blocks from exploitation blocks. The paper shows the update is a stochastic approximation to gradient ascent, which is the basis for its convergence claims. Two theorems are reported: asymptotic first-order stationarity under Robbins-Monro learning rates, and a finite-time stationarity result giving an order of 1 over the square root of T convergence rate for a horizon-dependent constant learning rate. The authors state that because the expected SNR objective induced by the product softmax probability is nonconvex, global optimality cannot be guaranteed, and a regret bound for gradient sampling is therefore not available.

Field tests show a five dB average SNR gain

The method was implemented on a real testbed at 3.5 GHz with 100 MHz bandwidth, using an IRS prototype of 10 by 10 meta-atoms per printed circuit board module, each meta-atom programmable to four phase states of 0, pi/2, pi and 3pi/2. Transmit power was 5 dBm by default, and the receiver sampled baseband at 500 kHz. The comparison set was CSM, epsilon-greedy, the proposed gradient sampling, and a random max sampling baseline that replaces the exploitation step with the best phase-shift vector seen during exploration.

Gradient sampling achieved the best average SNR in every reported comparison. Its average SNR rose from about 19 dB at T equal to 1000 blocks to nearly 23 dB at T equal to 3000, while CSM and epsilon-greedy stayed about 5 dB lower at T equal to 3000 and random max sampling was lowest throughout. Across reflecting-element counts, gradient sampling rose from about 14 dB at N equal to 100 to about 23 dB at N equal to 400, with the other methods improving less. At the block level with T equal to 3000, gradient sampling reached and held roughly 25 dB, whereas CSM and random max sampling settled near 22 dB and 19 dB respectively, and epsilon-greedy showed a longer low-SNR tail.

The same scheme transfers to other objectives

The paper tests whether the sampling scheme is specific to SNR maximization. In an SNR reduction task, where the reward is the negative SNR and transmit power was raised to 10 dBm, gradient sampling reached the lowest average SNR across all tested T, falling below 3 dB at T equal to 3000, while CSM and epsilon-greedy stayed mostly around 10 to 12 dB and random max sampling was about 8 dB higher than gradient sampling.

In a multi-user broadcast setting where the reward is the worst-case SNR among users, all methods degraded as the number of users grew from 2 to 4, but gradient sampling kept a clear margin. With 4 users it reached an average worst-case SNR of about 14.7 dB, against about 8 dB for CSM and epsilon-greedy and about 5 dB for random max sampling. The authors present these as evidence that the scheme extends to objectives beyond SNR maximization, including SNR reduction and max-min fairness.

What the gain costs and where it stops

The practical trade-off is that gradient sampling needs a learning rate and a preference variable per element and phase option, and it gives up the clean explore-then-exploit split that makes CSM easy to schedule. The paper sets the learning rate to 0.005 times the square root of T in the field tests, so the reported gains are tied to that schedule and to the tested block horizons of 1000 to 3000.

The convergence guarantees are also narrower than the field results might suggest. Stationarity is established, not global optimality, and the paper explicitly notes that no regret bound is available for gradient sampling because such a bound would require a global-optimality guarantee. The field evidence comes from one indoor 3.5 GHz testbed with a specific IRS prototype, transmit powers of 5 dBm and 10 dBm, and up to 4 users, so the size of the gain should be read as a result for that setup rather than a universal constant.

When gradient sampling is worth the complexity

The decision the paper supports is not whether blind beamforming works, but which sampling strategy to use once channel estimation has been ruled out. If the deployment can afford a preference-based sampler and a tuned learning rate, gradient sampling dominates uniform exploration on every metric reported here: cumulative regret, average SNR versus blocks, average SNR versus reflecting elements, block-level SNR stability, SNR reduction, and worst-case SNR with multiple users.

The clearest threshold in the evidence is the comparison at 1000 samples, where the proposed scheme beats the existing blind beamforming method by more than 5 dB in average SNR. The optimized exploration probability is the second lever: it falls from 0.498 at 1000 blocks to 0.336 at 3000 blocks, so a deployment with a longer horizon should explore less, and the paper's field tests show that fixed choices of 0.1 or 0.9 are worse than the optimized value. Readers weighing the two approaches should treat the gradient sampler as the higher-ceiling option and the epsilon-greedy or CSM formulation as the simpler one whose performance is bounded by how well epsilon is chosen.

Sources used in this answer