AERL-Based Control: Seamless Islanding Transition for Unbalanced Multimicrogrids

TL;DR

Transitioning a multimicrogrid (MMG) from grid-connected to islanded mode during a fault is a high-stakes race against time. This paper introduces an Artificial Emotional Reinforcement Learning (AERL) framework combined with a Merge-Sort-based phase balancing technique. This dual approach ensures that even in highly unbalanced three-/single-phase systems, voltage and frequency stabilize significantly faster (up to 25.6% improvement) than traditional model-based methods.

Motivation: The Unbalance Trap

In modern distribution networks, we don't just deal with balanced three-phase loads. We have a chaotic mix of single-phase photovoltaics (PV), battery storage (BES), and residential loads. When a distribution network fails, the MMG must "island" itself.

Current SOTA methods like MISOCPM (Mixed-integer second-order cone programming) or IEM (Implicit Enumeration) face two fatal flaws:

1. Complexity: They rely on heavy mathematical models that take too long to solve during a millisecond-level transient event.
2. Unbalance Neglect: They often treat the system as a balanced entity, leading to severe phase voltage offsets when single-phase components are disconnected.

Methodology: The Core Intuition

1. Merge-Sort for Phase Balancing

The authors treat single-phase components as building blocks. Instead of evaluating every possible combination (which grows exponentially), they use a Merge Sort algorithm to rank and combine phase-A, B, and C components into "Three-Phase Combinations" (TPCs) that naturally minimize the unbalance factor ($\epsilon$).

2. Artificial Emotional Reinforcement Learning (AERL)

Standard Q-learning can be slow to converge. AERL adds an "emotional" layer. The "emotion" here is a mathematical quantifier of the system's stress (power shortage, DG output).

• The Logic: If the system is in a "high-stress" state, the learning rate ($\alpha$) is dynamically adjusted.
• The Reward: The reward function incorporates a Fuzzy Comprehensive Evaluation System (FCES), which weighs load importance, outage losses, and unbalance factors to prevent cutting off critical infrastructure (like hospitals or command centers).

Experiments and Results

The researchers tested their strategy on the IEEE 37-bus and IEEE 118-bus systems.

Performance Gains

In the IEEE 118-bus system, the results were striking. While traditional methods struggled with oscillations, the AERL strategy provided a much smoother landing:

• Frequency Recovery: 12% to 25.6% faster than baselines.
• Voltage Stability: Fluctuations were kept within much tighter bounds compared to MISOCPM.

Adaptability

Crucially, the authors demonstrated that the AERL agent could "learn" and adapt even when the grid topology changed mid-simulation (e.g., adding new PV modules or losing a branch). The Q-values consistently converged to the optimal reward values regardless of the operating state.

Critical Insight & Conclusion

The brilliance of this paper lies in the shift from model-driven to data-driven decision-making with a "human-like" emotional heuristic. By pre-training the AERL model offline, the online execution becomes a simple look-up or low-latency inference, which is exactly what a microgrid needs during a transient fault.

Takeaway: Future microgrid controllers will likely move away from solving rigid optimizations in real-time, favoring adaptive agents that can "feel" the stress of the grid and act decisively to save critical loads.

Limitations: The study currently discretizes the state space, which might lead to "boundary issues" where state transitions aren't smooth. Moving toward Deep Reinforcement Learning (DRL) with continuous state spaces could be the next logical step.