基于多步自校正Q学习的孤岛微电网负荷频率控制策略

打开文本图片集

摘 要：在以绿色交通和间歇性清洁能源为主的未来城网中，其波动性对城网的安全性和供电可靠性提出了越来越高的要求.针对传统控制策略无法解决微电网中新能源规模化接入带来的频率不稳定，控制性能标准差的问题，提出了多步自校正Q学习算法.该算法中的自校正估计器能够准确估计系统的状态，有效提高机组控制精度.此外，资格迹机制可以实现多步备份，提高算法收敛速度，使得控制器能够满足指令信号与机组响应的时延性，减小时延所带来的调频影响.仿真部分，本文构建了包含储能系统、风力发电以及电动汽车的两区域负荷频率控制模型，并分别引入正弦波、阶跃和随机阶跃扰动来模拟电力系统中的负荷变化扰动.仿真结果表明与其他算法相比，所提算法在控制性能指标方面表现出更优的效果.

关键词：强化学习；微电网；负荷频率控制；清洁能源

中图分类号：TM732

文献标志码： A

A load frequency control strategy of island microgrid based on multi-step self-correcting Q-learning

WANG Qiang1，2，HUANG Zhen-wei1

（1.College of Electrical and New Energy， China Three Gorges University， Yichang 443002， China; 2.Hubei Provincial Engineering Research Center of Intelligent Energy Technology， China Three Gorges University， Yichang 443002， China）

Abstract：In the forthcoming era of power grids emphasizing clean energy and green transportation，stringent safety and reliability standards are imperative.This study addresses the limitations of traditional reinforcement learning in managing the control performance degradation due to the extensive integration of new energy sources in microgrid by proposing a multi-step self-correcting Q-learning algorithm.This algorithm features a self-correcting estimator for accurate system state estimation and an eligibility trace mechanism to expedite convergence，facilitating rapid controller responses to system fluctuations and minimizing the impact of frequency regulation delays.The simulation section of this paper presents an enhanced two-area load frequency control model，integrating wind power and electric vehicle modules，and subjected to various disturbances to mimic real-world power system load changes.The results demonstrate that the proposed algorithm excels in control performance metrics when compared to existing methods.

Key words：reinforcement learning；microgrid；load frequency control；clean energy

0 引言

随着全球资源紧缺与环境问题的日益严峻，“双碳”政策和相应行动已逐渐成为各国的共识.在此背景下，以清洁能源为核心的电源结构转型以及以纯电动为目标的交通工具发展，已成为全球能源和交通领域的主要趋势。（剩余12277字）