Machine Learning Applications in Microgrid Management System: History
Please note this is an old version of this entry, which may differ significantly from the current revision.
Contributor: ,

The advent of renewable energy sources (RESs) in the power industry has revolutionized the management of these systems due to the necessity of controlling their stochastic nature. Deploying RESs in the microgrid (MG) as a subset of the utility grid is a beneficial way to achieve their countless merits in addition to controlling their random nature. Since a MG contains elements with different characteristics, its management requires multiple applications, such as demand response (DR), outage management, energy management, etc. The MG management can be optimized using machine learning (ML) techniques applied to the applications. This objective first calls for the microgrid management system (MGMS)’s required application recognition and then the optimization of interactions among the applications.

  • active power distribution network
  • energy management system
  • microgrid management system
  • machine learning
  • deep reinforcement learning

1. Introduction

With the advent of RESs, the power industry could access infinite independent generators in feasibly electrified remote places [1]. RESs were initially integrated into MGs to supply rural and remote areas and were later deployed to campus and urban locations. A MG is capable of working independently in island mode and interacting with the utility grid in a grid-connected style. The main problem of utilizing RESs in MGs involves their intermittence characteristics, controlled by employing technologies, such as energy storage systems (ESSs), smart inverters, and responsive loads (RLs). Other challenges of MGMSs include the output prediction and load behavior of RES uncertainty. Therefore, it is necessary to coordinate the performances of all elements of MGs together, practiced by energy management systems (EMSs) [2]. EMSs are responsible for the generation and consumer elements of MG coordination by determining each generation unit portion in consumer supply, aside from preserving the stability of the system, such as voltage and frequency regulation [3]. It makes it necessary to provide a multi-objective optimization algorithm for EMSs.
As the main objective of a MGMS is to control energy consumption, prediction, and generation, ML can be used to control and predict in this system, dealing with the complex decision-making process. In the same line of thought, several studies have considered the application of ML in the optimization of MGMS [4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27]. Researchers in these studies considered MGMS implementation by investigating elements, forecasting tools, data and demand management, standards, and different sights of view in control levels, including communication, energy, and power electronics. The main focus points of these studies on MGMS optimization were on classical mathematics programming, heuristic optimization algorithms, and artificial intelligence (AI) methods, such as fuzzy logic, artificial neural networks (ANN), and game theory. Mosavi et al. in [16] proposed an overview of the ML application in the energy system, mainly focusing on RESs and their output predictions. The authors concluded that the hybridization of ML techniques leads to a better solution for modeling RESs. Han et al. [11], without the definition of the EMS role in the MG, divided MG control levels to primary, secondary, and tertiary levels, and introduced some optimization methods, such as linear programming, genetic algorithm (GA), and particle swarm optimization (PSO) for power management in MG. The authors in [9], after the classification of MGs based on control methodologies, operation modes, and applications, provided a review of the different strategies for EMSs in MGs, including dynamic programming, meta-heuristic approaches, fuzzy logic, and ANN, and showed the limitations of each procedure. The authors of [8] considered EMSs in the operation fashions of MGs, i.e., stand-alone and grid-connected modes. Fuzzy logic and linear programming (LP) were methods introduced to the implementation of EMSs in that study.
Reinforcement learning (RL) and its derivative methods with the combination of deep learning can defeat randomness in the element characteristic MGs; in recent years, trends have risen, with deep reinforcement learning (DRL) being applied in the MGMS. Multi-agent system (MAS) arrangements of power systems, energy management, distribution networks, economic dispatches, cyber security, MGs, demand side management (DSM), DR, and electricity markets are various power system applications that can be optimized by DRL [18,20,21,28]. The robustness of RL in stochastic behavior management of optimization problem elements also drives the use of RL in building energy management systems (BEMSs). Wang et al. [19] and Yu et al. [29] investigated DRL applications in BEMSs. Both studies in the same line of thought arranged the Markov decision process (MDP) of the BEMS issue and classified relevant research based on the DRL hired method, objectives, and implementation environments. Arwa et al. [30] investigated the formationg of a MDP for an integrated MG to the utility grid. After presenting a brief introduction of the RL methods, such as Q-learning, batch RL, actor–critic, and DRL, the authors summarized and classified related studies based on their applied techniques and scheduling MG objectives. The authors developed their study in a further attempt [31] to recognize RL method drawbacks and benefits. Another outcome of this paper involved the efficiency of applying MAS methods in the performance optimizations of MGs, in particular, multi-MGs.

2. Microgrid Management System

2.1. Microgrid Structure and Control Methodology

A MG involves a collection of loads and generators that should handle supplying consumers in grid-connected or islanded modes. MG suppliers include RESs and conventional generators, in which the latter are applied to dominate the intermittency and stochastic characteristics of the former [36]. The ESS is the other element of a MG; it is widely used to make RESs dispatchable, in other words, the ESS supports the grid in the unavailability situation of the RES. Controlling the active power generation and, consequently, the frequency deviation of the MG, facilitates these accessories. An ESS can be a hybrid ESS, e.g., batteries to support the steady power demand or an ultra-capacitor to support the transient power demand [37]. The electric vehicle (EV) is another form of an ESS in a MG [38]. According to the 2003 version of the IEEE 1547 standard, the MG should work in a grid-connected mode and supply its domain consumers by the utility grid. Then, in case of any failure situation in the utility grid, the MG should disconnect from the grid and supply a whole or a part of the loads autonomously by the ESS. In this scenario, the RES during the grid-connected mode charges ESS [39]. The IEEE 1547 standard, modified in 2018, is used to consider the MG as a member of an ADN, and transacts energy with the utility grid where the RES is equipped with voltage and frequency ride-through capabilities to control abnormal situations in a grid-connected mode [40]. If MG elements are considered, as depicted in Figure 1, they generally include conventional generators, e.g., RESs, ESSs, EVs, and loads. There are two groups for loads in the MG, including emergency loads and responsive loads. While emergency load-serving has priority in both grid-connected and islanded modes of a MG, responsive loads are sheddable and help the grid remain stable [41].
Figure 1. MG structure and elements.
The stability performances of MGs require each element to receive setpoints from the supervisory level to adjust their actions. There are three levels of control for MGs, including primary, secondary, and tertiary levels [42]. The primary level control of a MG is related to the adjustment of output-level resources. The interfaces of the resources and ESSs in the MG involve smart inverters. Smart inverters can follow grid forming, grid feeding, and grid-supporting strategies, as addressed in [43] extensively. Current source inverters (CSIs) connect to RESs with high impedance in parallel to implementing the grid-following strategy. This strategy tracks the maximum power point of a RES. The grid-forming involves a voltage source control inverter (VSI), which connects with a low impedance to the ESS in a “series” arrangement. CSIs and VSIs can control the voltages and currents of RESs through the inner loops and receive references from the primary control levels. The grid-supporting strategy is a droop control that is used in the primary level control and is provided by VSI to utilize dispatchable generators and ESSs in the MGs if grid-forming inverters in the islanded mode cannot maintain the frequency and voltage of the grid. Droop control methods make primary-level control communications "less", and are reliable by mimicking synchronous generator behaviors; however, due to distances between RESs, this method is not always feasible. Active power-sharing is another method used at the primary control level, which is communication-based. Since primary control specifications require fast responses, communication infrastructure requirement costs would be challenging [44]. The idea of ESS installation near to RESs has been researched because taking source installation space locations has altered the utilization of distributed ESSs [45]. The secondary control level is applied to restore voltage and frequency deviations of MGs to zero. This level of control, which is the MGMS, will determine the power-sharing of resources and ESSs for protecting the stability of the MG. The secondary control level, also known as the energy management system (EMS), interconnects with a supervisory level of the utility grid in the point of common coupling (PCC), which is a distribution management system (DMS). Secondary control is centralized or decentralized. The centralized controller approach follows the controlling and supervision of resources and ESSs, grid-connected and islanded mode determination, load management, contributions to the market, and forecasting the output power of sources centrally. Because of the multitude of information, decision-maker algorithms are complicated; fast, reliable, and high bandwidth communications in this trend are necessary [42]. Instead of a central control, MG actors in the decentralized method have a local controller. This scenario meets the MAS style requirements. In the MAS, each MG element is an agent, has a decision-maker algorithm in its local controller, and interacts with the neighbor and supervisory agents. Although both methods track hierarchical schemes and require MGMS to communicate with DMS, a centralized controller is commonly used to control distributed generators (DGs) inside MG. Additionally, the decentralized type is used in the grid with a broad horizon, including numbers of MGs incorporating the utility grid [46]. The tertiary control level is a DMS, which coordinates the performance of MG by assigning setpoints to the MGMS.

2.2. Microgrid Management System Requirements and Applications

According to the IEEE 1547 standard, a MG has several stakeholders, including aggregators, maintainers, operators, and distribution system operators (DSO). The MGMS, as a supervisory level of the MG, should be scheduled in a way that satisfies all stakeholders’ requirements, as shown in Figure 2. Moreover, Figure 3 reveals the related application of each stakeholder. Herein, the researchers mainly consider the MG role as an ADN member. This hypothesis is an opportunity to bring merits to whole stakeholders of MGs in several criteria. It facilitates the integration of RESs into the utility grid. MGs aid the role of prosumers for RES owners. Moreover, the power grid receives financial profits by getting rid of the development and refurbishment of transmission lines. MGs aid grid stability by providing services for the grid in the case of a failure in the system, such as frequency stability, voltage regulation, and the black start aid [47]. These characteristics would not be viable without a robust management system and applications coordinating the MG’s elemental performances and interactions with the utility grid. The following is a brief of the most notable applications.
Figure 2. EMS optimization function dependencies.
Figure 3. MG application requirements based on stakeholders.

2.2.1. Demand Response

Utilities deploy DR to control energy consumption during peak hours and contingencies through the dynamic contributions of consumers. DR has been applied in MGs to assist EMSs during contingencies in the islanded mode. DR also responds to the utility grid requirements when the MG behaves as an active element of the power system. DR is implemented by offering incentive regulations or time-dependent programs. Consumers decide to cooperate in peak shaving with utilities based on dynamic prices and schedules, such as time of use (TOU), critical peak pricing (CPP), and real-time pricing (RTP) [48].

2.2.2. EMS

The traditional power systems under the control of the generation company (GENCO), TSO, and DSO operated hierarchically. Centralized generation units supply passive loads through transmission and distribution networks in this arrangement. With the addition of smart grid technology and distributed generation, the power system transitioned from a hierarchical to a unidirectional distributed style. DGs are power generation units located at consumer sites and provide dynamic loads. DR programs will strengthen the new role of loads. The distribution system as an adjacent section of the power system to active consumers becomes more flexible, active, and complicated. The dynamic loads are controllable under the supervision of MGs. Both ADNs and MGs can collaborate to satisfy the control and protection requirements. In the ADN concept, one of the responsive elements of the system is the MG. The AS provided by MG EMS includes participation in the remedial action schema of the power system, such as the black start aid, network reconfiguration, frequency, voltage control, power loss reduction, and congestion management. These characteristics require EMS to be able to coordinate the performances of all MG elements. Energy consumption or production of RES, responsive or non-responsive loads, conventional generators (CGs), and ESSs should be determined. This can be done locally or by the ESS unit of the MGMS. Each entity may estimate its situation by its controller independently and send its information to the EMS, or the EMS calculates the portion of entities in the power generation and consumption and issues related commands. The EMS should coordinate its performance with supervisory-level control of the utility grid, which is the DMS [42]. The required input data and output scheduling that the EMS should provide are represented in Figure 4.
Figure 4. A MG EMS unit under the concept of ADN.

2.2.3. RES Output Prediction

This application is applied in a MG to reveal the amount of available energy from the RESs. They are the main suppliers of the MG and their performances depend on natural sources, such as solar irradiation and wind availability. Hence, forecasting their output assists EMSs in scheduling the MG contribution in market and storage charging and discharging [49].

2.2.4. Preventive Maintenance (PM)

Due to the huge number of elements in the power system, PM aids the grid in controlling risks and reducing the costs of outages and equipment breakdowns. Therefore, it improves the reliability indices of the grid. PM for the power grid in a smart environment is classified as passive and active. The passive class is concerned with the market, while the active class schedules PM without paying attention to the market or prices [50].

2.2.5. Distribution system Operation

The DSO implements the DMS by utilizing different applications, such as online power flow (OPF), fault location isolation–service restoration (FLISR), volt–var optimization (VVO), and smart connection arrangement (SCA). All of these applications help the DSO analyze the status of the power grid and predict its action requirements.

2.2.6. Market regulator

Any violation in the power system stability due to the difference between the scheduled power generation and consumption is known as an area control error (ACE). The electricity market regulator’s traditional task involves the ACE compensation with adjustable generation and consumption units in a market-based manner and determination of the electricity price for end users. The ADN is viable by providing AS services from the MG side. Therefore, in an enhanced power system, the electricity market regulator facilitates the MG participation in the ACE correction as a member of the ADN. The MGs can address the ACE by justifying the amount of generation and consumption according to the signal received from a market-based regulator.

3. Machine Learning Applications in MGMS

3.1. Machine Learning Techniques

The main challenge of obtaining access to the benefits of RESs and the high penetration of MGs is in providing a comprehensive MGMS, which is dependent on the embodiment of the EMS unit as a robust decision center through intelligent ML techniques. ML mimics how humans learn by deploying data and algorithms and improves the solution’s accuracy over time, typically in an iterative manner. ML algorithms, with the help of historical data, called data sets, train computers to output values that fall within a predetermined range. A MGMS, with the help of ML techniques, attempts to make the right decision for the control and management of the MG based on existing information and experiences. ML can detect all information patterns inside a MG and predict the behaviors of heterogeneous devices gathered under the supervisory of the MGMS [51]. ML has several sub-models, including supervised learning, unsupervised learning, semi-supervised learning, and RL, as shown in Figure 5.
Figure 5. Different types of machine learning.

3.2. Classification of Machine Learning Applications for the MGMS

Application requirements of each stakeholder should be satisfied by the MGMS. Easy access to ML modules in different language programming, such as Python-based scikit-Learn [52], MATLAB ML Toolbox [53], and R.Package caret [54], make it popular in optimization, prediction, and training issues in the MGMS.

This entry is adapted from the peer-reviewed paper 10.3390/en15228739

This entry is offline, you can click here to edit this entry!
Video Production Service