VRE resource forecasts are critical for minimizing the uncertainty of their generation, as this impacts all phases of decision-making, planning and funding. In the short-term, it helps with system stability through enhancing unit commitment and reducing reliability issues, in addition to being utilized in spot market electricity trading, and reducing the risk of incurring penalties for imbalances. In the long term it plays a role in preparations for extreme weather events through allocation of adequate balancing reserves, planning future expansions and the placement of VRE plants [76].

Forecast horizon is defined as the time period between actual and effective time of prediction. Four categories have emerged in recent literature, namely very-short-term (seconds to 30 min), short-term (30 to 360 min), medium-term (6 to 24 h), and long-term more than 24 h). Nowcasting refers to the process of producing short-range forecasts in the range of 4–6 h in the future. Nowcasting is projected to undergo significant improvements through the use of varied data sources, such as ground-based observations, radar data, remotely sensed observations, etc. These will result in challenges, including the handling of big data, quality control, and assigning weights to the various data sources [78].

Power forecasting is primarily done with three methods: physical, statistical, and hybrid. Physical methods rely on the systems design parameters to simulate the output power. Statistical methods encompass both traditional statistical modeling techniques and ML algorithms. Hybrid approaches are used to refer to the combination of two different methods [85].

Power forecasting generally includes two kinds of approaches: deterministic, which provides a unique value for the variable being forecasted at each future time-step, and probabilistic, which provides the full potential range of events using quantiles, prediction intervals (PIs), or distributions. Deterministic approaches have been explored for several decades, while probabilistic approaches have gained momentum in the past decade [86]. Probabilistic models provide a more comprehensive outlook on the possible scenarios resulting from forecasting processes, in the form of an interval where the point forecasts are expected to be found [87].

Reviews and studies on VRE power forecasting have found that classical ML models, such as linear regression, can be a decent choice for simplicity, but they may fail to capture non-linear relationships, making RF and SVM possible better choices. Hybrid models combining traditional time series forecasting with ML have also been used for VRE power forecasting [71]. The following subsections elaborate on studies conducted for power forecasting within each VRE domain.

Numerous reviews on the use of AI for solar power forecasting have been conducted [88,89,90]. The accurate forecasting of solar irradiance is of utmost importance for the power system designers and grid operators for efficient management of solar energy systems. An interesting observation from literature searches on solar power forecasting with AI is that most studies have been conducted for PV systems. This seems to primarily result from how forecasting methods have been developed largely for GHI, with few studies dedicated to DNI forecasting [54]. Reviews of solar power forecasting from PV systems provide insights into the current methodologies and future directions. The quantity of GHI is a primary influencing factor on the efficiency, in addition to the temperature, of the PV module. The efficient design of a PV forecasting system is also dependent on factors including the incorporation of forecast horizons, the selection of inputs with correlation analysis, pre- and post-processing of data, weather classification, network optimization, and uncertainty quantification [74].

GHI forecasting is performed primarily through two methodologies: the first utilizes cloud imagery with physical models, and the second utilizes ML techniques for statistical models [91]. Physical models utilize atmospheric variables that are directly related to solar power generation, making its process complex due to being affected by the uncertainty of the meteorological variables being used as input. Alternatively, statistical models utilize historical data to determine the relationship between meteorological variables and PV power generation, which is then utilized to build the power forecasting model [92].

Among the physical, statistical, AI, ensemble and hybrid models, extensive literature reviews have found that ANNs, and specifically convolutional neural networks (CNNs), are the most promising for short-term forecast accuracy and are covered most extensively in the literature [88]. A study on ML techniques for solar radiation forecasting envisioned the use of SVM, regression trees, and RF in the coming years due to their promising results, competing with ANN. A recommendation was for the use of ensemble predictors rather than simple ones [91]. Figure 1 shows the process of utilizing AI for solar power forecasting.

The estimation of PIs for point forecasts of solar power and their improvement is a topic that has gained interest in recent literature. The use of optimization techniques paired with ANNs can be used to customize PIs to different times of the day rather than have intervals, e.g., the power output during night hours is zero—thus, the interval of PIs during those hours can be narrower than during the day [93]. A study conducted on solar stations in Australia estimated prediction intervals using multi-objective PSO paired with ANNs. The prediction intervals were found to be improved when measured solar output was utilized as input alongside meteorological forecasts for short forecast horizons of 1–2 h, thus reducing uncertainty [79].

An interesting point discussed by Garus et al. [88] was on the models in the existing literature mainly being trained for the conditions of specific locations. The authors suggest that AI tools should be utilized to generalize models over a wider set of conditions with better prediction accuracy through integrating the existing AI approaches with optimization techniques such as GA, PSO, and analysis of variance (ANOVA). A complication to the generalization proposal lies in the impacts of different meteorological conditions in different locations. Climate conditions have been shown to influence the performance of different ML techniques for solar irradiance prediction. One example of this is a study conducted for solar power prediction at the Shagaya Renewable Energy Park in Kuwait, an arid desert region with predominantly sunny and clear sky conditions. The use of a regime-dependent approach, in which k-means clustering was used to independently classify regimes before applying an ANN, led to a degraded performance. The dominance of clear sky conditions in the meteorological conditions of Kuwait makes regime-identification approaches perform worse, due to minimal cases of cloudy sky conditions, and such approaches could be better suited to climate regimes with more diverse cloud conditions [94]. Another example is provided for the Nordic climate, which is characterized by daylight hours that are long in the summer and short in the winter, heavy snow, and highly variable weather conditions due to fast-moving clouds. These cloud movements can cause significant issues for PV plants integrated with low-voltage grids. Additionally, the snow-caused soiling effect during the winter is an important factor to consider. The estimation of the reduction in power generation due to soiling is difficult due to the complex optical characteristics of snow. A review of ML approaches to forecasting concluded that the choice of ML algorithm depended on the weather conditions of the study area. The deterministic component is more dominant than the stochastic component during stable weather conditions, making conventional ML algorithms such as SVM and RF viable choices. In conditions of unstable weather, in which the stochastic component is as important as the deterministic, the conventional algorithms mostly perform poorly, and DL methods are found to better capture the complex nature of the processes [95].

The estimation of behind-the-meter (or what is known as invisible) solar power has drawn attention in recent published literature. Invisible solar power refers primarily to small-scale rooftop solar resources for a single building, which is invisible to system operators due to privacy concerns or the lack of measurement infrastructure. Invisible power can lead to the underestimation of power demand during extreme weather conditions, in addition to impacting the stability of the power system. A review of methods conducted utilizing historical data discusses studies using fuzzy models, ANNs, SVR [96]. The authors note the importance of employing simplified approaches that do not require the historical records of many variables due to the difficulty of their collection for grid operators.

The unstable and random nature of wind speed is the primary contributor to the complexity of creating a stable supply of energy from wind resources. Wind speed is impacted by multiple atmospheric elements, including wind direction and atmospheric pressure [80]. Power generation in wind farms fluctuates sharply with changes in wind speed due to the non-linear generation between power generation and wind speed. Enhanced forecasting capabilities for wind energy are therefore critical for wind farm site selection, energy production planning, and grid stability. Literature searches yielded a number of reviews conducted on the use of AI for wind power forecasting, with AI methods creating breakthroughs in the forecasting process [97]. Relevant results from these studies are highlighted in this section.

AI techniques such as ANNs and SVM have been applied for wind speed forecasting, primarily generating point forecasts. The stochastic nature of both wind speed and the conversion of wind to power make uncertainty forecasts with a probabilistic framework a necessary area of research for wind power forecasting. PIs are therefore employed to quantify the uncertainty through upper and lower bounds of the forecasted variable [98]. The use of more than one ANN to forecast wind speed is recommended, and appropriate choices for pre- and post-processing techniques to increase accuracy. Ensemble methods have also shown promise for future use [99].

Big data research is becoming increasingly relevant to wind speed studies due to the increase in data sources that can be utilized, including weather satellites, equipment images, and time series. For example, forecasting using integrated information from wind farms in various geographic locations of a region is recommended to be studied as an alternative to only utilizing on-site data to forecast for a single farm [100]. Wind power forecasting models are generally classified into three categories: physics-based, data-based, and hybrid [101]. Data-based methods include AI approaches that assist in integrating big data to forecast wind energy output. For data-based wind forecasting, the most common approaches applied in studies are those employing AI methods and AI-based hybrid methods [102]. The hybrid approach of coupling NWP with ML methods, such as ANNs, is attracting attention due to its potential to produce more accurate forecasts. [101]. Hybrid applications often lack sufficient interpretability, leading to recommendations for future work to consider explainable AI methods for wind power forecasting [102].

ML and DL can be applied to a variety of areas in the field of marine energy, varying from perception in remotely sensed data, forecasting/prediction, optimization of design, and autonomous operations using reinforcement learning. Tidal energy at present is in need of more accurate energy forecasting methods to efficiently design and locate tidal turbines. Traditional forecasting methods do not have the full potential to meet this requirement. Currently, tidal currents are predicted using four method categories: statistical methods, dynamic models, AI, and hybrid models. Reviews of work utilizing AI for tidal energy forecasting discuss the utilization of DL for analyzing and extracting the change rules of tidal currents and using the learned rules for forecasting. DL algorithms are touted as a method that is not constrained by the weaknesses of current statistical methods and numerical models. MLP has been utilized for forecasting tidal height, long short-term memory (LSTM) for tidal water level prediction and meridional and zonal components of tidal current velocity [103]. Forecasting of significant wave height is an important element for wave energy management and requires heavy computational power in conventional numerical simulation methods. ANNs have found applications in this field, with recent advances including empirical mode decomposition techniques and transformer-based encoders [81]. LSTM has been used for the prediction of power generation from wave energy converters and has been shown to be faster and more accurate than the utilization of numerical simulations [82].

Energy transition initiatives have prompted power planning scenarios to move from traditional versions to integrated ones, to account for the characteristics of VRE. In addition to power generation forecasting, various elements exist which must be considered, including power demand forecasting, energy storage systems, performance of energy systems, and maintenance.

Accurate demand forecasting is critical to ensure reliability of power systems and provide an uninterrupted power supply to end users. It is important for grid stability and reliability, enabling grid operators to balance supply and demand in real-time in addition to assisting with efficient allocation of resources to avoid over-generation or under-generation. It also helps with minimizing costs associated with the purchase of power at high prices during peak demand times. Demand forecasting allows for the effective planning of the integration of VRE sources, including the needed infrastructure and charging schedules for energy storage systems [104].

ML methods have undergone improvements due to advancements in data analytics and have become a more standardized method for forecasting projected changes in energy demand. Reviews of ML techniques for demand forecasting have classified the most accurate forms based on a system level: on a microgrid/smart building level, ANNs or hybrid ANNs should be deployed; on a smart grid/smart city level, hybrid ANNs are found to perform best; and on a national/regional level, linear models display the best accuracy [24]. Literature surveys of load forecast model research have shown advancements in ANN to improve their capabilities over traditional methods [105]. A study conducted for Queensland, Australia employed ANN models to forecast 6 h and daily electricity load demand using climate data (e.g., temperature, rainfall, solar radiation) and determined that the best performance was obtained from a hybrid ANN approach with multivariate adaptive regression spline (MARS), multiple linear regression (MLR), and autoregressive integrated moving average (ARIMA) models [83]. kNN is another method that has been utilized for power demand forecasting [106,107]. A study conducted on analysis and short-term forecasting of energy demand for industrial facilities utilized a modeling approach based on clustering and kNN, with an error of 3% [108]. A predictive model for energy consumption applying kNN was utilized in Malaysia and had a minor difference in error compared to SVM while outperforming ANN [109]. KNN was applied in a study on predicting the stability of the grid linked with VRE, which involved conducting supply and demand predictions [110].

Energy storage options that are commonly deployed include transient variation options such as pumped storage hydro, adiabatic compressed air, lithium-ion and redox-flow batteries, and long-term storage such as hydrogen. Forecasting is important for scenarios in which storage is available and decision-making capabilities are required for when to charge and discharge batteries. The discussions in previous sections on AI techniques for forecasting of power generation and demand are therefore applicable for energy storage scenarios. Cost minimization is the primary planning goal, with the incorporations of flexibility strategies for real-time scheduling and deployment [111]. Optimum battery configuration is determined through the optimization of power matching and energy management algorithms [112]. PSO is widely employed in this framework [113].

For off-grid applications, hybrid VRE systems and microgrids are utilized to compliment energy storage options. AI methods are under study for use in optimization of coupling VRE resources in Saudi Arabia [72] and supplementing physics-based forecasting in Kuwait [114]. Some of the issues that arise with the development of these systems include stability analysis, big data analytics, and optimization of the combined components. Optimal sizing of hybrid microgrid systems is an example of where AI techniques can be implemented, where metaheuristic algorithms including PSO and GA have been utilized with good results. Evolutionary algorithms have been shown to achieve good results when using three or fewer objectives and therefore should be designed based on the number of objectives and constraints in the microgrid sizing problem [84].

The role of AI is becoming increasingly significant in the space of VRE systems, including the system design and materials, system monitoring and performance assessments, and overall security. The optimized design and sizing of VRE system components is another field in which AI has found applications. An example is ANN-based modelling of solar-grade silicon under wide temperature variations. Electrical parameters of the studied solar-grade silicon vary non-linearly with temperature. The ANN-based models allowed for their prediction using a limited amount of data over a wide temperature range [115]. An extensive list of more applications on AI for VRE design is provided in other reviews [116].

ML methods have found applications in renewable energy material studies, namely in the development of materials and devices for energy harvesting, storage, conversion and power grid optimization. Neural networks have become a recent subject of focus in the field of physical system modeling with the underlying property physics. The most common application in this field is the prediction of properties for material screening, which shortens the time needed. Examples of uses for the developed descriptors include material design for CO₂ capture, battery electrolytes and electrode discovery, and material screening for solar cells [117]. Closed-loop ML methods are being studied for applications in material discovery, as they enable the expansion of explored chemical space without the typical costs of time and effort. This is achieved through pattern detection in material structure–property relationships to create databases for training models, which will then produce predictions for other candidates in the chemical space [118]. Perovskites are a material type for which ML methods are greatly advantageous due to having a large chemical space from which constituents are selected [119].

AI can be of great assistance in applications for managing the performance and maintenance of VRE systems. An example of performance management is the use of an ANN to predict the temperature of the water outlet in a solar collector, through ingesting seven input variables. The ANN serves to better understand the behavior of the heating fluid, which can facilitate better use of mathematical models [120]. Another example is the application of ANNs to enhance the performance of a hybrid distributed generation VRE systems, in which the ANN was applied as a controller to enhance the quality of the power network [121].

Predictive maintenance is the augmentation of the system’s current operation states with the forecasting of future failure states. Predictive maintenance, along with condition-based monitoring, assists in lowering system maintenance costs, minimizing downtime, and increasing their useful life. AI methods have been incorporated for studies on developing prognostic maintenance systems, such as SVM and RF, for reliability assessment and maintenance optimization [25]. AI techniques allow for monitoring and anomaly detection of solar energy systems in real time through constant evaluation of performance data. Variations from the predicted working behavior can be detected swiftly, such as PV module failures, shading issues, and inverter malfunctions. This allows for increased reliability of energy systems through minimizing downtime and reducing losses [122]. Wind turbine maintenance is an application of AI which utilizes ANNs, GA, PSO, and fuzzy logic most frequently. ANNs are employed for monitoring, optimization, forecasting and decision-making, resulting in them being the most adaptable method. Optimization and decision-making are mostly performed utilizing GA and PSO, while risk mitigation employs fuzzy logic [41]. For smart grids, fault detection and classification are a critical component of self-healing and mitigating system failures. ANNs have been studied for intelligent fault detection, classification, and localization, with results indicating high success rates, and they have the potential to significantly improve power system reliability [123].

With regards to performance prediction, literature reviews have found that ANN and FL have been used more extensively than other approaches for solar energy performance prediction. The number of studies conducted on hybrid approaches, such as adaptive neuro-fuzzy interface system (ANFIS), are few despite their higher prediction accuracy due to their significant costs and computational time requirements, in addition to the complexity they add to the prediction process [88]. Another area for applications of AI is in system optimization incorporating batteries. Optimum configurations for batteries and ultra-capacitors have been done with PSO, artificial bee colony optimization, and harmony search algorithms [124].

Weather variations, in addition to playing a governing role in power generation forecasting, also impact the resilience and performance of energy systems in addition to the demand on them. Extreme weather conditions have been the main focus in modelling weather impacts on power distribution systems, and weather causing specific faults. ML methods employed in the integration of these conditions for system resilience include DL, ANNs, and probabilistic modelling. Despite weather variables playing a prominent role in the degraded reliability of VRE systems, they are often overlooked in reliability analysis. A lack of modelling of the collective effects of weather conditions for forecasting total system disruptions has been noted in the literature [125].

With the increased popularity of adopting smart grids along with VRE systems, attention has been brought to several critical issues, including individual privacy, security, and reliability in terms of communication and performance [126]. The cyber-physical system of a smart grid integrating VRE can be made more secure using AI. Example studies have looked into utilizing neural networks to identify the point of attack and impact of cyber attacks, with the breach of consumer data privacy being identified as a significant threat [127].

The costs involved in integrating VRE into power systems are not taken into account for the levelized cost of electricity (LCOE), which can result in them negatively impacting the economic feasibility of VRE. Several cost components control the integration of VRE systems based on their characteristics, including uncertainty and variability. The uncertainty stems from the differences between VRE forecasted output and actual generation, and the need to balance the differences in a short time period. The variability relates to power being generated in specific weather conditions, which does not always match demand, making the frequent ramping up and down of backup generators necessary additional profile costs [128]. As discussed in previous sections, AI can assist in more accurate VRE power generation and demand forecasting, thus assisting in the mitigation of uncertainty and variability costs. With increased availability of data on energy demand and supply, AI will assist in optimized scheduling based on weather conditions and consumer patterns, enabling further cost reductions [129].