Rice is a globally significant staple food crop. Therefore, it is crucial to have adequate tools for monitoring changes in the extent of rice paddy cultivation. Such a system would require a sustainable and operational workflow that employs open-source medium to high spatial and temporal resolution satellite imagery and efficient classification techniques. This study used similar phenological data from Sentinel-2 (S2) optical and Sentinel-1 (S1) Synthetic Aperture Radar (SAR) satellite imagery to identify paddy rice distribution with deep learning (DL) techniques. Using Google Earth Engine (GEE) and U-Net Convolutional Neural Networks (CNN) segmentation, a workflow that accurately delineates smallholder paddy rice fields using multi-temporal S1 SAR and S2 optical imagery was investigated. The study′s accuracy assessment results showed that the optimal dataset for paddy rice mapping was a fusion of S2 multispectral bands (visible and near infra-red (VNIR), red edge (RE) and short-wave infrared (SWIR)), and S1-SAR dual polarization bands (VH and VV) captured within the crop growing season (i.e., vegetative, reproductive, and ripening). Compared to the random forest (RF) classification, the DL model (i.e., ResU-Net) had an overall accuracy of 94% (three percent higher than the RF prediction). The ResU-Net paddy rice prediction had an F1-Score of 0.92 compared to 0.84 for the RF classification generated using 500 trees in the model. Using the optimal U-Net classified paddy rice maps for the dates analyzed (i.e., 2016–2020), a change detection analysis over two epochs (2016 to 2018 and 2018 to 2020) provided a better understanding of the spatial–temporal dynamics of paddy rice agriculture in the study area. The results indicated that 377,895 and 8551 hectares of paddy rice fields were converted to other land-use over the first (2016–2018) and second (2018–2020) epochs. These statistics provided valuable insight into the paddy rice field distribution changes across the selected districts analyzed. The proposed DL framework has the potential to be upscaled and transferred to other regions. The results indicated that the approach could accurately identify paddy rice fields locally, improve decision making, and support food security in the region.