Surgical therapy, encompassing liver resection and liver transplantation, represents the optimal approach for treating malignant liver tumors. Notably, postoperative recurrence is a critical prognostic factor, drawing the attention of clinicians.

For HCC, the prediction of recurrence has garnered considerable interest. Ji et al. [34] harnessed contrast-enhanced CT images from 470 patients with solitary HCC who underwent surgical resection to construct a recurrence prediction model using a machine-learning framework, achieving a concordance index (C-index) of 0.633–0.699. By integrating clinical features into the predictive model, superior prognostic performance was realized with a C-index of 0.733–0.801. Several similar studies employed various machine learning modeling methods, including random forest and SVM [20,25,27,50], to establish predictive models and achieved remarkable results in predicting HCC recurrence, with area under the receiver operator characteristic curves (AUCs) ranging from 0.834 to 0.948. These advancements facilitate a more precise evaluation of recurrence in HCC patients following surgical resection.

Deng et al. [43] demonstrated that radiomics models hold an advantage in predicting overall survival (OS) following radical resection. By constructing a radiomics prediction model using pre-surgery CT images, they achieved AUCs of 0.905, 0.884, and 0.911 for predicting 1-year, 3-year, and 5-year OS, respectively, in the validation cohort. This research may hold potential implications for informing clinical treatment decisions and prognostic assessments post surgery. Furthermore, the assessment of functional liver reserve before surgical resection is of critical importance, as it relates to the risk of post-hepatectomy liver failure [61]. Zhu et al. [32] utilized preoperative MRI and CT images to develop radiomics models for evaluating the functional liver reserve of HCC patients. In groups with indocyanine green retention rates at 15 min set at 10%, 20%, and 30%, the MRI radiomics model outperformed CT, with AUCs of 0.917 vs. 0.822, 0.979 vs. 0.860, and 0.961 vs. 0.938, respectively. Similarly, Wang and collaborators [51] employed preoperative gadoxetic acid-enhanced MRI radiomics features and an unsupervised machine learning approach to assess the risk of liver failure in HCC patients with varying functional liver reserves, revealing significant distinctions among functional liver reserve subgroups.

For ICC, Jolissaint et al. [28] retrospectively analyzed 138 ICC patients who underwent surgical resection, incorporating CT texture features and tumor size to predict early intrahepatic recurrence. The model achieved an AUC of 0.840 for recurrence prediction in the validation cohort. This result was successfully replicated in a multicenter study by Bo et al. [29], who enrolled 127 ICC patients undergoing curative surgery from three institutions. Their machine learning radiomics models, based on CT images, exhibited a mean AUC of 0.87 ± 0.02 for predicting early recurrence. Additionally, Qin et al. [35] conducted a retrospective study involving 274 perihilar cholangiocarcinoma patients who underwent contrast-enhanced CT and curative resection. They developed a multilevel predictive model that performed impressively in quantifying the risk of early recurrence, with AUCs reaching 0.883. The accuracy of the multilevel predictive model was 0.826, significantly surpassing the accuracy of conventional staging systems (0.641 for the American Joint Committee on Cancer 8th TNM, 0.617 for Memorial Sloan-Kettering Cancer Center, and 0.581 for Gazzaniga systems) [62,63,64].

Liver metastases are widespread, with colorectal metastases being the most common [65]. Granata et al. [16] harnessed radiomics features extracted from multiphase MR images, employing both traditional machine learning and deep learning frameworks to predict clinical outcomes in colorectal liver metastases (CLM) patients following liver resection. These features and models demonstrated significant prognostic value in evaluating recurrence, mutational status, pathological characteristics, and surgical resection margin, with accuracy ranging from 82% to 95%.

Nonsurgical resection therapy serves as a vital complementary treatment for malignant liver tumors, encompassing therapies like ablation therapy, TACE, radiotherapy, and systemic therapy. The following are some radiomics studies associated with these treatment modalities.

Ablation therapy represents one of the radical treatment approaches for liver malignant tumors, particularly suitable for small HCC and liver metastases.

In the context of HCC, Tabari et al. [31] collected pre-ablation MR images to predict post-ablation pathologic treatment responses in early-stage HCC patients undergoing liver transplant. By constructing a radiomics model using machine learning, they discovered that pre-ablation MRI radiomics features could predict the pathologic treatment response of tumors in HCC patients undergoing ablation therapy, achieving an AUC of 0.830. Peng et al. [49] enrolled 149 HCC patients who underwent curative ablation with the goal of predicting recurrence-free survival. The random survival forest model, which integrated MRI radiomics and clinicopathological features, demonstrated strong prognostic value for evaluating early recurrence, with a C-index ranging from 0.733 to 0.801. This effort may hold potential in stratifying patients for the adoption of the most appropriate follow-up and intervention strategy.

In the case of liver metastases, Taghavi et al. [18] found that a CT radiomics model could predict local tumor progression for CLM before thermal ablation, with a C-index of 0.780. Their analysis included 90 CLM patients with 140 lesions. Shahveranova et al. [39] constructed a combined model based on MRI radiomics and clinical characteristics and arrived at a similar conclusion, with AUCs ranging from 0.927 to 0.981. Subsequently, Taghavi et al. [17] sought to validate whether radiomics features derived from pre-ablation CT images of patients with colorectal cancer could predict the development of new CLM after successful thermal ablation. However, they were unable to identify an effective predictive model, with AUCs ranging from 0.520 to 0.570, only achieving inferior performance in an external validation cohort.

TACE is a common treatment for HCC and is particularly suitable for intermediate-stage HCC [61]. However, predicting the responses of HCC to TACE remains a challenge.

Liu et al. [9] used contrast-enhanced US cine images to predict the personalized response of HCC to the initial TACE treatment. They constructed a radiomics contrast-enhanced US model using deep learning, achieving an AUC of 0.930 in the validation cohort. In parallel, Shi et al. [15] and Peng et al. [38] each conducted single-center and multicenter studies to explore the ability of CT to predict the response of HCC to TACE. Remarkably, the radiomics models developed by these research teams achieved AUCs ranging from 0.949 to 0.994 in validation cohorts. Additionally, Bernatz et al. [21] found that a CT radiomics model could identify HCC patients responding to repetitive TACE, thereby contributing to the refinement of treatment algorithms. Similar studies [36,45] in the field of MRI combined with machine learning also reported promising results. These endeavors may contribute to selecting appropriate HCC patients who respond to TACE and enhancing the patient prognosis.

In terms of selecting suitable HCC patients for TACE, Wang et al. [11] retrospectively enrolled multicenter HCC patients who underwent TACE treatment. They found that a CT radiomics model could effectively discriminate between suitable and unsuitable HCC patients for TACE, achieving an AUC of 0.894 in the validation cohort.

For survival prognosis, Liu et al. [40] and Wang et al. [23] utilized CT images to develop CT radiomics-based survival prognosis models using a deep learning framework to predict the overall survival of HCC patients after TACE treatment. These models achieved C-index values ranging from 0.649 to 0.730. Although TACE in combination with tyrosine kinase inhibitor has been shown to improve outcomes in HCC patients, identifying patients who might benefit from the combined treatment remains challenging [66]. Ren et al. [44] recruited HCC patients who received the combined treatment and exacted radiomics and deep learning features from pretreatment CT images to constructed models for predicting outcomes, achieving AUCs ranging from 0.870 to 0.940. The results may offer a rapid and supportive method to identify patients likely to benefit from the combined treatment and have the potential to improve precision oncology.

In an effort to predict which HCC patients might develop extrahepatic spread or vascular invasion after initial TACE monotherapy, Jin et al. [33] retrospectively enrolled 256 patients and developed a combined model that integrated clinicoradiological predictors and a CT radiomics signature. The radiologic characteristics of tumors were evaluated by two experienced radiologists, blinded to patient information, who jointly reviewed all CT images to validate nine radiographic phenotypes, including (a) tumor number, (b) tumor size, (c) enhancement pattern. This combined model exhibited superior discrimination performance compared to the clinicoradiological model (AUCs 0.911 vs. 0.772 in the training set; AUCs 0.847 vs. 0.746 in the testing set). Importantly, it demonstrated the capacity to effectively stratify HCC patients based on their risk levels, potentially refining follow-up strategies for these patients.

These outstanding efforts underscore the enormous potential of radiomics in improving patient selection and treatment outcomes in the context of TACE.

Radiotherapy is categorized into external radiotherapy, such as stereotactic body radiation therapy (SBRT), and internal radiotherapy, such as transarterial radioembolization (TARE). It is a common and suitable treatment for unresectable HCC and liver metastases.

Fontaine et al. [12] conducted a retrospective multicenter study utilizing both unsupervised and supervised clustering methods to construct an MRI radiomics model for predicting overall survival in HCC patients after SBRT. However, the model’s performance was suboptimal, with a sensitivity of 0.52 and specificity of 0.71. Some researchers have also dedicated their efforts to the study of liver metastases. Hu et al. [46] retrospectively acquired data from 97 CLM patients after SBRT and developed an automated model to predict progression-free survival using CT radiomics and machine learning, achieving a C-index of 0.68.

Stüber et al. [14] collected CT images from 491 CLM patients who underwent TARE to extract radiomics features and create models. Nevertheless, they did not observe significant additional prognostic value in these radiomics features for predicting overall survival when compared to information obtained solely from clinical parameters. Kobe et al. [42] aimed to predict treatment response to TARE in patients with liver metastases using pre-treatment CT images, employing both traditional machine learning and deep learning algorithms. The model achieved an AUC of 0.850, a sensitivity of 94.2%, and a specificity of 67.7% in a testing set.

Based on the findings from these studies, it is evident that the applications of radiomics combined with machine learning face several challenges in the field of liver malignant tumors after radiotherapy, particularly regarding the inferior performance of these models. Therefore, there is a pressing need for more advanced methods and innovative research in this area.

Systemic therapy encompasses various anti-tumor treatments, primarily including molecular targeted drug therapy, immunotherapy, and chemotherapy. Numerous researchers have been exploring the applications of radiomics combined with machine learning in the systemic treatment of liver malignant tumors, as follows.

In terms of HCC, several notable studies have leveraged radiomics and deep learning techniques. Tian et al. [10] developed a preoperative MRI model that integrated radiomics and deep learning features to predict the programmed death-ligand 1 (PD-L1) expression level in HCC patients. This model exhibited robust predictive performance, achieving an AUC of 0.897, surpassing the performance of the radiomics-only MRI model with an AUC of 0.794. Dong et al. [37] aimed to predict the efficacy of anti-programmed death-1 (PD-1) antibodies in combination with tyrosine kinase inhibitors for advanced HCC. Their CT radiomics model achieved an AUC of 0.792 in the testing cohort, and radiomic features were found to be associated with overall survival. Bo et al. [41] also constructed CT radiomics models to predict the response to lenvatinib monotherapy for unresectable HCC patients. In this retrospective multicenter study involving 109 patients, the optimal radiomics model achieved impressive AUCs of 0.970 in the training cohort and 0.930 in the external validation cohort. Similarly, in the case of ICC, Zhang et al. [30] used combined models based on MRI radiomics and clinical features to predict PD-1 and PD-L1 expression of ICC, achieving AUCs of 0.897 and 0.890, respectively.

With respect to CLM, which are typically treated with first-line chemotherapy, a subset of patients benefits from this standard treatment [67,68]. Giannini et al. [22] developed a CT delta-radiomics score to predict the response of CLM patients to first-line chemotherapy, achieving a sensitivity of 85% and a specificity of 92%. Qi et al. [19] employed artificial neural networks and machine learning algorithms to create a predictive model based on CT images and clinical features, identifying CLM responses to first-line chemotherapy with AUCs of 0.754 in the training cohort and 0.752 in the validation cohort. Additional studies [13,26] used radiomics to predict CLM responses to first-line chemotherapy, yielding favorable results.

Antiangiogenic drugs are increasingly combined with chemotherapy in CLM patients [69]. Qu et al. [24] used a dynamic radiomics feature extraction method to construct a CT radiomics model for predicting the efficacy of antiangiogenic therapy in CLM patients, achieving a promising AUC of 0.945 and accuracy of 0.855. To identify CLM patients sensitive to therapy targeting the anti-epidermal growth factor pathway, Dercle et al. [48] built a CT radiomics model using a deep learning and machine learning framework, achieving an AUC of 0.800 for predicting sensitivity.