2.1.1 Individual fairness

Individual fairness in an AI model refers to the similarity in outcomes expressed for similar individuals within a dataset [21], ensuring fair treatment for each individual at the individual level, free from discrimination arising from individual characteristics such as personal lifestyle habits. In the context of individual fairness and feature extraction, if two individuals in a dataset are similar, the spatial distance between the feature vectors extracted by the model should be small. For example, in multi-organ segmentation, if the feature matrices extracted by the model from the input have small distances, or if the individuals share similar characteristics (they are all male or female), the model's effectiveness (measured by the Dice coefficient) on these individuals should also be similar. This ensures that the model treats similar individuals with consistent accuracy and fairness.

However, individual fairness has shortcomings in regard to fairness measurement. First, the feature distance between individuals is a crucial metric for individual fairness, but the way that this distance is measured can vary and is often subjective. For instance, both the Euclidean distance and Manhattan distance can be used to measure the distance between individual features. However, these different distance metrics emphasize different aspects, which can lead to situations where samples that appear close in Euclidean distance might be far apart when measured by Manhattan distance. This discrepancy can introduce bias into fairness assessments. Second, in fairness improvement, the individual distance is used as a learning target for model training. Because AI models tend to lack interpretability, misusing measurement standards can compromise fairness and even exacerbate model biases. Furthermore, calculating the distance between individuals is a highly time-consuming task because of the need for a big dataset to train the segmentation model and the complex data features of imaging data. Moreover, the measurement of individual fairness is susceptible to outlier data and may guide the model towards learning outlier data with extreme tolerance, thereby weakening the model.

2.1.2 Group fairness

The concept of group fairness is widely applied in medical image analysis. Braocas et al. [22] proposed that the goal of group fairness was to minimize performance imbalances among various subgroups, differentiated by sensitive attributes. Group fairness emphasizes the model's consistent performance across all individuals with various sensitive attributes, contrary to individual fairness, which focuses on consistent output for similar individual inputs. Strategies such as fairness indicator computation and disparity measurement have been proposed to assess and improve group fairness. For example, Puyol-Antón et al. [7] used standard deviation and the social equality ratio to quantify the fairness of nnUNet in cardiac MRI segmentation and assess the fairness of the model when applied to different races.

However, group fairness has limitations. In the downstream tasks of multi-organ segmentation, various sensitive attributes are often used, which may even contradict each other in certain cases. In such a situation, satisfying group fairness for a subgroup may disrupt fairness in other subgroups. Furthermore, a few fairness assessment indicators are overly idealized, which may make it challenging to fully meet the conditions of data security and privacy.

2.1.3 Counterfactual fairness

Counterfactual fairness is a concept based on causal inference put forth by researchers [7]. Alternatively, it focuses on studying the causes of bias in models. Counterfactual fairness is typically associated with a sensitive attribute and explores the consistency of outcomes under the counterfactuals of the sensitive attribute. This consistency reveals the degree of correlation between the model bias and the current sensitive attribute, thereby assessing the fairness of the model. Generative models such as generative adversarial networks or variational autoencoders (VAEs) are used to generate counterfactual samples for further assessments. For example, to assess the fairness of multi-organ segmentation across different races, it is necessary to control for other sensitive attributes and use a generation strategy to create data representing various races. This approach allows for the study of variations in segmentation effectiveness. If the multi-organ segmentation task satisfies counterfactual fairness, the variation in segmentation effectiveness should remain insignificant because the proportion of samples from different races in the dataset changes. However, this process can generate unrealistic features, showing less convincing fairness assessments.

2.1.4 Max–min fairness

Max–min fairness, much like its counterpart in network resource allocation, aims to minimize the gaps in benefits from machine learning between all groups or individuals [23]. More specifically, for a particular task, if a model performs poorly on certain subgroups, improving max–min fairness means guiding the AI model to improve its performance on these subgroups, thereby ensuring a reduction in the overall bias. For fairness assessments, max-min fairness tends to be used to assess the fairness of multiple models for multi-organ segmentation under certain sensitive attributes. For example, higher fairness will appear in the model when its poorest segmentation effectiveness in the specialized subgroup is the best among all the poorest effectiveness cases of the models. Therefore, max–min fairness relies heavily on the selection of models for specialized tasks. Furthermore, the model may compromise its excellent performance on other subgroups while improving its minimum benefit for a group or individual.

2.2.1 Imbalanced training data

2.2.2 Model design

A model's design, encompassing its structure, loss function, and optimizer hyperparameters, dictates its feature extraction ability and influences performance bias across different segmentation tasks [44]. It is noteworthy that in multi-organ segmentation tasks, models often prioritize understanding the masks provided by technicians and the manually set assessment metrics within medical images, rather than comprehensively capturing the global characteristics of the image data or the complex relationships between organ morphology and sensitive attributes [45]. For example, if male samples dominate the training set, the model may develop a bias towards male organ characteristics during segmentation, leading to poor performance on female subjects, particularly those with smaller organ volumes. This is especially problematic if the model fails to learn the differences in organ characteristics between men and women, or lacks the ability to distinguish whether the input data is from a male or female subject, relying solely on the image masks. It is possible for a segmentation model to be highly sensitive to data and gender, achieving minimal performance differences between male and female groups. However, the model might overly emphasize gender fairness, neglecting organ characteristics associated with race, which could result in more significant bias between different racial groups. In summary, a model's ability to extract image data features—including organ location, contour, size, and the relationships between organ characteristics and sensitive attributes—directly influences its segmentation performance across different subgroups. This performance is also directly affected by the model's architecture [44].

Afzal et al. [25] suggested that model construction impacts the fairness of organ segmentation. They used standard deviation and skewed error rate to assess fairness discrepancies in models, such as U-Net, V-Net, Attention U-Net, and nnU-Net. Their results indicated that Attention U-Net is a favorable choice for fairness concerning gender attributes, whereas the classic U-Net is better suited for age-related fairness. This finding highlights that different models have varying abilities to extract sensitive attribute information from data, leading to differing levels of fairness across groups. Interestingly, the most effective model, nnU-Net, did not demonstrate outstanding fairness.

Beyond structural design, the loss function also plays a critical role in determining fairness. The loss function establishes the optimization objectives of the model, and its composition is closely related to the assessment metrics used to assess model effectiveness. It also directly impacts model fairness. Therefore, incorporating fairness assessment metrics (such as the standard deviation of intergroup effects for each batch) into the loss function and providing the model with the attribute values of samples during the fairness calculation process can help improve fairness. Additionally, the loss function can guide the model in better understanding the feature differences among samples from different groups, thereby improving the fairness of intergroup segmentation performance. For instance, in multi-organ segmentation, introducing an additional classification task that focuses on sensitive attributes allows the model to identify which subgroup, such as men or women, the current sample belongs to before generating the segmentation output. This process enables the model to learn data features more comprehensively, thereby improving its encoding capabilities.

Other aspects of model design, such as the optimizer and its hyperparameters, influence the network's granularity, including the number of neurons involved in computation, the number of network layers, the number of iterations, the activation functions, and the model's learning rate, all of which affect segmentation fairness. Current studies have found that changes in the coefficients of the loss function can impact model fairness [25]. However, the effects of other potential factors (as mentioned above) on multi-organ segmentation fairness have not yet been assessed.

Although models can be improved to better extract data features and increase fairness, this can potentially reduce the model's effectiveness. For example, while nnUNet has demonstrated remarkable accuracy in kidney tumor segmentation, its fairness performance is not optimal [25]. This indicates that the existing models' effectiveness and fairness are not necessarily aligned, underscoring the need for researchers to consider fairness when striving to improve segmentation accuracy.

3.1.1 Data resampling

Imbalances in datasets can lead to disparities in a model's ability to learn from and extract features from various subgroups of data and are one of the main causes of bias. For example, an imbalance in the number of subjects in different subgroups used for model training will lead to differences in the model's inference effects for these different groups, thus causing fairness issues. Data resampling is often used to address problems of dataset imbalance, which can be categorized into undersampling and oversampling [61]. Undersampling involves randomly reducing data to achieve overall dataset balance, in which, for fairness, the data for various subgroups are mapped to a multi-dimensional point, and the points with larger numbers are adaptively reduced to keep the data distribution in the spatial mapping and balance the dataset. Oversampling involves adding data through unsupervised learning to reduce the imbalance between subgroups. Similar to undersampling, a mapping process is used; however, generation strategies such as applying a generative adversarial network are used to add data [62, 63].

A few studies have argued that oversampling is more effective for balancing data [64-66], whereas others have stated that oversampling did not operate appropriately [67]. Medical imaging data are valuable, and the extraction and labeling of imaging data from volunteers or patients require substantial human, financial, and time resources. The imbalance between subgroups (such as race and age) can be remarkable in popular research fields such as AI disease diagnosis and AI cancer site segmentation and prediction. Choosing to delete data in pursuit of balance can substantially increase the relative cost of a project because the data collection process is time-consuming, labor-intensive, and costly. Reducing the amount of data is also a waste of resources, and it can degrade the generalization ability of the model. As an alternative, oversampling is a balancing strategy based on adding data, but the usability and feasibility of the data itself may be somewhat compromised, and experimental results are hard to reproduce because of randomness. Moreover, the data generated by the generation strategy in order to balance the dataset may have data quality problems, and the introduction of outlier data will lead to an overall decrease in the model training effect. However, a few studies have innovatively combined two data-resampling strategies to further improve a model's performance [68].

Brown et al. [46] used undersampling to generate a biased dataset and a balanced dataset from original chest radiography data and found that the balanced dataset achieved a balanced distribution of negative and positive labels across all age groups. Puyol-Antón et al. [7] used batch sampling to ensure racial group balance within each batch to improve the fairness of cardiac MRI segmentation.

3.1.2 Data aggregation

Imbalances in the datasets used for model training can arise because of various real-world factors such as geography, policy, and even culture. For instance, multi-organ segmentation models designed in African countries often use data primarily from Black populations. In contrast, similar models in Europe and America mainly use data from White or Mixed-race individuals. However, these datasets can achieve balance by aggregation and complementation, thereby reducing model bias. In summary, data aggregation involves a strategy combining multiple datasets to solve the data imbalance problem. Because most datasets, especially those used for multi-organ segmentation, have considerable uniformity, achieving balance among various subgroups through data aggregation is highly feasible. Additionally, data aggregation is commonly applied in the multimodal training of AI networks to help the network fully learn data features and achieve more accurate predictions.

Seyyed-Kalantari et al. [47] aggregated MIMC Chest X-ray, Chest-Xray8, and CheXpert datasets to form an integrated dataset, which was then used for model training. Their experimental results indicated that the model trained on this integrated dataset exhibited lower disparity of true positive rate, thereby implying higher model fairness.

Data aggregation is also commonly applied in the multimodal training of AI networks to help them fully learn data features and achieve more accurate predictions. Different data modalities focus on capturing distinct characteristics of the target organ. For example, CT images provide comprehensive information on the density of an organ, whereas MRI images better represent the contrast in soft tissues, and these image features may be influenced by different sensitive attributes. Moreover, single-modality data may suffer from noise issues; for instance, patients from certain racial or age groups may exhibit poor image quality on a specific modality, which can hinder accurate multi-organ segmentation. By integrating multiple modalities, the model can leverage data from other modalities to compensate for this deficiency, thereby balancing multi-organ segmentation performance across different groups. Zhou et al. [42] aggregated CT scans with electronic health records for model training and compared the results with those obtained from training on individual datasets. They observed that the models trained on the aggregated datasets generally performed better across multiple subgroups than the control group, with relatively lower bias among groups, that is, a lower true positive rate gap between the groups. The results of this experiment demonstrate the feasibility of using multimodal dataset integration to address fairness issues.

However, data aggregation is often accompanied by other preprocessing strategies. While on the one hand there may be inconsistencies in the modalities and instrument parameters of different image datasets that requires preprocessing strategies such as data augmentation or image resampling to reduce their impact on the fairness of the model, on the other hand, different datasets may have different measurement units for the same sensitive attribute, which requires unification. For example, BMI may be recorded in one dataset for fairness assessment, whereas in another dataset the classes of overweight, healthy, and underweight are recorded for each sample. In order to aggregate the two datasets, the metric for measuring human obesity needs to be standardized.

3.1.3 Data embellishment

Data embellishment is used for image preprocessing and refers to the filtering out of original data noise through artificially specified rules. For instance, data embellishment can be applied in the preprocessing phase of facial recognition. Noise data, such as skin and hair color, can be filtered out by transforming original facial data into sketch data, reducing the generation of bias [42]. Data embellishment also has considerable application value for multi-organ segmentation. Multi-organ segmentation is often used as an intermediary process in research to facilitate subsequent feature extraction to improve model performance in tasks such as disease diagnosis and lesion analysis. During the data aggregation process, data from various regions may show inconsistencies in image size, brightness, and color due to equipment or parameter differences, affecting the performance of the model and creating bias in various sub-datasets. Image resampling is often employed as a solution for image size discrepancies; however, brightness and color inconsistencies, which may be a form of noise in imaging data, can be filtered out through data embellishment, thereby reducing the bias due to data specification inconsistencies in models trained on diverse sub-datasets. Sidhu et al. [48] used principal component analysis (PCA) to process retinal blood vessel images into grayscale images during the preprocessing step, and then applied contrast-limited adaptive histogram equalization to highlight retinal blood vessels in the grayscale image, effectively eliminating the impact of color on model training.

Data embellishment, which includes data augmentation strategies such as PCA, can be used to eliminate data features that contribute to unfairness issues, resulting in image data with similar features across subgroups. For example, in populations with organ lesions, the use of PCA can reduce the representation of lesion characteristics in the images while preserving the original features of the organs to be segmented. This makes the model less likely to be influenced by the lesion characteristics, allowing it to achieve better segmentation performance, even in areas affected by lesions. However, this approach may lead to the loss of original features that are beneficial for segmentation. For instance, when PCA is used, image features are compressed, and while this reduces the influence of factors such as health conditions, it can also blur organ contours, making segmentation more difficult. Additionally, there are challenges in training and fine-tuning the segmentation model on the processed data; the data improvement process is time-consuming, especially for multi-dimensional CT and MRI data.

3.2.1 Representation learning

Representation learning is the process of converting data types that computers cannot recognize, such as raw images, natural language, and sound, into numerical vectors. This process helps models learn the fundamental features of the data and ignore noise data, such as gender, skin color, and age, thus reducing the gap between various subgroups in the model. For fairness, representation learning performs feature extraction on the original data and focuses on reducing the spatial distance of data features in various subgroups in the spatial mapping, to reduce or even eliminate the model's data prejudice across different subgroups. Adeli et al. [50] proposed a feature extraction model based on adversarial learning and introduced a loss function based on the adversarial learning mechanism to reduce the correlation between the bias and the learned features. Furthermore, data features will contain confounding factors, and these covariates will affect the fairness of the model. For example, a model may be trained for multi-organ segmentation and the training data for a particular organ may be smaller in an elder subgroup. In this case, age performs as a confounding effect in the multi-organ segmentation process. To reduce the effect of confounding factors, Zhao et al. [49] proposed an end-to-end training strategy to extract features in the data that did not change with confounding factors such as age and gender.

3.2.2 Disentanglement learning

Disentanglement learning decomposes the factors of variation in the feature representations of a dataset into independent subfactors, each representing a certain real-world meaning. In machine learning, the bias a model exhibits across various subgroups arises from its inadequate learning of the dataset's features. This means that a few features related to sensitive attributes are also integrated into representations generated by the model, leading to a correlation between the model's output and the sensitive attributes. The subfactors unrelated to sensitive attributes in the dataset can be separated out through disentanglement learning, and the unbiased features so-obtained are particularly feasible for fairness improvement. Sarhan et al. [69] disentangled meaningful and sensitive representations from the original autism brain imaging data exchange dataset, using a forced orthogonal constraint as a proxy for independence. They enforced a reduction in the correlation between meaningful representation and sensitive representation through entropy maximization to obtain unbiased representations. Afzal et al. [25] incorporated a DenseNet architecture as a novel branch for gender classification into their segmentation model, thereby introducing a gender detection module into an existing multi-organ segmentation framework. This improvement assisted the model in distinguishing between sensitive attributes and common features, thereby mitigating spurious correlations between sensitive attributes and the representations learned for the segmentation task. Deng et al. [53] adjusted the relationship between the target representation and sensitive representation in the column space and row space, respectively, when disentangling the features of the CheXpert dataset, thereby achieving orthogonality in the representation space for the independence between target and multi-sensitive representations.

3.2.3 Contrastive learning

Contrastive learning is based on the unsupervised learning paradigm, with the objective of generating an encoder to encode similar data within a dataset in a similar manner while encoding various data as distinctively as possible [70]. Du et al. [54] introduced an additional neural network branch into disentanglement learning to act as a contrastive learning module, and this aided in the disentanglement of target representation and sensitive representation. The disentangled target representation demonstrated a lower correlation with the sensitive attribute owing to the contrastive learning mechanism, thereby further reducing the model bias across various subgroups.

3.2.4 Federated learning

In machine learning, federated learning is a distributed computing strategy that allows for the training of models on data distributed across various devices and servers while preserving data privacy [71]. Compared with traditional machine learning strategies, federated learning necessitates maintaining a distributed data step on each local server. The initial model from the central server is shared with all local servers that hold data for model training, and the central server ultimately aggregates and updates the model. This not only protects data privacy but also addresses the problem of data imbalance by aggregating the training results from multiple local servers, thereby substantially improving model fairness. Conversely, model fairness may decrease due to a decline in data diversity when the model relies on a single dataset. In radiology, federated learning has been applied in areas such as multi-organ segmentation, lesion detection, organ image classification, and disease tracking based on specific modalities through multi-institutional collaboration [72, 73].

Jiang et al. [57] proposed federated training through contribution estimation and used standard deviation, Pearson correlation, and Euclidean distance to assess fairness in two organ segmentation tasks, demonstrating the effects federated learning had on improving organ segmentation.