\cite{sarhan2020fairness} propose a method of learning representations that can be used in downstream tasks, yet that are independent of certain sensitive attributes, such as race or sex. The learned representations can be considered “fair” as they are independent of sensitive attributes. The authors report results on five different datasets, which most notably include (1) the ability of the representations to be used for downstream tasks (target prediction accuracy) and (2) the extent to which sensitive information is present in these representations (sensitive prediction accuracy).
In this text we report and compare the obtained results as well as highlight any difficulties encountered in reproducing the reference paper by \cite{sarhan2020fairness}.
As there was no openly available code base, we re-implemented the work. We
included scripts to automatically download the required data, designed the
dataloaders, and implemented the models as described in the reference
paper. Code is available https://github.com/paulodder/fact2021.
We were able to reproduce some of the results of the paper, but a significant part of our results was inconsistent with the findings of \cite{sarhan2020fairness}. For some of the simpler datasets we found similar patterns, but for the more complex tasks the models training became unstable, leading to results that varied significantly across random seeds. This made reproduction infeasible.
Conceptually, the paper was interesting and, given some prior knowledge on variational autoencoders and the math involved, it was also relatively straightforward to understand.
The most difficult aspect of the project was dealing with missing information. Many essential implementation details were missing, and there were inconsistencies in the pseudo code provided. Resolving these issues provided significant difficulties.
Various emails were exchanged with the original authors, in which we received explanation about unclear aspects of the paper. In general the authors were very helpful, but despite the fact that a few emails were exchanged, some aspect of the paper still remained unclear.
\newpage
A challenging problem in machine learning is learning representations that are fair with regards to a sensitive attribute present in the original samples. A common definition of fairness in this context is that the model’s output is statistically independent from the sensitive attribute \citep{xie2017controllable, roy2019mitigating, quadrianto2019discovering, fairmlbook}. \cite{sarhan2020fairness} proposed a new way to learn fair representations that are invariant towards the sensitive attribute, but are still useful for the task at hand. The proposed method to achieve this invariability is to disentangle the latent representation into independent target and sensitive representations \citep{locatello2019fairness}. As a proxy for independence, orthogonality is enforced between these individual representations. Furthermore, in order to prevent sensitive information leaking into the target representation, the model is trained to learn a target representation which is agnostic to the sensitive information, maximizing the entropy of our sensitive attribute given the target representation.
In order to consolidate the claims brought forth in the reference work of this report \cite{sarhan2020fairness}, and to assess the reproducibility of this work, our research attempts to reproduce the achieved results by re-implementing the proposed method. In the next section, we specify the parts of the original work that we attempt to reproduce. In Section 3, we summarize the method as proposed by \cite{sarhan2020fairness} that we attempt to reimplement. In Section 4, we report the results we attain. Finally, in Section 5, we discuss the results, and we analyse the reproducibility of the reference work.
The reference work by \cite{sarhan2020fairness} works towards a method of generating embeddings of data which are useful for downstream tasks, while they remain invariant towards a particular (sensitive) feature. The efficacy of this proposed method is assessed using two evaluative questions. First, how well can the learned target representation be used in the target task? Second, to what degree is information which might reveal the sensitive attribute still present in the learned target representation?
In the reference work, a collection of three binary- and two multi-class classification tasks are considered for a total of five classification tasks, corresponding to five different datasets. For each of these tasks, a version of the proposed model is trained, and evaluated using two metrics: target accuracy and a sensitive accuracy. The target accuracy measures how well we are able to predict the target attribute based on the produced target representation, and the sensitive accuracy measures how well we are able to predict the sensitive attribute based on the target representation – note that we want the sensitive accuracy to be as low as possible, because this implies that the target representation is independent of the sensitive attribute. Both of these accuracies, for each of the five tasks at hand, are included in our reproduction.
Furthermore, an ablative study is conducted in the reference work, in which specific components of the loss function used to train the model are excluded (i.e., ablated), in order to observe the behaviour of the model and, in doing so, understand the role of each of these loss components within the training process. This ablative study, which entails the evaluation of the impact of five unique combinations of loss components, is performed on each of the five datasets, and is included in our reproduction.
Finally, the authors perform a sensitivity analysis on the hyperparameters that control the relative importance of two of the loss terms they used, for one of the five tasks. For each combination of these hyperparameters, the model is trained, and the resulting target and sensitive accuracies achieved are displayed on a heatmap. We include this sensitivity analysis in our reproduction.
Since the code of the original implementation is not available, it is our goal to reproduce the method, based on all implementation details expounded in the reference work. The essential elements of the model are described in the next section. For a more detailed explanation, we refer the reader to the reference work \cite{sarhan2020fairness}.
Let
The target label
\mathcal{L}_{S}(θ_{S}^{*},φ_{S}) &= KL(p(\bm{s}|\bm{x})\parallel
q_{φ_{S}}(\bm{\hat{s}}|\bm{z}_{S}))
\end{align}
Here
To ensure that no sensitive information can leak into the target representation, we maximize the entropy of the sensitive attribute given the target representation, following \cite{roy2019mitigating, sarhan2020fairness}. This is achieved by minimizing \begin{equation} \label{eq:entropy-loss} \mathcal{L}_{E}(φ_{S},θ_{T}) = KL(q_{φ_S}(\bm{s}|\bm{z}_{T})\parallel\mathcal{U}(\bm{s})) \end{equation}
Last, we want to ensure that there is some level of independence between the
two representations. Ideally, we want the posterior
Here
We combine these two loss terms into a single term, which we call the Orthogonal Disentangled (OD) loss: $$ \mathcal{L}_{OD}(θ_{T}, θ_S) = \mathcal{L}_{z_{T}}(θ_{T}) + \mathcal{L}_{z_{S}}(θ_{S}) $$
We use the re-parameterization trick \citep{kingma2013auto} to sample from the approximated posterior distribution to obtain the latent representations, which can then be fed to the respective discriminators.
All of the aforementioned individual loss terms are further worked out in Appendix A. We combine all of them into one loss term and arrive at the following objective:
\begin{equation} \label{eq:total-loss} \underset{θ_{T},θ_{S},φ_{T},φ_{S}}{argmin} \mathcal{L}_{T}(θ_{T},φ_{T}) + \mathcal{L}_{S}(θ_{S^{*}},φ_{S}) + λ_{E}\mathcal{L}_{E}(θ_{T}, φ_{S}) + λ_{OD}\mathcal{L}_{OD}(φ_{T},φ_{S}) \end{equation}
Here
λ_{E}^{(t)} = λ_{E}^{(0)} γ_{E}^{t/t_s}
\end{equation}
Here
In order to reproduce the results obtained by \cite{sarhan2020fairness} it was necessary to apply the model to five datasets. Below, we outline some basic properties of the datasets and we explain the sensitive and target attributes that are to be modeled. For detailed information about the datasets such as train/test splits, number of samples and dimensions we refer to Table \ref{tab:data_details} in the Appendix.
The Adult and German dataset were obtained from the UCI repository \citep{uci}. Both of these datasets contain census data, and include categorical and continuous attributes which contain information about the person’s gender, education, and occupation. For both datasets, preprocessing consisted of representing categorical columns in a one-hot encoding, where missing values were explicitly encoded as a separate category, while continuous variables were left unchanged.
For the Adult dataset, the task is to predict whether a persons income exceeds
The Extended YaleB dataset was collected from the University of Toronto computer science department website \cite{georghiades2000few}. Specifically, the `Cropped’ version of the dataset was used \citep{KCLee05}, which contains grayscale images of 38 human faces under different lighting conditions. The task is to identify to which of the 38 humans an image corresponds. We constructed a sensitive attribute by clustering the illumination conditions into 5 clusters loosely corresponding to top left, bottom left, top right, bottom right and center. We defined these classes ourselves as we were unable to find detailed information on how this was done in the study by Sarhan et al. More details about the clustering of the illumination conditions can be found in the Appendix. Note that our majority class is not in line with the paper by Sarhan et al, who mention that a majority class classifier could attain 50% accuracy, in our case this is around 35%. Unfortunately, we were unable to find sufficient information to be able to replicate the ratios mentioned in the reference paper, and instead constructed our own sensitive attributes.
Our training dataset comprised of 190 images corresponding to one lighting position from each cluster, following \citep{sarhan2020fairness, louizos2015variational}. It is important to note that our testing dataset contained 2243 images, while the testing set in the reference work contained only 1096. The reason for this is unclear, as we used the full dataset, and found no mention of the omission of certain images in the reference paper.
The CIFAR-10 and CIFAR-100 datasets were also collected from the University of Toronto computer science department website \citep{georghiades2000few}. CIFAR-10 consists of colour images that are divided into 10 classes such as airplane, automobile and bird. For our purposes, we construct a new target attribute, one that denotes whether the subject of the image is alive or not, following \citep{roy2019mitigating}. The sensitive attribute, then, is the original label of the image. The CIFAR-100 dataset is similar to CIFAR-10, except that images are categorized as one of 100 total fine-grained classes. These 100 fine classes are split into 20 coarse classes that cluster similar concepts into one category. For example: `beaver’, `dolphin’ and `otter’ all belong to the coarse class `aquatic mammals’ (c.f. \citep{proteek}). Here, the coarse class of an image is used as the target attribute, while its fine class is used as the sensitive attribute.
Following the paper of \cite{sarhan2020fairness}, we implement the following networks for the several datasets. Note that, for every MLP mentioned below, ReLU’s are used as (non-final) activation functions. For the CIFAR-10 and CIFAR-100 tasks, the encoder used was the ResNet-18 architecture \citep{he2016identity}.
Most used hyperparameters were taken directly from the reference work, or its supplement provided by Sarhan et al. However, optimal values for some hyperparameters were not reported, and thus, we empirically set these to values that seemed to result in satisfactory performance. We discuss which hyperparameters we were missing in Section 5, and report all hyperparameters that we used in the Appendix.
Our implementation and instructions to run the code are available at
https://github.com/paulodder/fact2021. The repository contains a folder
scripts that contains all the scripts necessary to perform several tasks. All
instructions for setting up are in the README and instructions for
reproducing any of the numbers or figures reported in this text can be found in
produce_results.pdf in the aforementioned repository.
Evaluation of the embeddings learned by our model is non trivial, as we must gather whether the embeddings adequately represent the data for the downstream task (e.g. classification of target attribute), while also ensuring that the embeddings contain no sensitive information. In order to quantitively evaluate our model after completing training, we train two classifiers. These classifiers use the test data that is embedded using our trained model in the target space.
The first classifier, known as the target predictor is trained to predict the target label from the target embeddings. In accordance with the reference paper, we evaluated the target predictor using accuracy as metric. The details of the target predictors used are reported in Table \ref{tab:preds} in the Appendix. It is desirable that the target predictor performs as well as possible, as this means that the target embeddings embed the information necessary for the downstream task well.
The second classifier, known as the sensitive predictor is trained to predict the sensitive attribute from the target representation. For the sensitive predictor we use the exact same architecture and hyperparameters as for the sensitive discriminator. It is desirable that this classifier performs poorly, as we would like there to be no information pertaining to the sensitive attribute in our target embedding. As such, we would like the model to be as close to a `majority classifier’ as possible, where the model is forced to simply predict the majority label for each data row as it has no meaningful information with which to make a prediction about the sensitive attribute. Again, we use solely accuracy as evaluation metric.
For the sake of completeness, we briefly report alternatives that were explored but did not yield improved results, and were therefore abandoned. None of the features described below were used to generate results.
In order to select the best performing model to evaluate, two independent selection mechanisms were implemented, but not used in the final experiments. (1) We attempted to select the best iteration of the proposed model (over all epochs) by keeping track of the version in which performance was best. We first defined performance as train target accuracy (higher is better). Later, to also take into account the extent of sensitive information leakage in the target representations, we also included the accuracy of predicting sensitive attributes based on target representations. (2) we attempted to select the best iteration of target and sensitive predictors during their training, again by tracking based on their performance. Here, performance was defined as test target accuracy. However, this augmentation was discarded as we were unsure whether this was implemented correctly, as results did not improve (even though it should, in theory).
For YaleB, various model architectures were implemented in an attempt to amend
performance on this dataset. We experimented with variations in the
dimensionality and number of hidden layers of the encoder and discriminators,
activation functions (specifically, we tried Tanh), and the hyperparameters
learning rate, max epochs, batch size,
For CIFAR-10 and CIFAR-100, we experimented with freezing the ResNet-18 encoder (with the exception of the final, Linear layer, which was reinitialized), but despite faster training, the model’s performance did not increase.
We used Google Colab Pro to train our models, which supplies one Tesla
V100-SXM2-16GB GPU, and 2 Intel(R) Xeon(R) CPU @ 2.00GHz CPUs. Average
run-times are specified in Table \ref{tab:computational_details}. In order to train
all models over various seeds for all results, this would be the estimated
required run-time:
$$
(3 * 62) + (5 * 5 * 62) + (2*8^2 * 5 * 0.8) = 4,296 \text{ minutes}
$$
To judge the reproducibility of the model proposed by \cite{sarhan2020fairness}, we compare their results with those results we were able to attain using our implementation. First, we compare target and sensitive accuracy attained by training and evaluating the proposed model on each of the five datasets. Second, we compare the results of the ablative study. Finally, we make the same comparison for the sensitive study.
The results of our ablative study are shown in Figure \ref{fig:ablative}, which can be compared with the ablative study of Sarhan et al. in Figure \ref{fig:sarhan_ablative} in Appendix B. As a discussion of the potential implications of the various combinations explored in this ablative study forego the scope of this paper, we refer to \cite{sarhan2020fairness} for a detailed overview. The baseline measurement was omitted as it was unclear from the text what it entailed.
In comparison to Sarhan et al., for German, we see that varying loss components seems to have less impact on performance; for Adult, we see similar invariability for target accuracy but a lower impact on sensitive accuracy; for CIFAR-10, we observe a larger variance in performance over seeds and loss components; and lastly, CIFAR-100 and YaleB results are significantly different. In summary, our ablative study results generally do not exhibit the same patterns as those of Sarhan et al. This may, however, be attributed to our use of random seed averaging.
The results of our sensitivity study are shown in Figure \ref{fig:sensitive_adult}, which can be compared with the sensitivity study of Sarhan et al. in Figure \ref{fig:sarhan_sensitive_adult} in Appendix B.
When comparing these sensitivity analyses, it can easily be observed that there is very little in common between the two. First off, there is, for each subfigure, a sizeable difference in the accuracy ranges. This difference is in line with differences encountered in Figures \ref{fig:adult_target} and \ref{fig:adult_sens}. More importantly, however, there is very little similarity to be found in any of the accuracy landscapes displayed, with peaks and valleys located in different places. In the reference sensitivity analysis, these landscapes are smooth. However, this is also not reflected in our sensitivity analysis. Note that the smoothness of the reference sensitivity analysis might be visually exaggerated due to a relatively low number of coordinate samples compared to ours.
The claim of the original authors are as follows: by disentangling the latent representation of a data sample into two subspaces that are orthogonal to each other, as well as training the model using a loss function that encourages it to encode sensitive information into one of these subspaces, and meaningful information for the task at hand into the other of those subspaces, it is possible to create meaningful representations that do not contain any information from which a protected, or sensitive, attribute can be inferred.
In order for our results to support this claim, they would need to show that the proposed model is able to create representations that perform well on the target task (i.e. attains a high target accuracy), while it performs poorly in the inference of the sensitive attribute using the target representation (i.e. attains a sensitive accuracy close to the accuracy of majority voting). When looking at our results, we observe that this is indeed the case for the German dataset. However, for the Adult and CIFAR-10 datasets, the attained sensitive accuracy is substantially higher than the majority vote baseline; and for the CIFAR-100 and YaleB datasets, the model does not achieve a satisfactory performance in terms of target accuracy; and so, results from these four datasets do not appear to support the original claim of the authors. Likewise, those patterns that the authors observe in their ablative studies are reproduced in our own ablative studies.
This means that there is a discrepancy between our results and the original results from \cite{sarhan2020fairness}. Thus, when considering the large effort undertaken in this research to minutely reimplement their proposed method, we conclude that the original paper is relatively difficult to reproduce, and can in fact not be reproduced based solely on its contents.
We experienced especially the theoretical part of the paper to be well structured and though out. The set-up of the two types of representations and notions of disentaglement and orthogonality makes sense intuitively. Additionally, all loss terms are well described and were therefore easy to implement.
\paragraph{Performance fluctuations and training instability} One of the issues we ran into is that for these models training seems to be unstable, which is evident from the high fluctuation in performance when we vary the random seed or the number of maximum epochs. This is not addressed in the paper and therefore there is no information on how to deal with it. To add to this, it was unclear what trade-off between target and sensitive accuracy was used by the authors to select the best model during training. This trade-off ultimately determines which model is selected for testing which can have a large influence on performance.
\paragraph{Implementation} There were a few unclear aspects of the model
implementation that we resolved either by making a choice that seemed logical
to us, or through contacting the original author. For example, there was
limited information on how certain losses were backpropagated with a shared
encoder network. Besides this, the implementation of the decay the two
\paragraph{Hyperparameters} The amount of epochs that the model was trained was
not reported in either the paper or its supplementary material. This was quite
an important value given that no explicit stopping criterion was mentioned
either. In correspondene with Sarhan, we were able to set values for the
step_size hyperparameter that correspond to those used by the original
team. Furthermore, amongst the not reported hyperparameters were those involved
the training of the network-based target and sensitive predictors. These
include the optimizer used, the learning rate, weight decay, amount of epochs
as well as the nonlinearities, to name a few.
\paragraph{Dataset details} As mentioned in YaleB paragraph of the Datasets section we have made a number of assumptions about how to set up the classes corresponding to the sensitive attributes, which might have some influence on the performance of our approach for this datasets. We were unsure about some other details concerning the data as well. Namely, the type of data-normalization is not specified, and for the German dataset there is not a train-test split reported. However, these details were not as vital for reproduction as the aforementioned issue concerning the YaleB dataset.
We have had the pleasure of communicating with the original authors of the
paper. This helped getting our hands on some additional hyperparameters, such
as the stepsize
Due to the large scope of the research performed in our reference paper, our approach was diverse from the start. Many different avenues were explored from the beginning, dataloaders for all of the datasets were implemented and we had quickly written code to produce many of the figures necessary to asses the reproducibility of the research. While this meant that we gained a better understanding of the models’ performance and behaviour on all of the datasets and tasks from the beginning, it was complicated to work on all the tasks and datasets simultaneously.
The representation target loss can be computed as follows:
The second part of this expression solely depends on the true posterior of our data and hence does not depend on our neural network. Therefore, we drop it here. What remains is equal to the cross-entropy loss:
This is the same as the cross-entropy loss over the output of the discriminator. The representation sensitive loss can be computed in similar fashion.
We can compute the entropy loss as follows:
The second term is a constant and will be the same for every loss no matter the network, hence we drop it: \begin{equation} \label{eq:entropy_loss} \mathcal{L}_{E}(φ_{S},θ_{T}) = ∑_{\bm{s}} q_{φ_S}(\bm{s}|\bm{z}_{T}) log q_{φ_S}(\bm{s}|\bm{z}_{T}) \end{equation} Note that by dropping the last term, the entropy loss will always be negative.
We can write out the OD target loss as follows,
because both the prior and the encoder posterior are independent Gaussian distributions, the
KL divergence between the two is simply a sum over KL divergences between the
univariate Gaussians
One KL divergence terms can be computed as follows:
In practice, we will compute the element-wise KL divergence between the prior and posterior and sum over the result. The OD losses therefore require the output means and variances of the encoder network and the prior distributions of the latent variable. The OD sensitive loss can be computed in a similar way.
In order to construct the sensitive attributes for the YaleB dataset, we define a five-class clustering for the lighting positions, which corresponds to a five-class sensitive attribute. These clusters, as well as the lighting positions that are selected for the train partition, are displayed in Figure fig:yaleb_lighting_positions.
The hyperparameters that we used for our reported results can be found in table \ref{tab:hps1} and \ref{tab:hps2}. Note that for all experiments we used the Adam optimizer \citep{kingma2014adam}. <<TODO>> We should check whether these are in fact the last hyperparameters we used
We have reported the architectures and hyperparameters of the target predictor networks in Table \ref{tab:pred}. We used the Adam optimizer \citep{kingma2014adam} to optimize all MLP based predictor networks.
For ease of comparison, we include two Figures from the reference paper. All rights for Figures fig:sarhan_ablative and fig:sarhan_sensitivity_adult reserved by Sarhan et al.
