Defense Against Attribute Inference

[Jun 27, 2022] Vanilla Model Training

Predicting Hispanic ethnicity from Texas-100X (I)

Threat Setting:
Training distirbution is general data set distribution except the most populous hospitals.

Adversarial Knowledge:

• High: Adversary knows all but one record from training set.
• Medium: Adversary knows the training distribution (i.e. can sample records with no intersection with training set).
• Low: Adversary knows a skewed distribution (that has records from hospitals with highest number of patients).

Model uses race as a training feature

As reported in the paper draft, the model trained on Texas-100X data set uses the race attribute as a feature which could have a positive correlation with the ethnicity attribute. Regardless of this, it is still useful to compare the attribute inference attacks with the imputation (IP) attack in this setting.

WB$\cdot$IP multiplies the outputs of WB and IP
WB$\diamondsuit$IP uses a decision tree model to combine WB and IP
WB$\square$IP uses a decision tree model to combine WB, IP and WB$\cdot$IP
IP$^\dagger$ is the imputation with access to class label

Candidate Sets:

• Train: Subset of 10,000 candidate records drawn from the training set.
• Test: Subset of 10,000 candidate records randomly sampled from the training distribution.
• OOD: Subset of 10,000 candidate records randomly sampled from a distribution different from the training distribution.

Model doesn’t have access to race

Here we study the impact of removing race from the model training. Since race and ethnicity have implicit correlation, it would be more realistic to assume the adversary trying to infer ethnicity would not know the race of the query record.

Remark: From the results in Table 2, we can see the impact of removing race attribute on ethnicity inference. The imputation PPV drops from ~80% to ~40%.

Results on out-of-distribution candidate set

Predicting Hispanic ethnicity from Texas-100X (II)

Here we consider the scenario where the training data comes from skewed distribution, while an adversary (in low adversarial knowledge setting) might have access to the general pupulation distribution.

Threat Setting:
Training distirbution is a skewed distribution limited to records from most populous hospitals.

Adversarial Knowledge:

• High: Adversary knows all but one record from training set.
• Medium: Adversary knows the training distribution (i.e. can sample records with no intersection with training set).
• Low: Adversary knows the general data set distribution except the most populous hospitals.

Model uses race as a training feature

Remark: As shown in Table 3, IP$^\dagger$ performs slightly worse than IP. Hence the class label seems to have a negative correlation with the Hispanic ethnicity in this setting where training records come from a skewed distribution.

Model doesn’t have access to race

Remark: Similar to previous case, race’s correlation to ethnicity has a huge impact on the inference task.

Results on out-of-distribution candidate set

Comments

• In all the above settings, the white-box attacks perform similar on both training and test candidates, unlike the membership inference case where the attacks perform dissimilar across training and test candidate sets.
• White-box attack outperforms even the imputation attack that has access to the class label. Thus, the model is leaking something beyond what an imputation can learn from the same data that the model is trained on. Is this because the imputation model is not learning effectively?

[Jul 14, 2022] Vanilla Model Training [Contd.]

Here we study the inference of Asian race from Texas-100X data set to see if the trend is same as what we observed for ethnicity attribute above. For this experiment, we remove the ethnicity attribute from the model training, since it is correlated to the target race attribute as we saw above.

Predicting Asian race from Texas-100X (I)

The model is trained on general distribution and the adversary might have access to either the training set, or training distribution or a different distribution based on the adversarial knowledge. In the general distribution, 22% of records have Asian race.

Remark: We observe the same trend— white-box attack does better than imputation and combining drastically improves the PPV of the combined attack on the general distribution candidate sets. Thus model is learning something useful about the distribution beyond the imputation. In the low setting, where the adversary only has access to skewed distribution, white-box attack still learns something useful, but combining with imputation does not further improve the PPV, suggesting that the attack signal is not different from imputation.

Predicting Asian race from Texas-100X (II)

Here the model is trained on the skewed distribution that has 15% of records with Asian race.

Remark: Here also the observation is the same. White-box attack does better than imputation when the model is trained on the skewed distribution and the candidate sets are also from the skewed distribution.

Results on out-of-distribution candidate set

Here we evaluate the attacks on out-of-distribution candidate set. For the model trained on general distribution, the OOD set is from the skewed distribution, and vice-versa.