HR Data: Confidentiality, Pseudonymization and Anonymization

First lesson of the HR Analytics course, free to read: about 15 minutes, 3 exercises and 3 quiz questions. This is the catalogue's base lesson; in the student space, it is rewritten for your profile.

Objectives

  • Apply personal data protection principles to an HR analysis
  • Distinguish pseudonymization from anonymization and pseudonymize a file with a secret key
  • Publish aggregates without allowing a person to be re-identified (disclosure threshold)

HR data confidentialityPseudonymizationAggregate disclosure threshold

Data About People

Salaries, absences, evaluations, departures: HR analytics works on the most personal data in the company. A leak or poor disclosure can directly harm an employee, and teams’ trust in the HR department depends on it. This course therefore starts with data protection, before any calculation.

HR data is personal data within the meaning of the GDPR. A few established principles guide any analysis:

  • purpose: you know why you process the data (manage turnover, measure pay equity) and you do not reuse it for anything else;
  • minimization: you keep only the columns needed for that purpose;
  • limited access: only authorized people see individual data;
  • limited retention and security: protected files, no copy on a USB stick or in an email.

Some information falls under the special categories of data (notably health or trade union membership) and benefits from enhanced protection. A medical reason for absence is one of them: it is analyzed only in aggregate, if necessary, and discussed with the company’s data protection officer (DPO).

Minimize and Pseudonymize

To analyze salaries by department, you need neither the name nor the email address. Step one: remove them. Step two: replace the employee ID with a pseudonym, so you can link several files without exposing the real identifier.

A simple hash (sha256 of the employee ID) is not enough: employee IDs are few and predictable, and it is enough to hash all possible numbers to recover the correspondence. Use a function with a secret key, HMAC:

import hmac, hashlib

def pseudo(employee_id, secret):
    digest = hmac.new(secret.encode(), str(employee_id).encode(), hashlib.sha256).hexdigest()
    return digest[:12]

clean = raw.drop(columns=["name", "email"])
clean["pid"] = clean["employee_id"].map(lambda i: pseudo(i, SECRET))
clean = clean.drop(columns=["employee_id"])

The same employee ID always gives the same pseudonym with the same key, which allows joins. The key is kept separately, by an authorized person, never in the notebook or in the code repository.

Pseudonymization Is Not Anonymization

  • Pseudonymized: the person is no longer named, but can be found again (with the key, or by cross-referencing columns). The data remains personal and the GDPR continues to apply.
  • Anonymized: re-identification is impossible, irreversibly, by any reasonable means. The data then falls outside the scope of the GDPR.

Real anonymization is hard: in a small business, “the only female manager in the legal department over 50” is identifiable without any name. Cross-referencing a few columns (department, age, sex, hire date) is often enough. The usual techniques: generalize (age bands rather than exact age), remove rare columns, and publish only aggregates.

The Disclosure Threshold

Publishing “average salary of the legal department” when that department has two people amounts almost to publishing their salaries: each knows their own and deduces the other’s. Apply a threshold rule: any indicator calculated on fewer than N people (often 5; the value is an internal rule to set with the DPO) is masked.

def safe_aggregate(df, by, value, min_n=5):
    out = df.groupby(by)[value].agg(n="size", mean_value="mean")
    out["suppressed"] = out["n"] < min_n
    out.loc[out["suppressed"], "mean_value"] = float("nan")
    return out

Beware of deduction by difference: if you publish the company total and all departments but one, the masked department can be recalculated. Then mask a second group or group the small departments into an “other” category.

Key Takeaways

  • Minimize: keep only the columns useful for the purpose.
  • Pseudonymize with a secret key kept separately; the data remains personal.
  • Never disclose an aggregate calculated on too small a group.

Exercises

Solutions and tests stay on the server. In the student space, your code is run and checked, and the tutor gives hints before the solution.

  1. CodeDifficulty 2Python in the browser2 hints

    Pseudonymize an extract

    Write pseudonymize(df, secret) that returns a copy of df:

    • without the columns name and email;
    • with a column pid = the first 12 hexadecimal characters of the HMAC-SHA256 of str(employee_id) with the key secret;
    • without the column employee_id.

    Do not modify df.

    import pandas as pd
    
    raw = pd.DataFrame({
        "employee_id": [1001, 1002, 1003, 1004],
        "name": ["Camille Roux", "Sacha Morel", "Lou Perrin", "Noa Lemaire"],
        "email": ["[email protected]", "[email protected]", "[email protected]", "[email protected]"],
        "department": ["Ventes", "Tech", "Tech", "Ventes"],
        "salary": [34000, 48000, 52000, 36000],
    })
    
    import hmac
    import hashlib
    
    
    def pseudonymize(df, secret):
        ...
  2. CodeDifficulty 2Python in the browser2 hints

    Aggregates with a disclosure threshold

    Write safe_aggregate(df, by, value, min_n=5) that returns, per group, n (headcount), mean_value (mean of value) and suppressed (True if n < min_n); the mean of masked groups is replaced by NaN. Then calculate report = safe_aggregate(staff, "department", "salary") and hidden, the list of masked groups.

    import pandas as pd
    
    staff = pd.DataFrame({
        "department": ["Ventes"] * 6 + ["Tech"] * 7 + ["Juridique"] * 2,
        "salary": [32000, 34000, 36000, 38000, 40000, 42000,
                   45000, 47000, 50000, 52000, 55000, 58000, 60000,
                   48000, 70000],
    })
    
    
    def safe_aggregate(df, by, value, min_n=5):
        ...
    
    
    # report = ...
    # hidden = ...
  3. TaskDifficulty 22 hints

    Prepare a shareable extract in a spreadsheet

    Enter this fictitious file starting at A1:

    Matricule;Nom;Service;Age;Salaire
    1001;Roux;Ventes;26;30000
    1002;Morel;Ventes;34;32000
    1003;Perrin;Ventes;41;34000
    1004;Lemaire;Ventes;52;36000
    1005;Garnier;Ventes;29;38000
    1006;Blanc;Juridique;38;45000
    1007;Faure;Juridique;47;50000
    1008;Robin;Juridique;31;55000
    1. Delete the columns Matricule and Nom.
    2. Add a column Tranche with the formula =SI(B2<30;"<30";SI(B2<45;"30-44";"45+")) (SI (IF)), copy it down, then delete the column Age after converting it to values (paste special, values).
    3. Create a pivot table by Service with the row count and the average of Salaire.
    4. Apply the 5-person threshold rule: in the summary table to be shared, replace the average of departments with fewer than 5 people with “secret”.
    • The extract no longer contains a name, an employee ID or an exact age (columns Service, Salaire, Tranche)
    • The bands contain 2 people under 30, 4 aged 30 to 44 and 2 aged 45 and over
    • The summary table shows Ventes: 5 people, average 34,000
    • The average for the Juridique department (3 people) is replaced by "secret"

End-of-lesson quiz

  1. A file where the name has been replaced by a code, with a correspondence table kept by HR, is:

    • Anonymized
    • Pseudonymized: the data remains personal
    • Outside the scope of the GDPR
  2. Why use HMAC rather than a simple sha256 of the employee ID?

    • It is shorter
    • Without a secret key, an employee ID can be recovered by hashing all possible numbers
    • sha256 does not exist in Python
  3. Why mask the average salary of a two-person department?

    • Because the average is wrong
    • Because it allows an individual salary to be deduced
    • Because pandas cannot calculate it

The rest of the course

  1. Headcount and Turnover18 min
  2. Absenteeism: Rate, Frequency and Duration15 min
  3. Recruitment: Time to Fill, Application Funnel and Sources15 min
  4. Pay Equity: Raw Gap, Gap for Comparable Roles and Limits20 min

HR Analytics · Intermediate · 1 h 23 min