Removing duplicates

drop_duplicates() removes duplicate rows from a DataFrame. subset limits the comparison to certain columns and keep chooses the row kept (first, last). You often sort beforehand to keep the most recent record, and you check the number of rows removed.

Example

df.sort_values('maj').drop_duplicates('client_id', keep='last')

Related concepts

The whole glossary