Auditing online software for bias: A controlled experiment in the domain of car insurance

Auditing online software systems for biAs automation increasingly becomes a part of society, processes that reverse engineer and uncover key aspects of algorithms and automated decision systems become essential. Algorithmic auditing plays a crucial role in ensuring fairness, accountability, and transparency. It allows communities to monitor technology and decision-making systems, ensuring they align with specific values and requirements. The Italian car insurance industry plays a critical role in the country’s economic, social, and political landscape. With Italy ranking second only to Luxembourg in car ownership among EU states, auto insurance is vital in making vehicle ownership and usage less financially hazardous for individuals. The industry is primarily governed by the Motor Vehicle Liability system (Responsabilità Civile Autoveicoli - RCA), which mandates the purchase of RCA coverage for all motor vehicles used or kept on public roads. Comparison websites, also known as aggregators, have gained prominence as primary channels for RCA subscriptions, accounting for a significant portion of the total gross written premiums in the Italian vehicle insurance market. These websites have consequently faced increased scrutiny and regulation, especially concerning how they present insurance options and pricing to consumers. In this thesis work, we carried out an audit of an Italian car insurance comparison website by collecting quotes. We show that some protected attributes such as birthplace have a considerable impact on the prices received by each profile. This feature is used to the disadvantage of non-Italian profiles but also to Italians born in some specific cities. We also showed that riskier profiles (for instance profiles born in a foreign country) were given fewer quotes in the result page of the comparison website. This work confirms the presence of strategic behavior by insurers, affecting some users and putting at advantage other types of profiles.