RightsCon 2018
Session: Overcoming the black box problem: Scrutinizing automated decisions in the real world
Role: Speaker.
Description
Automated decisions increasingly mediate civic life. Governments use algorithms to screen immigrants and allocate social services. Corporations rely on software to help in making decisions in vital areas like hiring, credit scoring, and political discourse. There is a growing desire to “open the black box” of complex algorithms and hold the institutions using them accountable. But across the globe, civil society faces a range of challenges as they pursue these goals. Testing the societal impact of algorithms from the outside is one of the biggest challenges for advocacy organizations: A critical amount of data may be needed and privacy provisions must be accounted for. Analysis can be time-consuming and expensive. Additionally, the potential of reidentification and different data formats can make it difficult for researchers to share results and effectively collaborate. Nevertheless, many researchers, journalists, and public interest groups have been able to make real progress in understanding these systems.
In this session, we’ll explore what it means to scrutinize automated systems in the real world, using both technical and non-technical methods. We’ll look at case studies where public actors were able to discover meaningful insights about an automated decision — without having access to the underlying code. And we’ll get a flavor for the many different ways to “hold algorithms accountable.” The moderator will first share an overview of findings from research about different methods of scrutiny — both technical and nontechnical — and invited session participants will then describe the work they each did to investigate an automated system. One case study presented will be AlgorithmWatch’s “openSCHUFA” project, investigating Germany’s leading credit scoring company. A low SCHUFA score means banks will reject your credit card application, car rentals will reject you as a customer and network providers will say ‘no’ to a new Internet contract. No one knows how accurate SCHUFA’s data is and how it computes its scores, and OpenSCHUFA wants to change this by analyzing thousands of credit records. The project has successfully crowdfunded more than 43,000 Euros and almost 18,000 people have requested their score.
Session leaders will then encourage attendees to share their own work investigating algorithms to promote an understanding that there are many ways to gain more insight into automated decision systems without having access to or being able to understand code. Participants will then have the opportunity to brainstorm other systems worth scrutinizing and how to do so.
Session: Fingerprint on the pulse: Challenging the lack of privacy protections for biometric data
Role: Speaker.
Description
Entities all over the world are adopting the use of biometric technology for an array of purposes like identification, electoral registries and policing, amongst others. A significant number of those entities are in states that do not have sufficient data protection laws. This means that this data is collected, is shared and further processed by various public and private entities, without subjecting them to any regulations which normally require them to protect the data throughout its lifecycle and respecting the rights of individuals, amongst other obligations and duties. This enables these actors to evade any sort of accountability and oversight thus denying individuals any access to complaint mechanisms and redress. This context raises grave concerns of abuse and misuse of biometric data which could have adverse negative effects on individuals’ enjoyment of their fundamental rights.
Biometric data is categorized as sensitive information and it is currently being processed in a complete legal and regulatory void. Not only that, it is extremely difficult to find out exactly what is being collected and how it is being stored and shared, particularly by private companies. There is an urgent to adopt and enforce need for practical legal and regulatory frameworks in these regulate and oversee effectively the processing of all data by both public and private actors in their capacities as data controllers and/or processors.
In countries like India, Kenya and even Somaliland, state agencies are collecting biometric data massively for voter registration and other purposes yet the countries lack proper laws on data protection and privacy rights. In this session, we will bring together international civil society panelists each of whom are in varying stages of researching on the use of biometrics in their jurisdictions, the risks data subjects are exposed to due to lack of legal framework and proposed solutions in getting data protection laws finally being legislated or enforced. The panelists will to identify and share the motivations of the adoption of biometric technologies in their jurisdictions such as fool proof electoral systems and national identification programs, the legal, cultural and ethical issues that arise; and what they think will be the best way forward lobbying and pushing for legislation of a data protection law. The panelists will also exchange ideas, share lessons learned and, in collaboration with active participants in the audience, work toward addressing the issue of use of biometric data and data protection mechanisms.