ParakhAI

CivicDataLab has been selected by the Ministry of Electronics and Information Technology and Digital India Programme to co-create ParakhAI — an open-source framework for Participatory Algorithmic Auditing of AI solutions, especially high-risk algorithmic decision-making systems (ADS). Enabling data driven decision-making processes is key to creating sustainable interventions for inclusive development.

There is an increasing need for citizens to have more agency over how their data is used in Artificial Intelligence (AI) models, especially the ones deployed to impact lives and livelihoods. We are witnessing citizen-centric AI models now being developed through both public policy interventions and by private entities to deliver essential citizen services like public health, education, banking, e-commerce and others.

These models are also known as high-risk ADS (Algorithmic Decision-making Systems) that directly or indirectly impact benefits, punishments, or opportunities individuals or groups can receive. The risk here being that individuals or groups are harmed by the output of the decision-making system, resulting in incorrect or unfair outcomes.

Most high-risk ADS may undermine the fundamental rights of citizens related to equality, privacy, freedom of speech and expression, and may even lack protection from any potential discrimination on grounds of religion, race, caste, gender or place of birth.

ADS may lack participatory auditing frameworks that can help citizens report issues related to discrimination, exclusion, unfair treatment and more potentially due to different types of biases arising from the training data, technical constraints as well as societal or individual biases.

Our proposed framework will enforce the adoption of a continuous lifecycle with Public Consultation, Development, Deployment and Assessment as key steps built into the lifecycle. This ensures adequate avenues for impact groups to timely and effectively participate, collaborate and co-create the ADS system.

The citizen engagement models for high-risk ADS auditing need to be tailored keeping in mind the diverse nature of Indian citizens and groups, especially looking at the issues related to volume of usage, language, regional diversity, digital literacy, access and usage of internet & smartphones and more.

Citizen participation in ADS needs to be formulated from both formal, which is instituted into governance and legal regulation as well as the informal modelling that is without legally-binding engagement requirements.