How is open data changing India?

Recently I attended an India Open Data Community meeting organised by the World Bank in New Delhi that brought together government officials, academics, corporates, developers and a few development sector professionals to discuss social and economic Open Data opportunities in India and the emerging partnerships forming around them.

Organized at the highly regarded Indian Institute of Technology, the meeting was focused on three key areas; experiences of institutions using open data around the world, how organisations need to prepare to tap into the growing potential of Open Data, and how to build and strengthen the community of data users and providers. The aim was to help assess the challenges and opportunities for extracting and using open government data in India, and to then communicate these at a subsequent National Conference on Open Data and Open API.

India – Open Data opportunity

One of the key speakers at the meeting was Professor Jeanne Holm, a senior Open Data consultant at the World Bank and former evangelist for Data.gov in the US. In a brief presentation, she summarized the key reasons for governments’ willingness to open their data. These include improved internal efficiency and effectiveness, transparency, innovation, economic growth and better communication with citizens and other stakeholders.

She highlighted some key observations about the opportunities for Open Data in India: the availability of a vast resource of data; a stable, open source platform for open government data; rich technological expertise and knowledge; and opportunities to design specific data sciences programmes in educational institutions. A rapidly growing community of open data enthusiasts in India, DataMeet, is also shaping the discourse on data and its civic uses and exploring engagement opportunities with a wide spectrum of Open Data users.

Barriers to Open Data in India

However, there are certain challenges that are currently preventing these opportunities from being tapped. The infrastructure to support efficient data collection, processing and management needs to be strengthened. Issues of privacy and data anonymisation are a concern.  There is also a lack of standardization of data collection formats that make it difficult to aggregate and make sense of data.

Recently I stumbled across a detailed, informative report by the Centre for Internet and Society that backs up this view, observing “Open Government Data (OGD) in India must be looked at differently from what it has so far been understood as in countries like the UK and the US.”

Global trends

Oleg Petrov, Senior Program Manager at the World Bank, talked about two significant global trends in the Open Data movement. The first was the data revolution, which is set to put data at the heart of the way the post-2015 Sustainable Development Goals are evaluated. Secondly, he discussed Smart Digital Government, which “focuses on doing more with less” and is inherently characterised as being digital and open in design, data-driven and easy to integrate with other systems.

Data is the new oil

Open Data is increasingly seen as the raw material for entrepreneurial activity, innovation and economic growth. A McKinsey report on Open Data, published in 2013, estimates its annual potential value to be more than US$3 trillion. The legendary investor Ann Winblad has described data as the “new oil.”

Laura Manley, project manager at Open Data 500, shared insights on the private sector trend in the US of innovative businesses and products delivering commercial value from open government data. Some small and medium enterprises use Open Data to improve their decision-making and efficiency. Others are capitalizing on new opportunities to create saleable products and services. One example is Enigma, which cleans, visualises and aggregates large amounts of public data produced by governments and other organisations for wider usage.

How are open government data initiatives taking shape in India?

India was one of the first countries to join the Open Data movement after the US and UK. Since then, more than 20 countries have opted for open government data (OGD) and released their datasets online.

The idea of OGD in India evolved over a period of time before it took the shape of a formal, robust National Data Sharing and Accessibility Policy (NDSAP) in March 2012. Some of the key milestones that created a conducive environment for the open government data policy in India were the widespread usage of computers by the central government from 1975 and the creation of the National Informatics Centre (NIC) for launching e-governance initiatives to move from manual, paper-based systems to automated processes.

This was followed by landmark legislation, The Right to Information Act in 2005 that transformed the citizen-government relationship with the agenda to promote transparency and accountability within government. Due to the changing global financial climate, many prominent Indian technology firms like Infosys started collaborating with the government to develop e-governance systems geared towards Open Data.

In 2012, the Indian government approved NDSAP, declaring a proactive disclosure of “all sharable, non-sensitive datasets in open formats by various ministries, departments, subordinate offices, organizations and national government agencies as well as individual states.”

The NIC, in collaboration with the US government, has created the Open Government Data Platform India as an open source portal, data.gov.in, for Indian government departments and ministries to publish their datasets for easy and open access by citizens.

Digital India, a recent initiative of the newly formed government in India, plans to prepare the country for a knowledge future by increasing efficiency and improving interactions between government departments and citizens. It aims to ensure government services are made available electronically to reduce paperwork. It is structured on nine pillars of which Pillar Six (Information for All) is of prominence with respect to Open Data in India. It entails the online hosting of information and documents and the proactive engagement and interaction of government and citizens through social media, online messaging, etc.

Currently 85 government ministries, departments and agencies have contributed more than 12,000 datasets across segments such as population census, water and sanitation, health and family welfare, transportation and agriculture to data.gov.in. One of the unique features of the Open Data portal in India is that citizens or users can demand a specific dataset from the government and others looking for similar data can endorse these requests. It then becomes mandatory for a department to release that data if 100 such endorsements are raised for a particular dataset.

Limitations and reliability concerns

However, a large volume of existing government data is still not accessible in digital formats. NDASP is still a policy, but not a mandatory policy. As a result, a lot of departments are reluctant or slow to share their respective datasets. Issues of capacity, attitude, lack of demand, confidence to release the data, and the existing state of data (in paper-based formats) explain this reluctance.
​Even when data is made available in a machine-readable format, in most cases the reliability is questionable; the raw data needs to go through rigorous editing and aggregation before it can be used. Different departments collect and collate information in their respective silos using diverse formats and terminology, making it tough to use that data effectively. Adoption of data sharing at the state level has also been slow with only four out of 29 Indian states contributing data to the national portal.

The granularity of open government data in India has been a concern, as it fails to satisfy the users to access and use only micro-level data. There is also serious demand for geospatial data for visualizing and communicating issues as they exist on the ground. The unavailability of official map data due to the conservative map policy of the government and lack of interoperability in sharing this data has discouraged a discourse about better planning, tracking progress in the ‘real space’ and pushing government to take remedial steps.

Where are the NGOs?

The discussions around Open Data also highlight the absence of non-profit organizations among the technology-focused groups, entrepreneurs and businesses. There is a need for such organizations to be more visible in Open Data circles and proactively get up to speed with the technological trends set to define the next steps forward. Observing this need, Akvo facilitates the use of open source technology tools in helping development aid organisations move towards a more collaborative and effective way of working. We help organizations working in diverse fields of activity to build their capacity to collect, manage and disseminate their data and enhance their overall data literacy, usage and practices. As Thomas Bjelkeman, one of our founders, has said, ‘Going open has huge benefits.’

This post first appeared on The World Bank’s Information and Communications for Development Blog. Publication does not imply endorsement of views by the World Economic Forum.

To keep up with Agenda subscribe to our weekly newsletter.

Author: Isha Parihar is the Programme Manager of Akvo Asia Hub.