Why measuring digital safety can protect us online — and how we do it

Too often, debates about digital safety are long on slogans and short on specifics. Much like with the financial performance of companies, where we look to detailed reporting to assess performance and measure progress, we need a data-driven approach to online safety.

By tracking and reporting on digital safety impacts, risks and processes, companies can concretely demonstrate progress to their users, regulators, investors and the wider public.

In an era where digital safety is a necessity for navigating our increasingly online lives, reaching an agreement on what sort of metrics most meaningfully measure safety is imperative. How companies, governments and civil society think about this topic is top of mind due to a combination of fast-moving regulatory developments.

These include the EU's Digital Services Act and the UK Online Safety Act, among others, which highlight the escalating emphasis on accountability and responsibility within the digital sphere. This trend is not solely driven by governmental interventions; private entities, exemplified by Google's pioneering 2010 transparency report, are also taking proactive measures. Google's initiative, which inaugurated the practice of transparency reporting, offers valuable insights into the concerted efforts undertaken by tech giants to combat harmful content and conduct on digital platforms.

But what exactly constitutes digital safety? At its core, digital safety is about preventing and reducing harm in the digital space. This includes moderating illegal or harmful content, driving responsible platform design and governance and empowering users to tailor their online experiences. Sound metrics lay the foundation for a safer digital ecosystem. These metrics not only foster accountability but also facilitate evidence-based decision-making, guide resource allocation, enable benchmarking and progress monitoring, promote transparency and engagement and allow for the evaluation of intervention effectiveness.

Nevertheless, a fundamental hurdle lies in effectively measuring digital safety. The measurement of digital safety is constantly evolving, shaped by technological advances and regulatory interventions. This evolution also brings forth challenges: malicious actors persistently refine their tactics to circumvent detection, underscoring the necessity for continuous innovation in digital safety measures. Another challenge arises in striking a delicate balance between access to data and privacy concerns, as well as acknowledging the diversity among services.

The challenge of crafting effective metrics to define and quantify phenomena that elude concrete and causal measurement is what we refer to as “measuring the immeasurable.”

Recognizing the complexity of measuring digital safety, we have written a paper titled “Making a Difference: How to Measure Digital Safety Effectively to Reduce Risks Online” where we propose grouping metrics into three categories:

– Impact: Metrics that illuminate the impacts on individuals and provide insights into characteristics and patterns of lived experiences.

– Risk: Metrics that enable the detection and mitigation of potential harms.

– Process: Metrics that cover the approach, implementation and outcomes of systems relating to digital safety.

This categorization is designed to clarify their application and facilitate stakeholders’ tracking and reporting of these crucial aspects. It proves relevant for evaluating the efficacy of interventions and monitoring outcomes. Metrics and measurements for digital safety must align and adapt with the goals and challenges of the digital landscape.

The report is not designed to be exhaustive; its primary aim is to serve as a guiding framework, offering structured thinking rather than presenting a rigid, exhaustive list of metrics. The framework also demonstrates how intertwining insights from impact, risk and process metrics can improve digital safety measures. Digital service providers should prioritize metrics that can yield the most significant outcomes, rather than perceiving the recommendations in this report as an exhaustive roster of options for adoption.

The paper represents a part of the ongoing efforts undertaken by the World Economic Forum’s Global Coalition for Digital Safety. Comprising a diverse array of stakeholders, including experts from various sectors, the Coalition is dedicated to addressing critical issues surrounding digital safety. Past works by the Coalition include the Global Principles on Digital Safety, Digital Safety Risk Assessment in Action, and a Toolkit for Digital Safety Design Interventions and Innovations: Typology of Online Harms.

The pragmatic application of digital safety metrics is essential for assessing the real-time efficacy of interventions and safety protocols. Such metrics bolster accountability, equipping NGOs and regulatory bodies to oversee service providers effectively. Moreover, they serve as benchmarks for compliance monitoring, thereby augmenting user trust in platforms, provided privacy considerations are balanced and service differentiations are duly acknowledged. Nevertheless, metrics represent merely one facet of the comprehensive strategy required to shield users from digital threats. By fostering collaboration and leveraging diverse perspectives, we can collectively forge a safer and more resilient digital landscape for all.

With contributions from Agustina Callegari, Project Lead, Global Coalition for Digital Safety.