Exploring the unknown unknowns: A critical perspective on disasters and climate risks

Authors: Perrine Hamel, Robert Soden, Karen Barns, David Lallemant, Sneha Krishnan, Dane Carlson, Martin Demaria

Critical perspectives on technical work in the field of disaster risk management and climate adaptation are important: they can help discover “unknown unknowns”. This article presents some lessons learnt in a working group on Responsible Engineering, Science & Technology for Disaster Risk Management.

What’s in a flood?

Floods affect rural and urban communities all around the world – and will increasingly do so given our changing climate. How do we, as a society, attempt to reduce flood risk? Ask an engineer and s/he may work on three components of risk: reducing hazards (why flood water accumulates), reducing exposure (physical assets or populations located in flood-prone areas), and reducing vulnerability (how exposed assets and population might cope with a flood event). 

One conception of risk common in engineering and disaster risk management

This simple framework is taught in engineering schools around the world and has its merits, perhaps most importantly to provide a starting point to design a flood risk mitigation intervention. Technical experts can map hazard and exposure (where inundation is likely to occur), assess vulnerability based on socio-demographic criteria (e.g., populations without access to early warning systems) then design a more resilient system – one where the intersection of these three dimensions is minimized. 

This sounds good… or does it? As it becomes increasingly clear, applying this framework comes with a lot of implicit decisions. For example, decisions about what counts as hazard and exposure (is 10 cm of water considered a flood?), what qualifies as vulnerability (should people over 65 be considered as vulnerable?), and what do we measure as flood impact (is $ value lost the only meaningful measure?). Importantly, in using this framework, one defines what is considered as risk in a (inherently narrow) quantitative way. How about mapping social networks? Individual stories and contradictory perceptions of floods based on different viewpoints? How about using art to better understand what is at stake, beyond the physical process of inundation?

These questions are starting to emerge in disaster risk management (DRM) and resilience research and practice, where practitioners acknowledge that technical decisions are not neutral but political. Resettlement projects designed on the basis of technical (presumably “value-free”) risk analyses, for example, have received increased attention from the DRM community. Researchers have also challenged traditional risk perceptions by considering and visualizing alternative ways of knowing: for example, in the WhyDAR project researchers and local communities have used qualitative GIS mapping, in the form of geolocalized recordings and social networks, to enrich the definition of flood resilience.

Such participatory mapping approaches can help expand a traditionally narrow understanding of risk. Importantly, they provide legitimacy for new risk reducing interventions to be implemented. Yet, practitioners also need to understand how the approaches they use (e.g. how we design a participatory process) and the tools and data we use (e.g. framing risk as the intersection of hazard, exposure, and vulnerability) shape the answers we give – and ultimately the decisions we influence. 

Not only these decisions have short-term implications for social justice, they also have long term repercussions for development as they may perpetuate inequalities or misallocate long-term investments. Despite some of these socio-technical linkages being now acknowledged, how do we start to discover other linkages, the unknown unknowns of DRM practice?

A critical technical practice for DRM

Scholars in the field of science and technology studies have examined ethical issues related to various epistemological frameworks and in fact there’s a niche area of research, called critical technical practice (CTP), whose work could provide excellent insights for DRM practice.The underlying idea behind CTP is to keep “one foot in critique, one foot in practice”, drawing on critical literature to help one understand the possible implications of their modeling frameworks. (Here we use the term “modeling” in a broad sense, encompassing mental models – the way one perceives and analyses information – and scientific tools – or “formalisms” – aimed at synthesizing and translating knowledge.) 

Originally applied to the field of artificial intelligence, CTP proposes to identify key “metaphors” in a scientific field and uncover what is lost, or marginalized when using these metaphors. In our flood example, the metaphor may be that risk is at the intersection of hazard, exposure, and vulnerability. Using this metaphor, a critical technical practitioner could ask: what if risk was outside this intersection, in a (physical or non-physical) location where the flood hazard doesn’t overlap with exposed assets? This would force him/her to challenge the idea of an asset, or what it means to be exposed, perhaps leading him/her to consider alternative mapping and participatory methods to answer these questions. 

One characteristic of CTP is that it can take myriad forms, as it is defined more by its principles, the path of inquiry and critical examination of one’s field, than its methods. We propose that one way to start applying these principles is to find like-minded people and start scrutinizing a project through different lenses. A non-exhaustive list of methods for such critical work, drawn from the literature on critical thinking, includes: historical research, design methods (e.g. speculative design, design justice), ethnography, journaling, participatory action research, philosophical inquiry.  

Conceptual representation of Critical Technical Practice: There are distinct, sometimes overlapping, communities studying and practicing DRM: one comprising mostly scholars in disaster studies and related fields, the other one comprising engineers, scientists, technicians practicing DRM on the ground. Critical Technical Practices (CTP) have the potential to bridge these interdisciplinary gaps. [Credit: Rachel Ooi]. 

We have experimented with this idea over the summer of 2020. With the COVID-19 pandemic affecting our personal and professional lives, the challenges of starting an online working group were huge. Yet, over four months from May to September 2020, we met with about 20 “critical technical practitioners” in a virtual space to understand how methods that promote self-reflexivity and criticality could be implemented in practice. 

The Responsible Engineering, Science, Technology for Disaster Risk Management experiment

Participants were recruited through an online call. The only requirements for participation were to engage with other participants (in the virtual space Slack) throughout the project, and to have an ongoing DRM project that they could use as a case study. Most participants were freelance consultants and academics. We organized monthly webinars and facilitated the formation of online working group meetings, self-organized around topics of interest, which could be methods (e.g. historical research, participatory approaches, speculative design) or themes (e.g. feminism, nature-based solutions).

Despite the challenges of engagement for participants (including organizers), this experiment taught us three main lessons. First, the loosely structured, interdisciplinary, and voluntary nature of the initiative was conducive to unique conversations. The working group meetings allowed participants to discuss their work and challenges from the lens of specific themes (e.g. environmental history, feminism, landscape architecture), giving them fresh perspectives one rarely gets from collaborators. For some of us, the experiment helped realize that different technical backgrounds do not tend to sort out cultural diversity. Thus, socializing every week on Zoom was key to build the necessary bridges to reach mutual understanding, and a reminder of the value of carving time for deep thinking in the midst of busy professional lives. The working group sessions were also a time to find support in a difficult period when the COVID-19 pandemic disturbed about every aspect of our lives. 

Second, we found that developing a CTP requires commitment and an open mindset. When starting the project, we set on a path with unclear milestones. Almost by definition, the process of discovering the “metaphors” and their implications is largely unknown at the beginning, which requires commitment. A few working groups produced tangible outcomes, such as academic articles (e.g. an annotated bibliography, a draft of a polyvocal journal article), speculative design (a novel flood early warning system highlighting specific perspectives on disaster), and a grant proposal. For the majority of the participants, however, the main outcome was the ability to engage with stimulating conversations, outside their day to day work, with people from various backgrounds. Engaging with such conversation requires one to be open to unclear “rewards”, without a predetermined plan, a mindset that is not usual in engineering and science.

Third, the discussions helped broaden the horizons of disaster imagination. Thinking about challenges collectively, and to face those uncertainties as a group, was found to be a very supportive and safe way to “take that step towards new imaginaries”. Sharing our own disaster imaginaries was already enlightening. Yet, thinking and working around speculative design proved to be an approach as helpful as intriguing, and maybe a great technique to transform our ontological positions the way modelling does as well.

Finally, the concept of CTP itself often seems abstruse at first. Because it relies on different levels of cognitions (the abstract world of ‘critical ideas” and the “practice”), and because most of the existing documentation of CTP remains academic, we found articulating and vulgarizing the concept to be challenging, especially for an online group. Clear examples helped but extrapolating and replicating the approaches in different projects proved challenging. Relatedly, because CTP relies on a range of methods, it was difficult to know beforehand which ones were best suited to each participant. We invited all participants to keep a diary as one simple way to reflect on their progress and mindset. More facilitation and feedback on the diary could help CTP practitioners, and if we were to redo the experiments we would probably ask participants to be more intentional in their goals and methods. 

Based on this experiment, we feel encouraged to develop more avenues to facilitate the adoption of CTP practices in DRM. These could take the form of general education materials in plain language, but also seeds of exploration in traditional curricula (e.g. engineering or urban planning). As technical decisions are not (merely) technical, DRM researchers and practitioners should be better equipped to understand and unravel the implications of their work, and propose alternatives that meet social justice and long term development goals. CTP, originally applied to the field of AI, has a useful history and provides useful resources for practitioners interested in starting this reflective journey.

Call for Proposals – Critical Perspectives on Disaster Data


Data and information and communication technologies (ICTs) are central to how we understand disaster risks and impacts. Technologies and approaches ranging from crowdsourcing to drones to artificial intelligence are reshaping societal relationships to disaster, changing what we can know about them and the kinds of responses we can imagine. We know that these changes aren’t neutral. Decisions about what data to collect, how to collect it, how to analyze it, and how to share the results are expressions of politics and values as much as they are technical and practical. Yet we understand little about the consequences of these decisions for who and what receives protection from disasters, or assistance in their aftermath. An emerging area of interdisciplinary research is beginning to explore these questions in ways that develop a robust understanding of current practice and explore alternatives.

This call is designed to support early-career and non-academic researchers. Available funds will award up to 6 submissions at the level of $2,500 US each. Successful proposals will describe compelling research projects that offer fresh perspectives on the data and ICTs used to understand and respond to disasters and climate change. 

Examples of Relevant Topics

  • Promises and limits of participation – What do participatory practices, considered broadly to include crowdsourcing, citizen science, counter-mapping, participatory design, and action research, offer to expand the range of worldviews and imaginaries that are included in the design of disaster data.
  • Historical research – How can understanding the historical lineage of current data standards, technologies, and practices inform contemporary debates or expand our perspectives?
  • Uncertainty – Uncertainty is a central problem in disaster and climate data, but often narrowly conceived. How might we draw on recent work in the humanities and social sciences to better evaluate and cope with current challenges?
  • Decolonizing disaster studies – How might current approaches to disaster data relate to efforts to decolonize disaster studies?
  • Speculative & critical design – How can speculative and other critical design practices help to raise questions about current practices and imagine alternatives?

Eligibility & Award Requirements

We are specifically seeking to recognise late-stage PhD students from any discipline, early-career academics, and researchers in non-academic positions. Applicants from diverse backgrounds are encouraged. Awarded researchers will present the results of their work as part of an online symposium in mid 2021.

Submission Requirements

Short proposals should be submitted online by 12:00am on 12/09/2020 (Dec 09, 2020) AoT here: 


How the Funding Will be Issued

Selected submissions will receive direct payment from the fiscal sponsor of this grant, the Humanitarian OpenStreetMap Team (HOT). Awardees are responsible for any tax implications of these awards. Payment arrangements must be finalized by 12/21/2020.

Further Questions

Email: disasterdata@co-risk.org 

Rethinking flood risk management: The Chiang Mai Field Lab 2019 experiment

What happens if you put 100 people interested in flood risk management in a room for a full month? This is in essence the question we asked ourselves 18 months ago, when we started planning for the Field Lab. The answer is: a lot!  

The Understanding Risk: Chiang Mai Field Lab, an interdisciplinary “unconference” organized last June, aimed to explore critical design practices in disaster risk management by generating and testing new ideas, tools, artistic pieces, and communication products. Since flooding is an “all-of-society” challenge, so too we sought to bring together people of various disciplines to work together, including engineers, scientists, artists, policy-makers and more. The result was a 1-month event in Chiang Mai, Thailand, where participants created ad-hoc working groups — on topics as varied as risk communication, nature-based solutions, sensing technologies, etc. The event format was flexible to allow for new ideas to emerge and new groups to form. Every day’s schedule was created organically based on the momentum of each working group.

Participants joined from all over the world, staying anywhere from one week to one month. A lot of effort was made to make the event accessible regardless of financial capacity. We raised funds to make the event free to participate, and provided scholarships to nearly 20 participants to join for the entire month (thank you to our sponsors for making this possible!) Our only ask was to: (1) produce something, (2) document the work, and (3) contribute to the community. And indeed a lot was produced in the event, as documented on the website, in blog posts and interactive story maps: from policy briefs and academic papers to physical models of the City of Chiang Mai and original art pieces.

So what did we learn from the experiment? Now the dust has settled a bit, here are four key insights that may apply well beyond the Field Lab.

One month is both long and short. The month-long duration was an essential part of the experiment as we realized that the process of building collaborations and understanding different perspectives takes time. Participants stayed between one week and one month, which allowed friendships to form, an essential ingredient to developing short- and long-term collaborations. The duration of the event enable the flexible format to work with at least one week dedicated to each theme. We typically found that the first couple of days served to exchange perspectives and generate ideas while the remaining days could be used to design and create new products. From one week to another, some of the materials and ideas could also be re-used and refined.

On the downside, even accounting for the scholarships that defrayed the costs for many attendees, we found that only participants with flexible schedules could attend for the full month  (about 20% of the participants). They were mainly graduate students or professors, independent consultants, or individuals in between jobs, which made their contribution disproportionate compared to other participants. In addition, one month felt short to develop some types of products, especially given the constant flow of new people and new ideas. For example, only early drafts of academic papers were written (rather than full papers), possibly because the collaborative activities may have been considered as a better use of everyone’s time. 

Flexibility is a skill, and it can be learnt. The entire event utilised an “open-space” model where participants themselves create the schedule, objectives and work-plan. They were completely empowered to organise their collective work, while we provided tools, advice and resources to facilitate this (as described in more detail here). The flexible schedule, though challenging to some participants, encouraged collective ownership of the event, emergent project ideas, and enabled attendees to contribute based on their skills and experience in ways that a pre-planned agenda would not have. Flexibility does not mean absence of preparation, and if anything it may be more work for facilitators who need to anticipate a range of eventualities rather than a specific development. Yet we found that this skill could be learnt and we heard from participants that accepting the flexible structure became easier with time.

The open structure of the event puts a lot of responsibility on individual participants to independently manage their own schedules, lead projects, and collaborate with other attendees. The ability to do this successfully is influenced by experience and cultural background, and facilitators can play a role in helping ensure all participants are able to have positive experiences. We also found that the flexible structure could become a challenge for working groups. For example, one participant reported in the exit survey that “the fluid timeframe allowed a lot more people to attend, which was great, but also made it challenging to keep momentum for some work as teams were broken and rebuilt many times.” 

Interdisciplinarity can be designed, but it’s hard to fight instincts. One goal of the event was to support interdisciplinary exchanges, recognizing the silos in flood risk management (e.g. the fields of communication, technical assessment, social assessment…) The products of the Field Lab illustrate that this goal was reached, at least to some extent: for example, the personas [link] developed within the nature-based solutions group were derived from the somewhat technical discussions on the role of nature-based solutions. 

On the other hand, we observed a clear self-selection between working groups, with “technical profiles” being drawn to working groups such as sensors or machine-learning. This instinct is hard to fight and the longer timeframe, with ample opportunities to socialize and learn from other attendees, was one attempt to create multiple levels of connection and encourage interdisciplinary collaboration. Another approach would have been to assign participants to teams and oriented projects around challenges that required interdisciplinary collaboration, perhaps losing some of the creativity (flexibility?) in the process.

Focusing the event on the city of Chiang Mai was critical to the success. In a spectrum ranging from local events (focusing on a city or region) to international events (where the host city merely serves as a meeting point), the Field Lab seems to stand in the middle: while some working groups could have worked in almost any other location, Chiang Mai and the region provided inspiration and a “sandbox” to explore new ideas. The partnerships with strong local institutions (e.g. Chiang Mai University School of Public Policy, Oasys lab, FOPDEV) certainly participated in the development of a “sense of place”, with the participants and projects engaging deeply with local people and the environment. 

For the purposes of the Field Lab, this relationship to place seemed to work well. In fact, some of the most exciting projects happened as a result of collaboration with local organizations, as exemplified by the City Dialogue organized by the School of Public Policy. Of course, building the relationships that facilitated these projects required significant upfront investment. Funding for these projects made available by the Field Lab also helped establish joint ownership of the outcomes between the Field Lab organizers and local partners.

So was it a success? By many metrics, yes. And we are grateful for our funders who took a risk in supporting such an unusual project. Our main goal was to test a unique event format and analyze how the process, outputs, and outcomes might advance the practice of flood risk management. The event produced over 100 documented outputs (equivalent to 3 per day!) ranging from draft academic papers, art pieces, new modeling approaches, etc. The lessons above were drawn from the experiment and form part of a larger body of work being documented on the website and in a report and academic paper, which we will continue to explore in our future research. Long-term collaborations were formed, a testimony of the strong links that were created between participants. And importantly, the experience constituted a great opportunity for professional development for all participants, irrespective of their level of experience, since they were all confronted to novel format and dynamics.

As instigators of the Field Lab, the event allowed us to gain insights into the type of “action research” that can positively influence the field of disaster risk management. We presented some of these insights above: balancing slow and fast timelines for deeper engagement, training a flexible mindset, thinking critically about interdisciplinarity, and developing a sense of place for all stakeholders. Bringing 100 people in Chiang Mai was one way to explore these insights, and we look forward to future Field Lab-type events to continue this learning journey.

Rapid Assessment of Disaster-Induced Vulnerability in Nepal – Project Planning

Last month I spent a week in Nepal setting up an exciting project with collaborators at Kathmandu Living Labs. The project is funded through a “Collaborative Data Innovations for Sustainable Development” award by the World Bank and Global Partnership for Sustainable Development Data. Our proposal focused on better understanding the ways in which disaster impact is currently measured, and how it could be improved to foster more effective and equitable disaster recovery. There are two main aspects to this: (1) improving the statistical accuracy of post-disaster impact assessment using more and better data (e.g. remote-sensing, crowd-sourcing, rapid field surveys) and better statistical modeling; and (2) defining better metrics of disaster-induced vulnerability and need.

We focus on metrics because the specific ways in which we measure the impact of disasters shapes the way that response and recovery are orchestrated. In numerous large-scale disasters including the 2015 earthquake in Nepal, we witnessed that early damage assessments often fall short in properly describing the scale and geographic distribution of damage. More importantly, we saw that the way that disaster impact is measured is in itself problematic.

By describing impact purely as a measure of damage and loss to physical assets (e.g. number of houses and bridges destroyed, etc), this focuses attention (and funding) on communities that have lost the most, but not necessarily those that need the most. Seen another way, measuring impact in terms of counting assets that have been damaged focuses attention away from those who had lost little simply because they had few assets to lose (typically the poorest and most vulnerable). In this way we saw that disasters not only reflect inequality (since the poor tend to live in more precarious conditions), but disaster recovery can often deepen inequality further.

This does not arise from malicious intent from anyone involved, but simply as an outcome of the way we measure and build our databases of disaster impact. So to correct this, we’d like to change the way we measure disaster impacts: shifting from disaster-induced damage to disaster-induced vulnerability and need.

This latest trip was my first time back to Nepal since immediately following the April 2015 earthquake. Following the earthquake I was working with the World Bank to support various Nepali government agencies on developing the early plans for recovery, and organising the housing sector analysis for the Post-Disaster Needs Assessment (PDNA). In part, this new projects aims at addressing some of the limitations of the work I had done then. I won’t attempt to comment on the reconstruction progress three years later. I spent the entire week in Kathmandu while most of the earthquake damage was in the rural areas, and even so one week is much too short to say anything beyond generalizations. So I’ll save that for some future post.

For now I just wanted to share my excitement for this project and working on it with the team at Kathmandu Living Labs. Spending time with the team was quite inspiring. They are a incredibly dynamic group, passionate about improving the society they live in by combining data, people and technology. They are building Open Government platforms to increase effectiveness and transparency of local government; developing citizen-engagement apps to promote citizen-government collaboration and deliberation; mapping hazards and risks of municipalities to promote their resilience; training government surveyors on housing reconstruction monitoring; etc. What better people to partner with on this project?

The project includes other eminent collaborators. Led by the Earth Observatory of Singapore (my academic home), the project also brings together engineers from the Stanford Urban Resilience Initiative, disaster informatics experts from GFDRR (Robert Soden), data scientists from the World Bank Big Data Program, humanitarian data specialists from the Humanitarian OpenStreetMap Team, and remote sensing experts from NASA-JPL.

Stay tuned for more as the project evolves.

Co-Risk Labs at UR2018

The 2018 Understanding Risk Forum kicked off this week in Mexico City. The bi-annual event brings together scientists, engineers, remote sensing experts, software developers, social scientists, and representatives of government, civil society, and development agencies to discuss emerging approaches for making sense of disaster and climate risk. The Forum is convened by the World Bank’s Global Facility for Disaster Reduction and Recovery and over the years has been an important source of inspiration and collaboration for our team. Robert and Karen are there this week, come say hi if you see us around!

On Tuesday morning, we led a Simulation of the Post-Disaster Needs Assessment (PDNA) as part of a training by GFDRR’s Resilient Recovery team on post disaster recovery planning. 20 participants worked together in groups attempting to complete an assessment of damages, losses, and recovery needs in the housing sector following an earthquake on a fictional island in the Atlantic. The simulation forced participants to deal with the very real world problems of limited time and information in post-disaster settings, and helped them to understand some of the politics of disaster information. As one of the participants stated, “unless you put a dollar value on these things, there will be no means to support recovery.” The guidelines that inform post disaster assessments, and the ways in which they are put into practice, thus play a central role in shaping disaster recovery processes.  

The simulation was the first large-scale pilot of an activity developed by the Co-Risk Labs team, with the support of GFDRR, and we hope to find opportunities to develop it further with other groups in the future. Some of the game materials are attached to the bottom of this post.

PDNA Simulation at UR2018

We’re looking forward to participating in the rest of the event. Some of the other highlights are shaping up to be:

Danger! Weird ways engineers think and talk about disasters in cities



02 disastersslide2

03a slidefinal

03b disastersslide3bCROPPED

04 disastersslide4CLEAN

04b disastersslide4b

05a disastersincities2b

05 disastersslide5CROPPED

06 new slide

07 newslide

08 newslide

09 newslide

10 disastersslide10

10b newslide

11 disastersslide11

12 disastersslide12

13 newslide

14 disastersslide14

14a disastersslide14a

14b newslide

14c disastersslide14v2a

15 newslide

16 disastersslide16



18 newslideIllustrations by the brilliant Veronica Wood.

Thanks to all the friends and colleagues who gave encouragement and feedback.