From October 2022 to March 2023, we ran a forecasting tournament with a total of 54 questions. Almost all of our forecasting questions were developed in cooperation with 16 different public institutions and ministerial departments. Each institution or department defined its most useful forecasting topics, participated in a workshop to define specific questions with us, and was later provided with the results. This was intended as a proof of concept of one possible approach to incorporating forecasting in public decision-making.
Once defined, our forecasting questions were then posted on a private Metaculus sub-domain (in Czech), where an average of 72 forecasters had the opportunity to address them as they would any other question on Metaculus (median of 18 predictions per user). Throughout the tournament, we produced 16 reports detailing the rationales and forecasts, to be used by the cooperating institutions.
A handful of our partners have already reported acting on the information/judgment presented in our reports. This has concerned, for example, the national foreclosure issue (some 6% of the total population have debts in arrears) where the debt relief process is being redesigned midst strong lobbying and insufficient personal capacities; or the probabilities of outlier scenarios for European macroeconomic development, which was requested by the Slovak Ministry of Finance to help calibrate their existing judgements.
It also seems useful to explore various approaches to grow the number of policymakers with personal experience and skills in forecasting. In our case, we found curiosity and willingness to try forecasting even in unexpected institutional locations (i.e. the Czech R&I funding body). This makes us more confident that the “external forecasts” approach (as compared to building internal prediction tournaments or focusing on advancing forecasting skills of public servants) is worth investigating further precisely because it allows us to detect and draw on this interest irrespective of institutional and seniority distinctions and resource constraints.
While we hope that any readers with an interest in forecasting may find our experience useful, we expect that both this and any future projects of ours make it easier for other teams to work towards similar goals. To that effect, the write-up also contains an Annex of “Methodological Guidelines,” where we outline in more explicit terms the questions and decisions that we found were important to tackle when running the project, and what they may entail.
de introdus homepage