The Bank of Canada COVID‑19 stringency index: measuring policy response across provinces

Introduction

Policy measures that governments have put in place to protect people against the community transmission of COVID‑19 have been crucial to public health. But they have also had a large impact on the economy. In Canada, responsibility for these measures falls primarily on the provinces. Provincial governments have taken different approaches to containing the spread of COVID‑19. These have varied over time as governments have responded to changing rates of infection and hospitalization and new information about the effectiveness of different measures. Tracking these measures is important for understanding the impact of the pandemic on Canada’s economy.

This note focuses on the Bank of Canada’s stringency index—a measure of containment policies and public information campaigns. Since summer 2020, staff at the Bank have been collecting publicly available information on government policies and creating daily government response indexes for the 10 provinces. This allows the Bank to systematically measure, track and compare government policy responses. These indexes follow the methodology of the Oxford COVID‑19 Government Response Tracker (OxCGRT) developed by the University of Oxford’s Blavatnik School of Government.1 Bank staff adjusted the measures to make them more appropriate for the Canadian context and to capture more granular differences in policy responses.

The provincial indexes show that most government containment policies in response to COVID‑19 were strictest during the first months of the pandemic in spring 2020, despite low case counts relative to late 2020. Throughout the pandemic, Ontario and Quebec have had the strictest containment measures and the highest case counts.

The stringency index does not measure the efficacy of a province’s COVID‑19 response or provide a direct measure of the impact of government policies on the economy, including the impact on output or consumption. While there appears to be an association between economic activity and both the stringency index and its components, more research is needed to precisely identify their impacts.

The composition of the Bank of Canada’s stringency index

What the index measures

Our stringency index measures the severity of containment policies and public information campaigns. For each province, the index is the simple average of 12 sub-indexes. Sub-indexes are normalized values of corresponding policy indicators. Table 1 lists the 12 policy indicators currently included in our stringency index.

Because most containment measures are issued by provincial governments, the stringency index captures mostly differences across provincial policies over time. The main exception is the international travel restrictions indicator (C8), which captures mainly federal policies imposed across the country.

Table 1: Policies included in the Bank of Canada’s stringency index

How a sub-index is constructed

Following the methodology of OxCGRT, we construct a sub-index for each province as follows:

1. For each policy indicator listed in Table 1, we assign a coding value along an ordinal scale for the most restrictive measure in place within the province based on the coding guidance (see Table A‑1 in the Appendix for details). Higher values indicate stricter measures.
2. Each policy indicator has additional binary flags to identify whether the policy is general (1) or targeted (0) within a province along various dimensions (e.g., geography). We treat targeted measures as fractional decreases between the ordinal integers (i.e., less stringent than general, non-targeted policies). The more targeting dimensions an indicator has, the more steps exist between each coding value.2
3. Finally, we calculate a daily sub-index that normalizes the coding values of each indicator between 0 and 100 (higher values mean stricter measures).

Equation 1 below describes the calculation for each sub-index, I_j, for policy indicator, j:

\(\displaystyle{Sub-Index}_j\) \(\displaystyle=\,100\ast\left(\frac{a_j-\sum_{i=1}^{n}\left(\frac{1-f_{ji}}{n+1}\right)}{b_j}\right),\ \ \ \ \ \ \ \ \ \ (1)\)

where:

• \(a_j\) is the ordinal value for the most restrictive measures in place within the province (see Table A‑1 for codes)
• \(b_j\) is the maximum coding value
• \(n_j\) is the total number of targeting dimensions (e.g., by geography)
• \(f_{ij}\) is the flag for targeting dimension, i, which is equal to 1 if the policy is general or 0 if targeted

See Table A‑2 in the Appendix for an example of the calculation.

Adjustments to the OxCGRT methodology

We adapted the methodology of OxCGRT to make the index more appropriate for the Canadian context and to capture more granular differences in provincial policy responses.3 (All adjustments are marked by asterisks in Table A‑1.)

1. We added three policy indicators (beyond the nine included in OxCGRT).4
   • C9—Restrictions on interprovincial travel: We revised OxCGRT's policy indicator on internal movement restrictions (C7) to capture only intra-provincial travel restrictions and added a separate indicator of interprovincial travel. Adding this indicator mechanically gives greater weight to travel restrictions in the stringency index.
   • C10—Enforcement mechanisms for individuals: OxCGRT does not include a measure identifying differences in provinces that have similar containment policies but varying punishments or enforcement of those policies. Thus, we created a new indicator to capture penalties for individuals violating public health orders.
   • C11—Enforcement mechanisms for firms: For the same reason as above, we created a new indicator to capture penalties for businesses violating public health orders.
2. We added more targeting dimensions. These increase the granularity in the variation across provinces and over time. Beyond the geographic dimension included in OxCGRT, we include targeting dimensions for, among other things, types of public events, outdoor versus indoor events and exceptions for certain travellers.5 (See Table A‑1 for the complete list.)
3. We refined the categories of coding values for several policy indicators. This was based on a more detailed examination of the policies in Canada. Where we added categories to our indicators, the denominator of the sub-index calculation (b from equation 1) increases. This often means a lower sub-index compared with OxCGRT (an exception is when both code the maximum value). Below, we present some of the main changes to categories (see Table A‑1 for the complete list):
   • Workplace closures (C2): Governments have issued lists of workplaces designated as “essential services” that can remain open, and these vary by province. Whereas OxCGRT defines the strictest category as the requirement to close “all-but-essential workplaces,” we divide this category into two:
     • broad—when the government defines the essential service list by sector (e.g., retail and construction), which was more frequent during the spring 2020 lockdowns
     • narrow—when the government defines the essential service list by subsector (e.g., pharmacies and commercial construction), which was more common after the spring 2020 lockdowns
   • Public event cancellations (C3): Whereas OxCGRT assigns the maximum ordinal value to any cancellation of events no matter the size, we added two categories to differentiate among event restrictions based on the maximum number of people allowed at one event:
     • events limited to a maximum of 50 to 100 attendees
     • events limited to a maximum of more than 100 attendees
   • Restrictions on private gatherings (C4): We adjusted the categories to allow for finer distinctions among restrictions for gatherings of between 10 and 100 people, based on the realities of different provinces’ policies.
   • International travel (C8): Unlike OxCGRT, we combine the categories of “banned arrivals from some regions” and “banned arrivals from all regions” into one category and, instead, include a targeting dimension for exceptions.

In addition to the above methodological adjustments, we interpret some policies differently than Oxford researchers. These differences are reflected in some discrepancies between our index and OxCGRT. For example, when provinces mandate the closure of primary and secondary schools but not post-secondary institutions, Oxford researchers code the indicator of school and university closures (C1) at the most restrictive level of 3—“Required closing all levels.” Their rationale is that while post-secondary institutions are not required to close, other policies in place (i.e., gathering restrictions) make it impossible for them to remain open. Meanwhile, we code this indicator at the less restrictive level of 2—“Require closing some school types”—because the closures of post-secondary institutions are not mandated and the gathering restrictions are captured in the coding of C4.

How our index compares with OxCGRT

Chart 1 shows a national stringency index weighted by provinces’ gross domestic product. It compares our calculations with OxCGRT's from January 2020 to January 2021.6 While they generally move together, our index is lower than OxCGRT for the entire period, albeit sometimes very slightly. This is the result of a number of factors:

• Under our approach, having additional categories for some indicators increases the maximum ordinal value for the indicator (i.e., the denominator of the sub-index calculation). This reduces the sub-index. The exception is when both versions code the maximum value.
• Some of our adjustments (e.g., around public gatherings) imply coding a lower ordinal value under the Bank’s version compared with OxCGRT.
• When the additional targeting dimensions apply, the sub-indexes will be lower than OxCGRT.

Chart 1: The Bank’s national stringency index is lower than OxCGRT for most of the pandemic

Note: Both indexes are GDP‑weighted averages of the provincial indexes. OxCGRT refers to the Oxford COVID‑19 Government Response Tracker.
Sources: Blavatnik School of Government and Bank of Canada calculationsLast observation: January 10, 2021

Interpreting the stringency index

COVID‑19 containment measures across provinces

Chart 2 presents our stringency index for each of the five regions in Canada.7

The indexes show that most provincial government containment policies in response to COVID‑19 were the strictest during the first months of the pandemic in spring 2020, despite low case counts relative to late 2020. In late May to early June, restrictions began to ease significantly in all regions, and they generally remained relatively lenient throughout the summer months, even loosening further in some regions. However, in the autumn, policies in all regions tightened in response to an increase in the number of COVID‑19 infections.

Chart 2: Across most regions, containment measures have increased recently in response to higher virus transmission rates

* The indexes for Prairies (Alberta, Manitoba and Saskatchewan), Atlantic (New Brunswick, Newfoundland and Labrador, Nova Scotia and Prince Edward Island) and Canada (all 10 provinces) are GDP‑weighted averages of the indexes for their corresponding provinces.
Source: Bank of Canada calculationsLast observation: February 17, 2021

By the end of 2020 and in early 2021, case numbers were significantly higher than earlier in the pandemic. Despite this, containment measures were no tighter than at the beginning of the pandemic. Ontario is an exception, with slightly stricter policies in January 2021. However, the range of policy stringency varies widely across the country, increasing to its highest point in January 2021 from one of its lowest levels in August 2020. This happened at the same time as an increasing range of case counts. Throughout the pandemic, governments in Ontario and Quebec have generally put in place the strictest containment measures. Both of these provinces have also recorded the highest total case counts.

Restrictions in the Prairies region and in British Columbia have generally been slightly more lenient compared with those in the rest of the country. In fact, during autumn 2020, the Prairies region had some of the most lenient containment policies in Canada—even though it had the highest case numbers per capita. Policies in the Atlantic region have generally been less strict than those in Ontario and Quebec but, despite similar or lower COVID‑19 case counts, stricter than those in British Columbia and the Prairies.

The value and challenges of using the index

The index captures changes in the severity of the policies in a systematic way across jurisdictions and over time. It creates new opportunities for future analysis of how the economy is responding to the pandemic. Indeed, the stringency indexes appear correlated with various measures of economic activity across provinces, particularly in the early stages of the pandemic. These measures include hours worked, unemployment rates, the number of businesses that are currently open and consumer spending.

However, mapping changes in the stringency index to effects on the Canadian economy is not a straightforward exercise for several reasons:

• The index measures the severity of containment policies. It does not consider how businesses and consumers respond and adjust to government policies.
• To preserve simplicity and uniformity, the index uses relatively coarse measures of government policies that cannot incorporate all the details of these policies.
• The index does not weight the 12 broad policy categories to adjust for their impact on the economy. Changes in the stringency index may affect the economy differently depending on which policies are driving them. For instance, stricter workplace closures, school closures, public transit closures, stay-at-home requirements and enforcement for individuals appear associated with more negative economic outcomes compared with other policies.
• Other related factors affecting the economy at the same time need to be considered, such as COVID‑19 case counts and related hospitalizations, voluntary business closures, physical distancing and government support to individuals and businesses.

Next steps for further research

Further empirical analysis is needed to understand how policy stringency affects the economy and to disentangle its impact from other factors at play (e.g., changing transmission rates, voluntary business closures, physical distancing and government support to individuals and businesses). Future research may include using the stringency index to:

• further investigate how government polices affect business behaviour across regions and sectors over the pandemic
• assess which individual policies have the most impact on economic outcomes

Researchers can also use the index for analysis beyond economics, particularly within epidemiology. For example, the index could help researchers study the impact of policy actions on health outcomes, such as virus transmission, hospitalizations and mental health, among many others.

Appendix

Table A-1: Coding guidance for the Bank of Canada stringency index

Note: * represents our adjustments to methodology of the Oxford COVID‑19 Government Response Tracker.

Table A-2: Calculating Bank of Canada stringency sub-indexes: example of how the sub-index in equation 1 is calculated for workplace closures

Note: Coding guidance for workplace closures policy indicator (C2): 0—no measures; 1—recommend closing (or recommend work from home), or require closing for one or two subsectors; 2—require closing (or work from home) for some sectors; 3—require closing (or work from home) for all but a broad set of essential workplaces categorized by sector*; 4—require closing (or work from home) for all but a narrow set of essential workplaces categorized by subsector*
* represents our adjustments to the methodology of the Oxford COVID-19 Government Response Tracker.

Endnotes

References

1. Bank of Canada. 2021. Monetary Policy Report (January).
2. Hale, T., N. Angrist, E. Cameron-Blake, L. Hallas, B. Kira, S. Majumdar, A. Petherick, T. Phillips, H. Tatlow and S. Webster. 2020. “Variation in Government Responses to COVID‑19.” Blavatnik School of Government, University of Oxford, Working Paper No. 2020/032 Version 10.0.

Acknowledgements

We thank Jamal Dumas, Robert Ialenti, Luke Robitaille and Kate Waslen for their contributions collecting the data. We also thank Tatjana Dahlhaus, Brigitte Desroches, Joshua Slive, Jaime Trujillo and Laurent Martin for valuable comments and suggestions. The views expressed in this note are those of the authors and do not necessarily reflect those of the Bank of Canada or its Governing Council.