The Bank of Canada COVID‑19 stringency index: measuring policy response across provinces

Introduction

Policy measures that governments have put in place to protect people against the community transmission of COVID‑19 have been crucial to public health. But they have also had a large impact on the economy. In Canada, responsibility for these measures falls primarily on the provinces. Provincial governments have taken different approaches to containing the spread of COVID‑19. These have varied over time as governments have responded to changing rates of infection and hospitalization and new information about the effectiveness of different measures. Tracking these measures is important for understanding the impact of the pandemic on Canada’s economy.

This note focuses on the Bank of Canada’s stringency index—a measure of containment policies and public information campaigns. Since summer 2020, staff at the Bank have been collecting publicly available information on government policies and creating daily government response indexes for the 10 provinces. This allows the Bank to systematically measure, track and compare government policy responses. These indexes follow the methodology of the Oxford COVID‑19 Government Response Tracker (OxCGRT) developed by the University of Oxford’s Blavatnik School of Government.1 Bank staff adjusted the measures to make them more appropriate for the Canadian context and to capture more granular differences in policy responses.

The provincial indexes show that most government containment policies in response to COVID‑19 were strictest during the first months of the pandemic in spring 2020, despite low case counts relative to late 2020. Throughout the pandemic, Ontario and Quebec have had the strictest containment measures and the highest case counts.

The stringency index does not measure the efficacy of a province’s COVID‑19 response or provide a direct measure of the impact of government policies on the economy, including the impact on output or consumption. While there appears to be an association between economic activity and both the stringency index and its components, more research is needed to precisely identify their impacts.

The composition of the Bank of Canada’s stringency index

What the index measures

Our stringency index measures the severity of containment policies and public information campaigns. For each province, the index is the simple average of 12 sub-indexes. Sub-indexes are normalized values of corresponding policy indicators. Table 1 lists the 12 policy indicators currently included in our stringency index.

Because most containment measures are issued by provincial governments, the stringency index captures mostly differences across provincial policies over time. The main exception is the international travel restrictions indicator (C8), which captures mainly federal policies imposed across the country.

Table 1: Policies included in the Bank of Canada’s stringency index

Identifier Description
C1 School and university closures
C2 Workplace and office closures
C3 Public event cancellations and restrictions
C4 Restrictions on private gatherings
C5 Public transport closures
C6 Stay-at-home requirements
C7 Restrictions on intra-provincial travel (between cities or regions within a province)
C8 Restrictions on international travel
C9 Restrictions on interprovincial travel (between provinces)
C10 Enforcement mechanisms for individuals
C11 Enforcement mechanisms for firms
H1 Public information campaigns

How a sub-index is constructed

Following the methodology of OxCGRT, we construct a sub-index for each province as follows:

  1. For each policy indicator listed in Table 1, we assign a coding value along an ordinal scale for the most restrictive measure in place within the province based on the coding guidance (see Table A‑1 in the Appendix for details). Higher values indicate stricter measures.
  2. Each policy indicator has additional binary flags to identify whether the policy is general (1) or targeted (0) within a province along various dimensions (e.g., geography). We treat targeted measures as fractional decreases between the ordinal integers (i.e., less stringent than general, non-targeted policies). The more targeting dimensions an indicator has, the more steps exist between each coding value.2
  3. Finally, we calculate a daily sub-index that normalizes the coding values of each indicator between 0 and 100 (higher values mean stricter measures).

Equation 1 below describes the calculation for each sub-index, Ij, for policy indicator, j:

\(\displaystyle{Sub-Index}_j\) \(\displaystyle=\,100\ast\left(\frac{a_j-\sum_{i=1}^{n}\left(\frac{1-f_{ji}}{n+1}\right)}{b_j}\right),\ \ \ \ \ \ \ \ \ \ (1)\)

where:

  • \(a_j\) is the ordinal value for the most restrictive measures in place within the province (see Table A‑1 for codes)
  • \(b_j\) is the maximum coding value
  • \(n_j\) is the total number of targeting dimensions (e.g., by geography)
  • \(f_{ij}\) is the flag for targeting dimension, i, which is equal to 1 if the policy is general or 0 if targeted

See Table A‑2 in the Appendix for an example of the calculation.

Adjustments to the OxCGRT methodology

We adapted the methodology of OxCGRT to make the index more appropriate for the Canadian context and to capture more granular differences in provincial policy responses.3 (All adjustments are marked by asterisks in Table A‑1.)

  1. We added three policy indicators (beyond the nine included in OxCGRT).4
    • C9—Restrictions on interprovincial travel: We revised OxCGRT's policy indicator on internal movement restrictions (C7) to capture only intra-provincial travel restrictions and added a separate indicator of interprovincial travel. Adding this indicator mechanically gives greater weight to travel restrictions in the stringency index.
    • C10—Enforcement mechanisms for individuals: OxCGRT does not include a measure identifying differences in provinces that have similar containment policies but varying punishments or enforcement of those policies. Thus, we created a new indicator to capture penalties for individuals violating public health orders.
    • C11—Enforcement mechanisms for firms: For the same reason as above, we created a new indicator to capture penalties for businesses violating public health orders.
  2. We added more targeting dimensions. These increase the granularity in the variation across provinces and over time. Beyond the geographic dimension included in OxCGRT, we include targeting dimensions for, among other things, types of public events, outdoor versus indoor events and exceptions for certain travellers.5 (See Table A‑1 for the complete list.)
  3. We refined the categories of coding values for several policy indicators. This was based on a more detailed examination of the policies in Canada. Where we added categories to our indicators, the denominator of the sub-index calculation (b from equation 1) increases. This often means a lower sub-index compared with OxCGRT (an exception is when both code the maximum value). Below, we present some of the main changes to categories (see Table A‑1 for the complete list):
    • Workplace closures (C2): Governments have issued lists of workplaces designated as “essential services” that can remain open, and these vary by province. Whereas OxCGRT defines the strictest category as the requirement to close “all-but-essential workplaces,” we divide this category into two:
      • broad—when the government defines the essential service list by sector (e.g., retail and construction), which was more frequent during the spring 2020 lockdowns
      • narrow—when the government defines the essential service list by subsector (e.g., pharmacies and commercial construction), which was more common after the spring 2020 lockdowns
    • Public event cancellations (C3): Whereas OxCGRT assigns the maximum ordinal value to any cancellation of events no matter the size, we added two categories to differentiate among event restrictions based on the maximum number of people allowed at one event:
      • events limited to a maximum of 50 to 100 attendees
      • events limited to a maximum of more than 100 attendees
    • Restrictions on private gatherings (C4): We adjusted the categories to allow for finer distinctions among restrictions for gatherings of between 10 and 100 people, based on the realities of different provinces’ policies.
    • International travel (C8): Unlike OxCGRT, we combine the categories of “banned arrivals from some regions” and “banned arrivals from all regions” into one category and, instead, include a targeting dimension for exceptions.

In addition to the above methodological adjustments, we interpret some policies differently than Oxford researchers. These differences are reflected in some discrepancies between our index and OxCGRT. For example, when provinces mandate the closure of primary and secondary schools but not post-secondary institutions, Oxford researchers code the indicator of school and university closures (C1) at the most restrictive level of 3—“Required closing all levels.” Their rationale is that while post-secondary institutions are not required to close, other policies in place (i.e., gathering restrictions) make it impossible for them to remain open. Meanwhile, we code this indicator at the less restrictive level of 2—“Require closing some school types”—because the closures of post-secondary institutions are not mandated and the gathering restrictions are captured in the coding of C4.

How our index compares with OxCGRT

Chart 1 shows a national stringency index weighted by provinces’ gross domestic product. It compares our calculations with OxCGRT's from January 2020 to January 2021.6 While they generally move together, our index is lower than OxCGRT for the entire period, albeit sometimes very slightly. This is the result of a number of factors:

  • Under our approach, having additional categories for some indicators increases the maximum ordinal value for the indicator (i.e., the denominator of the sub-index calculation). This reduces the sub-index. The exception is when both versions code the maximum value.
  • Some of our adjustments (e.g., around public gatherings) imply coding a lower ordinal value under the Bank’s version compared with OxCGRT.
  • When the additional targeting dimensions apply, the sub-indexes will be lower than OxCGRT.

Chart 1: The Bank’s national stringency index is lower than OxCGRT for most of the pandemic

Note: Both indexes are GDP‑weighted averages of the provincial indexes. OxCGRT refers to the Oxford COVID‑19 Government Response Tracker.
Sources: Blavatnik School of Government and Bank of Canada calculationsLast observation: January 10, 2021

Interpreting the stringency index

COVID‑19 containment measures across provinces

Chart 2 presents our stringency index for each of the five regions in Canada.7

The indexes show that most provincial government containment policies in response to COVID‑19 were the strictest during the first months of the pandemic in spring 2020, despite low case counts relative to late 2020. In late May to early June, restrictions began to ease significantly in all regions, and they generally remained relatively lenient throughout the summer months, even loosening further in some regions. However, in the autumn, policies in all regions tightened in response to an increase in the number of COVID‑19 infections.

Chart 2: Across most regions, containment measures have increased recently in response to higher virus transmission rates

* The indexes for Prairies (Alberta, Manitoba and Saskatchewan), Atlantic (New Brunswick, Newfoundland and Labrador, Nova Scotia and Prince Edward Island) and Canada (all 10 provinces) are GDP‑weighted averages of the indexes for their corresponding provinces.
Source: Bank of Canada calculationsLast observation: February 17, 2021

By the end of 2020 and in early 2021, case numbers were significantly higher than earlier in the pandemic. Despite this, containment measures were no tighter than at the beginning of the pandemic. Ontario is an exception, with slightly stricter policies in January 2021. However, the range of policy stringency varies widely across the country, increasing to its highest point in January 2021 from one of its lowest levels in August 2020. This happened at the same time as an increasing range of case counts. Throughout the pandemic, governments in Ontario and Quebec have generally put in place the strictest containment measures. Both of these provinces have also recorded the highest total case counts.

Restrictions in the Prairies region and in British Columbia have generally been slightly more lenient compared with those in the rest of the country. In fact, during autumn 2020, the Prairies region had some of the most lenient containment policies in Canada—even though it had the highest case numbers per capita. Policies in the Atlantic region have generally been less strict than those in Ontario and Quebec but, despite similar or lower COVID‑19 case counts, stricter than those in British Columbia and the Prairies.

The value and challenges of using the index

The index captures changes in the severity of the policies in a systematic way across jurisdictions and over time. It creates new opportunities for future analysis of how the economy is responding to the pandemic. Indeed, the stringency indexes appear correlated with various measures of economic activity across provinces, particularly in the early stages of the pandemic. These measures include hours worked, unemployment rates, the number of businesses that are currently open and consumer spending.

However, mapping changes in the stringency index to effects on the Canadian economy is not a straightforward exercise for several reasons:

  • The index measures the severity of containment policies. It does not consider how businesses and consumers respond and adjust to government policies.
  • To preserve simplicity and uniformity, the index uses relatively coarse measures of government policies that cannot incorporate all the details of these policies.
  • The index does not weight the 12 broad policy categories to adjust for their impact on the economy. Changes in the stringency index may affect the economy differently depending on which policies are driving them. For instance, stricter workplace closures, school closures, public transit closures, stay-at-home requirements and enforcement for individuals appear associated with more negative economic outcomes compared with other policies.
  • Other related factors affecting the economy at the same time need to be considered, such as COVID‑19 case counts and related hospitalizations, voluntary business closures, physical distancing and government support to individuals and businesses.

Next steps for further research

Further empirical analysis is needed to understand how policy stringency affects the economy and to disentangle its impact from other factors at play (e.g., changing transmission rates, voluntary business closures, physical distancing and government support to individuals and businesses). Future research may include using the stringency index to:

  • further investigate how government polices affect business behaviour across regions and sectors over the pandemic
  • assess which individual policies have the most impact on economic outcomes

Researchers can also use the index for analysis beyond economics, particularly within epidemiology. For example, the index could help researchers study the impact of policy actions on health outcomes, such as virus transmission, hospitalizations and mental health, among many others.

Appendix

Table A-1: Coding guidance for the Bank of Canada stringency index

Identifier and policy indicator name Policy indicator coding values Targeting dimension Targeting coding guidance
Containment and closure policies
C1 – School and university closures 0 – No measures
1 – Recommend closing or leave all schools open with significant differences compared with non-COVID‑19 operations
2 – Require closing some school types (at least one school type is not required to close) (e.g., grade schools, post-secondary, trade schools)
3 – Require closing all school types (e.g., grade schools, post-secondary, trade schools)
Geography 0 – Targeted region
1 – Entire province
Demographic* 0 – Targeted grade
1 – All grades
Class* 0 – Online classes replace in-person classes
1 – All lesson types cancelled
C2 – Workplace closures 0 – No measures
1 – Recommend closing (or recommend work from home). Or require closing for one or two subsectors (e.g., only bars and clubs required to close)*
2 – Require closing (or work from home) for some sectors
3 – Require closing (or work from home) for all but a broad set of essential workplaces (sectors)*
4 - Require closing (or working from home) for all but a narrow set of essential workplaces (subsectors)*
Geography 0 – Targeted region
1 – Entire province
Opening* 0 – Partial opening allowed (i.e., curbside pickup, delivery)
1 – No partial opening allowed
C3 – Public event cancellations 0 – No measures
1 – Recommend cancelling or require cancelling of select types of events*
2 – Cancel events with more than 100 attendees*
3 – Cancel events with more than 50 attendees*
4 – Cancel all events*
Geography 0 – Targeted region
1 – Entire province
Event* 0 – Some events exempt
1 – No events exempt
Outdoor/Indoor* 0 – Difference between outdoor/indoor event restrictions
1 – Policy applies across both indoor and outdoor events
C4 – Restrictions on private gatherings 0 – No measures
1 – Restriction limit on gatherings is greater than 100 people*
2 – Restriction limit on gatherings is greater than 50 but less than or equal to 100*
3 – Restriction limit on gatherings is greater than 15 but less than or equal to 50*
4 – Restriction limit on gatherings is less than or equal to 15*
5 – Prohibit gatherings of any size
Geography 0 – Targeted region
1 – Entire province
Bubble* 0 – Dynamic group bubble
1 – Stable group bubble (same people every time)
Outdoor* 0 – Larger gatherings allowed outdoors
1 – Same gathering size applies across both indoor and outdoor groups
C5 – Public transport closures 0 – No measures
1 – Recommend closing (or significantly reduce volume/route/means of transport available)
2 – Require closing (or prohibit most citizens from using it)
Geography 0 – Targeted region
1 – Entire province
C6 – Stay-at-home requirements 0 – No measures
1 – Recommend not leaving house
2 – Require not leaving house with exceptions for daily exercise, grocery shopping and essential trips
3 – Require not leaving house with minimal exceptions (e.g., allow to leave once a week, only one person can leave at a time)
Geography 0 – Targeted region
1 – Entire province
C7 – Restrictions on intra-provincial travel 0 – No measures
1 – Recommend not travelling between regions/cities
2 – Restrict internal movement
Geography 0 – Targeted region
1 – Entire province
Exceptions* 0 – Some exceptions (i.e., essential travel, certain borders open)
1 – No exceptions
C8 – Restrictions on international travel 0 – No restrictions
1 – Screen arrivals / Recommend quarantine*
2 – Require quarantine for arrivals*
3 – Ban arrivals*
Geography 0 – Targeted (i.e., people from certain regions are exempt from this policy, whether departures or arrivals)
1 – General (i.e., no region is exempt from departing/arriving)
Demographic exceptions* 0 – Targeted (i.e., some people are exempt regardless of their region for departures or arrivals)
1 – General (i.e., no group from a non-exempt region is exempt from departing/arriving)
C9 – Restrictions on interprovincial travel * 0 – No restrictions*
1 – Screen arrivals / Recommend quarantine*
2 – Require quarantine for interprovincial travellers*
3 – Ban interprovincial travel*
Geography 0 – Targeted (i.e., people from certain regions are exempt from this policy whether departures or arrivals)
1 – General (i.e., no region is exempt from departing/arriving)
Demographic exceptions* 0 – Targeted (i.e., some people are exempt regardless of their region for departures or arrivals)
1 – General (i.e., no group from a non-exempt region is exempt from departing/arriving)
C10 – Enforcement mechanisms for individuals* 0 – No enforcement mechanism*
1 – Small fine of less than or equal to $500*
2 - Moderate fine of less than or equal to $2,000, greater than $500*
3 – Large fine of greater than $2,000*
4 – Potential imprisonment*
Geography 0 – Targeted region
1 – Entire province
Enforcer* 0 – Specific group can enforce (only health authorities)
1 – General enforcement (e.g., police, bylaw officers)
Enforcement* 0 – Enforcement does not apply to all COVID‑19 policies (i.e., most stringent fine is only for certain violations)
1 – Enforcement applies to all COVID‑19 policies
Punishment* 0 – Punishment enforced later by courts
1 – Enforced immediately by police
C11 – Enforcement mechanisms for firms* 0 – No enforcement mechanism*
1 – Small fine of less than or equal to $2,000*
2 - Moderate fine of less than or equal to $5,000, greater than $2,000*
3 – Large fine of greater than $5,000*
4 – Can be shut down (forced shutdown for non-compliance)
Geography 0 – Targeted region
1 – Entire province
Enforcer* 0 – Specific group can enforce (only health authorities)
1 – General enforcement (e.g., police, bylaw officers)
Enforcement* 0 – Enforcement does not apply to all COVID‑19 policies (i.e., most stringent fine is only for certain violations)
1 – Enforcement applies to all COVID‑19 policies
Punishment* 0 – Punishment enforced later by courts
1 – Enforced immediately by police
Health system policies
H1 – Public information campaigns 0 – No COVID‑19 public information campaign
1 – Public officials urging caution about COVID‑19
2 – Coordinated public information campaign (e.g., across traditional and social media)
Geography 0 – Targeted region
1 – Entire province

Note: * represents our adjustments to methodology of the Oxford COVID‑19 Government Response Tracker.

Table A-2: Calculating Bank of Canada stringency sub-indexes: example of how the sub-index in equation 1 is calculated for workplace closures

Code Geographically targeted Partial opening allowed Normalized
0 No No 0.00
1 Yes Yes 8.3
1 Yes No 16.7
1 No Yes 16.7
1 No No 25.0
2 Yes Yes 33.3
2 Yes No 41.7
2 No Yes 41.7
2 No No 50.0
3 Yes Yes 58.3
3 Yes No 66.7
3 No Yes 66.7
3 No No 75.0
4 Yes Yes 83.3
4 Yes No 91.7
4 No Yes 91.7
4 No No 100.0

Note: Coding guidance for workplace closures policy indicator (C2): 0—no measures; 1—recommend closing (or recommend work from home), or require closing for one or two subsectors; 2—require closing (or work from home) for some sectors; 3—require closing (or work from home) for all but a broad set of essential workplaces categorized by sector*; 4—require closing (or work from home) for all but a narrow set of essential workplaces categorized by subsector*
* represents our adjustments to the methodology of the Oxford COVID-19 Government Response Tracker.

  1. 1. Since spring 2020, OxCGRT has provided indexes related to various containment, economic support and health system policies for more than 180 countries and the US states. For more details on their methodology and coding guidance, see Hale et al. (2020) and their “Codebook for the Oxford Covid-19 Government Response Tracker,” respectively. The tracker initially provided only national indexes for Canada; these lacked the granularity to sufficiently account for policy differences across provinces. In late December 2020, OxCGRT added provincial indexes for Canada—see subsection on How our index compares with OxCGRT.[]
  2. 2. If an indicator has n targeting dimensions, then it has n equal steps between coding values (or each flag decreases the numerator in equation 1 by 1/(n+1)). For example, an indicator with two targeting dimensions has two steps between each coding value, and so each targeting dimension decreases the numerator by one-third if policies are targeted. For example, if a policy coded at a value of 1 is targeted along only one of its two targeting dimensions, then the numerator in equation 1 is 0.67 (or 1 – (1/3) = 2/3).[]
  3. 3. As a result of these changes, our stringency index is not directly comparable with OxCGRT. To compare policies internationally, it is best to use the latter.[]
  4. 4. A policy change within our sub-indexes will have a smaller effect on the stringency index since each sub-index has a weight of one-twelfth, compared with a weight of one-ninth in OxCGRT.[]
  5. 5. In OxCGRT, the number of targeting dimensions (n, in equation 1) is at most one, and the steps between coding values is always one-half if a sub-index has a targeting dimension. Conversely, our sub-indexes have up to four targeting dimensions, and thus steps between ordinal values can range from one-half to one-fifth.[]
  6. 6. The weighted national aggregate stringency index that appears in Chart 1 differs slightly from the version that appeared in the Bank’s Monetary Policy Report published in January 2021. These differences mainly reflect some adjustments that were made to the categories of some policy indicators following the publication.[]
  7. 7. The five regions include the individual provinces of British Columbia, Ontario and Quebec, as well as the Prairies (Alberta, Manitoba and Saskatchewan) and Atlantic Canada (New Brunswick, Newfoundland and Labrador, Nova Scotia and Prince Edward Island).[]

References

  1. Bank of Canada. 2021. Monetary Policy Report (January).
  2. Hale, T., N. Angrist, E. Cameron-Blake, L. Hallas, B. Kira, S. Majumdar, A. Petherick, T. Phillips, H. Tatlow and S. Webster. 2020. “Variation in Government Responses to COVID‑19.” Blavatnik School of Government, University of Oxford, Working Paper No. 2020/032 Version 10.0.

Acknowledgements

We thank Jamal Dumas, Robert Ialenti, Luke Robitaille and Kate Waslen for their contributions collecting the data. We also thank Tatjana Dahlhaus, Brigitte Desroches, Joshua Slive, Jaime Trujillo and Laurent Martin for valuable comments and suggestions. The views expressed in this note are those of the authors and do not necessarily reflect those of the Bank of Canada or its Governing Council.

Disclaimer

Bank of Canada staff analytical notes are short articles that focus on topical issues relevant to the current economic and financial context, produced independently from the Bank’s Governing Council. This work may support or challenge prevailing policy orthodoxy. Therefore, the views expressed in this note are solely those of the authors and may differ from official Bank of Canada views. No responsibility for them should be attributed to the Bank.