Advertisement

The effect of state-level stay-at-home orders on COVID-19 infection rates

      Highlights

      • We examined the effect of state-level stay-at-home orders on COVID-19 diagnoses.
      • Rates of confirmed cases were tracked before and after orders were put in place.
      • Logged infection rates were 0.113/day preorder and 0.047/day postorder.
      • Results were consistent across states supporting the use of stay-at-home orders.
      State-level stay-at-home orders were monitored to determine their effect on the rate of confirmed COVID-19 diagnoses. Confirmed cases were tracked before and after state-level stay-at-home orders were put in place. Linear regression techniques were used to determine slopes for log case count data, and meta analyses were conducted to combine data across states. The results were remarkably consistent across states and support the usefulness of stay-at-home orders in reducing COVID-19 infection rates.

      Key Words

      Background

      The emergence of the COVID-19 virus and the lack of therapeutic responses have led to widespread implementation of nonpharmaceutical interventions (NPIs) across the United States. Substantial evidence exists for the effectiveness of NPIs in preventing the spread of infection.
      • Aledort JE
      • Lurie N
      • Wasserman J
      • Bozzette SA
      Non-pharmaceutical public health interventions for pandemic influenza: an evaluation of the evidence base.
      ,
      • Torner N
      • Soldevila N
      • Garcia JJ
      • et al.
      Effectiveness of non-pharmaceutical measures in preventing pediatric influenza: a case–control study.
      However, significant controversy still exists about the value and cost-benefit of these interventions in response to the current pandemic. Specifically, it is unclear what social isolation actions must be mandated by government policy to reach desired mitigation outcomes.
      • Anderson RM
      • Heesterbeek H
      • Klinkenberg D
      • Hollingsworth TD
      How will country-based mitigation measures influence the course of the COVID-19 epidemic?.
      In response to the rising death toll associated with COVID-19, 42 states and the District of Columbia have implemented some variation of a state-wide stay-at-home order. In each case, there was controversy regarding the appropriateness of the order, and it is widely understood that these decisions incur human and economic costs.

      Ozili PK, Arun T. Spillover of COVID-19: impact on the global economy [Published online ahead of print 30 Mar 2020]. SSRN Electr J. Available at:https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3562570. Accessed July 10, 2020.

      In the current analysis, we present data from the states with stay-at-home orders and examine the effect of these policies on the rate of increase in COVID-19 diagnoses.

      Methods

      Data sources

      Google searches were conducted daily to assess state-level responses to the COVID-19 pandemic. Once stay-at-home orders were sighted, state government webpages were searched to confirm the dates that the orders went into effect. Forty-two states and the District of Columbia were identified to have issued a stay-at-home order between March 19, 2020 and April 7, 2020. Stay-at-home orders were defined as statewide mandates that included (a) mandatory nonessential business closures, (b) furloughs enforced for most government and commercial employees, (c) prohibition of public events and gatherings, and (d) travel restrictions including orders to avoid leaving the home except for necessities such as groceries and medical care.
      • Gostin LO
      • Wiley LF
      Governmental public health powers during the COVID-19 pandemic: stay-at-home orders, business closures, and travel restrictions.
      Daily confirmed COVID-19 cases were obtained from the Johns Hopkins Center for Health Security Application and downloaded via GitHub.

      Analysis

      COVID-19 confirmed case counts were matched to stay-at-home order dates and divided into preorder and postorder datasets for every state. Eight states without state-wide stay-at-home orders were excluded from this analysis. For the pre- and postorder sets, slopes were calculated from both raw and logged case count data using linear regression techniques and R2 fit statistics were obtained. As expected, R2 fits were substantially better for the logged data and this approach was used throughout the analysis. Ninety-five percent confidence intervals were obtained for all parameters. Meta-analytic techniques were used to combine data across states using the METAN command suite in Stata. A sensitivity analysis was conducted using the same linear regression and meta-analytic approach to determine whether delaying the days counted as postorder by 1 week would impact the infection rates and R2 fit statistics.

      Results

      Table 1 displays the dates stay-at-home orders went into effect as well as logged, preorder and postorder slopes and 95% confidence intervals for all 42 states and the District of Columbia. R2 fit statistics are also shown. In general, the fit of the slopes to the logged data is excellent, as would be expected from an exponentially growing pandemic in a naive population. The average rate of increase preorder was 0.113 (95% C.I.: 0.110, 0.115) per day and postorder was 0.047 (95% C.I.: 0.045, 0.048) per day
      Table 1Infection rates before and after stay at home orders went into effect*,†
      StateOrder dateNumber of days before orderInfection rate and confidence interval (before order)R squared (before order)Number of days after orderInfection rate and confidence interval (after order)R squared (after order)
      Alabama4/4/2020210.099 (0.088, 0.109)0.95190.042 (0.039, 0.045)0.994
      Alaska3/28/202090.11 (0.095, 0.126)0.976160.03 (0.027, 0.032)0.982
      Arizona3/31/2020180.134 (0.124, 0.143)0.981130.03 (0.025, 0.036)0.932
      California3/19/2020280.084 (0.077, 0.091)0.957250.055 (0.05, 0.06)0.959
      Colorado3/26/2020170.11 (0.1, 0.121)0.973180.04 (0.035, 0.044)0.951
      Connecticut3/23/2020110.154 (0.136, 0.172)0.976210.065 (0.059, 0.07)0.968
      Delaware3/24/202080.111 (0.096, 0.126)0.983200.064 (0.061, 0.066)0.993
      District of Columbia4/1/2020170.091 (0.082, 0.099)0.972120.044 (0.038, 0.049)0.974
      Florida4/3/2020270.12 (0.114, 0.127)0.981100.029 (0.028, 0.03)0.997
      Georgia4/3/2020260.109 (0.102, 0.116)0.979100.039 (0.032, 0.045)0.959
      Hawaii3/25/202090.117 (0.095, 0.14)0.956190.04 (0.034, 0.046)0.92
      Idaho3/25/202070.097 (0.072, 0.122)0.951190.054 (0.043, 0.064)0.874
      Illinois3/21/2020120.158 (0.142, 0.173)0.981230.059 (0.054, 0.064)0.965
      Indiana3/24/2020140.123 (0.108, 0.138)0.964200.061 (0.054, 0.069)0.94
      Kansas3/30/2020150.105 (0.098, 0.112)0.987140.039 (0.036, 0.043)0.982
      Kentucky3/26/2020150.102 (0.095, 0.109)0.987180.046 (0.042, 0.049)0.98
      Louisiana3/23/2020120.159 (0.141, 0.177)0.976210.061 (0.054, 0.068)0.945
      Maine4/2/2020180.072 (0.065, 0.08)0.961110.019 (0.018, 0.021)0.991
      Maryland3/30/2020190.115 (0.109, 0.12)0.991140.057 (0.053, 0.06)0.991
      Massachusetts3/24/2020170.098 (0.086, 0.109)0.958200.057 (0.052, 0.063)0.965
      Michigan3/24/2020120.197 (0.183, 0.211)0.99200.054 (0.047, 0.061)0.938
      Minnesota3/27/2020150.102 (0.09, 0.114)0.962170.034 (0.033, 0.036)0.992
      Mississippi4/3/2020190.107 (0.091, 0.123)0.919100.034 (0.032, 0.037)0.992
      Missouri4/6/2020210.113 (0.099, 0.128)0.93470.029 (0.023, 0.034)0.971
      Montana3/28/2020110.113 (0.101, 0.125)0.982160.027 (0.023, 0.03)0.956
      Nevada4/1/2020210.104 (0.097, 0.111)0.979120.029 (0.026, 0.032)0.982
      New Hampshire3/27/2020120.081 (0.07, 0.092)0.964170.042 (0.036, 0.049)0.924
      New Jersey3/21/2020120.182 (0.169, 0.196)0.989230.066 (0.058, 0.074)0.927
      New Mexico3/24/2020120.09 (0.084, 0.096)0.99200.058 (0.053, 0.064)0.966
      New York3/22/2020190.164 (0.153, 0.174)0.985220.045 (0.04, 0.049)0.955
      North Carolina3/30/2020190.115 (0.107, 0.124)0.98140.038 (0.035, 0.042)0.982
      Ohio3/23/2020110.15 (0.138, 0.161)0.99210.053 (0.047, 0.059)0.948
      Oregon3/23/2020160.082 (0.076, 0.087)0.985210.04 (0.034, 0.045)0.93
      Pennsylvania4/1/2020230.126 (0.122, 0.131)0.993120.047 (0.042, 0.053)0.975
      Rhode Island3/28/2020160.086 (0.08, 0.091)0.988160.062 (0.058, 0.065)0.991
      South Carolina4/7/2020280.091 (0.082, 0.099)0.94860.028 (0.014, 0.042)0.887
      Tennessee4/2/2020220.112 (0.104, 0.121)0.973110.026 (0.024, 0.028)0.99
      Texas4/2/2020260.114 (0.108, 0.12)0.984110.041 (0.037, 0.045)0.985
      Vermont3/25/2020100.125 (0.109, 0.141)0.975190.038 (0.035, 0.042)0.968
      Virginia3/30/2020190.099 (0.094, 0.105)0.988140.051 (0.047, 0.054)0.986
      Washington3/23/2020230.101 (0.093, 0.11)0.966210.033 (0.029, 0.036)0.951
      West Virginia3/24/202040.079 (-0.002, 0.16)0.899200.061 (0.054, 0.067)0.951
      Wisconsin3/25/2020130.133 (0.12, 0.147)0.977190.036 (0.032, 0.039)0.964
      * The COVID-19 infection rates above are logged, preorder and postorder slopes.
      † Results shown above were updated through April 13, 2020.
      Fig 1 presents meta analyses results using combined data across states. Two meta analyses were conducted using different sets of weights: number of days during the period and final number of cases during the period. Weighing by number of days yielded a pooled standardized mean difference of 3.847 (95% C.I.: 3.653, 4.041; P < .0001). Weighing by final number of cases yielded a similar result, with a pooled standardized mean difference of 6.847 (95% C.I.: 6.831, 6.863; P < .0001). Overall, these slope changes translate to a reduction from about 12% more cases per day (and thus a 5 to 6-day doubling rate) to 5% more cases per day (and thus a 14-day doubling rate).
      Fig 1
      Fig 1(a) Meta-analysis weighed by the number of days. 95% confidence intervals (CI) and standardized mean difference (SMD) are shown. (b) Meta-analysis weighed by the final number of cases 95% confidence intervals (CI) and standardized mean difference (SMD) are shown.
      The same analyses were repeated with postorder datasets starting 1 week after stay-at-home orders went into effect. In this sensitivity analysis, the pooled standardized mean difference was 4.229 (95% C.I.: 4.011, 4.448; P < .0001) after weighing by number of days and was 6.811 (95% C.I.: 6.798, 6.824; P < .0001) after weighing by final number of cases.

      Discussion

      These data suggest a remarkably consistent and important effect associated with the issuing of stay-at-home orders and are generally supportive of such measures. In combination with other social isolation approaches and NPIs, these orders may play a significant role in “flattening the curve.” However, this study has a number of important limitations. First, it would be impossible to isolate the effect of these orders against the background of numerous other local, state, and federal interventions occurring at the same time. Second, the expected COVID-19 expansion curve in the absence of interventions to reduce the transmission is unknown. As such, it is possible that the observed slope changes are driven by the natural history of the pandemic as well as the specific policies in question. Two sensitivity analyses were conducted to examine whether the number of days in the pre- and postperiods were drivers of either the slopes or the change in the slopes between periods. In both cases, we found no relationship. Third, states implemented stay-at-home orders in response to the pandemic and these observations are thus profoundly threatened by selection and indication biases. Fourth, even with the use of trend data in this analysis, there is a threat of regression to the mean if stay-at-home orders were consistently placed at the peak of epidemic growth.
      Finally, and perhaps most importantly, a major limitation of this work is the endogenous relationship between case counts and both the availability and use of testing. The availability of testing was a significant barrier to COVID-19 diagnoses in the early stages of the pandemic in the United States and improved gradually over the month of March. Tests per day increased roughly 10-fold between March 15th and March 31st and have increased at a more gradual rate of 4-fold over the last 6 weeks. We conducted an additional sensitivity analysis to examine whether the decrease in case incidence between pre- and postorder periods was different if the order occurred before or after April 1st and found the change in slopes to be nearly identical. As mentioned earlier, we also conducted a sensitivity analysis to account for the effect of time on these slopes and found no effect. Overall, the results must be interpreted in the context of this threat. Not only was the availability of tests evolving over this time period, but there is a direct relationship between the number of infections and public awareness of the pandemic and the number of tests performed. As such, it would be difficult to begin to parse out the effect of testing on the rate of incidence of COVID-19. That said, it is likely that the availability of tests may have an unmeasured impact on these results, but we cannot know if the impact was to increase or decrease the incidence rates. Despite these limitations, the consistency and strength of the results is notable. The results presented here will be updated daily and available for public download at www.hpmcovidpolicy.org.

      Conclusions

      Stay-at-home orders are generally supported by our results. This data is publicly available and provides daily updates on national infection rates. By tracking stay-at-home orders and the resulting infection rates, information is gathered on the effectiveness of such orders in mitigating the spread of COVID-19. The data may also inform policy and mitigation efforts for future infection outbreaks. Further research is needed to examine the effect of these policies in the longer term and in combination with other interventions.

      References

        • Aledort JE
        • Lurie N
        • Wasserman J
        • Bozzette SA
        Non-pharmaceutical public health interventions for pandemic influenza: an evaluation of the evidence base.
        BMC Public Health. 2007; 7: 208
        • Torner N
        • Soldevila N
        • Garcia JJ
        • et al.
        Effectiveness of non-pharmaceutical measures in preventing pediatric influenza: a case–control study.
        BMC Public Health. 2015; 15: 543
        • Anderson RM
        • Heesterbeek H
        • Klinkenberg D
        • Hollingsworth TD
        How will country-based mitigation measures influence the course of the COVID-19 epidemic?.
        The Lancet. 2020; 395: 931-934
      1. Ozili PK, Arun T. Spillover of COVID-19: impact on the global economy [Published online ahead of print 30 Mar 2020]. SSRN Electr J. Available at:https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3562570. Accessed July 10, 2020.

        • Gostin LO
        • Wiley LF
        Governmental public health powers during the COVID-19 pandemic: stay-at-home orders, business closures, and travel restrictions.
        JAMA. 2020; https://doi.org/10.1001/jama.2020.5460
      2. Daily COVID-19 Tests. Our World in Data. Available at:https://ourworldindata.org/grapher/full-list-covid-19-tests-per-day?time=2020-03-17..&country=UnitedStates,testsperformed+USA. Accessed May 18, 2020.