Immunized Doubly-Robust Synthetic Control (BEAST)

Immunized Doubly-Robust Synthetic Control (BEAST)#

When to Use This Estimator#

A classic synthetic control chooses donor weights to make a weighted average of the control units track the treated unit’s pre-treatment outcomes, then reads the treatment effect off the post-treatment gap. That works well when a few donors clearly resemble the treated unit, but it says nothing about how to use covariates when there are many of them, and it gives no easy standard error for the resulting effect.

Use BEAST, due to Bléhaut, D’Haultfœuille, L’Hour and Tsybakov [BEAST], when you have a moderately rich set of unit-level covariates – economic predictors, lagged outcomes – of which only a handful truly matter, and you want a treatment effect that comes with an honest, analytic confidence interval. BEAST picks donor weights by covariate balancing rather than outcome fitting: it finds exponential-tilting weights that make the covariates of the weighted donor pool match the treated unit, with an ℓ₁ penalty that selects the informative covariates and discards the rest. It then corrects the effect with an immunizing outcome regression, which makes the estimator doubly robust – valid if either the balancing or the outcome model is right – and asymptotically normal, so its standard error is a closed-form expression rather than a placebo distribution.

The name is the authors’: BEAST is the immunized doubly-robust estimator built from the Calibration-Lasso balancing weights and the immunizing WLS-Lasso outcome model.

Do not use this estimator when#

  • Your covariate set is over-saturated – as many (or nearly as many) covariates as control units. The exponential-tilting calibration then degenerates: the balancing weights no longer sum to one, so they are not a valid synthetic control. BEAST detects this and raises rather than returning a meaningless answer (see The balance-validity guard). Reduce to a sparse, informative covariate set, or use a high-dimensional-donor estimator built for that regime (Sparse Synthetic Control (SparseSC), Cluster Synthetic Controls (CLUSTERSC)).

  • You have no covariates worth balancing on. BEAST is the covariate machinery; with nothing to balance, a plain synthetic control (Vanilla Synthetic Control (VanillaSC)) is the right tool.

  • The post-period is a single period, or you need period-by-period sharp-null inference. BEAST’s interval is for the average post-treatment effect under an approximately-independent-errors argument; conformal inference (Vanilla Synthetic Control (VanillaSC) with inference="conformal") targets the sharp null.

Notation#

Let \(d_j \in \{0, 1\}\) indicate whether unit \(j\) is the treated one, with \(n_1 = \sum_j d_j\) treated units (here \(n_1 = 1\)) and \(\pi = n_1 / n\) the treated share. Each unit carries a covariate vector \(\mathbf{x}_j \in \mathbb{R}^{p}\) whose first entry is a constant; these are pre-treatment covariate means and, optionally, a few lagged outcomes, all normalized. Stacked over units these form \(\mathbf{X} \in \mathbb{R}^{n \times p}\). For a single post period \(t\) write \(y_{jt}\) for unit \(j\)’s outcome.

BEAST has two nuisance parameters and one target. The balancing coefficients \(\boldsymbol{\beta} \in \mathbb{R}^{p}\) define the donor weights; the outcome-model coefficients \(\boldsymbol{\mu} \in \mathbb{R}^{p}\) immunize the effect; the target is the average treatment effect on the treated, \(\theta\), estimated as \(\widehat\theta\).

Assumptions#

Assumption 1 (sparsity). The treated-vs-control covariate contrast is driven by a small number \(s \ll p\) of the covariates – \(\boldsymbol{\beta}\) and \(\boldsymbol{\mu}\) are sparse.

Remark. This is what lets the method admit many covariates: the ℓ₁ penalty selects the few that carry the signal, so \(p\) may be large relative to the sample as long as the active set stays small. It is also the assumption the balance guard protects – when it fails and every covariate is active, the calibration has no sparse solution to find and the weights stop summing to one.

Assumption 2 (overlap / valid balancing). There exist tilting weights that balance the covariates between the treated unit and the donor pool; equivalently the exponential-tilting calibration has a finite solution whose weights sum to one.

Remark. The intercept (the constant column of \(\mathbf{X}\)) is the balance condition for the total mass: balancing it forces \(\sum_j W_j = 1\), which is exactly what makes the tilting weights a synthetic control. When covariates are so rich that no tilting balances them, this fails and the design is not a synthetic control at all.

Assumption 3 (double robustness). Either the balancing model or the outcome model is correctly specified (not necessarily both).

Remark. This is the payoff of immunization. The effect is built from a moment that is Neyman-orthogonal to both nuisances, so a first-order error in \(\widehat{\boldsymbol\beta}\) or \(\widehat{\boldsymbol\mu}\) does not bias \(\widehat\theta\); only if both are wrong does the estimate suffer. The same orthogonality is what makes \(\widehat\theta\) asymptotically normal with the analytic variance below.

The estimator#

Step 1: Calibration-Lasso balancing weights. The donor weights are an exponential tilt of the covariates,

\[W_j = \frac{(1 - d_j)\,\exp(\mathbf{x}_j^\top \boldsymbol\beta)}{n_1},\]

with \(\boldsymbol\beta\) solving the ℓ₁-penalized calibration

\[\widehat{\boldsymbol\beta} = \operatorname*{arg\,min}_{\boldsymbol\beta} \ \frac{1}{n}\sum_j \bigl[(1 - d_j)\exp(\mathbf{x}_j^\top\boldsymbol\beta) - d_j\,\mathbf{x}_j^\top\boldsymbol\beta \bigr] \ +\ \lambda \, \lVert \boldsymbol\Psi \boldsymbol\beta \rVert_1 ,\]

the intercept left unpenalized. The first-order conditions are the covariate balancing equations \(\frac{1}{n}\sum_j (d_j - (1-d_j) \exp(\mathbf{x}_j^\top\boldsymbol\beta))\,\mathbf{x}_j = \mathbf{0}\); the unpenalized intercept condition is \(\sum_j W_j = 1\). The penalty level \(\lambda = c\,\Phi^{-1}(1 - g/2p)/\sqrt{n}\) is the Belloni–Chernozhukov– Hansen choice with data-driven loadings \(\boldsymbol\Psi\) iterated to convergence (mlsynth solves it by proximal-gradient descent, the authors by OWL-QN; the two agree to five decimals).

Step 2: immunizing outcome model. For each post period, regress the outcome on the covariates over the weighted control pool by a weighted-least-squares Lasso,

\[\widehat{\boldsymbol\mu} = \operatorname*{arg\,min}_{\boldsymbol\mu} \ \frac{1}{n}\sum_j W_j\,(y_{jt} - \mathbf{x}_j^\top\boldsymbol\mu)^2 \ +\ \lambda' \lVert \boldsymbol\mu \rVert_1 .\]

Step 3: immunized ATT and its standard error. The effect is the sample analogue of the orthogonal moment,

\[\widehat\theta = \frac{1}{\pi}\,\frac{1}{n}\sum_j \bigl(d_j - (1 - d_j)\exp(\mathbf{x}_j^\top\widehat{\boldsymbol\beta})\bigr) \bigl(y_{jt} - \mathbf{x}_j^\top\widehat{\boldsymbol\mu}\bigr),\]

with influence function \(\psi_j = (d_j - (1-d_j) \exp(\mathbf{x}_j^\top\widehat{\boldsymbol\beta}))(y_{jt} - \mathbf{x}_j^\top\widehat{\boldsymbol\mu}) - d_j\widehat\theta\) and analytic standard error \(\widehat\sigma = \sqrt{\tfrac{1}{n}\sum_j \psi_j^2}\,/\, (\pi\sqrt{n})\). Setting \(\boldsymbol\mu = \mathbf{0}\) (immunity=False) gives the non-immunized plug-in, which is consistent only if the balancing model is right.

The post-treatment ATT reported by fit averages \(\widehat\theta\) over the post periods; its interval combines the per-period standard errors treating them as independent, which is optimistic under serially correlated post residuals.

The balance-validity guard#

Assumption 2 is checkable from the fit: a valid tilting synthetic control has \(\sum_j W_j = 1\). BEAST computes \(\lvert \sum_j W_j - 1 \rvert\) and, if it exceeds balance_tol, raises MlsynthEstimationError rather than returning weights that are not a synthetic control. This is the concrete signature of the over-saturated regime: when the covariate set is as rich as the donor pool, the exponential tilt has no sparse balancing solution and the mass condition breaks. The guard turns that failure into an explicit error instead of a silently wrong estimate.

Example#

California’s Proposition 99 (Abadie, Diamond & Hainmueller 2010): California raises its cigarette tax in 1989; the donors are the other US states, the covariates are four economic predictors plus a few lagged sales.

import pandas as pd
from mlsynth import BEAST

url = ("https://raw.githubusercontent.com/jgreathouse9/mlsynth/main/"
       "basedata/augmented_cali_long.csv")
df = pd.read_csv(url)
df["treated"] = ((df.state == "California") & (df.year >= 1989)).astype(int)

res = BEAST({
    "df": df, "outcome": "cigsale", "treat": "treated",
    "unitid": "state", "time": "year",
    "covariates": ["loginc", "p_cig", "pct15-24", "pc_beer"],
    "outcome_lags": [1975, 1980, 1988],
    "display_graphs": False,
}).fit()

print(f"ATT = {res.att:.2f}")                              # -22.42
print(res.inference.ci_lower, res.inference.ci_upper)      # ~[-23.2, -21.6]
det = res.inference.details
print("sum of weights =", round(det["sum_weights"], 4))    # 1.0  (valid SC)
print("covariates selected =", det["n_selected"])          # sparse
print("effect by 2000 =", round(float(det["tau"][-1]), 2)) # -31.51

The average post-1989 reduction is about 22 packs per capita, growing to roughly 32 by 2000 – the classic Proposition 99 result, here with an analytic interval that excludes zero. The balancing weights sum to one (a valid synthetic control) and the ℓ₁ calibration selects a sparse covariate set.

Verification#

BEAST is cross-validated against the authors’ own R code (Path: cross- validation). On the Proposition 99 panel, mlsynth’s Calibration-Lasso / immunizing-regression / immunized-ATT port reproduces a live run of jeremylhour/alternative-synthetic-control-sparsity (CalibrationLasso.R / OrthogonalityReg.R / LassoFISTA.R / ImmunizedATT.R) on the identical design matrix: the balancing coefficients match to five decimals, and the immunized ATT reproduces the R (mean \(-22.44\), matched to 0.02 packs; \(-31.51\) by 2000, to 0.006), with the largest per-year gap on the path about 0.15 packs and the standard errors matched to ~0.05. The durable check is benchmarks/cases/beast_prop99.py, run against the pinned R reference under benchmarks/reference/beast_prop99/:

python benchmarks/run_benchmarks.py --case beast_prop99

See BEAST – Immunized Doubly-Robust Synthetic Control (Proposition 99) for the demonstrate-first port story, the cell-by-cell comparison, and the finding that maps out BEAST’s operating envelope (why the over-rich augmented specification breaks the balancing).

References#

[BEAST]

Bléhaut, M., D’Haultfœuille, X., L’Hour, J., & Tsybakov, A. B. (2021). An alternative to synthetic control for models with many covariates under sparsity. arXiv:2005.12225. https://arxiv.org/abs/2005.12225

Core API#

BEAST: immunized doubly-robust synthetic control (Bléhaut et al. 2021).

A synthetic control whose donor weights come from an ℓ₁-penalised covariate balancing (exponential tilting) rather than an outcome-fitting simplex QP, and whose ATT is immunized by a Lasso outcome regression, making it doubly robust and asymptotically normal with an analytic standard error. Best suited to a sparse, informative covariate regime; a balance-validity guard rejects the over-saturated high-dimensional case the method is not built for.

Reference: Bléhaut, D’Haultfœuille, L’Hour & Tsybakov (2021), “An alternative to synthetic control for models with many covariates under sparsity” (arXiv 2005.12225); authors’ code jeremylhour/alternative-synthetic-control-sparsity.

class mlsynth.estimators.beast.BEAST(config: BEASTConfig | dict)#

Bases: object

Immunized doubly-robust synthetic control estimator.

Parameters:

config (BEASTConfig or dict) – Configuration. See mlsynth.utils.beast_helpers.config.BEASTConfig.

Examples

>>> from mlsynth import BEAST
>>> res = BEAST({"df": panel, "outcome": "cigsale", "treat": "treated",
...              "unitid": "state", "time": "year",
...              "covariates": ["loginc", "p_cig", "pct15-24", "pc_beer"],
...              "outcome_lags": [1975, 1980, 1988]}).fit()
>>> res.att
fit() → BaseEstimatorResults#

Estimate the immunized ATT and return standardized results.

Configuration#

class mlsynth.utils.beast_helpers.config.BEASTConfig(*, df: ~pandas.DataFrame, outcome: str, treat: str, unitid: str, time: str, display_graphs: bool = True, save: bool | str = False, counterfactual_color: ~typing.List[str] = <factory>, treated_color: str = 'black', plot: ~mlsynth.config_models.PlotConfig = <factory>, covariates: ~typing.List[str], outcome_lags: ~typing.List[~typing.Any] | None = None, c_cal: ~typing.Annotated[float, ~annotated_types.Gt(gt=0)] = 0.03, c_ort: ~typing.Annotated[float, ~annotated_types.Gt(gt=0)] = 0.3, immunity: bool = True, alpha: ~typing.Annotated[float, ~annotated_types.Gt(gt=0.0), ~annotated_types.Lt(lt=1.0)] = 0.05, balance_tol: ~typing.Annotated[float, ~annotated_types.Gt(gt=0)] = 0.05)#

Configuration for mlsynth.BEAST.

BEAST (Bléhaut, D’Haultfœuille, L’Hour & Tsybakov 2021) is a doubly-robust synthetic control: covariate-balancing (exponential-tilting) donor weights with an ℓ₁-penalised calibration, plus an immunizing outcome regression, giving an ATT that is asymptotically normal with an analytic standard error.

It targets a sparse, informative covariate regime (a handful of predictors truly drive selection). It is not built for over-saturated p ~ n sets, where the balancing degenerates; that is guarded by balance_tol.

alpha: float#
balance_tol: float#
c_cal: float#
c_ort: float#
covariates: List[str]#
immunity: bool#
model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid'}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

outcome_lags: List[Any] | None#

Helper Modules#

Data preparation – dataprep then the treated / donor covariate design matrix (covariate pre-means plus optional lagged outcomes, normalized).

Long-panel ingestion for BEAST (via mlsynth.utils.datautils.dataprep()).

mlsynth.utils.beast_helpers.setup.prepare_beast_inputs(config) → BEASTInputs#

Build BEASTInputs from the config’s long DataFrame.

X is [constant, pre-treatment covariate means, lagged outcomes] per unit (covariates raw, as in the reference: the calibration’s penalty loadings absorb scale). d is the single-treated indicator aligned to the outcome matrix’s unit order.

Core numerics: the Calibration-Lasso balancing weights, the immunizing WLS-Lasso outcome model, the immunized ATT with its analytic standard error, and the balance-validity check.

Core numerics for the BEAST immunized doubly-robust synthetic control.

Bléhaut, D’Haultfœuille, L’Hour & Tsybakov (2021), “An alternative to synthetic control for models with many covariates under sparsity” (arXiv 2005.12225). A NumPy port of the authors’ R (CalibrationLasso.R / OrthogonalityReg.R / LassoFISTA.R / ImmunizedATT.R in jeremylhour/alternative-synthetic- control-sparsity), validated against a live run of that code on the Abadie, Diamond & Hainmueller (2010) Proposition 99 panel: the calibration weights match to five decimals and the immunized ATT path to ~0.15 packs (mean to ~0.02).

Three steps:

  • calibration_lasso() – covariate-balancing (exponential-tilting) weights W = (1 - d) exp(X beta) / n_1 with an ℓ₁-penalised calibration (Neyman-orthogonal, sparse in high dimensions);

  • orthogonality_reg() – a weighted-least-squares Lasso outcome model mu (the immunizing regression of the outcome on X for the controls);

  • immunized_att() – the doubly-robust ATT theta = mean[(d - (1 - d) exp(X beta))(y - X mu)] / pi with a closed-form asymptotic standard error.

The calibration is a valid synthetic control only when the weights balance the constant, i.e. sum(W) == 1; balance_ok() exposes that check (the exact signal that flags the over-saturated high-dimensional regime the method is not built for).

mlsynth.utils.beast_helpers.estimator.balance_ok(weights: ndarray, tol: float = 0.05) → bool#

True when the balancing weights are valid (|sum(W) - 1| <= tol).

mlsynth.utils.beast_helpers.estimator.balance_weights(d: ndarray, X: ndarray, beta: ndarray) → ndarray#

Donor balancing weights W = (1 - d) exp(X beta) / n_1 (should sum to 1).

mlsynth.utils.beast_helpers.estimator.calibration_lasso(d: ndarray, X: ndarray, c: float = 0.03, max_iter_pen: int = 10000, tol_psi: float = 0.01) → ndarray#

ℓ₁-penalised covariate-balancing (calibration) coefficients beta.

Minimises mean[(1 - d) exp(X beta) - d (X beta)] + lambda ||Psi beta||_1 (the intercept, column 0, is unpenalised) by proximal-gradient descent, with the penalty loadings Psi iterated as in the reference. The first column of X must be a constant.

mlsynth.utils.beast_helpers.estimator.immunized_att(y: ndarray, d: ndarray, X: ndarray, beta: ndarray, mu: ndarray | None = None)#

Immunized doubly-robust ATT theta and its asymptotic standard error.

theta = mean[(d - (1 - d) exp(X beta)) eps] / pi with eps = y - X mu (or eps = y when mu is None, the non-immunized plug-in).

mlsynth.utils.beast_helpers.estimator.orthogonality_reg(y: ndarray, d: ndarray, X: ndarray, beta: ndarray, c: float = 0.3, max_iter_pen: int = 1000, tol_psi: float = 0.01) → ndarray#

Weighted-Lasso immunizing outcome model mu (regress y on X, control weights W = (1 - d) exp(X beta)).

End-to-end orchestration: fit the immunized ATT path, apply the balance guard, and assemble the standardized results.

BEAST driver: fit the immunized ATT path and assemble standardized results.

mlsynth.utils.beast_helpers.pipeline.run_beast(config) → BaseEstimatorResults#

Fit BEAST and return BaseEstimatorResults.