Synthetic Control with Differencing (SCD)#

Overview#

SCD is a synthetic control estimator for settings where the treated and control “units” are not single aggregated series but groups observed through repeated cross-sections of individuals – states in a labour survey, regions in a repeated household survey, markets sampled afresh each period. It is the method of Rincon & Song (2026), with the repeated-cross-section inference of Canen & Song (2025).

The idea is a two-step one. First collapse the microdata to a survey-weighted mean per group and period, so that a panel of group means emerges from the individual records. Then run a synthetic control on those means – but on their within-group differences rather than their levels. Differencing off a pre-period baseline removes each group’s fixed level, so the donor weights are fit on de-trended trajectories; this is the “D” in SCD, and it nests classical synthetic control (no differencing) and a difference-in-differences style baseline (difference off the last pre-period) as the two ends of a dial.

The payoff of keeping the microdata is inference. Classical synthetic control has one treated series and draws its uncertainty from permuting the handful of donors – a coarse grid with no power below one over the donor count. SCD instead recycles the individual observations as influence functions, giving a genuine \(\sqrt{n}\) (number of individuals), fixed-\(T\) standard-error band on the per-period effect, together with a confidence set for the counterfactuals the data cannot rule out. The result is naturally read as an event study.

When to use this estimator#

The treated and control units are groups, and you observe individuals within each group and period (microdata / repeated cross-sections), optionally with survey weights – not a single number per group-period.
You want a per-period effect path with honest confidence bands, and the number of donor groups is too small for credible permutation inference.
A group-level fixed effect (a persistent level gap between the treated group and the donors) is plausible, so differencing it out before fitting improves the match.

If you only have one aggregated outcome per unit and period, use the aggregate synthetic-control estimators (Vanilla Synthetic Control (VanillaSC), Cluster Synthetic Controls (CLUSTERSC)) – there are no individuals to form the \(\sqrt{n}\) band. If you care about the whole counterfactual distribution rather than the group mean, use Distributional Synthetic Control (DSC).

Notation#

Index groups by \(j\), with \(j = 0\) the treated group and \(\mathcal{N}_0 \coloneqq \{1, \dots, K\}\) the donor pool of \(K\) groups. Time runs over \(t \in \{1, \dots, T\}\), 1-indexed; treatment takes effect at \(T^\star = T_0 + 1\), splitting time into the pre-period \(\mathcal{T}_1 \coloneqq \{t \le T_0\}\) and the post-period \(\mathcal{T}_2 \coloneqq \{t \ge T^\star\}\).

Within group \(j\) at period \(t\) we observe individuals \(i\) with outcome \(Y_{i}\) and survey weight \(\pi_{i}\). The survey-weighted group mean is

\[\widehat m_{j,t} \coloneqq \frac{\sum_i \pi_{i}\, Y_{i}\, \mathbf{1}\{G_i = j,\ t_i = t\}} {\sum_i \pi_{i}\, \mathbf{1}\{G_i = j,\ t_i = t\}} ,\]

with \(\widehat n_{j,t} \coloneqq \sum_i \pi_i \mathbf{1}\{G_i=j, t_i=t\}\) the weighted cell total and \(n\) the total number of individuals.

Differencing is set by a weight vector \(\boldsymbol{\lambda} \in \mathbb{R}^{T_0}\) over the pre-period. The baseline level of group \(j\) is \(\bar m_j \coloneqq \sum_{t \le T_0} \lambda_t\, \widehat m_{j,t}\), and the differenced series is \(\mu_{j,t} \coloneqq \widehat m_{j,t} - \bar m_j\). Three schemes are offered: did uses \(\boldsymbol{\lambda} = (0, \dots, 0, 1)\) (difference off the last pre-period), uniform uses \(\lambda_t = 1/T_0\) (off the pre-period average), and sc uses \(\boldsymbol{\lambda} = \mathbf{0}\) (no differencing – classical synthetic control on levels).

Donor weights are the simplex vector \(\mathbf{w} \in \Delta^{K-1}\) (\(w_j \ge 0\), \(\sum_j w_j = 1\)). The per-period effect is \(\widehat\theta_t \coloneqq \mu_{0,t} - \sum_{j=1}^K \mu_{j,t}\, w_j\), and the ATT is \(\widehat\tau \coloneqq |\mathcal{T}_2|^{-1} \sum_{t \in \mathcal{T}_2} \widehat\theta_t\). The significance level is \(\alpha\), split by a Bonferroni share \(\kappa\) for the weight confidence set.

Identifying assumptions#

Pre-treatment fit of the differenced means. There exist simplex weights \(\mathbf{w}\) under which the donors reproduce the treated group’s differenced pre-period path, \(\mu_{0,t} \approx \sum_j \mu_{j,t} w_j\) for \(t \in \mathcal{T}_1\).

Remark. As in every synthetic control, a good pre-fit is the empirical certificate of a credible counterfactual. Differencing changes what is being matched: with did or uniform the fit ignores persistent level gaps and targets the co-movement of trajectories.
No anticipation. Treatment has no effect before \(T^\star\), so the pre-period group means reflect the no-intervention path.

Remark. Date \(T^\star\) at the first plausible response, not the nominal policy date; a contaminated pre-period biases the extrapolation.
No interference and untreated donors (SUTVA at the group level). Treating group \(0\) changes no donor group’s outcomes, and every donor group is untreated throughout.

Remark. With grouped microdata this also rules out individuals sorting between the treated and donor groups in response to treatment.
Sampling within cells. Individuals are sampled (with weights \(\pi_i\)) within each group-period cell, and the cells are populated. The repeated-cross-section variance treats the periods as independent samples; individuals are not tracked across time.

Remark. This is the assumption that earns the \(\sqrt{n}\) asymptotics. It is what distinguishes SCD from an aggregate panel method: the uncertainty comes from sampling individuals into the group means, with \(T\) fixed.

Mathematical formulation#

Write the pre-period differencing as a linear operator. For a series \(x = (x_1, \dots, x_T)\) and pre-period weights \(\boldsymbol\lambda\), let \((\mathcal D_\lambda x)_t \coloneqq x_t - \sum_{s \le T_0} \lambda_s x_s\), so that \(\mu_{j,t} = (\mathcal D_\lambda \widehat m_{j,\cdot})_t\). The weights are fit by simplex least squares on the transformed pre-period,

\[\widehat{\mathbf{w}} = \operatorname*{argmin}_{\mathbf{w} \in \Delta^{K-1}} \; \sum_{t \le T_0} \Bigl( \mu_{0,t} - \sum_{j=1}^K \mu_{j,t}\, w_j \Bigr)^2 ,\]

and the effect path \(\widehat\theta_t = \mu_{0,t} - \sum_j \mu_{j,t} \widehat w_j\) is read off over all periods. Because the pre-period residuals are near zero by construction, the pre-period \(\widehat\theta_t\) trace the fit quality and the post-period \(\widehat\theta_t\) are the dynamic effects.

Differencing as an intercept. Because \(\mathcal D_\lambda\) subtracts a constant in \(t\) from each series, the fitted gap can be written on the raw means with an additive offset,

\[\widehat\theta_t = \Bigl( \widehat m_{0,t} - \sum_j \widehat w_j \widehat m_{j,t} \Bigr) - \alpha, \qquad \alpha = \bar m_0 - \sum_j \widehat w_j\, \bar m_j ,\]

so SCD is a synthetic control with an intercept \(\alpha\): it matches the donors to the treated group’s movement, not its level. The scheme sets how the intercept is chosen. Under uniform (\(\lambda_t = 1/T_0\)) both sides are demeaned by their pre-period averages, and by the Frisch-Waugh-Lovell theorem the resulting \(\widehat{\mathbf w}\) is exactly that of a synthetic control with a freely estimated (least-squares) intercept – profiling \(\alpha\) out of \(\min_{\mathbf w \in \Delta,\,\alpha} \sum_{t \le T_0}(\widehat m_{0,t} - \alpha - \sum_j w_j \widehat m_{j,t})^2\) returns the demeaned objective. Under did (\(\boldsymbol\lambda = (0, \dots, 0, 1)\)) the intercept is instead pinned so the counterfactual meets the treated group exactly at \(t = T_0\) (the difference-in-differences normalisation), which is not the pre-period error-minimising \(\alpha\). Under sc (\(\boldsymbol\lambda = \mathbf 0\)) the intercept is \(\alpha = 0\): classical synthetic control on levels.

Generalized matching condition#

Differencing is what lets SCD tolerate a persistent level gap between the treated group and the donors, and it does so by weakening the matching requirement. Suppose the group means follow a linear factor (interactive fixed-effects) model

\[\widehat m_{j,t} = a_j + \mathbf F_t^\top \boldsymbol\Gamma_j + \varepsilon_{j,t},\]

with a group fixed effect \(a_j\), common factors \(\mathbf F_t\), group loadings \(\boldsymbol\Gamma_j\), and mean-zero sampling noise \(\varepsilon_{j,t}\). Applying \(\mathcal D_\lambda\) with \(\sum_{s \le T_0} \lambda_s = 1\) cancels the fixed effect,

\[\mu_{j,t} = (\mathbf F_t - \bar{\mathbf F}_\lambda)^\top \boldsymbol\Gamma_j + \widetilde\varepsilon_{j,t}, \qquad \bar{\mathbf F}_\lambda = \sum_{s \le T_0} \lambda_s \mathbf F_s ,\]

so the pre-period matching \(\mu_{0,t} = \sum_j w_j \mu_{j,t}\) reduces to a condition on loadings alone. The generalized matching condition is that the treated group’s loading lies in the convex hull of the donors’,

\[\boldsymbol\Gamma_0 = \sum_{j=1}^K w_j\, \boldsymbol\Gamma_j, \qquad \mathbf w \in \Delta^{K-1} .\]

Under it the no-treatment gap \(\mu_{0,t} - \sum_j w_j \mu_{j,t}\) is mean zero for every \(\boldsymbol\lambda\) with \(\sum_s \lambda_s = 1\), so \(\widehat\theta_t\) is consistent for the post-period treatment effect and the choice between did and uniform is a finite-sample efficiency choice, not an identification one. The three schemes are the two ends and the middle of a single dial:

did and uniform (\(\sum_s \lambda_s = 1\)) need only loading matching \(\boldsymbol\Gamma_0 = \sum_j w_j \boldsymbol\Gamma_j\); the fixed effect \(a_j\) is differenced away and may differ freely across groups;
sc (\(\boldsymbol\lambda = \mathbf 0\)) does not remove the fixed effect, so classical synthetic control additionally requires level matching \(a_0 = \sum_j w_j a_j\) – the stronger demand that the donors reproduce the treated group’s level, not only its factor structure.

Differencing therefore generalises the matching requirement: it buys identification under the weaker loadings-only condition, at the price of leaving the level unidentified – which is immaterial, because the reported object is itself a difference.

Inference#

Pointwise variance. Each individual enters its group mean through the influence function

\[\psi^\star_{i,t} = \pi_{i}\, \frac{n}{\widehat n_{G_i,t}}\, \bigl( Y_{i} - \widehat m_{G_i,t} \bigr),\]

and its contribution to \(\widehat\theta_t = \mu_{0,t} - \sum_j \mu_{j,t} w_j\) carries the sign and weight of its group, \(\psi^\theta_{i,t} = c_{G_i}\, \psi^\star_{i,t}\) with \(c_0 = 1\) (treated) and \(c_j = -\widehat w_j\) (donor \(j\)). Writing \(S_t \coloneqq \sum_{i:\,t_i = t} (\psi^\theta_{i,t})^2\) for the per-period sum and \(\delta_t = \lambda_t\) for \(t \le T_0\) (and \(0\) after), the repeated-cross-section pointwise variance is

\[\widehat\sigma^2_t = \frac{1}{n}\Bigl[ \sum_{s \le T_0} \delta_s^2\, S_s + (1 - 2\delta_t)\, S_t \Bigr], \qquad \widehat{\mathrm{se}}_t = \sqrt{\widehat\sigma^2_t / n} .\]

The first term is the sampling noise of the differencing baseline \(\bar m_j\); the second is the period’s own noise (at a post-period, \(\delta_t = 0\) and only \(S_t\) remains). Because the group means are \(\sqrt{n}\)-consistent with \(T\) fixed, this is a genuine standard-error band, not a permutation p-value on a coarse donor grid.

Weight confidence set. Let \(\mathbf G \in \mathbb R^{T_0 \times K}\) hold the differenced donor means \(\mu_{j,t}\) and \(\mathbf g\) the treated column, and form the pre-period moment operators

\[\widehat H = \tfrac{1}{T_0}\, \mathbf G^\top \mathbf G, \qquad \widehat h = \tfrac{1}{T_0}\, \mathbf G^\top \mathbf g, \qquad \varphi(\mathbf w) = \widehat H \mathbf w - \widehat h ,\]

so \(\varphi(\widehat{\mathbf w}) = \mathbf 0\) at the interior optimum. The weights carry a \((K-1)\)-dimensional asymptotic variance

\[\widehat V = \frac{1}{T_0} \sum_{t \le T_0} \widehat v_t\, B_2^\top M_t B_2, \qquad \widehat v_t = S_t / n ,\]

with \(\mu_t\) the \(t\)-th row of \(\mathbf G\), \(\bar\mu\) its pre-period mean,

\[M_t = \tfrac{1}{T_0}\, \mu_t \mu_t^\top - \lambda_t\bigl( \bar\mu\, \mu_t^\top + \mu_t\, \bar\mu^\top \bigr) + T_0\, \lambda_t^2\, \bar\mu\, \bar\mu^\top ,\]

and \(B_2 \in \mathbb R^{K \times (K-1)}\) an orthonormal basis of the sum-zero subspace \(\{\mathbf v : \mathbf 1^\top \mathbf v = 0\}\) (the non-null eigenvectors of the centring matrix \(I_K - \tfrac1K \mathbf 1 \mathbf 1^\top\)), which absorbs the degree of freedom the simplex constraint removes. Set \(P = B_2 \widehat V^{-1} B_2^\top\).

A candidate \(\mathbf w\) is in the level-\((1 - \kappa)\) set when its projected moment is small. Project \(\varphi(\mathbf w)\) onto the cone the active simplex constraints allow,

\[\widehat r = \operatorname*{argmin}_{r \ge 0,\; \mathbf w^\top r = 0} \; (\varphi - r)^\top P\, (\varphi - r) ,\]

form the statistic and its data-dependent degrees of freedom

\[T(\mathbf w) = n\, (\varphi - \widehat r)^\top P\, (\varphi - \widehat r), \qquad d(\mathbf w) = \bigl\lvert \{ j : w_j = 0,\; \lvert [P(\varphi - \widehat r)]_j \rvert < \text{tol} \} \bigr\rvert ,\]

and accept \(\mathbf w \in \mathcal C_{1-\kappa}\) iff \(T(\mathbf w) \le \chi^2_{1-\kappa}(K - 1 - d(\mathbf w))\). For an interior \(\mathbf w\) (all \(w_j > 0\)) the constraints are slack, \(\widehat r = 0\), and the test is the plain quadratic form on \(K - 1\) degrees of freedom – the reason a dense grid is swept in one batched form and only the sparse candidates need the projection.

Band. With \(z = \Phi^{-1}\!\bigl(1 - (\alpha - \kappa)/2\bigr)\) the period-\(t\) band runs over the accepted set,

\[\Bigl[ \min_{\mathbf w \in \mathcal C_{1-\kappa}} \bigl( \mu_{0,t} - \textstyle\sum_j w_j \mu_{j,t} \bigr) - z\, \widehat{\mathrm{se}}_t,\;\; \max_{\mathbf w \in \mathcal C_{1-\kappa}} \bigl( \mu_{0,t} - \textstyle\sum_j w_j \mu_{j,t} \bigr) + z\, \widehat{\mathrm{se}}_t \Bigr] ,\]

so the width combines weight uncertainty (the spread of counterfactuals the data cannot reject) with sampling noise (\(z\, \widehat{\mathrm{se}}_t\)), and the two shares \(\kappa\) and \(\alpha - \kappa\) add to \(\alpha\) by Bonferroni.

A note on the RC standard error. The published reference code builds the treated/donor multiplier \(c_{G_i}\) with an indexing expression that, in R, silently drops the treated rows and misaligns the donor weights; mlsynth uses the corrected length-\((K+1)\) lookup (matching the reference’s own weight-variance routine \(\widehat V\)). The point estimator, the weight variance, and the confidence set are unaffected; only the pointwise standard error differs, and mlsynth’s value is cross-validated against a corrected base-R reference.

Example#

import pandas as pd
from mlsynth import SCD

df = pd.read_parquet(
    "https://raw.githubusercontent.com/jgreathouse9/mlsynth/"
    "refs/heads/main/basedata/cps_lawa_arizona.parquet"
)
# treatment is applied at the group-period level (Arizona from period 55)
df["treat"] = ((df["state_name"] == "Arizona") & (df["D"] == 1)).astype(int)

res = SCD({
    "df":          df,
    "outcome":     "wklyearn",     # individual log weekly earnings
    "treat":       "treat",
    "unitid":      "state_name",   # the group
    "time":        "period",
    "weight_col":  "weight",       # survey weights (optional)
    "differencing": "did",         # "did", "uniform", or "sc"
    "compute_inference": True,
    "alpha":       0.10,
    "kappa":       0.05,
    "display_graphs": False,
}).fit()

print(res.att)                                  # post-period mean effect
print(res.time_series.estimated_gap)            # the effect path (event study)
det = res.inference.details
print(det["lower"], det["upper"])               # per-period confidence band
print(det["se"])                                # RC pointwise standard errors

Verification#

SCD is cross-validated, value for value, against a from-scratch base-R reproduction of the estimator (point weights, effect path, corrected RC standard error, weight-variance trace, and confidence-set membership decisions) on the Arizona LAWA CPS extract, for all three differencing schemes. The upstream package is GPL, so the method is reproduced on public-domain CPS microdata rather than vendored. Every quantity matches to solver tolerance (\(\sim 10^{-9}\)). See the durable benchmark case benchmarks/cases/scd_cps.py and the SCD – Synthetic Control with Differencing (Arizona LAWA, CPS microdata) page.

Core API#

Synthetic Control with Differencing (SCD) estimator.

SCD (Rincon & Song 2026, “Synthetic Control with Differencing”, arXiv:2510.26106; repeated-cross-section inference from Canen & Song 2025) targets settings where the treated and control “units” are groups observed through repeated cross-sections of individuals – states in a labour survey, say – rather than a single aggregated series each.

The estimator collapses the microdata to survey-weighted group means, applies a within-group differencing (the “D”: off the last pre-period, off the pre-period average, or none), and fits a simplex synthetic control on the differenced pre-period. It reports the per-period effect path as an event study. Because the group means are estimated from individuals, the individual rows re-enter as influence functions to give \(\sqrt{n}\) (number of individuals), fixed-T confidence bands – a genuine standard-error band, together with a weight confidence set for the counterfactuals the data cannot rule out – rather than the coarse donor-permutation inference of classical SCM.

Data requirements#

SCD operates on grouped microdata: one row per individual observation, with an optional survey-weight column. The input is treated as repeated cross-sections (individuals are not tracked across periods). Treatment is applied at the unit-time level via the treat column.

Output#

SCD.fit() returns a BaseEstimatorResults whose estimated_gap is the effect path \(\hat\theta_t\), att its post-period mean, and (when compute_inference) whose inference.details carry the per-period lower/upper bands and the standard errors.

class mlsynth.estimators.scd.SCD(config: SCDConfig | dict)#

Bases: object

Synthetic Control with Differencing estimator.

Parameters:: config (SCDConfig or dict) – Configuration object. See mlsynth.config_models.SCDConfig.

fit() → BaseEstimatorResults#: Run SCD and return BaseEstimatorResults.

Configuration#

class mlsynth.utils.scd_helpers.config.SCDConfig(*, df: ~pandas.DataFrame, outcome: str, treat: str, unitid: str, time: str, display_graphs: bool = True, save: bool | str = False, counterfactual_color: ~typing.List[str] = <factory>, treated_color: str = 'black', plot: ~mlsynth.config_models.PlotConfig = <factory>, differencing: ~typing.Literal['did', 'uniform', 'sc'] = 'did', weight_col: str | None = None, compute_inference: bool = True, alpha: ~typing.Annotated[float, ~annotated_types.Gt(gt=0.0), ~annotated_types.Lt(lt=1.0)] = 0.1, kappa: ~typing.Annotated[float, ~annotated_types.Gt(gt=0.0), ~annotated_types.Lt(lt=1.0)] = 0.05, n_grid: ~typing.Annotated[int, ~annotated_types.Ge(ge=2)] = 3000, grid_radius: ~typing.Annotated[float, ~annotated_types.Gt(gt=0)] = 0.05, tolerance: ~typing.Annotated[float, ~annotated_types.Gt(gt=0)] = 1e-06, random_state: int = 0)#

Bases: BaseEstimatorConfig

Configuration for Synthetic Control with Differencing (SCD).

SCD (Rincon & Song 2026, arXiv:2510.26106; repeated-cross-section inference from Canen & Song 2025) fits a simplex synthetic control on within-group-differenced survey-weighted group means, and reports a per-period effect path with confidence bands built from individual influence functions (a \(\sqrt{n}\), fixed-T asymptotic).

Unlike the aggregate-panel estimators, SCD expects grouped microdata: each (unit, time) cell carries many individual observations, supplied as one row per individual, optionally with a survey weight. Treatment is still applied at the unit-time level via the treat column.

Parameters:

differencing ({“did”, “uniform”, “sc”}) – Within-group differencing scheme (the “D” in SCD). "did" (default) differences every group series off its last pre-period mean; "uniform" differences off the pre-period average; "sc" applies no differencing (classical synthetic control on levels).
weight_col (str, optional) – Column with individual survey weights used to form the group means. If None the group means are unweighted (equal weight per row).
compute_inference (bool) – If True (default), compute the repeated-cross-section variance and the weight confidence set, producing per-period confidence bands. If False, only the point estimator runs.
alpha (float) – Overall (one minus coverage) level for the confidence bands. Default 0.10 (90% bands).
kappa (float) – Bonferroni split allotted to the weight confidence set; the pointwise term uses the remaining alpha - kappa. Must satisfy 0 < kappa < alpha. Default 0.05.
n_grid (int) – Number of candidate weight vectors sampled per family (a Gaussian cloud around the fitted weights and a Dirichlet cloud) when sweeping the weight confidence set. Default 3000.
grid_radius (float) – Standard deviation of the Gaussian cloud around the fitted weights. Default 0.05.
tolerance (float) – Numerical tolerance for the confidence-set active-constraint count. Default 1e-6.
random_state (int) – Seed for the confidence-set grid sampler. Default 0.

alpha: float#

compute_inference: bool#

differencing: Literal['did', 'uniform', 'sc']#

grid_radius: float#

kappa: float#

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid'}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

n_grid: int#

random_state: int#

tolerance: float#

weight_col: str | None#

Synthetic Control with Differencing (SCD)

Contents