The BHF Model in Python with svy-sae

Unit-level EBLUP of district means

Tutorials
Unit-level
EBLUP
Python
Fit the Battese-Harter-Fuller (BHF) unit-level model in Python with svy-sae: EBLUP of district means from household records and census covariates, REML or ML, analytical and bootstrap MSE, and out-of-sample districts.
Author

Mamadou S. Diallo, Ph.D.

Modified

November 8, 2026

Keywords

Battese-Harter-Fuller model, BHF model Python, unit-level small area estimation, EBLUP Python, nested error regression model, bootstrap MSE small area

WarningAbout these examples

This tutorial illustrates the svy-sae API; it is not a rigorous small area analysis. The models are kept simple so each step stays readable, and in many places, from the choice of covariates to the variance model, better choices could be made to improve them.

The Battese–Harter–Fuller (BHF) model (Battese, Harter, and Fuller 1988), or nested error regression model, fits the survey records directly. Each household’s outcome is a regression on its covariates plus an effect shared by its district:

\[ y_{ij} = x_{ij}'\beta + u_i + e_{ij}, \qquad u_i \sim N(0, \sigma^2_u), \quad e_{ij} \sim N(0, \sigma^2_e), \]

for household \(j\) of district \(i\). The EBLUP of the district mean adds the predicted area effect to the regression at the census mean of the covariates, \(\bar X_i\):

\[ \hat{\bar Y}_i = \bar X_i'\hat\beta + \hat u_i, \qquad \hat u_i = \gamma_i\,(\bar y_i - \bar x_i'\hat\beta), \qquad \gamma_i = \frac{\hat\sigma^2_u}{\hat\sigma^2_u + \hat\sigma^2_e/n_i}, \]

where \(\bar y_i\) and \(\bar x_i\) are the means of the \(n_i\) sampled households. The factor \(\gamma_i\) shrinks the district’s mean residual toward zero, the more so the smaller its sample. This is why census means are enough. It treats the sampled households as a negligible share of the district, as they are here: about 136 of 41,000 on average.

This tutorial demonstrates how to fit the model with UnitLevel.eblup() to estimate district mean expenditure per person, read its results, predict districts outside the sample, and measure uncertainty with the analytical and bootstrap MSE.

Setting up the inputs

We continue with the household records and census covariates from Preparing unit-level inputs:

Rebuild the inputs from Preparing unit-level inputs
import polars as pl
import svy

import svy_sae as sae

yes_no = ["electricity", "car", "refrigerator", "tv", "cell", "bank"]
covariates = [
    "urban",
    "piped_water",
    "flush_toilet",
    *yes_no,
    "hhsize",
    "rooms",
]


def household_covariates(frame):
    return frame.with_columns(
        (pl.col("tot_exp") / pl.col("hhsize")).alias("exp_pp"),
        (pl.col("urbrur") == "Urban").cast(pl.Float64).alias("urban"),
        (pl.col("piped_water") != "No piped water")
        .cast(pl.Float64)
        .alias("piped_water"),
        (pl.col("flush_toilet") == "Flush toilet")
        .cast(pl.Float64)
        .alias("flush_toilet"),
        *[(pl.col(c) == "Yes").cast(pl.Float64).alias(c) for c in yes_no],
        pl.col("hhsize").cast(pl.Float64),
        pl.col("rooms").cast(pl.Float64),
    )


hld_data = household_covariates(svy.datasets.load("hld_sample_wb_2023").data)
hld_census = household_covariates(svy.datasets.load("hld_pop_wb_2023").data)

census_units = hld_census.select("geo1", "geo2", *covariates)
census_means = (
    hld_census.group_by("geo2")
    .agg(*[pl.col(c).mean() for c in covariates], pl.len().alias("n_hh"))
    .sort("geo2")
)

truth = (
    hld_census.group_by("geo2")
    .agg(
        pl.col("exp_pp").mean().alias("true_mean"),
        (pl.col("exp_pp") < 1800).mean().alias("true_rate"),
    )
    .sort("geo2")
)

Fitting the model

UnitLevel takes the survey records. eblup() takes the outcome, the covariates, the district key, and the population data. With the census means, pop_size names the number of households:

model = sae.UnitLevel(hld_data)

fit = model.eblup(
    y="exp_pp",
    x=covariates,
    area="geo2",
    pop_data=census_means,
    pop_size="n_hh",
)

print(fit)
╭──────────────── SAE: BHF (EBLUP) - MSE: Analytical - Unit Level ────────────────╮
│ Method     REML  Log Likelihood  -71416.2426                                    │
│ No. Areas    59  AIC             142860.4853                                    │
│ No. Obs    8000  BIC             142958.3060                                    │
│                  KIC             142874.4853                                    │
│                                                                                 │
│                                                                                 │
│  Term            Estimate   Std.Err        z   p-value      [0.025      0.975]  │
│  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  │
│  Intercept      3221.0084   93.6050    34.41    0.0000   3037.5426   3404.4743  │
│  urban           621.4706   64.5135     9.63    0.0000    495.0240    747.9171  │
│  piped_water    -173.5577   49.9744    -3.47    0.0005   -271.5075    -75.6080  │
│  flush_toilet    689.0086   52.2410    13.19    0.0000    586.6161    791.4010  │
│  electricity    -219.0911   74.1058    -2.96    0.0031   -364.3384    -73.8437  │
│  car            1949.3656   56.8769    34.27    0.0000   1837.8869   2060.8443  │
│  refrigerator    483.2891   61.8377     7.82    0.0000    362.0872    604.4909  │
│  tv              249.7701   66.2702     3.77    0.0002    119.8806    379.6596  │
│  cell            960.2579   52.1781    18.40    0.0000    857.9889   1062.5270  │
│  bank           -117.4160   68.5322    -1.71    0.0867   -251.7392     16.9071  │
│  hhsize         -594.0049   10.4648   -56.76    0.0000   -614.5159   -573.4939  │
│  rooms           340.9177   14.0264    24.31    0.0000    313.4259    368.4095  │
│                                                                                 │
│                                                                                 │
│ Variance components: area variance: 148686; residual variance: 3.32915e+06      │
│                                                                                 │
│ Findings:                                                                       │
│ - Area effects not normal                                                       │
│ - 19 influential areas                                                          │
╰─────────────────────────────────────────────────────────────────────────────────╯

The model is fitted to the 8,000 survey households and predicts all 61 districts of the census.

Census means or census records

The EBLUP of a mean is linear in the covariates, so the district means of the census are enough. Passing the census records instead, without pop_size, gives the same estimates:

fit_records = model.eblup(
    y="exp_pp", x=covariates, area="geo2", pop_data=census_units
)

difference = (
    fit.to_polars("predictions")
    .join(
        fit_records.to_polars("predictions").select(
            "area", pl.col("pred").alias("records")
        ),
        on="area",
    )
    .select((pl.col("pred") - pl.col("records")).abs().max())
    .item()
)

print(f"Largest difference: {difference:.2e}")
Largest difference: 0.00e+00

Reading the results

Fixed effects, with standard errors and confidence intervals:

fit.to_polars("fixed")
shape: (12, 7)
term est se t_score p_value lci uci
str f64 f64 f64 f64 f64 f64
"Intercept" 3221.008445 93.605018 34.410639 0.0 3037.542611 3404.474279
"urban" 621.470581 64.51355 9.633179 0.0 495.024023 747.917139
"piped_water" -173.557733 49.974354 -3.472936 0.000515 -271.507467 -75.608
"flush_toilet" 689.008565 52.241042 13.189028 0.0 586.616123 791.401007
"electricity" -219.091064 74.105807 -2.956463 0.003112 -364.338445 -73.843683
"car" 1949.365577 56.876879 34.273428 0.0 1837.886896 2060.844259
"refrigerator" 483.289073 61.837665 7.815448 5.5511e-15 362.08725 604.490897
"tv" 249.770099 66.27017 3.768967 0.000164 119.880566 379.659632
"cell" 960.257915 52.17809 18.40347 0.0 857.988859 1062.526971
"bank" -117.416022 68.532228 -1.713296 0.086658 -251.739189 16.907145
"hhsize" -594.004901 10.464816 -56.762097 0.0 -614.515941 -573.493862
"rooms" 340.917726 14.026426 24.305387 0.0 313.425931 368.409522

Variance components, the area variance \(\sigma^2_u\) and the residual variance \(\sigma^2_e\):

fit.to_polars("components")
shape: (2, 4)
name kind est se
str str f64 null
"area" "variance" 148686.240375 null
"residual" "variance" 3.3292e6 null

Their standard errors are not estimated for unit-level models, and are reported as missing. The area variance is small next to the residual one: most of the variation in expenditure is between households of the same district, not between districts.

Predictions

pred = fit.to_polars("predictions")

pred.select("area", "pred", "mse", "cv", "lci", "uci", "in_sample").head()
shape: (5, 7)
area pred mse cv lci uci in_sample
str f64 f64 f64 f64 f64 bool
"geo_01_01" 4952.333874 12408.489589 0.022493 4734.002809 5170.664938 true
"geo_01_02" 4284.671918 47510.398354 0.050872 3857.453068 4711.890769 true
"geo_01_03" 4312.707174 10468.771801 0.023725 4112.165823 4513.248526 true
"geo_01_04" 4148.458611 35305.101023 0.045293 3780.181439 4516.735784 true
"geo_01_05" 4753.730365 27850.306259 0.035106 4426.637509 5080.823222 true

Districts outside the sample

The two districts the survey did not reach have no sampled households and no area effect to predict:

pred.filter(~pl.col("in_sample")).select("area", "pred", "mse", "cv")
shape: (2, 4)
area pred mse cv
str f64 f64 f64
"geo_02_02" 2404.290979 152403.04181 0.162371
"geo_09_03" 3148.453447 151977.354439 0.12382

Their estimate is the synthetic \(\bar X_i'\hat\beta\) from their census means, with an MSE that includes \(\sigma^2_u\).

Estimation options

Table 1: Options of eblup()
Argument Values Description
method "reml" (default), "ml" How the variance components are estimated
mse "analytical" (default) Prasad–Rao approximation
"bootstrap" Parametric bootstrap with n_reps replicates (default 1,000); rstate seeds it
intercept True (default) Include an intercept; x=[] with an intercept fits a model without covariates
pop_size column of pop_data Number of units per area when pop_data holds area means

refit() reruns a fit with the arguments you change. Compare the variance components by REML and ML:

fit_ml = fit.refit(method="ml")

pl.concat(
    [
        fit.to_polars("components").select(
            pl.lit("REML").alias("method"), "name", "est"
        ),
        fit_ml.to_polars("components").select(
            pl.lit("ML").alias("method"), "name", "est"
        ),
    ]
)
shape: (4, 3)
method name est
str str f64
"REML" "area" 148686.240375
"REML" "residual" 3.3292e6
"ML" "area" 145019.821963
"ML" "residual" 3.3246e6

With 8,000 households and twelve fixed effects, the two hardly differ.

Refit with the bootstrap MSE:

fit_boot = fit.refit(mse="bootstrap", n_reps=1000, rstate=12345)

pred.select("area", "mse").join(
    fit_boot.to_polars("predictions").select(
        "area", pl.col("mse").alias("mse_boot")
    ),
    on="area",
).select(
    (pl.col("mse_boot") / pl.col("mse")).alias("bootstrap / analytical")
).describe()
shape: (9, 2)
statistic bootstrap / analytical
str f64
"count" 61.0
"null_count" 0.0
"mean" 0.994447
"std" 0.046958
"min" 0.859147
"25%" 0.962847
"50%" 1.006726
"75%" 1.026974
"max" 1.082905

The two agree closely, within the Monte Carlo error of the bootstrap.

TipReplicates are cheap here

BHF bootstrap replicates run in batches and compilation dominates the cost, so 1,000 replicates take about as long as 100. Use a generous count: the Monte Carlo error of the MSE is what more replicates reduce.

How close did we get?

Compare the direct estimates, a model without covariates, and the model with all eleven, against the true district means:

hld_design = svy.Design(stratum=("geo1", "urbrur"), psu="ea", wgt="hhweight")
hld_sample = svy.Sample(data=hld_data, design=hld_design)

direct = (
    hld_sample.estimation.mean(y="exp_pp", by="geo2")
    .to_polars()
    .select(
        pl.col("geo2").alias("area"),
        pl.col("est").alias("pred"),
        (pl.col("se") ** 2).alias("mse"),
        "lci",
        "uci",
    )
)

fit_base = model.eblup(
    y="exp_pp", x=[], area="geo2", pop_data=census_means, pop_size="n_hh"
)


def scores(name, frame):
    scored = frame.join(truth, left_on="area", right_on="geo2")
    error2 = (pl.col("pred") - pl.col("true_mean")) ** 2
    covered = (pl.col("true_mean") >= pl.col("lci")) & (
        pl.col("true_mean") <= pl.col("uci")
    )
    return scored.select(
        pl.lit(name).alias("estimates"),
        pl.len().alias("districts"),
        error2.mean().sqrt().alias("rmse"),
        pl.col("mse").mean().alias("mean mse"),
        error2.mean().alias("mean squared error"),
        covered.sum().alias("interval covers truth"),
    )


pl.concat(
    [
        scores("direct", direct),
        scores("BHF, no covariates", fit_base.to_polars("predictions")),
        scores("BHF, eleven covariates", pred),
    ]
)
shape: (3, 6)
estimates districts rmse mean mse mean squared error interval covers truth
str u32 f64 f64 f64 u32
"direct" 59 836.336741 465721.647721 699459.14369 49
"BHF, no covariates" 61 749.288835 100183.58477 561433.757995 28
"BHF, eleven covariates" 61 332.553882 31953.060347 110592.08452 40

The covariates cut the error of the direct estimates by more than half. Yet on the same districts the area-level Fay–Herriot model of Transformations does better, with an RMSE of 197, and the model’s MSE is well below the actual squared error: many intervals miss the true mean. The checks in the next tutorial show why: expenditure is far from normal around the regression, and the model assumes it is. The EBP tutorial fits the same model on a transformed scale.

Summary

This tutorial fitted the BHF model with UnitLevel.eblup():

  1. Population data as census means with pop_size, or census records, which give the same estimates
  2. Results: fixed effects, the area and residual variances, and predictions with MSE, CV, and intervals
  3. Districts outside the sample predicted from their census means
  4. Options: REML or ML, analytical or bootstrap MSE
  5. Accuracy against the truth, and an MSE that understates it when the model’s assumptions fail

Next steps

Check the model
Continue to Checking Unit-Level Models →

References

Battese, G E, R M Harter, and W A Fuller. 1988. “An Error-Components Model for Prediction of County Crop Areas Using Survey and Satellite Data.” J. Amer. Statist. Assoc. 83: 28–36.