View code
d = women.data
print(f"strata: {d['v022'].n_unique()}")
print(f"clusters: {d['v021'].n_unique()}")
print(f"women: {d.height:,}")
print(f"sum(wgt): {d['wgt'].sum():,.1f}")strata: 27
clusters: 797
women: 21,395
sum(wgt): 21,395.0svy on the 2024–25 Ethiopia DHS
Validation
Ethiopia
Global health
Reproducing the sampling error appendix of the 2024–25 Ethiopia Demographic and Health Survey — estimates, standard errors, and design effects — with Taylor linearisation in Python.
Keywords
DHS sampling errors, Ethiopia DHS 2024-25, Taylor linearization Python, design effect DEFT, complex survey analysis Python, stratified two-stage cluster sample, domain estimation, svy validation
Summary
TipTL;DR
The 2024–25 Ethiopia Demographic and Health Survey publishes a sampling error appendix — estimate, standard error, design effect, and confidence limits for about 130 indicators, nationally and for 16 domains (Appendix B of the final report).
Using svy, we can specify the sample design with three arguments:
sample = svy.Sample(
data=women,
design=svy.Design(stratum="v022", psu="v021", wgt="wgt"),
)We ran 50 comparisons: 23 of Table B.2’s women’s-survey indicators — every row that standard recode variables define, with rates, medians, and chapter-constructed indicators excluded (see the note below the tables) — plus 3 household-population indicators and 24 domain cells across urban/rural and four regions:
- Estimates and standard errors: 49 of 50 cells at published precision. The one exception is a regional subpopulation cell that matches exactly once the data is subset the way the producer subsets it — which is its own finding, below.
- Design effects: at published precision on 34 of 47 comparable cells. Every one of the 13 deviations is on a subpopulation-based cell, is at most 0.4%, and traces to two producer conventions (subset files and the SRS reference); the 3 household cells are excluded because ICF’s DEFT there answers a household-SRS question, not the person-SRS one.
- An independent run of the same cells in a second implementation agrees to 2.9e-14 on estimates and 5.3e-15 on standard errors — floating-point round-off — pinning the remaining deltas on conventions, not estimation.
Note that the svy estimators have been validated separately and in depth against R’s survey package — see svy vs R’s survey.
Background
The 2024–25 EDHS is the fifth Demographic and Health Survey in Ethiopia, after 2000, 2005, 2011, and 2016. The women’s file carries 21,395 respondents age 15–49; the household member file 97,262 records.
The sample was stratified by the 14 regions and by urban and rural areas. For security reasons, 8 of the 75 clusters selected in the Amhara region were never interviewed. The 805 selected clusters became 797 with data, and the sampling weights absorb the loss through cluster-level nonresponse adjustment within strata. Every variance computed below runs on those 797 clusters.
The design in three arguments
Appendix A specifies a stratified two-stage design: 27 strata (region × urban/rural, with Addis Ababa urban-only), 805 EAs drawn PPS from the 2019 census frame, and a fixed 28 households per cluster. The recode files carry it in three variables, and the DHS Program’s own published analysis code confirms the mapping — its Stata idiom is svyset v021 [pw=wt], strata(v022), which in svy is:
women_data = svy.read_stata("ETIR8AFL.dta").with_columns(
(pl.col("v005") / 1e6).alias("wgt") # Normalize the sample weights
)
women = svy.Sample(
data=women_data,
design=svy.Design(stratum="v022", psu="v021", wgt="wgt"),
)Before estimating anything, check the file against Appendix A’s own numbers:
These statistics much the published numbers.
Note the DHS weights are normalised so that weighted and unweighted totals agree nationally.
ImportantNormalised weights estimate rates, not totals
v005/1e6 sums to the sample size, not to Ethiopia’s population.
Means, proportions, and ratios are unaffected; population totals cannot be estimated from it, and the report says so itself.
svy makes the same point structurally: a design effect referenced to without-replacement SRS needs a population size, and with normalised weights svy refuses to fabricate one — deff="wr" (Kish’s deft², which is scale-invariant) is the option that is actually defined here.
Reproducing Appendix B
The target is Appendix B (pp. 463–502) which computes every mean and proportion as a Taylor-linearised ratio.
In svy, we can provide y as a sequence, to compute a given statistics (e.g. proportion) for a batch of variables of interest in one call. For example, we can estimate eight All women 15–49 indicators at ounce:
View code
print(
women.estimation.prop(
y=[
"urban",
"no_edu",
"secondary_plus",
"literate",
"internet_12m",
"married",
"pregnant",
"mobile_phone",
],
deff="wr",
drop_nulls=True,
)
)╭──────────────────────────── Estimate: PROP (TAYLOR) ────────────────────────────╮ │ │ │ y level est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ urban no 0.6536 0.0103 0.6331 0.6735 1.58 10.0134 │ │ urban yes 0.3464 0.0103 0.3265 0.3669 2.97 10.0134 │ │ no_edu no 0.6753 0.0094 0.6566 0.6934 1.39 8.5689 │ │ no_edu yes 0.3247 0.0094 0.3066 0.3434 2.89 8.5689 │ │ secondary_plus no 0.7175 0.0089 0.6997 0.7346 1.24 8.3420 │ │ secondary_plus yes 0.2825 0.0089 0.2654 0.3003 3.15 8.3420 │ │ literate no 0.4332 0.0101 0.4136 0.4530 2.32 8.8177 │ │ literate yes 0.5668 0.0101 0.5470 0.5864 1.77 8.8177 │ │ internet_12m no 0.8527 0.0073 0.8378 0.8665 0.86 9.1344 │ │ internet_12m yes 0.1473 0.0073 0.1335 0.1622 4.97 9.1344 │ │ married no 0.3870 0.0069 0.3735 0.4006 1.79 4.3151 │ │ married yes 0.6130 0.0069 0.5994 0.6265 1.13 4.3151 │ │ pregnant no 0.9397 0.0028 0.9340 0.9449 0.30 2.9189 │ │ pregnant yes 0.0603 0.0028 0.0551 0.0660 4.61 2.9189 │ │ mobile_phone no 0.5114 0.0107 0.4903 0.5324 2.10 9.8935 │ │ mobile_phone yes 0.4886 0.0107 0.4676 0.5097 2.20 9.8935 │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────╯
Subpopulations are where=, never a filter — the base changes, the design does not. Currently married women:
View code
print(
women.estimation.prop(
[
"cpr_any",
"cpr_modern",
"want_no_more",
"unmet_spacing",
"unmet_limiting",
"unmet_total",
"employed_12m",
],
where=MARRIED,
deff="wr",
drop_nulls=True,
)
)╭─────────────────────────── Estimate: PROP (TAYLOR) ────────────────────────────╮ │ where: married == 1 │ │ │ │ │ │ y level est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ cpr_any no 0.6456 0.0111 0.6236 0.6670 1.71 6.9633 │ │ cpr_any yes 0.3544 0.0111 0.3330 0.3764 3.12 6.9633 │ │ cpr_modern no 0.6554 0.0110 0.6335 0.6767 1.68 6.9737 │ │ cpr_modern yes 0.3446 0.0110 0.3233 0.3665 3.19 6.9737 │ │ want_no_more no 0.8066 0.0065 0.7935 0.8190 0.81 3.5150 │ │ want_no_more yes 0.1934 0.0065 0.1810 0.2065 3.36 3.5150 │ │ unmet_spacing no 0.8936 0.0049 0.8836 0.9028 0.55 3.2699 │ │ unmet_spacing yes 0.1064 0.0049 0.0972 0.1164 4.59 3.2699 │ │ unmet_limiting no 0.9520 0.0030 0.9457 0.9576 0.32 2.6179 │ │ unmet_limiting yes 0.0480 0.0030 0.0424 0.0543 6.32 2.6179 │ │ unmet_total no 0.8456 0.0060 0.8334 0.8570 0.71 3.6175 │ │ unmet_total yes 0.1544 0.0060 0.1430 0.1666 3.90 3.6175 │ │ employed_12m no 0.5153 0.0129 0.4899 0.5405 2.50 8.6568 │ │ employed_12m yes 0.4847 0.0129 0.4595 0.5101 2.66 8.6568 │ │ │ ╰────────────────────────────────────────────────────────────────────────────────╯
Women 20–49, for the age-at-event indicators:
View code
print(
women.estimation.prop(
["married_b15", "married_b18", "sex_b18", "birth_b18"],
where=W2049,
deff="wr",
drop_nulls=True,
)
)╭────────────────────────── Estimate: PROP (TAYLOR) ──────────────────────────╮ │ where: v013 >= 2 │ │ │ │ │ │ y level est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ married_b15 no 0.7939 0.0072 0.7794 0.8076 0.90 5.1828 │ │ married_b15 yes 0.2061 0.0072 0.1924 0.2206 3.47 5.1828 │ │ married_b18 no 0.5446 0.0090 0.5270 0.5622 1.65 5.3666 │ │ married_b18 yes 0.4554 0.0090 0.4378 0.4730 1.97 5.3666 │ │ sex_b18 no 0.5086 0.0090 0.4909 0.5262 1.77 5.3664 │ │ sex_b18 yes 0.4914 0.0090 0.4738 0.5091 1.83 5.3664 │ │ birth_b18 no 0.7367 0.0073 0.7221 0.7507 0.99 4.5128 │ │ birth_b18 yes 0.2633 0.0073 0.2493 0.2779 2.76 4.5128 │ │ │ ╰─────────────────────────────────────────────────────────────────────────────╯
And the means: children ever born, living children, ideal family size (numeric responses), and children ever born to women 40–49:
View code
print(
women.estimation.mean(
["ceb", "living_children", "ideal_children"],
deff="wr",
drop_nulls=True,
)
)
print(
women.estimation.mean(
"ceb",
where=AGE4049,
deff="wr",
drop_nulls=True,
)
)╭──────────────────────── Estimate: MEAN (TAYLOR) ─────────────────────────╮ │ │ │ y est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ ceb 2.4323 0.0406 2.3527 2.5119 1.67 5.0311 │ │ living_children 2.2442 0.0359 2.1738 2.3146 1.60 4.7192 │ │ ideal_children 5.0973 0.0709 4.9582 5.2364 1.39 14.5039 │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯
╭────────── Estimate: MEAN (TAYLOR, deff=wr) ───────────╮ │ where: v013.is_in([[6, 7]]) │ │ │ │ │ │ est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ 5.4359 0.0881 5.2630 5.6089 1.62 3.8967 │ │ │ ╰───────────────────────────────────────────────────────╯
In the two tables below, we list every women’s-survey row of Table B.2 whose indicator derives from standard recode variables. The reasons for excluding some of teh variables from Table B.2 follow the tables. The base population is all women 15–49 unless the indicator says otherwise.
The proportions:
| Indicator | svy | Published | svy SE | Published SE | svy DEFT | Published DEFT |
|---|---|---|---|---|---|---|
| Urban residence | 0.3464 | 0.346 | 0.0103 | 0.010 | 3.164 | 3.164 |
| No education | 0.3247 | 0.325 | 0.0094 | 0.009 | 2.927 | 2.927 |
| Secondary education or higher | 0.2825 | 0.283 | 0.0089 | 0.009 | 2.888 | 2.888 |
| Literacy | 0.5668 | 0.567 | 0.0101 | 0.010 | 2.969 | 2.969 |
| Used the internet in the last 12 months | 0.1473 | 0.147 | 0.0073 | 0.007 | 3.022 | 3.022 |
| Currently married/in union | 0.6130 | 0.613 | 0.0069 | 0.007 | 2.077 | 2.077 |
| Currently pregnant | 0.0603 | 0.060 | 0.0028 | 0.003 | 1.708 | 1.708 |
| Owns a mobile phone | 0.4886 | 0.489 | 0.0107 | 0.011 | 3.145 | 3.145 |
| Married before age 15 (women 20–49) | 0.2061 | 0.206 | 0.0072 | 0.007 | 2.277 | 2.276 |
| Married before age 18 (women 20–49) | 0.4554 | 0.455 | 0.0090 | 0.009 | 2.317 | 2.316 |
| Sexual intercourse before age 18 (women 20–49) | 0.4914 | 0.491 | 0.0090 | 0.009 | 2.317 | 2.316 |
| First birth before age 18 (women 20–49) | 0.2633 | 0.263 | 0.0073 | 0.007 | 2.124 | 2.124 |
| Want no more children (married women) | 0.1934 | 0.193 | 0.0065 | 0.006 | 1.875 | 1.875 |
| Using any contraceptive method (married women) | 0.3544 | 0.354 | 0.0111 | 0.011 | 2.639 | 2.638 |
| Using any modern method (married women) | 0.3446 | 0.345 | 0.0110 | 0.011 | 2.641 | 2.640 |
| Unmet need for spacing (married women) | 0.1064 | 0.106 | 0.0049 | 0.005 | 1.808 | 1.808 |
| Unmet need for limiting (married women) | 0.0480 | 0.048 | 0.0030 | 0.003 | 1.618 | 1.618 |
| Unmet need, total (married women) | 0.1544 | 0.154 | 0.0060 | 0.006 | 1.902 | 1.902 |
| Employed in the last 12 months (married women) | 0.4847 | 0.485 | 0.0129 | 0.013 | 2.942 | 2.941 |
And the means:
| Indicator | svy | Published | svy SE | Published SE | svy DEFT | Published DEFT |
|---|---|---|---|---|---|---|
| Mean number of children ever born (women 40–49) | 5.4359 | 5.436 | 0.0881 | 0.088 | 1.974† | 1.968 |
| Mean number of children ever born | 2.4323 | 2.432 | 0.0406 | 0.041 | 2.243 | 2.243 |
| Mean number of living children | 2.2442 | 2.244 | 0.0359 | 0.036 | 2.172 | 2.172 |
| Mean ideal number of children | 5.0973 | 5.097 | 0.0709 | 0.071 | 3.808 | 3.807 |
† svy’s DEFT (confirmed digit-for-digit by R) differs from the printed value in the third decimal on 1 subpopulation-based cell — producer conventions, discussed below.
NoteWhat is excluded, and why
Appendix B carries about 130 indicators; this table reproduces the women’s-survey rows that standard recode variables define unambiguously. Excluded, by category:
- rates (total fertility, mortality, discontinuation — computed by jackknife with specialised estimators, a different exercise from Taylor proportions); - constructed indicators (vaccination coverage, ANC content, media exposure, HIV knowledge composites — their definitions live in topic chapters, and a definition mismatch would masquerade as a validation failure);
- medians (DHS uses a grouped-median convention);
- separate-weight modules (domestic violence, HIV testing); and
- anthropometry (a measured subsample with its own base). The men’s tables repeat the same machinery on the men’s recode.
The household file, same three arguments
Household-population indicators come from the member recode with the base restricted to de jure members. The unweighted bases land exactly on the published counts (95,594 members; 13,560 children under 5):
View code
print(
members.estimation.prop(
["elec_light", "open_def"],
where=pl.col("dejure"),
deff="wr",
drop_nulls=True,
)
)╭────────────────────────── Estimate: PROP (TAYLOR) ───────────────────────────╮ │ where: dejure │ │ │ │ │ │ y level est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ elec_light no 0.5983 0.0188 0.5610 0.6345 3.14 140.2329 │ │ elec_light yes 0.4017 0.0188 0.3655 0.4390 4.67 140.2329 │ │ open_def no 0.7278 0.0161 0.6951 0.7582 2.21 124.4588 │ │ open_def yes 0.2722 0.0161 0.2418 0.3049 5.90 124.4588 │ │ │ ╰──────────────────────────────────────────────────────────────────────────────╯
View code
print(
members.estimation.prop(
"birth_reg",
where=pl.col("dejure") & (pl.col("hv105") < 5),
deff="wr",
drop_nulls=True,
)
)╭────────────────────────────────── Estimate: PROP (TAYLOR, deff=wr) ──────────────────────────────────╮ │ where: dejure & ([hv105 < 5]) │ │ │ │ │ │ Births registered with civil authority (<5) est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ no 0.6940 0.0140 0.6658 0.7208 2.02 12.5775 │ │ yes 0.3060 0.0140 0.2792 0.3342 4.59 12.5775 │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────╯
| Indicator | svy | Published | svy SE | Published SE |
|---|---|---|---|---|
| Electricity as primary source of lighting | 0.4017 | 0.402 | 0.0188 | 0.019 |
| Open defecation | 0.2722 | 0.272 | 0.0161 | 0.016 |
| Births registered with civil authority (<5) | 0.3060 | 0.306 | 0.0140 | 0.014 |
Published DEFT is omitted for these rows deliberately: for a household-level trait measured on persons, ICF’s SRS reference is not the member-level one svy computes (a person-SRS and a household-SRS answer different questions), so the columns are not comparable. Estimates and standard errors are.
Domains: urban, rural, and the regions
by= runs all published domains in one pass over the intact design. Urban and rural (Tables B.3–B.4):
View code
print(
women.estimation.prop(
"cpr_modern",
where=MARRIED,
by="v025",
deff="wr",
drop_nulls=True,
)
)╭────────────────────────────────────── Estimate: PROP (TAYLOR, deff=wr) ──────────────────────────────────────╮ │ where: married == 1 │ │ │ │ │ │ type of place of residence Using any modern method est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ rural no 0.6996 0.0126 0.6743 0.7238 1.80 6.7334 │ │ rural yes 0.3004 0.0126 0.2762 0.3257 4.19 6.7334 │ │ urban no 0.5556 0.0203 0.5152 0.5952 3.66 6.8523 │ │ urban yes 0.4444 0.0203 0.4048 0.4848 4.58 6.8523 │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
And all fourteen regions in the same single call — shown here for the no-education indicator:
View code
print(
women.estimation.prop(
"no_edu",
by="v024",
deff="wr",
drop_nulls=True,
)
)╭───────────────────────────── Estimate: PROP (TAYLOR, deff=wr) ─────────────────────────────╮ │ │ │ region No education est se lci uci cv (%) deff │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ addis ababa no 0.9333 0.0094 0.9118 0.9498 1.01 2.3647 │ │ addis ababa yes 0.0667 0.0094 0.0502 0.0882 14.07 2.3647 │ │ afar no 0.4184 0.0250 0.3692 0.4693 5.98 3.4902 │ │ afar yes 0.5816 0.0250 0.5307 0.6308 4.30 3.4902 │ │ amhara no 0.6764 0.0168 0.6421 0.7089 2.48 2.2372 │ │ amhara yes 0.3236 0.0168 0.2911 0.3579 5.18 2.2372 │ │ benishangul-gumuz no 0.6830 0.0276 0.6252 0.7357 4.04 4.1010 │ │ benishangul-gumuz yes 0.3170 0.0276 0.2643 0.3748 8.70 4.1010 │ │ central ethiopia no 0.7461 0.0217 0.7002 0.7871 2.91 3.7896 │ │ central ethiopia yes 0.2539 0.0217 0.2129 0.2998 8.55 3.7896 │ │ dire dawa no 0.7633 0.0182 0.7247 0.7980 2.38 2.1228 │ │ dire dawa yes 0.2367 0.0182 0.2020 0.2753 7.67 2.1228 │ │ gambella no 0.8260 0.0247 0.7705 0.8703 2.99 5.2779 │ │ gambella yes 0.1740 0.0247 0.1297 0.2295 14.19 5.2779 │ │ harari no 0.7845 0.0241 0.7318 0.8292 3.07 3.7782 │ │ harari yes 0.2155 0.0241 0.1708 0.2682 11.17 3.7782 │ │ oromia no 0.6333 0.0207 0.5912 0.6736 3.27 3.9275 │ │ oromia yes 0.3667 0.0207 0.3264 0.4088 5.66 3.9275 │ │ sidama no 0.7879 0.0166 0.7527 0.8193 2.11 3.1163 │ │ sidama yes 0.2121 0.0166 0.1807 0.2473 7.84 3.1163 │ │ somali no 0.3358 0.0387 0.2632 0.4171 11.52 9.8361 │ │ somali yes 0.6642 0.0387 0.5829 0.7368 5.83 9.8361 │ │ south ethiopia no 0.6702 0.0273 0.6135 0.7225 4.08 5.6022 │ │ south ethiopia yes 0.3298 0.0273 0.2775 0.3865 8.29 5.6022 │ │ south west ethiopia no 0.6381 0.0360 0.5632 0.7069 5.65 8.6544 │ │ south west ethiopia yes 0.3619 0.0360 0.2931 0.4368 9.96 8.6544 │ │ tigray no 0.7169 0.0178 0.6800 0.7510 2.49 2.7296 │ │ tigray yes 0.2831 0.0178 0.2490 0.3200 6.29 2.7296 │ │ │ ╰────────────────────────────────────────────────────────────────────────────────────────────╯
Against the published tables, across urban/rural and four regions chosen to stress different corners — Tigrai, Afar, Amhara (the region with the dropped clusters), and Addis Ababa (the single-stratum region):
| Indicator | Domain | svy | Published | svy SE | Published SE | svy DEFT | Published DEFT |
|---|---|---|---|---|---|---|---|
| No education | Urban | 0.1628 | 0.163 | 0.0110 | 0.011 | 2.630 | 2.629 |
| No education | Rural | 0.4105 | 0.411 | 0.0132 | 0.013 | 3.126 | 3.125 |
| No education | Tigrai | 0.2831 | 0.283 | 0.0178 | 0.018 | 1.652 | 1.651 |
| No education | Afar | 0.5816 | 0.582 | 0.0250 | 0.025 | 1.868 | 1.867 |
| No education | Amhara | 0.3236 | 0.324 | 0.0168 | 0.017 | 1.496 | 1.495 |
| No education | Addis Ababa | 0.0667 | 0.067 | 0.0094 | 0.009 | 1.538 | 1.537 |
| Using any contraceptive method | Urban | 0.4651 | 0.465 | 0.0201 | 0.020 | 2.573† | 2.571 |
| Using any contraceptive method | Rural | 0.3054 | 0.305 | 0.0128 | 0.013 | 2.621 | 2.620 |
| Using any contraceptive method | Tigrai | 0.2615 | 0.262 | 0.0176 | 0.018 | 1.282 | 1.281 |
| Using any contraceptive method | Afar | 0.0886 | 0.089 | 0.0177 | 0.018 | 1.988† | 1.985 |
| Using any contraceptive method | Amhara | 0.4387 | 0.439 | 0.0253 | 0.025 | 1.650 | 1.649 |
| Using any contraceptive method | Addis Ababa | 0.5115 | 0.512 | 0.0344 | 0.034 | 1.851† | 1.848 |
| Using any modern method | Urban | 0.4444 | 0.444 | 0.0203 | 0.020 | 2.618† | 2.616 |
| Using any modern method | Rural | 0.3004 | 0.300 | 0.0126 | 0.013 | 2.595 | 2.594 |
| Using any modern method | Tigrai | 0.2523 | 0.252 | 0.0178 | 0.018 | 1.307 | 1.307 |
| Using any modern method | Afar | 0.0877 | 0.088 | 0.0178 | 0.018 | 2.007† | 2.004 |
| Using any modern method | Amhara | 0.4332 | 0.433 | 0.0250 | 0.025 | 1.633† | 1.631 |
| Using any modern method | Addis Ababa | 0.4735 | 0.473 | 0.0383 | 0.038 | 2.060† | 2.055 |
| Mean number of children ever born | Urban | 4.0076 | 4.008 | 0.1932 | 0.193 | 2.474† | 2.464 |
| Mean number of children ever born | Rural | 5.9482 | 5.948 | 0.1014 | 0.101 | 2.017† | 2.015 |
| Mean number of children ever born | Tigrai | 5.3918 | 5.392 | 0.1663 | 0.165 | 1.242† | 1.230 |
| Mean number of children ever born | Afar | 6.6293 | 6.629 | 0.2874 | 0.287 | 1.458† | 1.450 |
| Mean number of children ever born | Amhara | 4.9659 | 4.966 | 0.1817 | 0.182 | 1.339† | 1.337 |
| Mean number of children ever born | Addis Ababa | 2.2975 | 2.298 | 0.1031 | 0.103 | 1.079 | 1.078 |
† svy’s DEFT (confirmed digit-for-digit by R) differs from the printed value in the third decimal on 12 subpopulation-based cells — producer conventions, discussed below.
One practical wrinkle: the region codes in v024 are not contiguous — they skip 11, and Addis Ababa is 14, not 13. Match domains by their published unweighted Ns before trusting any code-to-name mapping.
The cell that would not match, and what it taught
One regional cell resisted: mean children ever born to women 40–49 in Tigrai. Estimates identical, standard errors apart, which should be impossible if both sides run the same formula on the same data. The likely explanation is that the two computations do not see the same clusters. Two of Tigrai’s 70 clusters contain no woman age 40–49:
View code
tigrai_4049 = AGE4049 & (pl.col("v024") == 1)
d = women.data
print(f"Tigrai clusters: {d.filter(pl.col('v024') == 1)['v021'].n_unique()}")
print(f"with a woman age 40-49: {d.filter(tigrai_4049)['v021'].n_unique()}")Tigrai clusters: 70
with a woman age 40-49: 68The published value behaves as if the computation ran on a subset file, a working file restricted to the base population, in which those two clusters simply do not exist. Dropping them changes the \(m_h/(m_h-1)\) factors in the variance. svy’s where= keeps them, contributing zeros: textbook domain estimation. Run it both ways:
View code
domain = women.estimation.mean("ceb", where=tigrai_4049, deff="wr")
subset = svy.Sample(
data=d.filter(tigrai_4049),
design=svy.Design(stratum="v022", psu="v021", wgt="wgt"),
).estimation.mean("ceb", deff="wr")
for name, e in [("domain (where=)", domain), ("subset (filter)", subset)]:
r = e.to_dicts()[0]
print(f"{name}: se={r['se']:.4f} deft={r['deff'] ** 0.5:.3f}")
pub = DOMAINS[("ceb4049", "age4049")]["tigrai"]
print(f"published: se={pub[1]:.3f} deft={pub[2]:.3f}")domain (where=): se=0.1663 deft=1.242
subset (filter): se=0.1648 deft=1.231
published: se=0.165 deft=1.230To reproduce the published tables, subset first, as the published values imply.
Design effects
DEFT is the ratio of the design standard error to the standard error a simple random sample of the same size would have given. It is reported for every cell, and DHS reports the standard-error form rather than DEFF purely for presentational convenience alongside the CI columns (per the DHS lead statistician on the user forum).
svy’s deff=“wr” reproduces the published DEFT to all three printed decimals on 34 of the 47 comparable cells — including every cell whose base is the full file. The 13 exceptions, marked † in the tables above, are all subpopulation-based cells (currently married women, women 40–49), and none differs from the printed value by more than 1%.
The 13 flagged cells, quantified
Every †-flagged cell, computed both ways — as domain estimation (where=) and on a subset file restricted to the cell’s base population — against the printed value. svy and R agree digit-for-digit on both computed columns, so one column serves for both:
| Indicator | Domain | svy, domain | svy, subset | Published | Δ subset vs published |
|---|---|---|---|---|---|
| Mean number of children ever born (women 40–49) | National | 1.974 | 1.968 | 1.968 | +0.02% |
| Using any contraceptive method (married women) | Urban | 2.573 | 2.573 | 2.571 | +0.07% |
| Using any contraceptive method (married women) | Afar | 1.988 | 1.988 | 1.985 | +0.15% |
| Using any contraceptive method (married women) | Addis Ababa | 1.851 | 1.851 | 1.848 | +0.16% |
| Using any modern method (married women) | Urban | 2.618 | 2.618 | 2.616 | +0.06% |
| Using any modern method (married women) | Afar | 2.007 | 2.007 | 2.004 | +0.16% |
| Using any modern method (married women) | Amhara | 1.633 | 1.633 | 1.631 | +0.10% |
| Using any modern method (married women) | Addis Ababa | 2.060 | 2.060 | 2.055 | +0.22% |
| Mean number of children ever born (women 40–49) | Urban | 2.474 | 2.470 | 2.464 | +0.24% |
| Mean number of children ever born (women 40–49) | Rural | 2.017 | 2.016 | 2.015 | +0.07% |
| Mean number of children ever born (women 40–49) | Tigrai | 1.242 | 1.231 | 1.230 | +0.06% |
| Mean number of children ever born (women 40–49) | Afar | 1.458 | 1.455 | 1.450 | +0.33% |
| Mean number of children ever born (women 40–49) | Amhara | 1.339 | 1.339 | 1.337 | +0.13% |
Three things the table shows. First, every difference is on a subpopulation-based cell and none exceeds 1% even before any accounting. Second, the subset column closes most of the children-ever-born gaps — those are the empty-cluster cells — and leaves the contraceptive-use cells untouched, because every cluster contains a married woman and subsetting changes nothing there. Third, what remains after subsetting is at most a third of a percent, and some of it may be nothing at all: a published DEFT of 2.571 stands for any true value in [2.5705, 2.5715), so part of each remaining gap can be plain print-rounding.
The residue sits in the denominator of DEFT — the hypothetical-SRS variance — which is a software convention, not a single formula: R and SAS define it differently, and each offers more than one variant. svy’s deff="wr" is Kish’s with-replacement design effect, the same convention as R’s deff="replace" (the appropriate one for DHS’s normalised weights). SAS’s SURVEYMEANS does not report a design effect at all, and Appendix B says the sampling errors come from SAS programs developed by ICF — which could mean that the DEFT computation is ICF’s own rather than a stock SAS procedure, and its SRS formula for subpopulations does not appear to be published; we are asking the DHS Program about it. At under 1% on every cell, this is a footnote about conventions, not a validation issue: estimates, standard errors, and confidence intervals are unaffected.
Ruling out the estimator: svy and R survey produce identical results to machine precision
Published tables round to three decimals, so agreement there has a ceiling — and it leaves open whether the small design-effect deltas above are svy’s doing or come from the published side. To close that, we recomputed the same cells in R’s survey package, under identical domain semantics. The two implementations agree to a maximum relative difference of 2.9e-14 on estimates and 5.3e-15 on standard errors — floating-point round-off. And on the 13 †-flagged cells specifically, R reproduces svy’s DEFT to every compared digit: both packages land on the same value, together, away from the printed one. Whatever separates svy from the printed tables separates R from them identically — the difference lies in how the published values were produced (subsetting, the SRS reference), not in the estimation.
The estimators themselves are validated separately and in depth against R’s survey package — see svy vs R’s survey.
What this validates
| checked against | result | |
|---|---|---|
| estimates and SEs, 50 cells | Appendix B (Tables B.2–B.4, B.5–B.7, B.17) | 49/50 at published precision; the exception matches exactly when computed on a subset file |
| DEFT, full-base cells | Appendix B | published precision |
| DEFT, subpopulation cells | Appendix B | 34/47 at published precision; the 13 flagged cells differ by under 1% — partly subset computation and print rounding, SRS reference still open |
| domain estimation | R survey |
agreement to 2.9e-14 |
All results computed with svy 0.27.0 at render time; the page’s tables are rebuilt from the microdata on every render, so nothing svy-side is transcribed. The scripts alongside this post reproduce the run given registered access to the microdata.
References
Ethiopian Statistical Service (ESS) and ICF (2025). Ethiopia Demographic and Health Survey 2024–25 Final Report. DHS Program FR399.
Verma, V. and Pearce, M. (1986). CLUSTERS: A Package Program for the Computation of Sampling Errors for Clustered Samples. International Statistical Institute.
The DHS Program. Guide to DHS Statistics. https://dhsprogram.com/Data/Guide-to-DHS-Statistics/
This study is based on data from the 2024–25 Ethiopia Demographic and Health Survey, made available by the DHS Program. The analysis and any errors are the author’s alone and do not represent the views of the DHS Program, ICF, or the Ethiopian Statistical Service. DHS microdata may not be redistributed; the code published here runs against files obtained through the DHS Program’s registration process.
TipCommunity signal
Help make svy the standard for survey analysis in Python
If rigorous, design-based survey inference in Python matters to you, starring the repository helps signal demand and prioritize validation and stability work.