from urllib.request import urlretrieve
urlretrieve(
"https://gss.norc.org/content/dam/gss/get-the-data/documents/spss/2024_spss.zip",
"data/2024_spss.zip",
)Read SPSS .sav files in Python: GSS 2024
read sav file Python, SPSS sav Python, GSS Python, General Social Survey 2024, SPSS missing values Python, SPSS value labels Python, complex survey analysis Python
An SPSS .sav file carries variable labels, value labels and declared missing values. With svy you read all three in one call and go straight to survey estimation.
The example is the 2024 cross-section of the General Social Survey (GSS): 3,986 respondents, 989 variables.
Get the file
NORC publishes each year as a zip on the GSS SPSS page:
Read it
pip install svysvy.read_spss returns a polars DataFrame. It reads the .sav inside the zip, which also holds the codebook PDFs, so there is nothing to unpack:
import svy
gss = svy.read_spss("data/2024_spss.zip")
print(gss.shape)
gss.select("id", "age", "sex", "degree", "happy", "vstrat", "vpsu", "wtssnrps_as").head(3)(3986, 989)
| id | age | sex | degree | happy | vstrat | vpsu | wtssnrps_as |
|---|---|---|---|---|---|---|---|
| f64 | f64 | f64 | f64 | f64 | f64 | f64 | f64 |
| 1.0 | 33.0 | 1.0 | 3.0 | 3.0 | 3401.0 | 1.0 | 1.663284 |
| 2.0 | 64.0 | 1.0 | 4.0 | 2.0 | 3401.0 | 1.0 | 1.506088 |
| 3.0 | 69.0 | 2.0 | 2.0 | 2.0 | 3401.0 | 1.0 | 1.363951 |
Read only what you need with columns=:
svy.read_spss("data/2024_spss.zip", columns=["id", "happy"]).shape(3986, 2)
svy.read_sav is the same function under another name.
Keep the labels
svy.create_from_spss reads the data and the labels into a Sample:
sample = svy.create_from_spss("data/2024_spss.zip")
print(sample.meta.var_labels["happy"])
{code: text for code, text in sample.meta.value_labels["happy"].items() if not code.startswith("-")}General happiness
{'1': 'very happy', '2': 'pretty happy', '3': 'not too happy'}
Missing values are already null
GSS codes “don’t know”, “no answer”, “skipped on web” and similar as negative numbers, and the file declares them as SPSS missing values. svy reads declared missing values as nulls, so there is nothing to recode:
sample.data["happy"].value_counts().sort("happy")| happy | count |
|---|---|
| f64 | u32 |
| null | 32 |
| 1.0 | 797 |
| 2.0 | 2303 |
| 3.0 | 854 |
The 32 nulls are those answers. Stata and SAS files have nothing like this, so codes like these arrive as ordinary numbers that you have to clear yourself (see the Stata note). Pass drop_nulls=True when estimating to leave them out.
Set the design and the weight
The GSS is a stratified, clustered sample with variance strata and PSUs in the file:
sample = sample.set_design(
svy.Design(stratum="vstrat", psu="vpsu", wgt="wtssnrps_as")
)
print(sample.n_strata, "strata,", sample.n_psus, "PSUs")68 strata, 137 PSUs
The weight needs care. The 2024 file adds 677 respondents from an oversample of Hispanic, Black and Asian American, Native Hawaiian and Pacific Islander adults. The usual weight, wtssnrps, is null for them. NORC built wtssnrps_as so that all 3,986 respondents can be analysed together:
print("wtssnrps nulls:", gss["wtssnrps"].null_count())
print("wtssnrps_as nulls:", gss["wtssnrps_as"].null_count())wtssnrps nulls: 677
wtssnrps_as nulls: 0
GSS weights add up to the sample size, not the population. Use them for means and proportions, not totals.
Estimates now carry design-based standard errors and the value labels:
print(sample.estimation.prop("happy", drop_nulls=True))╭──────────────────── Estimate: PROP (TAYLOR) ─────────────────────╮ │ │ │ General happiness est se lci uci cv (%) │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ not too happy 0.2094 0.0094 0.1912 0.2288 4.50 │ │ pretty happy 0.5606 0.0106 0.5393 0.5817 1.90 │ │ very happy 0.2300 0.0113 0.2082 0.2533 4.91 │ │ │ ╰──────────────────────────────────────────────────────────────────╯
print(sample.estimation.prop("happy", by="sex", drop_nulls=True))╭───────────────────────────── Estimate: PROP (TAYLOR) ──────────────────────────────╮ │ │ │ Respondents sex General happiness est se lci uci cv (%) │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ │ female not too happy 0.2018 0.0117 0.1795 0.2261 5.80 │ │ female pretty happy 0.5671 0.0139 0.5392 0.5945 2.45 │ │ female very happy 0.2311 0.0152 0.2022 0.2629 6.58 │ │ male not too happy 0.2174 0.0152 0.1887 0.2492 6.98 │ │ male pretty happy 0.5544 0.0164 0.5216 0.5868 2.95 │ │ male very happy 0.2281 0.0129 0.2034 0.2549 5.67 │ │ │ ╰────────────────────────────────────────────────────────────────────────────────────╯
Other formats
The same steps work for the other two formats. Only the reader changes, along with what each format keeps:
- SAS: Read NHANES DEMO_L.xpt in Python. Transport files have variable labels but no value labels.
- Stata: Read Stata .dta files in Python: MEPS 2023. Value labels come through, but reserve codes are ordinary numbers.
Data: General Social Survey 2024, Release 3a, NORC at the University of Chicago.