Replication code and public data for constructing a longitudinal micro price panel from the Swiss Consumer Price Index (CPI) microdata.
This repository accompanies the following paper:
Seiler, Pascal (2026). Swiss CPI Microdata for Research: Data Structure, Panel Construction, and Applications. Mimeo, ETH Zurich.
The repository provides a fully reproducible pipeline for transforming the raw Swiss CPI microdata---as supplied by the Federal Statistical Office (FSO)---into a research-ready longitudinal price panel. The pipeline covers the period from May 2000 to December 2025 and implements all data-processing steps described in the accompanying paper: raw data import and cleaning, basket and classification merging, weight assignment, price series identification, plausibility corrections, sales treatment, price change computation, and sampling decisions.
All public input files (concordance tables, weight files, and classification mappings) are included in the repository. The confidential microdata---individual price quotes and the basket of goods and services---must be obtained separately from the FSO under a data-use agreement and are not included.
git clone https://github.com/pascalseiler/swiss-cpi-microdata.git
cd swiss-cpi-microdataThe raw CPI microdata are confidential and subject to data-use agreements with the Federal Statistical Office. To obtain access, contact the FSO to conclude a data-use agreement. Further information is available on the FSO website.
Once access has been granted, place the confidential files in the directories specified in Repository structure below.
The pipeline requires R (version 4.0 or later). All package dependencies are managed via renv. To restore the exact package environment used in the paper, run the following from the repository root:
install.packages("renv")
renv::restore()The pipeline uses the following packages: tidyverse, data.table, lubridate, zoo, here, readxl, readr, Hmisc, plyr.
The file R/00_config.R defines all paths, the sample period, and treatment switches. This is the only file that needs to be modified to change the sample definition. Ready-made configuration files for each case study are provided in R/90_case_studies/ (see Replicating the paper).
Key parameters in R/00_config.R:
| Parameter | Description | Default |
|---|---|---|
YEAR_START |
First year of the sample | 2008 |
YEAR_END |
Last year of the sample | 2025 |
TREATMENT$substitutions |
TRUE: treat substitutions as continuous; FALSE: start new spell at each substitution |
FALSE |
TREATMENT$sales |
TRUE: retain sale prices; FALSE: replace with last non-sale price (carry-forward) |
FALSE |
TREATMENT$index |
TRUE: include index-based prices; FALSE: exclude |
FALSE |
TREATMENT$administered |
TRUE: include administered prices; FALSE: exclude |
FALSE |
TREATMENT$scanner |
TRUE: include scanner-collected prices; FALSE: exclude |
TRUE |
TREATMENT$outlier_quantity |
Factor threshold for quantity jump detection | 10 |
TREATMENT$outlier_price_change |
Percentile threshold for price change trimming | 0.01 |
source("main.R")The pipeline runs sequentially through all modules and writes the final panel to output/cpi_microdata.Rda. Running the full pipeline for the complete sample (May 2000--December 2025) takes approximately 45 minutes on a standard workstation (24 GB RAM) and requires approximately 1.5 GB of free disk space for raw, intermediate, and output files.
.
├── main.R # Master script; runs the full pipeline
├── R/
│ ├── 00_config.R # Paths, sample period, treatment switches
│ ├── 10_raw_import/
│ │ ├── load_raw_data.R # Stack yearly CSV files into one dataset
│ │ └── clean_raw_data.R # Harmonize dates, rename variables, drop HICP
│ ├── 20_classifications/
│ │ ├── add_basket.R # Merge basket hierarchy (oid x year)
│ │ ├── add_coicop.R # Add COICOP codes and sector aggregates
│ │ └── add_item_indicators.R # Scanner, index, service, administered indicators
│ ├── 30_weights/
│ │ ├── add_item_weights.R # Item-level expenditure weights (IP x year)
│ │ └── add_region_weights.R # Regional weights (region x year)
│ ├── 40_identification/
│ │ └── identify_price_series.R # Spell identification; substitution treatment
│ ├── 50_plausibility/
│ │ ├── make_quantity_plausible.R # Correct implausible quantity jumps
│ │ └── make_price_plausible.R # Correct zero and negative prices
│ ├── 60_sales/
│ │ └── treat_sales.R # Retain or carry forward last regular price
│ ├── 70_prices/
│ │ └── compute_price_changes.R # Log prices, price changes, periodicity filter
│ ├── 80_sampling/
│ │ └── sampling_decisions.R # Apply treatment switches to final sample
│ └── 90_case_studies/
│ ├── config_case_I.R # Configuration for Case Study I
│ ├── config_case_II.R # Configuration for Case Study II
│ ├── case_study_I.R # Reconstruct CPI inflation from microdata
│ └── case_study_II.R # Price-setting moments
├── input/
│ ├── confidential/ # Not included---must be obtained from FSO
│ │ ├── micro_prices/
│ │ │ └── YYYY.csv # One file per year (e.g. 2008.csv ... 2025.csv)
│ │ ├── basket/
│ │ │ └── basket.xlsx # One sheet per year
│ │ └── LIK-Positionen-ohne-Einzeldaten.xlsx
│ └── public/ # Included in repository
│ ├── cpi.xlsx
│ ├── item_administered.xlsx
│ ├── item_covid.csv
│ ├── item_weights.xlsx
│ ├── mapping_basket_COICOP.xlsx
│ ├── mapping_COICOP_sector.xlsx
│ └── region_weights.xlsx
├── output/ # Created on first run; not version-controlled
│ └── cpi_microdata.Rda
├── renv.lock # Package versions for reproducibility
└── README.md
One file per calendar year. Each file contains all price quotes collected in that year. The unit of observation is a price quote: the transaction price of a narrowly defined good or service observed in a specific outlet and reference month.
| Pipeline name | Raw name | Type | Description |
|---|---|---|---|
date |
Datum |
Date | Month of price collection (YYYY-MM-DD) |
outlet |
PMS_Nummer |
Integer | Outlet identifier |
region |
Region |
Character | Price-collection region; available from January 2006 |
oid |
Schemapos_OID |
Integer | Variety identifier; links to the basket |
price_series |
Price_series_NB |
Integer | Price-series number within a variety--outlet combination |
description |
Description_g_TX |
Character | Variety description |
price |
Price_NB |
Numeric | Observed transaction price |
quantity |
Quantity_NB |
Numeric | Quantity associated with the price |
replacement_code |
Processing_CD |
Integer | Processing code for product replacements and quality adjustments |
price_pre |
Pre_period_corr_price_NB |
Numeric | Quality-adjusted price of the preceding period |
quantity_pre |
Pre_period_corr_quantity_NB |
Numeric | Quality-adjusted quantity of the preceding period |
sales_indicator |
Reduction_FLG |
Binary | Flag for temporary price reductions (sales) |
One sheet per calendar year, describing the full basket of goods and services. The hierarchy ranges from the aggregate CPI (level 1) to individual varieties (level 6). Higher-level identifiers are not repeated on child rows; the pipeline recovers the full hierarchy by carrying parent codes forward.
| Raw name | Description |
|---|---|
GP.Nr. |
Unique product identifier at each hierarchical level (levels 1--6) |
Schemapos_oid |
Variety-level identifier; links to oid in the micro price files |
Pos.Typ |
Hierarchical level (1 = aggregate CPI, 6 = variety) |
Periodizitaet.Code |
Collection frequency: A (aperiodic), H (semi-annual), J (annual), M (monthly), V (quarterly) |
Periodizitaetsmonat |
Twelve-character string: X = scheduled collection month, - = no collection |
Item-level metadata provided by the FSO. Used in R/20_classifications/add_item_indicators.R to construct time-varying scanner_indicator, index_indicator, and service_indicator columns. Each indicator is activated conditionally on the observation date, following the periods documented in the file.
Annual expenditure weights at the expenditure-item (IP) level. One sheet per basket revision period: LIK2000 (2000--2005), LIK2005 (2006--2010), LIK2010 (2011--2015), LIK2015 (2016--2020), LIK2020 (2021--2025). Merged by IP and year.
Source: FSO website.
Weights of each price-collection region. One sheet per basket revision period; weights are constant within each period. Merged by region and year.
Source: Own compilation from FSO documentation.
List of administered and semi-administered expenditure items (e.g. health insurance, public transport), by year. Used to construct admin_price_indicator in R/20_classifications/add_item_indicators.R; applied in R/80_sampling/sampling_decisions.R when TREATMENT$administered = FALSE. Merged by IP and year.
Source: Own compilation from FSO documentation.
Expenditure items for which prices were imputed during the COVID-19 pandemic, with associated date ranges. Used in R/40_identification/identify_price_series.R to flag likely imputations and introduce spell breaks when TREATMENT$substitutions = FALSE. Merged by IP and date.
Source: Own compilation from monthly FSO CPI press releases (2020--2021).
Concordance between FSO expenditure item numbers and five-digit COICOP codes. Merged by IP in R/20_classifications/add_coicop.R.
Source: Compiled from FSO weight documentation.
Mapping from COICOP codes to Eurostat special aggregates: food (FOOD), non-energy industrial goods (NEIG), energy (NRG), and services (SERV). Merged by COICOP in R/20_classifications/add_coicop.R.
Source: Own compilation from Eurostat special aggregates classification.
Official Swiss CPI monthly index series. Used in R/90_case_studies/case_study_I.R to compare the reconstructed inflation series against the official published index.
Source: FSO website.
Running main.R produces a single output file:
output/cpi_microdata.Rda---a longitudinal panel of micro prices. Each row is a price quote identified by variety, outlet, and month. Variables are ordered as follows: date and time variables; hierarchical product identifiers from broad to narrow (HG, WG, IP, EP, oid); outlet and region identifiers; spell identifiers (series_counter_number, series_counter_indicator); price and price-change variables (price, price_log, price_log_diff, price_change_indicator, and per-unit equivalents); auxiliary flags (replacement_code, sales_indicator, index_indicator, scanner_indicator, service_indicator, admin_price_indicator, collection_indicator, periodicity_code, periodicity_string); and weight variables (item_weight, region_weight).
The two case studies in the accompanying paper correspond to the following configurations.
| Case Study I | Case Study II | |
|---|---|---|
| Purpose | Reconstruct CPI inflation | Analyze price-setting behavior |
| Period | May 2000 -- Dec 2025 | Jan 2008 -- Dec 2025 |
substitutions |
TRUE |
FALSE |
sales |
TRUE |
FALSE |
index |
TRUE |
FALSE |
administered |
TRUE |
FALSE |
scanner |
TRUE |
TRUE |
outlier_quantity |
10 |
10 |
outlier_price_change |
0.01 |
0.01 |
| Config file | R/90_case_studies/config_case_I.R |
R/90_case_studies/config_case_II.R |
| Output | output/case_study_I.Rda |
output/case_study_II.Rda |
To reproduce Case Study I:
source("R/90_case_studies/config_case_I.R")
source("main.R")
source("R/90_case_studies/case_study_I.R")To reproduce Case Study II:
source("R/90_case_studies/config_case_II.R")
source("main.R")
source("R/90_case_studies/case_study_II.R")If you use this repository or the accompanying pipeline in your research, please cite the paper:
Seiler, Pascal (2026). Swiss CPI Microdata for Research: Data Structure, Panel Construction, and Applications. Mimeo, ETH Zurich.
The code in this repository is released under the MIT License. The public input data files are released under Creative Commons CC BY 4.0. The confidential CPI microdata are subject to the terms of the data-use agreement with the Federal Statistical Office and may not be redistributed.
Pascal Seiler
ETH Zurich
KOF Swiss Economic Institute
[email protected]
https://www.pascalseiler.ch