MACK is a lightweight connector written in R. It fetches data from external APIs and exports simple, structured results to JSON or YAML. It is intended to help standardise and streamline the process of gathering data inputs for energy systems models.
- Accepts and validates a structured request (what
sourceto connect to, whatparamsto pass to it, optionally what file to writeoutputto). - Routes this to a connector which calls the API, parses the response and normalises output.
- Returns the output as a list, and optionally writes it to JSON or YAML.
Currently, connectors are implemented for:
You can make your first API calls with just two lines of code.
Download this repo and then run:
source("path/to/mack/main.R")
uk_gdp <- run_mack('path/to/mack/examples/example_world_bank_indicators.yaml')This will load up MACK, read in an example call to the World Bank Indicators API and save the results with structured metadata to uk_gdp. uk_gdp$data will contain a list giving the UK's GDP (in local currency) from 2010 through to 2024.
You can run a request from an in-memory list, returning the result object as a list. This code yields the same result as above:
request <- list(
source = "world_bank", # from the world bank indicators
params = list( # ...
indicator = "NY.GDP.MKTP.KD", # request GDP
country = "GBR", # for the United Kingdom
years = c(2010, 2024) # from 2010 to 2024
)
)
result <- run_mack(request)The request used in the first example is defined by a simple YAML file structure:
request.yaml:
request:
source: world_bank
params:
country: GBR
indicator: NY.GDP.MKTP.KD
years:
- 2010
- 2024
output:
format: yaml
file: outputs/gbr_gdp.yamlBy defining the output block, calling run_mack() with this input file will save your results as YAML to disk.
The same is possible with JSON input and output:
request.json:
{
"request": {
"source": "world_bank",
"params": {
"country": "GBR",
"indicator": "NY.GDP.MKTP.KD",
"years": [2010, 2024]
},
"output": {
"format": "json",
"file": "outputs/gbr_gdp.json"
}
}
}Calling run_mack() with this input file will yield the same data as above, but save in a different format.
All connectors return the same top-level structure:
schema_version(set by dispatcher)connectortimestampquerydataunitsdimensions
Optional fields:
warningssource_metadata
For example:
{
"schema_version": "0.1.0",
"connector": "world_bank",
"timestamp": "2026-03-14T08:45:02Z",
"query": {
"url": "https://api.worldbank.org/v2/country/GBR/indicator/NY.GDP.MKTP.KD",
"params": {
"format": "json",
"date": "2018:2020",
"page": 1,
"per_page": 1000
}
},
"data": [
{ "country": "GBR", "indicator": "NY.GDP.MKTP.KD", "year": 2020, "value": 2868821517197.41 },
{ "country": "GBR", "indicator": "NY.GDP.MKTP.KD", "year": 2019, "value": 3189276674514.83 },
{ "country": "GBR", "indicator": "NY.GDP.MKTP.KD", "year": 2018, "value": 3149706986726.54 }
],
"units": null,
"dimensions": {
"temporal": { "start": "2018", "end": "2020", "resolution": "annual" },
"spatial": { "type": "country", "id": "GBR" },
"variable": "indicator",
"index": { "time": "year", "geography": "country", "variable": "indicator" }
}
}Two connectors are currently available.
A detailed guide has been written on how to extend MACK by writing connectors to additional APIs.
New connectors can also be created quickly with coding agents such as Codex or Claude Code by opening this repository locally and asking for the connector you want in plain language.
The World Bank Indicators connector provides quick access to development and infrastructure metrics, for example, GDP, GDP growth, electric power consumption, access to electricity, access to clean fuels and technologies for cooking, and the electricity generation mix. Indicators are typically annual country-aggregates, available from 1960 (with better availability from 1990 onwards). See World Bank Indicators for a full list of the indicators available.
The Renewables.ninja connector provides quick access to weather-driven wind and solar generation profiles which capture the real-world spatial and temporal variation in output. Several individual locations can be simulated within a single request, and aggregated together to form zonal or regional outputs. The wind and solar site configuration (technology, hub height, orientation) can be specified, with standard defaults applied if none are given. Simulations are available at hourly resolution from 1980 to 2024.
Renewables time series are returned in a column-oriented data structure. Results can be requested for an individual site, or the sum across several sites.
The Eurostat connector provides access to Eurostat's Statistics API using dataset codes and named dimension filters. It returns row-based observations with one field per returned Eurostat dimension, which makes it suitable for datasets such as household energy prices, population series, or other tabular Eurostat outputs once the dataset code and filter dimensions are known.
An optional aggregate_time = "annual_mean" mode is included for cases like semesterly energy price datasets where annual averages are more useful than the raw reporting periods.
Some APIs require authentication, for example, Renewables.ninja. This is handled by storing your tokens inside:
config/secrets.yaml
Template:
config/secrets_template.yaml
Remember not to share or commit your secrets.yaml file to a repository.
Runtime:
herehttr2yamljsonlite
Testing:
testthat
- Invalid request: clear error before API call.
- API failure: connector-specific error message with HTTP status when available.
- Missing data values: preserved as
NULL/NAin normalized output where appropriate.
Iain Staffell, Imperial College London