GDELT for researchers: a practical guide

GDELT is the largest open record of world news, and every index on this site is built from it. This guide covers what researchers need: what is in GDELT, how to get it, which fields matter, and the mistakes to avoid. It ends with the query that underlies a country-level conflict share like the African GPR Index.

What is GDELT?

The Global Database of Events, Language, and Tone (Leetaru and Schrodt, 2013) monitors news media worldwide and turns it into structured data. Two products matter most for research:

• The Event Database: “who did what to whom, where and when”, coded with the CAMEO event taxonomy. GDELT 1.0 runs from 1979; GDELT 2.0 adds 15-minute updates and a Mentions table linking each event to the articles that report it.
• The Global Knowledge Graph (GKG): one record per article with its themes, locations, people, organizations and tone. GKG 2.0 runs from February 2015 and covers 65 machine-translated languages.

Three ways to get the data

1. Raw files

GDELT posts every 15-minute file as a zipped, tab-delimited CSV. The file http://data.gdeltproject.org/gdeltv2/masterfilelist.txt lists every file ever published, and lastupdate.txt the newest. Raw files are free and suit a pipeline that keeps a local copy.

2. Google BigQuery

All GDELT datasets are public tables in BigQuery (project gdelt-bq), for example gdelt-bq.gdeltv2.events and gdelt-bq.gdeltv2.gkg, with date-partitioned versions that cut cost. BigQuery is the fastest route for country-day aggregates over many years. Queries are billed by data scanned, so select only the columns and date partitions you need; check the table list in the BigQuery console, as names and partitioning can change.

3. The DOC 2.0 API

The DOC API searches recent coverage without any setup and returns article lists or timelines as JSON or CSV:

https://api.gdeltproject.org/api/v2/doc/doc?query=(sanctions OR blockade) sourcecountry:egypt&mode=timelinevol&timespan=3months&format=json

Useful modes are artlist (up to 250 articles per call), timelinevol (share of coverage over time) and timelinetone. Operators such as sourcelang:, sourcecountry:, domain: and theme: go inside the query.

import requests, pandas as pd
r = requests.get("https://api.gdeltproject.org/api/v2/doc/doc", params={
    "query": '"strait of hormuz"', "mode": "timelinevol",
    "timespan": "3months", "format": "json"}, timeout=60)
data = r.json() if r.headers.get("content-type", "").startswith("application/json") else {}
if not data.get("timeline"):
    raise SystemExit("No results (or an error message): " + r.text[:200])
df = pd.DataFrame(data["timeline"][0]["data"])   # columns: date, value (% of coverage)
print(df.tail())

Fields that matter for risk research

Worked example: a daily conflict share by country

This BigQuery query computes, for three Sahel countries, the article-weighted share of each day's event coverage that is verbal or material conflict. It is the core of the event-based construction used for the African and European indices (before normalization).

SELECT
  PARSE_DATE('%Y%m%d', CAST(SQLDATE AS STRING)) AS day,
  ActionGeo_CountryCode                          AS fips,
  SAFE_DIVIDE(SUM(IF(QuadClass = 3, NumArticles, 0)), SUM(NumArticles)) AS threats_share,
  SAFE_DIVIDE(SUM(IF(QuadClass = 4, NumArticles, 0)), SUM(NumArticles)) AS acts_share,
  SAFE_DIVIDE(SUM(IF(QuadClass IN (3, 4), NumArticles, 0)), SUM(NumArticles)) AS conflict_share
FROM `gdelt-bq.gdeltv2.events_partitioned`
WHERE _PARTITIONTIME >= TIMESTAMP('2026-01-01')   -- date added to GDELT
  AND SQLDATE >= 20260101                          -- date the event happened
  AND ActionGeo_CountryCode IN ('ML', 'UV', 'NG')   -- Mali, Burkina Faso, Niger (FIPS)
GROUP BY day, fips
ORDER BY day, fips;

To turn the share into an index, divide by its average over a reference period and multiply by 100. If you would rather not run it yourself, the finished series for every country is free on the countries page.

Common pitfalls

• Source growth. The number of monitored sources rose sharply over time. Raw counts trend upward for that reason alone; use shares.
• Duplicates. The same event can be coded several times from different articles. Weight by articles and aggregate, rather than counting rows.
• FIPS vs ISO codes. Event country codes are FIPS 10-4. Map them before merging with other data.
• Geocoding. Locations resolve to a city, province or only a country. Sub-national work must drop country-level matches.
• Coding and translation errors. Everything is machine-coded. Validate against known events and treat single days with caution.
• Early years. Pre-2000 coverage is thinner and mostly English-language; prefer monthly aggregates for history.

Frequently asked questions

Citing GDELT

Leetaru, K., & Schrodt, P. A. (2013). GDELT: Global data on events, location, and tone, 1979–2012. ISA Annual Convention. If you use the GPR Data Hub's indices, please also cite the relevant working paper.