Clickhouse: Import compressed JSON fast

The MVG Observatory Project collects real-time departure data from Munich’s public transport system. The data is organized hierarchically: the top level contains date-based folders, each containing subfolders named after station IDs. These station folders hold multiple JSON files that capture departure information throughout the day. Each station’s data is stored in two types of files: *_body.json and *_meta.json. The body files contain either API error messages or JSON arrays of responses, which are imported into the mvg....

October 21, 2024 · 7 min · 1444 words · Robert