DESTATIS fetcher updated and fixed


The DESTATIS fetcher has not been working well since August 2023 (cf this post).

What I first thought to be an evolution in the textual format distributed by DESTATIS (called linearized datacube) was in fact a bug. I have been in contact with a nice person at DESTATIS doing user support, and he told me yesterday that the bug was going to be fixed soon (estimated february).

However in the meantime I did some attempts to fix the fetcher for DESTATIS, this took me a couple of days to dig into this textual format.
The fact that the only documentation available is written in german didn’t help :upside_down_face: (cf the format specification PDF file available on this help page). But thanks to a tool like DeepL translator, I could translate the PDF file in english, keeping its layout.

This situation emphasizes the need for standard data formats (e.g. SDMX) and reference implementations for parsing them in several languages, to ease data interoperability. The linearized datacube textual format of DESTATIS is documented but there is no official publically available library allowing to parse it, so every consumer has to dive into it.

Now the fetcher has been vastly rewritten to workaround the upstream bug, and at the time I’m writing, DESTATIS data is up to date on DBnomics.

1 Like