When there are missing values in a time series, there are two ways this can be handled:
A. Maintain a complete calendar (consecutive dates relative to the frequency of the
series) and insert “NA” (or some symbol defined as indicating “missing”).
B. Record only non-missing values, or in other words just skip dates for which there’s
no valid observation.
It’s my impression that dbnomics has mostly followed method A, but I recently came
across ECB/BSI/M.U2.N.R.LRE.X.1.A1.3000.Z01.E which uses method B. To be precise,
it uses method B in JSON output, though if you download CSV the calendar is complete
with NAs inserted for 21 missing monthly observations.
With gretl’s JSON-oriented dbnomics reader in mind, I’d really like to know if method B
is considered legitimate by dbnomics, or if it’s an anomaly that should be fixed. Gretl could
in principle handle JSON with skipped observations, but it’s more complicated than method
A. Moreover, method A – with its explicit NAs – gives the user greater confidence that
the data are actually correct, and not broken in some obscure way.
To make my point a little clearer, hopefully, I show below an extract from the “period” array in the JSON returned by dbnomics for ECB/BSI/M.U2.N.R.LRE.X.1.A1.3000.Z01.E. I haven’t seen this sort of “gappy” calendar before and it imposes an additional computational burden on third-party software seeking to make use of data obtained via dbnomics.
OK, apologies for replying to myself, but I think I see what’s going on now. If you want to ensure you get a complete calendar, with explicit “NA” entries for missing values, you have to include “complete_missing_periods=true” in your request. I’m not aware of having to do this before. Can anyone comment on whether something has changed in this regard?
You’re right: you have to pass the complete_missing_periods=true parameter in order to complete the calendar of the time series, as stated in https://api.db.nomics.world/v22/apidocs
As it’s a quite difficult information to find, and as the documentation is a single line, your question made me want to document this more.
We have a new documentation here and you may want to read about the features page and web API page. This is an ongoing process and we are open to contributions (especially if you tell us what is unclear, rather than actually writing documentation material!).
But this hasn’t changed since API v22: time series completion was never an automatically turned-on option.
As a summary, one of DBnomics design goals is to redistribute provider data as-is, and give access to tools allowing to cope with different representations.