About the Stata plugin

Community
21

Hi,
thank you for writing this stata plugin! I would like to use it, but run into the following error under any dbnomics command I try (e.g. dbnomics news):
Downloading recently added datasets…
Kernel panic. Abort
r(5100);

r(5100) signals a probelm with a Java runtime exception. However, I could not trace the problem using ‘set debug on’ and ‘set trace on’ to find a solution by myself (if needed, I can post the debug output, but it is just a quote from the ado file).

I have stata version 14.1 and installed libjson, moss and dbnomics from the ssc library.
My OS is Windows 10 (my colleague uses a macbook and does not have this problem)

Thanks for any hints!

Best,
Christoph

22

Hi all,
Agreed that the Stata plugin is a welcomed addition! For the most part it appears to be working, but I have run into a particular issue when attempting to import a complete World Bank dataset, specifically the GDP per capita data:

dbnomics import, provider(WB) dataset(WDI) indicator(NY.GDP.PCAP.KD) clear

For some reason it returns the error r(679) which I believe indicates some sort of web error. To be clear, I’m able to successfully import the data when I specify a specific country, but not the entire indictor. Any thoughts and/or suggestions would be greatly appreciated!

23

Hey Christoph, thanks a lot for using my command!

I often face the same issue, and in my case this is due to my Institution’s proxy. The problem seems to lie in the fact that the proxy uses a proprietary https certificate that is not recognised by the java https library used by Stata to connect to secure sites. Not sure this is the same issue that is affecting you, but here are a couple of solutions that worked for me.

One possibility could be to add the certificate to the Stata java path. To identify the certificate, open a mock page on the dbnomics website (e.g. https://api.db.nomics.world/v21/providers), see what secure certificate the browser is using and export it. You might need to convert the exported certificate to a .pem certificate file (or not).

Follow the procedure below to install the certificate (sent to me by the Stata tech support team):

We need to copy the certificate file to the “lib\security” folder in your Java directory found under the official Stata installation folder. For the instructions below I will use the fake certificate file name “fakecert.cer”, you will need to perform the steps and substitute the correct cert file name. On Windows you will need to have Administrator access to perform these tasks.

Note that on Windows there are two Java directories windows-i586 and windows-x64.
If you are using 64bit Stata choose the windows-x64 directory, otherwise choose the windows-i586 directory for 32-bit Stata. You can type:

about

in Stata to determine whether you are running 64 or 32 bit. The example below assumes you are on a 64-bit Stata 14. The java folder will look like:

C:\Program Files (x86)\Stata14\utilities\java\windows-x64\jre1.8.0_121\lib\security

Copy the “fakecert.cer” file in this directory. To import the certificate we will need to open a command terminal and run it as administrator. On a Windows 10 machine you can type:

cmd

into the search bar next to the Windows start button. Next right click on the best match and select “run as administrator”. Now navigate the directory where you copied the certificate file. On Windows the command you type in the terminal should be:

cd C:\Program Files (x86)\Stata14\utilities\java\windows-x64\jre1.8.0_121\lib\security

We can now execute the following command to register the certificate:

"C:\Program Files (x86)\Stata14\utilities\java\windows-x64\jre1.8.0_121\bin\keytool" -keystore cacerts -importcert -file "fakecert.cer"

It will ask you for a password which by default is: changeit

Then you will need to type: yes to the prompt below

Trust this certificate? [no]:

Finally close Stata, launch a fresh copy and test it with an HTTPS link.

Second option, and if you are comfortable with the lost secure connection, I could advise to change the main API endpoint in my ado-file. Replace the URL in line 42 with the following: http://api.db.nomics.world/v21. It should now work with or without your institution’s proxy (at the cost of the lack of secure data transfer).

Hope this helps.

Best,
Simone

24

Hi there, thanks a lot for using my command!

This is a very interesting issue. I also obtain an http error (r(675)) when running your query.

I am afraid the error might be on the server side, because if you type the underlying API call that is generated by my command (API call) I get a “502 bad gateway error”.

Interestingly, if I run the same query with version 22 of the API, the server returns the correct payload: v22 API call. So you should not expect this issue to be sorted out, because it only affects a legacy version of the API.

A new version of my command, compatible with v22 of the dbnomics API, is definitely due. Unfortunately, due to time constrains on my side, I won’t be able to push this before early next year.

As a short-term solution, you can use the v22 API and the option format to import this query directly in stata using import delimited:

import delimited using "http://api.db.nomics.world/v22/series/WB/WDI?dimensions=%7B%22indicator%22:%5B%22NY.GDP.PCAP.KD%22%5D%7D&limit=500&offset=0&&format=csv", clear varnames(1) encoding(utf-8).

Best,
Simone

1 Like
25

Hi all!

Apologies again for the long silence, I wanted to give you some quick feedback regarding my presentation of the DBnomics platform and the dbnomics stata plugin at the London Stata Conference last September.

The presentation went quite well, I had good feedback on my plugin and in general the audience seemed very interested in the platform, which is great. The command and the platforms were featured in a recent newsletter from Stata, which is also great!

As next steps, I definitely wish to update the command to the v22 of the API. At the same time I will start writing a technical article about the routine that I will then submit to the Stata Journal. Should the article be published, it would be a nice way to increase the outreach of both command and platform. Unfortunately I do not have much spare time right now, so the earliest I expect to progress on this is early 2020.

Best,
Simone

2 Likes
26

Hi all,

I wanted to flag that I released a new version of the dbnomics Stata command (https://github.com/dbnomics-stata/dbnomics). The new version is fully compliant with version 22.0.1 of the DB.nomics API and also addresses a number of bugs from the previous version.

I am excited to see where this project will go next. From my side, I’ll try to start working asap on a technical article to try and submit to the Stata Journal.

Let me also mention a few things I have noticed with the platform and the new API that you might want to consider addressing in the future:

  1. The dimensions_value_labels object in https://api.db.nomics.world/v22/series/UNDATA/DF_UNDATA_COUNTRYDATA is formatted differently than all other providers (i.e. a list of lists rather than a key-value pair). Not really sure why this is the case.
  2. It would be great to add an option to the /datasets/{provider_code}/{dataset_code} endpoint to remove the dimensions_value_labels from the payload, much like the metadata=false flag does for the /series/ endpoint. I think a great use of the /datasets/ endpoint is to complement the category tree, so that the user can see both the hierarchy of dataset and know which ones can be pulled. But since some providers have a huge set of dimensions, this solution was impractical due to how long the routine would take to put together the information.
  3. Not sure how it works with other programming languages, but everytime I push an incorrect request to the server, it responds with an HTTP error (typically a 404) and also a small payload that explains the error. Unfortunately, there seems to be no way for Stata to read the payload when the server also returns an HTTP error. So I can’t really parse server errors and can only return generic HTTP issues to the users. I wonder if it would be possible to have the server return an HTTP status 0 but retain the error message?
  4. Not really related to API issues, but it would be really great if you could extend the data collected from OECD to also include some more of their datasets. I would be so great, for example, to have the SMES_SCOREBOARD - which is among the Finance-themed datasets - available in DB.nomics.

Thanks again for your support and help so far!

Best,
Simone