The Open-DAI API creation

The pipeline to create and publish an API in Open-DAI is the following:

  • determine the source of the data set: it can be a legacy DB or a CSV file
  • use TEIID Designer to connect to the data and reverse engineer it so that it will be possible to have a new view of the data represented by virtual tables where the information is transformed or simply reduced to the needed data set that has to be exposed

In the following video is shown an example of generating a VDB from a CSV file

The new views have to have a primary key to be published as oData API.

The needed work to prepare a good view is:

  • defining meaningful columns name possibly changing the ones in the existing legacy DB that could be difficult to understand
  • defining the correct primary key
  • joining some table to present data that do not require further elaboration (denormalizing)

 

 

All this work should require no more than a few hours by a person that knows the data set is working with.

Once done that, the VDB can be produced.

This artefact, together with the connection information, will have to be deployed in the TEIID server.

In the Open-DAI environment this work has to be done as follows:

  • Using an SCP client connecting securely to the production environment and copying the artefact in the file system
  • Connecting to the server through SSH and issuing the shell commands to deploy firstly the connection to the data than the artefact

 

This automatically publishes in the Open-DAI environment the API, this means that there is an endpoint in the private network of the cloud environment that exposes the newly created API.

If needed specific REST API can be defined within the TEIID designer and a WAR can be egenrated and deployed in JBoss application server.

Now the API has to be published to the public store.

The user has to login to the API manager publisher and create the new API thus publishing it to the public store.

The following video show how to publish a REST API in the API manager

The API manager represent a gateway between the public internet and the internal endpoint and it represent a point where the system will enforce:

Usage throttling: a single user will be allowed just the designed amount of calls per second

Security: some endpoint can be exposed just to authorized users

 

The user has to insert information of the API so that it will be also used in the CKAN publishing tool.

In particular these fields have to be written:

Field meaning
Description Description of the API and of the returned data
Tags Comma delimited tags that will be used by the Homer federation index to allow people to find the API even searching in another european language
Credentials The endpoint is protected with basic authentication thus here will have to be placed the credentials that the API manager will use to connect to the endpoint
Business Owner The owner of the data
Business Owner Email The mail of the data owner
Technical Owner The technical entity responsible for publishing
Technical Owner Email The mail of the technical entity

 

The Open-DAI project decided to use the swagger document present in the API manager to add other information and facilitate thus the final developer in the API usage.

Particularly in the swagger file the user had to add the info element where the licence information has to be placed.

All this information will be used to automatically generate the CKAN documents that will allow the catalogs to index the APIs.

 

{
"apiVersion": "1.0.0",
"swaggerVersion": "1.1",
"basePath": "http://api.opendai.eu/api",
"resourcePath": "/airq",
"info": {
     "title": "Air Quality",
     "description": "",
     "termsOfServiceUrl": "",
     "contact": "info@opendai.eu",
     "license": "Attribuzione - Non commerciale 3.0 Italia",
     "licenseUrl": "http://creativecommons.org/licenses/by-nc/3.0/it/legalcode",
     "language": "it"
},
"apis": [
     {
     "path": "/airq/1.0.0/networks",
     "description": "API to get the list of networks",
     "operations": [
          {
          "httpMethod": "GET",
          "summary": "API to get the list of networks",
          "nickname": "networks",
          "parameters": [
              {
              "name": "startDate",
              "description": "Starting date from which getting entities, format (YYYYMMDD)",
              "paramType": "query",
              "required": false,
              "allowMultiple": false,
              "dataType": "String"
              },
              {
              "name": "endDate",
              "description": "Ending date from which getting entities, format (YYYYMMDD)",
              "paramType": "query",
              "required": false,
              "allowMultiple": false,
              "dataType": "String"
              }
          ]
          }
     ]
     },
.....

 

The swagger file will represent the final documentation of the API.

This will result in the API published in the Open-Dai store.

 

 

The project has demonstrated that given a simple dataset is possible to create virtual database the artefact, publish the endpoint and publish the API in the Open-DAI store in just 1 day work by a skilled resource.

 

The project is working to a web console that will automate the publishing steps thus reducing the required time and skill even more.