The goal of the Open-DAI platform is to deliver an environment in the cloud that allows Public Administrations to release its legacy data as APIs, thus to enable a PA to reproduce the environment either on premise or on a service provider.

The architecture requires a cloud environment where to place hosts. Actually the CloudStack software to deploy and manage networks of virtual machines (VMs) is supported by the project, but we are working to support also VCloud.

The approach of Open-DAI is not to bring data into the cloud, but accessing it from where it resides, on the Public Administration (PA) data center, without requiring any change to existing silos. The cloud environment make use of the cloud scalability to make on the fly transformation of the data and is published in the form of data service API in a web store.

The PA can thus decide wheather the API is a public/open data one, or reserved just to specific users. The APIs published in the Open-DAI store are protected and their access monitored. All this is accomplished through an architectural design that integrate various open source products and uses different virtual machines deployed in the environment.

The Open-DAI platform is composed of the 10 Virtual Machines represented in the deployed diagram below.

The diagram represents the maximum deployment possible depending on the SOA needs of the user the ESB and BPS VM could not be instantiated as the AS_Slave could be activated only if the user require particular loads or high availability.

The function of each machine is summarized in the following table, as is also the dependency that a machine has with the other, meaning that the NFS has to be created before than the AS_Master machine.

 

Machine name Function hostname Depends sizing[1]
OpenDaiMaster Host the DevOps tool that is able to install all the other virtual machines, host the Zabbix monitoring tool puppet RAM = 2 GB
Disk = 20 GB
CPU = 2 Cores
AS_Master Host the Db virtualization service, Geoserver and D2RQ tools and can also host an end user application server; host the cluster manager  jbossvdbmaster NFS, WEB-balancer-frontend RAM = 6 GB
Disk = 15 GB
CPU = 4 Cores
AS_Slave This is a machine generated for scaling and high availability that can be X  jbossvdbslave NFS, WEB-balancer-frontend RAM = 6 GB
Disk = 15 GB
CPU = 4 Cores
NFS Since the platform calls for a cluster there is the need of a NFS file system for keeping files visible to all clustered machines  nfs RAM = 512 kB
Disk = 20 GB
CPU = 1 Core
WEB-balancer-frontend This is the Apache machine that represent the frontend of the platform and manages Virtual Hosts both for internal and public use: the internal load balancer and the external API manager.  apache RAM = 2 GB
Disk = 15 GB
CPU = 2 Cores
API_Manager Host the tool that exposes, monitor, protect and manages the API  wso2api Registry, BAM RAM = 2 GB
Disk = 15 GB
CPU = 2 Cores
BAM Host the Business Activity Monitor to collect the usage information about the API  wso2bam Registry RAM = 2 GB
Disk = 20 GB
CPU = 2 Cores
Registry The central Governance Registry for all the SOA components  wso2greg RAM = 2 GB
Disk = 15 GB
CPU = 2 Cores
ESB Host the Enterprise Service Bus SOA component X  wso2esb Registry RAM = 2 GB
Disk = 20 GB
CPU = 2 Cores
BPS Host the Business Process Server component X  wso2bps Registry RAM = 2 GB
Disk = 20 GB
CPU = 2 Cores

 

Platform installation

To install the Open-Dai platform you need either a CloudStack environment or a generic cloud environment (the Open-DAI project tested it also with VCloud environment).

The difference between using CloudStack or another cloud management system to deploy and manage network machines is that in the first case, using the Open-DAI console, is possible to assign to the VM an arbitrary name, while in the latter case, on a generic cloud management system, the platform will use a naming convention to install the various VMs.

On the cloud domain you need to have the DNS service and UserData active at network offering level as shown in the following figure.

[1] Sizing information are minimum advised settings to use when generating machines.

The specific network of the user domain has to have defined the network domain as shown in the following figure.

This is important since the VMs have to configure their names and the name of some other machines using network services to get IP from known name.

To create the VMs you will have to download the two ISO images of the CentOS 6.5 Linux machines that will be used to generate them:

To create the VM the user will have import the Open-DAI Master machine into CloudStack or make it available to your cloud environment.

Then, using the CloudStack tools you will have to generate the new machine using the master ISO and choosing the appropriate service offering that is available in its environment.

No other operations will have to be executed, for the machine will automatically install all the needed software components.

The installation of this VM can take up to half an hour since it has to install lot of software.

With that active VM it will be possible to create the other VMs.

You will import the generic Open-DAI machine ISO in CloudStack as done with the master since machines will be created using that as a base: that ISO has some scripts that start on the first activation that allow the machine created to present itself to the master and to retrieve the correct information from the Puppet Master and transform itself from a generic CentOS virtual machine to a machine with the specific middleware required for the designed function in the Open-DAI environment.

At this moment user will have to configure the Open-DAI platform; the platform is automatically configured in all the known elements with standard settings, we advice user to change some of the parameters for security concerns (this reflect in changing the password of the DB and tools).

To do this he will have to fill the following checklist of needed information, some data have to be collected from the cloudstack environment (the first four elements of the following table) other has to be chosen.

Information Meaning
api_key The API_KEY of the user’s admin of the cloudstack domain
secret_key The SECRET_KEY of the user’s admin of the cloudstack domain
endpoint The endpoint of the cloudstack API
domain_id The DOM_ID of the user’s admin of the cloudstack domain
mc_pwd The password of MCollective
mc_stomp_pwd The password of the MCollective transport
postgres_pwd The password of the root postgresql DB
zabbixDBpwd The password of the zabbix DB
zabbixWEBpwd The password of the zabbix WEB Admin user
AM_admin_password The password of the API manager admin
AM_DB_password The password of the API DB
BAM_admin_password The password of the BAM manager admin
BAM_DB_password The password of the BAM DB
ESB_admin_password The password of the ESB manager admin
ESB_DB_password The password of the ESB DB
BPS_admin_password The password of the BPS manager admin
BPS_DB_password The password of the BPS DB
Reg_admin_password The password of the Registry manager admin
Reg_DB_password The password of the Registry DB

Once he will have all the listed information he will be able to run the configuration script by connecting in ssh to the Master machine

Once that the user will have done the configuration he will be able to create the Open-DAI environment.

To do so there are two cases:

  1. The user is in a CloudStack environment and has access to the cloudstack API to be able to control the creation of the VMs
    .
  2. The user either is in a CloudStack environment but does not have access to the API endpoint or is not in a CloudStack environment.

Cloudstack environment with access to API

In this case the user will have answered yes to the script and will find a Joomla site configured in the master machine so that it will be possible to use this web interface to generate the new machines of the Open-DAI environment.

User will have to check in the configuration panel of the com_odaienv component that the data is correctly filled.

Then he will find the list of the potential Open-DAI VMs that he can create in the component page where it will be possible to execute the deploy operation assigning the resources available in the environment.

Non CloudStack environment or no access to API

In this case the user will have to adopt a naming convention that is the one proposed in the “hostname” column in the first table of this document, with this naming convention the Puppet Master is able to recognize the machine as a machine of a specific role and assign it the correct configuration.

In this way the user will have to create the machines using the tools that the cloud gives to him.

 

The following pages provide a description of the pipeline to set up the Open-DAI platform in a cloud environment using the open source installation code available on GitHub at: https://github.com/open-dai: