Open Source PaaS Solutions Analysis

During this analysis, we will mainly focus on three points that are directly related to the use of PaaS solutions:

• Evaluation of the quality of the documentation and the open source community,
• Evaluation of the installation process and familiarization with the PaaS solution in terms of time and manpower.
• Evaluation of the behavior of a PaaS solution according to the defined scenarios (i.e. the continuous integration, scalability and elasticity).

Test environment characteristics
In order to test and evaluate the performance of the five PaaS solutions in a uniform and consistent manner, we propose to use a virtual environment based on the CETIC VMware cluster. Each solution will be installed and tested on a virtual machine equipped with a Ubuntu 16.04 Linux operating system (except for OpenShift which must be installed on CentOS 7), a 16Gb RAM and a 4vCPU processor.

Scenarios of execution
To test the behavior of a given PaaS solution, we propose to use two possible execution scenarios in the cloud environment. These scenarios have been proposed as part of two Belgian research projects and represent real use cases:

• Scenario 1: Nginx-Based Web Application: It allows a given patient to view and fill out their list of medicines based on data provided by their physicians (general practitioners or specialists) as well as pharmacists. These data are available via the Walloon Health Network (RSW). This application has been proposed as part of the Belgian SEAMPAT research project. We use this application to evaluate the deployment and continuous integration processes of applications in the PaaS solutions.
• Scenario 2: Big Data Analysis (TSimulus): TSimulus is an open source tool for random generation of time series. A time series is an ordered sequence of points in time where each point is associated with at most one value. TSimulus was proposed as part of the Belgian research project EAM-SDI. As part of this analysis, we propose to use the containerized version of TSimulus that we will deploy in each solution in order to test its behavior when TSimulus scales up.

PaaS solutions deployment
During the deployment of PaaS solutions on the CETIC infrastructure, we have discarded two solutions. The first one is PaaSTA. Indeed, its installation is too long. It requires the installation of at least seven tools (ie Docker, Mesos, Marathon, Chronos, SmartTask, Sensu and Jenkins). In addition, it is costly in terms of time and it is very resource hungry. Whereas the second solution is Flynn. Its installation relies on the use of an online remote server. This dependency does not meet our requirements in this analysis. In the rest of this blog post, we are interested in the three remaining solutions that are OpenShift Origin, Dokku and Deis.

Analysis criteria
In the following, we present the analysis criteria that we categorize according to the three points presented above:

• Quality of the documentation: In order to study the quality of the documentation of a given PaaS solution, it is very important to evaluate the open source community (i.e. its size, its level of responsiveness to the issues, etc.) and estimate the spent time to be familiar with the documentation.
• Deploying the PaaS solution: It is very important to analyze the process of deploying the PaaS solution. For this, it is necessary to evaluate the time of the deployment action, the cost of the necessary labor and the time of the familiarization with the PaaS solution once installed. In addition, it must be checked whether the PaaS deployment process is (1) automatic or manual and (2) declarative or imperative.
• Continuous integration of the application: Today the continuous integration of web applications in the cloud is highly recommended for the good maintenance of the application lifecycle. For this, we propose to study this aspect in each PaaS solution. Indeed, it is necessary to check if it is part of the native functionality of the PaaS solution and if it is possible to use a third solution for this fact. In addition, it is necessary to evaluate the deployment time of the application and the implementation of the continuous integration process.
• Scalability: Among the features most desired by users deploying their applications in the cloud is scalability. To do this, we propose to verify (1) the nature of the scalability provided (ie vertical and / or horizontal), (2) whether the scalability is automatic or not, and (3) the manner of the scalability. adding a new instance.

Synthesis
Based on the table above and our findings made during the installation and testing of PaaS solutions, we present a synthesis for this blog post:

• Both Flynn and PaaSTA solutions were discarded. Indeed, the installation of these two solutions is complicated and requires several dependencies. In addition, they are not too efficient.
• Deis is strongly related to the Kubernetes which is a container orchestration tool. This generates several limitations to the use of Deis. Indeed, the deployment process of Deis is very long as it is based on a pre-existing Kubernetes cluster. It takes 6d/h including 5d/h for Kubernetes. In addition, Deis was recently acquired by Microsoft. This has slowed down the production of new versions and the improvement of this PaaS solution. The information on Deis’ evolution in the future is too clear.
• The quality of the open source community of Dokku is very good as well as the quality of the documentation. It is the easiest solution to deploy. However, it is semi-declarative and it is based on the use of the command line. Dokku is not easily extensible with third party tools. Finally, the management of application scalability is manual.
• The open source community of OpenShift Origin is very large and good. It provides a stable quality of documentation available in several languages ​​(e.g. English, French, ...). Even if the deployment of OpenShift takes a lot of time, it is automatic and declarative. It should be noted that this solution is entirely based on Kubernetes and the installation of this latter is integrated in its deployment. This is a very good point because it facilitates the implementation of a PaaS solution and alleviates developers tasks. It natively enables seamless integration and it is extensible by third-party tools. Finally, Openshift allows vertical and horizontal scalability in an automatic way.

Thanks to this analysis, we make our decision and choose the most suitable PaaS solution for our requirements in the CETIC. Indeed, we select OpenShift Origin since it is one of the most popular open source PaaS solutions used in the industry today. In addition, it is stable and very interoperable because it allows the use of several third-party tools. It ensures automatic deployment and scalability.

• Introduction
• Part 1: Presentation of the PaaS solutions

References
[1] https://github.com/openshift/origin
[2] https://github.com/dokku/dokku
[3] https://github.com/deis/deis
[4] https://docs.openshift.org/latest/welcome/index.html
[5] http://dokku.viewdocs.io/dokku/getting-started/installation/
[6] https://deis.com/docs/workflow/