Posts

What is necessary to achieve interoperability in the Cloud?

As described in the previous sections, 3 major interoperability approaches arise. First, there is the standardisation approach, next there is the middleware approach and last but not least there is the API approach. This is also supported by [Hof09] and [Gov10]. In addition to that, [Gov10] suggests building abstraction layers in order to achieve interoperability and transportability.

There are two main aspects where interoperability is necessary. One level is the management level. This deals with handling the virtual machine(s), applying load balancing, setting DNS settings, auto scaling features and other tasks that come with IaaS solutions. However, this level is mainly necessary in IaaS solutions as PaaS solutions already take care of most of it. The other level is the services level. The services level is basically everything that comes with application services such as messaging, data storage and databases.

interop approaches

Figure: Cloud interoperability approaches

These requirements are described in several relevant papers such as [End10], [Mel09], [Jha09].

Parameswaran et al. describes similar challenges for Cloud interoperability. The authors see two different approaches, the first via a unified cloud interface (UCI) and the second via Enterprise Cloud Orchestration [Par09].

A unified cloud interface is basically an API that is written “around” other cloud APIs that is vendor specific. It requires some re-writing of and integration. This is similar to the approach selected by Apache jClouds and Apache libcloud.

The Enterprise Cloud Orchestration is a layer where different cloud providers register their services. The platform then provides these services for users like in a discovery. This is very similar to UDDI. The downside of this is that the orchestration layer still needs to integrate all different services (and built wrappers around them). However, it is transparent to the end-user.

Enterprise orchestration layer

Figure: Enterprise Orchestration Layer [Par09]

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

As discussed in the previous sections, there are several standards and interoperability frameworks available. Most of them are infrastructure related. The standards and frameworks can generally be clustered into 3 groups.

The first group is the “Standards” group, which consists of OCCI an the DMTF standards. The second group is the “Middleware” group. This group contains mOSAIC, the PaaS Semantic Interoperability Framework and Frascati. The third group is the “Library” group. This group is a concrete implementation that provides a common API for several cloud platforms. The two projects in here are Apache jClouds and Apache libcloud.

Interoperability Solutions for the Cloud

Interoperability Solutions for the Cloud

Figure: Interoperability in the Cloud

OCCI provides great capabilities for infrastructure solutions, but there is nothing done for individual services. The same applies to the standards proposed by the distributed management task force.

As with the libraries and frameworks, a similar picture is drawn. Apache jClouds and Apache libcloud provide some interoperability features for infrastructure services. As for platform services, only the blob storage is available. When developers build their applications, they still run into interoperability challenges.

mOSAIC offers a large number of languages and services, however, it is necessary to build a layer on top of an existing platform. This is not a lightweight solution, as a developer has to maintain both – the developed application and the middleware solution installed on top of the provider. The developer eliminates the vendor lock-in, but runs into operational management of the mOSAIC platform. This eliminates a problem on the one side but might create a new one on the other side.

The same problem exists with the PaaS Semantic Interoperability Framework. A user has to install a software on top of the cloud platforms, which then has to be maintained by the user. The goal of platform as a service is to relieve the user from any operational management like in IaaS platforms. Frascati is also a middleware that needs to be maintained.

All of the libraries or frameworks described work for IaaS services. None of them support the PaaS paradigm and services related to it. Apache libcloud and Apache jClouds offer some very fundamental support for the storage service. However, other services such as messaging and key/value storage are not supported.

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

PaaS Semantic Interoperability Framework (PSIF)

Loutas et al. defines semantic interoperability as “the ability of heterogeneous Cloud PaaS systems and their offerings to overcome the semantic incompatibilities and communicate” [Lou11]. The target of this framework is to give developers the ability to move their application(s) and data seamlessly from one provider to another. Loutas et al. propose a three-dimensional model addressing semantic interoperability for public cloud solutions [Lou11].

Fundamental PaaS Entities

The fundamental PaaS entities consist of several models: the PaaS System, the PaaS Offering, an IaaS-Offering, Software Components and an Application [Lou11].

Levels of Semantic Conflicts

Loutas et al. [Lou11] assumes that there are 3 major semantic conflicts that can be raised for PaaS offerings. The first one is an interoperability problem between the metadata definitions. This occurs when different data models describe one PaaS offering. The second problem is when the same data gets interpreted differently and the third is when different pieces of data have similar meaning. Therefore, [Lou11] uses a two-level approach for solving semantic conflicts. The first level is the Information Model that refers to differences with data and data structures/models. The other level is the data level that refers to differences in the data because of various representations.

Types of Semantics

Three different types of semantics are defined [Lou11]. The first type is the functional semantic. This is basically a representation of everything that a PaaS solution can offer. The second type, the non-functional semantic, is about elements such as pricing or Quality of Service. The third semantic is the execution semantic, which is describing runtime semantics.

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

mOSAIC

mOSAIC is a European project supported by the European union [Dan10]. The target of the project was to build a unified Application programing interface (API) for Cloud services that is not only available in Java but also for other languages. mOSAIC is platform and language agnostic. It supports a large number of platforms for this approach. The mOSAIC framework itself is a middleware that runs on top of each cloud provider and abstracts provider specifics. The platform then exposes it’s own API to clients.

The mOSAIC project is built in a layered architecture. On the lowest level, there is the native API or protocol. This is either a ReST, SOAP, RPC or a language-specific library. On the next level, a driver API is found. This API can now be exchanged easily with different platforms such as Amazon’s S3. On top of that is an interoperability-API that allows programming language interoperability. Cloud resources can be access via the connector API. This is also the entry-point for developers, as they access specific models and resources from that API. On top of the connector API is a cloudlet that provides a cloud compliant programming methodology.

Frascati-based Multi PaaS Solution

Frascati is an infrastructure solution that runs on 11 different cloud solutions: Amazon EC2, Amazon Elastic Beanstalk, BitNami, CloudBees, Cloud Foundry, Dot-Cloud, Google App Engine, Heroku, InstaCompute, Jelastic, and OpenShift [Par12].

Frascati follows 3 core principles: An open service model, a configurable Multi-PaaS infrastructure and several infrastructure services. The open service model is an assembly of loosely coupled services based on a Service oriented Architecture. The configurable federated Multi-PaaS Infrastructure is a configurable kernel. Infrastructure services take care of the node provisioning, the deployment of the PaaS service, the deployment of the SaaS service and a federation management service. The Frascati services are installed on top of existing IaaS-services. To work with these services, it is necessary to have access to virtual machines (either Linux or Windows).

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

Apache jClouds

Apache jclouds is a framework provided by the Apache Software Foundation. The framework is written in Java and serves the purpose to provide an independent library for typical cloud operations. At present (November 2014), Apache jclouds provides 2 kinds of services: a compute service and a blob service [Apa14b]. Apache jclouds can be used from Java and Clojure. The library offers an abstraction for more than 30 cloud providers, including AWS, Azure, OpenStack and Rackspace.

Apache jclouds is primarily built for infrastructure interoperability. As for the platform layer, only blob storage is currently supported. The focus of jclouds is to support a large variety of platforms over implementing a large variety of services. The Blob storage in Apache jclouds works with the concept of Containers, Folders and Blobs. The library supports access control lists for objects. The upload of Multipart objects is also supported, which allows jclouds to handle large files. [Apa14c]

Libcloud

[Apa14d] Apache libcloud is similar to Apache jClouds. The library is developed as an abstraction to different cloud providers. Libcloud currently supports more than 30 different providers. Libcloud is available as a Python library.

Libcloud is infrastructure-focused with most of it’s implementations done for computing, DNS and Load balancing. There is also an object storage implementation available.

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

Open Cloud Computing Interface (OCCI)

The Open Grid Forum created the Open Cloud Computing Interface (OCCI). They claim to have one of the first standards in the cloud. OCCI was initially built to deliver portability and interoperability for IaaS platforms. In its initial implementation, it was used for different tasks around deployment, scaling and monitoring for virtual machines in the cloud. The library also supports common tasks for other cloud layers such as SaaS and PaaS [Ope14a].

OCCI is not a specific library that enables interoperability and portability. Hence, it is a definition of standards that can be implemented by individual platforms. The standard exists of 3 elements: the core description, the infrastructure description for the IaaS domain and a rendering description. The rendering is used to provide a REST HTTP service [Ope14b].

OCCI is implemented by a large number of cloud platforms. Major platforms such as Apache Cloudstack, OpenStack, OpenNebula and Eucalyptus implement that standard. However, large public cloud providers such as Amazon Web Services, IBM, Microsoft and Google don’t implement that standard [Ope14c].

The OCCI standard focuses on IaaS. At present, it supports compute, networking, storage, and infrastructure templates. Storage does not define blob storage, as it is about the storage of virtual machines [Met11].

Distributed Management Task Force (DMTF) initiatives

Distributed Management Task Force proposed several standards for cloud computing [Dmt09]. The key focus of the DMTF is to improve standards around IaaS solutions. As a key standard, the DMTF developed the Open Virtualization Format (OVF). However, there are no standards for APIs and platform services available.

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

Gonidis [Gon11] defines an approach on how to enable interoperability for PaaS solutions. It is about building a standard for existing cloud platforms. However, this is a challenge given that every platform provider has it’s own proprietary API. Furthermore, services available on the one platform aren’t available on the other platform. In the following posts, existing interoperability frameworks, solutions and standards for Platform as a Service are evaluated.

Standard initiatives

Current standard initiatives are the Open Cloud Computing Interface (OCCI) and the Distributed Management Task Force (DMTF) initiatives. In the next posts, I will outline them in detail.

Libraries and Frameworks

Libraries and Frameworks for Cloud Interoperability are: Apache jClouds and Libcloud. They will be described in the following posts.

Middleware solutions

Middleware Solutions for Cloud interoperability are: mOSAIC, PaaS Semantic Interoperability Framework (PSIF), Frascati-based Multi PaaS Solution and SimpleCloud. They will be described in the following posts.

Interoperability Solutions for the Cloud

Interoperability Solutions for the Cloud

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

On the IaaS layer, work on cloud interoperability was already conducted [Ste13a], [Ste13b]. The authors described in “Challenges in the Management of Federated Heterogeneous Scientific Clouds” the problem and a feasible solution to migrate virtual machines between providers. Another challenge identified is the layer between the vendor API and the user. This problem is addressed by the same authors in the paper “Building an On-Demand Virtual Computing Market in Non-Commercial Communities”, where a market is introduced to handle that problem. The concept of the market is then described in “Take a Penny, Leave a Penny Scaling out to Off-premise Unused Cloud Resources” [Ste13b] in detail, where a solution is presented that allows users to use different cloud vendors with one abstract API.

Gonidis et al. [Gon11] give a first hint at challenges addressed for Platform as a Service interoperability. Platform as a Service gives the promise of speeding up application development [Mei11] by utilizing services. Platforms such as Microsoft’s Azure, Amazon’s Elastic Beanstalk or Google’s AppEngine offer a large number of services. The possibilities range from object storage, databases, messaging to analytics. A comprehensive overview is given in section 4. However, this leads to new challenges in terms of vertical interoperability. The challenge with PaaS is not like with IaaS, where one has to move the virtual machine. Now, it is about moving individual services and the corresponding data. The leading PaaS providers such as Amazon, Google and Microsoft provide their own frameworks and tools to access their platforms. When a customer decides to move the application to another provider, this becomes a challenge.

Loutas et al. [Lou11b] describe similar challenges. First, it is stated that cloud providers promote their own, incompatible formats over standards. This is largely due to the on-going battle of dominance in the cloud. Since each provider has their own standard, new and smaller providers cannot enter the market easily as there is no common standard yet. It is stated that interoperability is the missing element so far, even though it would benefit both – customers and providers [Lou11b].

Stravoskoufos et al. [Str] define different cloud interoperability challenges on each of the cloud layers. On the IaaS layer, interoperability is about controlling the infrastructure-specific services. For platform as a service, interoperability is about using the APIs and services. For Software as a s Service, the challenge is to exchange messages and data.

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

Interoperability is used in various levels of software and systems. With Cloud Computing, several major interoperability questions arise. First of all, interoperability can be seen as an infrastructure interoperability element. As for infrastructure interoperability, it is basically about the question on how to move virtual instances from one cloud provider to another.

Another interoperability question is about software and tools interoperability. This is not on how to transfer software from one provider to another, it is about how different software languages and tools collaborate in one cloud ecosystem or different cloud ecosystems. An example of that is a message-based communication between a Java-based solution and a Microsoft .NET solution. Mayrbäurl et al. described such a scenario on a Microsoft Azure based solution that itself communicates with an on-premise solution that is written in Java, whereas the Cloud service uses .NET [May11].

The third interoperability question is about the services offered by different distributors. When someone builds a cloud solution based on a specific provider and uses a software stack such as J2EE or Microsoft .NET, it is not that difficult to move the application to another instance or Cloud provider. However, when this application consumes platform-specific services such as a distributed storage service, a messaging service, e-mail and alike, migration becomes challenging. Therefore, providing interoperability on the platform services level is another key issue in Cloud computing interoperability.

Gonidis et al. defines Interoperability with two different characteristics [Gon11]. According to this paper, interoperability is when two components in different cloud environments are collaborating, whereas the possibility to move from one cloud platform to another is portability. Liu et al. defines portability itself with system portability, which means that a virtual machine can easily be moved to another cloud provider’s platform [Nis11]. In general, they define portability with the ability to move an application and all it’s data with little effort from one cloud provider to another cloud provider.

As for interoperability and portability, current literature knows different approaches. In Gonidis et al. portability and interoperability are distinct features [Gon11], whereas Dowel et al. defines portability to be a subset of interoperability [Dow11]. Petcu et al. discusses interoperability consisting of two dimensions: a horizontal dimension and a vertical dimension [Pet11]. Horizontal interoperability (as illustrated in figure 2) means that two cloud services on the same service level (e.g. PaaS) can communicate with each other, whereas vertical interoperability describes the ability to host an application on different cloud providers. When implementing vertical portability, it is necessary to support different cloud platforms.

cloud interoperability dimensions

Another description of interoperability for cloud solutions is given by Parameswaran et al. [Par12]. The authors describe 4 major factors of interoperability challenges. The first is portability, which was already discussed. The second is interoperability itself, which focuses on efficient and reliable protocols. The third is heterogeneity. This states that there are several protocols such as SOAP and Representational State Transfer (REST) [Fie00] and different formats such as XML and JSON. A PaaS application will need to deal with all of these protocols and formats in order to ensure interoperability in a heterogeneous environment. Last, there is geo-diversity. This means that smaller, but geographically more distributed datacentres might be more effective than large datacentres that are only available in some regions.

Why is interoperability so important to the cloud?

Leavitt [Lea09] addresses potential issues with the vendor Lock-In in cloud computing. When it is not possible to move from one platform to another at low cost and effort, it is very likely that one get’s locked into a certain platform. Should the provider decide to increase prices, the customer has to stick to the new conditions without the ability to move the application to a cheaper carrier. Another reason is when the cloud provider fails. Even large providers such as Microsoft [Oky14] and Amazon [Kep14] fail. This could mean that the platform is not available for several hours to days. In their SLA’s, cloud providers typically don’t reimburse the customers for lost revenue [Ama14b], [Mic14c]. But it could become worse: the cloud provider might actually disappear and customers have to move entirely to another platform entirely. This happened in February 2009 when Coghead, a cloud Platform went out of service [Kin09]. Users could export their data, but couldn’t port their applications.

This post is part of a work done on Cloud interoperability. You can access the full work here and the list of references here.

Platform as a Service is one of the major topics when it comes to cloud computing. However, it is well below the usage and acceptance of other levels such as SaaS and IaaS. The advantages are often not seen by the large majority. Furthermore, a definition of what PaaS is, is not always given. Therefore, I’ve started a discussion on PaaS – what it is and how it defines. Feel free to comment on the topics and add your points when you believe I’ve missed some of them.

A very simple definition could be: “Platform as a Service is talking away the pain of the Software Stack administration from IaaS platforms and allows to focus on your application”.

Platform as a Service attributes and characteristics

Basically, all attributes that apply to Cloud Computing also apply to PaaS. However, it is necessary to add some more specific attributes to PaaS itself. The below mentioned topics are my first findings:

  • Advanced Service Management. It is easy to run workflows on a PaaS Solution and they are often supported by a visual designer. Monitoring is very easy and comprehensive. It is often not based on the VM level since we are talking about abstraction here.
  • Elasticity, Flexibility and automated Resources. A key feature for PaaS is the elasticity, flexibility and resource automation that is going on in the background. In a PaaS environment you simply don’t realize the automation that is going on since this problem is abstracted from you.
  • Development-focused. Developers, Developers, Developers! A PaaS-Platform is all about developers. It is dedicated to them to get rid of the pain of using a complex deployment process or to install a software stack. With a PaaS platform, you simply don’t need to take care of your stack any more. You can focus on what is key – being a developer that delivers great apps.
  • Abstract and easy to use APIs. APIs, that are available are easy to use and abstracted. If you use a messaging service that comes with the platform, it is very simple and you don’t need to read a book first to get started.
  • Ease of operations. As stated earlier, it is also very easy to deploy new apps. In many cases, it is possible to have a test-environment and staging platforms. Deployment is often done from the Development environment itself.

Platform as a Service Types

When we look at currently available PaaS Solutions, we can see that there are some major differences in these solutions. Therefore, I believe that it is necessary to define them individually and focus on some key service types:

  • Application PaaS (aPaaS). This is a fully-featured PaaS-Platform that allows you to built applications on a pre-configured stack. With Application PaaS, it is possible to have all kinds of applications.
  • Dedicated PaaS (dPaaS). A dedicated PaaS platform is made for a specific SaaS-Application. The main focus of a dPaaS is to extend the SaaS-Application. The possibilities are somewhat limited to the aPaaS.
  • Integration PaaS (iPaaS). An iPaaS platform is made to integrate different platforms, services and applications. It is also often called a middleware.
  • Big Data PaaS (bPaaS). A bPaaS is built to handle all kinds of data driven applications. A common example is an easy to use Hadoop platform that can be run out of the box and enables the above described points.

Feel free to express your thoughts/opinion on this!