In this post, we attempt to define and score key criteria that should be used to measure and assess how well a particular Platform-as-a-Service (PaaS) offering meets the standard of “open”.
We examined five PaaS platforms: Cloud Foundry (from VMWare), DotCloud, Force.com, Google App Engine and OpenShift (from RedHat). We scored each against five criteria for openness: choice of infrastructure, choice of platform, portability, choice of support and open source licensing.
Cloud Foundry, DotCloud scored at the top with a B- on our open PaaS criteria. OpenShift was a little behind with a C+. Despite the fact that all three vendors claim to be Open PaaS, they still fall short on some of our key criteria of open source. In many cases they have announced the intent to provide for more choices and openness in the future. Google App Engine and Force.com don’t claim to be open, and not surprisingly scored a D and D-. Detailed explanations of the grades are provided below.
A Closed Beginning
As cloud computing has evolved, the early adoption has come primarily in the Infrastructure-as-a-Service arena. Early Platform-as-a-Service offerings (Google App Engine, force.com) gave developers little choice of languages, frameworks and runtime platforms. In addition, most of these PaaS offerings were tied to a particular IaaS provider, creating lock-in for those that chose them. As a result, developers and users gravitated towards more open IaaS offerings that let them pick and choose stacks and languages to meet their particular needs.
The Move to Open PaaS
The battle for adoption has now shifted, as a variety of vendors and cloud providers work to win hearts and minds of developers, thereby attracting developers and users to their offerings. Vendors in the PaaS arena are looking to define their offerings as “Open PaaS” – implying that they provide developers with choice and flexibility. However, current PaaS offerings still fall short of many of the criteria that might be expected for an “Open PaaS” label. In some cases, these early PaaS offerings have announced roadmaps that would expand their level of openness.
Criteria for Open PaaS
Below are the five key criteria that we used to score PaaS offerings.
- Choice of Infrastructure
PaaS offerings claiming to be open should allow developers a choice of infrastructure. This includes not only choice of cloud, but also choice of operating system and virtualization technology. - Choice of Platform
Open PaaS offerings should allow developers choice of language and platform stack. Developers should be able to pick and choose stack components from web framework to application server to database. Developers should also be able to customize the platform, adding in additional components as needed. - Portability
Open PaaS offerings should provide for portability, allowing developers to move data, scripts, and applications to other PaaS or IaaS offerings. - Choice of Support
To fully realize an open PaaS offering, developers and users should not be locked in to maintenance and support contracts that have been common in the world of proprietary software. Developers and users should be able to choose self-support or get support on software components from a different vendor. - Open Source Licensing
In order to ensure portability, the components and tools in an open PaaS offering should be under open source licenses. This guarantees developers can move to other platforms without requiring costly re-tooling of applications.
Summary
Open PaaS is a new, and we think positive, development in cloud computing. There are several offerings that are beginning to deliver on the promise of open PaaS, but there is still some room for improvement. Do you have other criteria that you would use to measure open PaaS? Would you grade some of the offerings differently? We’d love to hear your thoughts and ideas.
Related posts:
- Comparison of Open Source Application Servers
- The Rise of the App Store: Where Does Open Source Fit?
- Comparing Open Source Reporting Tools for Use in the Enterprise
- PostgreSQL vs. MySQL: Which Is the Best Open Source Database?
- Open Source Database Applications Comparison Matrix




















What do you mean by infrastructure? Cloudfoundry runs on a laptop, Amazon, Rackspace, ESX and bare metal. I would give that an “A”.
John
By “choice of infrastructure”, I mean the ability to easily deploy a stack on EC2, Rackspace, and others, ideally through a simple “point and click” or choosing a target via command line. Yes, I could install CloudFoundry on EC2 myself and monitor/manage it myself, but that’s a lot more work and complexity than most users would expect from an “open” PaaS platform.
Hi Rod.
I wouldn’t mix openness with ease of use. I’d rate the linux kernel “open” but it’s really not a “point and click” type of thing to configure. Actually Windows is king of “point and click” but I wouldn’t rate it as “open” (albeit it does have other advantages).
Nice table though.
Massimo Re Ferre’ (VMware).
[...] Excerpt from OLEX Wazi [...]
Hi Rod,
Cool scorecard! It’s nice to see these offerings compared side-by-side. I wanted to point out that while many of these services are in some form of beta today, one thing we think sets OpenShift apart is that we can and will support the complete stack of middleware used for all open source components that are not third party. So for example, MySQL, Ruby, Java & Java EE, Messaging, Perl, Python, PHP and a file system layer are all supportable by Red Hat ourselves (because we already support these in Red Hat Enterprise Linux).
Also curious why DotCloud and OpenShift have different “Open source licensing” grades when the approach is the same (as described in your notes.) We will open source other parts of OpenShift, but you should hold us to it and improve our grade when you see the code.
Issac (Red Hat)
Rod,
Any comment on why the DotCloud and OpenShift “Open source licensing” grades are different when the reality is the same (“all of the components are open source.”) ?
Yours,
Issac
You’re right. OpenShift should get a “B” in the “Open source licensing” category and an overall score of “B-”. This could improve further when everything is open source.