The Cloud and Software Engineering

In my role here at EMC, I have the opportunity to speak to many of our software development teams about how software development is changing. One of the presentations that I give discusses industry trends that are impacting the way we think about designing and developing software. In that particular presentation, there’s a rather boring graphic that looks like this:

This graphic represents the fact that “the cloud” is an important concept that software developers (especially those of us working in the IT Infrastructure management realm) need to be aware of and need to understand.

How “The Cloud” Impacts Software Engineering

Many people would argue that “The Cloud” (be it a private cloud, a public cloud or a hybrid) doesn’t really change anything about how software is engineered since it’s all about infrastructure, and after all, infrastructure is not the concern of the software engineer, right? The problem with that mode of thinking is that one big promise of the cloud movement is flexibility, and software that runs “in the cloud” needs to embrace that flexibility. Consider these points:

1. Cloud offerings billing themselves as “Platform as a Service (PaaS)” generally attempt to abstract the physical hardware environment from the virtualized application platform that software is built on. At a very high level, what this means is that developers who build applications for PaaS offerings don’t have any control over how the hardware is configured, maintained and protected.
2. Cloud offerings billing themselves as “Infrastructure as a Service (IaaS)” generally provide a the ability for hosting of virtual machines that are fully-controlled by the customer, but like PaaS offerings, abstract the physical implementation and associated maintenance from the customer.

In both cases above, software engineers need to understand that the environment that their software is executing on is subject to change without notice. In the case of PaaS offerings, developers need to think about persistent storage and volatile storage rather than simply writing to disk. For example, consider an normal application that writes data to a storage device. In many cases, the developer would choose to write that data to a logical disk hosted by the server that the application is executing on. The issue here is that for PaaS, in most cases the “disk” storage available to your application is volatile in nature and in the event that the environment hosting your app changes (failover, patch maintenance, network maintenance, etc) the state of the “disk” is not guaranteed, so for data that must be persisted, developers need to create and maintain non-volatile storage. In the case of IaaS offerings, in the event of a failover or other need to move the hosted virtual machine, there is no guarantee that the state of the VM will be maintained (i.e., it’s possible that the VM could be reset to its initial state, and thus developers need to consider building in mechanisms to deal with that possibility.

The point of the above is that software engineers need to consider the inherit flexibility requirements of the cloud infrastructure when designing and building software that is deployed “to the cloud”.