Embracing open source An expert look at the cutting edge of corporate technology
Open source solutions have changed the way people think about software and hardware alike.
As defined by Don Duet, head of the Technology Division at Goldman Sachs, "Open source is fundamentally the ability to take intellectual property and make it broadly available. People can look at it, enhance it, change it, incorporate it into their own solutions and really enable that IP to become much freer and much more able to create value over time."
In other words, open source computing lets users get the most out of their IP. They have the flexibility to apply the framework to several applications without starting from scratch.
"Open source has changed dramatically in the last several years. We've thought about these changes in three major categories: Variety, volume and velocity."
DON DUET
Variety speaks to how open source used to be confined to software – programming that could be improved or adjusted to fit different business needs – but has now evolved into hardware IP, like specs, servers, and data center designs.
Volume speaks to the amount of open source content that's available, which has grown astronomically in the past few years. Major growth in volume is largely due to the fact that open source IP isn't just created by individuals anymore – it's created by huge corporations, too.
Open source also must be viewed in terms of velocity, or how quickly it develops everyday use-cases. Duet says that open source is now fully permeated in technology, and points to the rise of the Internet of Things – made possible by the ability to analyze disparate data sets on a massive scale – as a triumph of open source philosophy.
As Duet mentions, open source began as a software innovation for individuals, but has now become a major asset for large companies like Goldman Sachs.
On June 22, 2015, Goldman Sachs took a leap forward in open source software by joining the Linux Foundation’s Open Container Initiative, a coalition of technology firms working to establish open industry standards for software containers.
A lightweight alternative to virtual machines, containers allow automation of application deployments. Instead of replicating a full compute stack, containers are highly available, horizontally scalable micro-services that can run on any server regardless of the underlying hardware. The container bundles all requirements and dependencies needed to run a process, delivers them to the cloud, and relies on cloud infrastructure to make sure that the service level agreements are met.
The Open Container platform leverages two elements native to the Linux O/S: Control groups which determine the size of available storage, compute and network resources; and kernel namespaces which ensure container isolation. As a result, multiple containers running as isolated processes on a server can share the same kernel and executables.
Containers are more efficient than virtual machines because they represent a fundamentally different architectural approach. We have found they yield several dramatic efficiencies:
- Quick iterations for our software development lifecycle: Provisioning time has been reduced from hours to minutes or seconds.
- Improved utilization: We can multiplex our infrastructure to allocate enough resources for any given process and for the exact duration it requires.
- Automated business continuity and failover: Machine-readable application modelling allows higher-level declarative statements to be intelligently handled by a platform; when a machine is lost, we resurrect the process it ran on a different machine or datacenter and update the runtime dependencies to ensure continuity of the applications and services it supports.
- DevOps efficiency: Standardizing process descriptors and embedding application owner knowledge into the application model provides a consistent, traceable link throughout the product lifecycle and runtime management.
- Infrastructure updates: By decoupling the application owners from infrastructure, platform owners can now update the underlying OS version, hardware specs and other components autonomously.
The story of open source doesn't end with software, however: Open source hardware is now an irreplaceable piece of the IT puzzle. For Goldman Sachs, the Open Compute Project (OCP) – originally developed at Facebook – represents the future of efficient hardware deployment.
The OCP is a community formed on the open source software model to drive the design, operation and management of server, storage, network and data center hardware and management. It consists of engineers from both the vendor and client communities.
Our goal is to create designs that are vanity-free, low in cost and use significantly fewer parts. At the same time, by keeping the management of these components simple and similar, we encourage designs that can scale to high-performance computing (HPC), cloud and other large deployment use cases.
RANJEEV, ENGINEER, GOLDMAN SACHS DATA CENTER PRODUCT GROUP
The OCP is organized into eight tracks: Server, Storage, Data Center Design, Networking, Hardware Management, Certification, Open Rack and Solution Providers.
A track is basically an open forum for vendors and clients.
"We typically have two meetings per month: One is a high-level hardware management meeting, the other is the working track meeting for the multi-node management spec," says Ranjeev. "We have 10 to 15 people who are regular participants, and of course others come and go for shorter periods of time. Like the OCP community, we have both vendors and clients, representing financial institutions, social networking companies and so forth. Clients come up with the wish lists; vendors explain what can and can’t be done and the costs for each feature; and collectively we agree on the feature set."
Goldman Sachs has already started deploying OCP hardware in both its dynamic computing and compute farm environments. The firm is also writing a common hardware management framework.
Open source is already a powerful business asset, but part of its beauty is how easy it can evolve with changing business needs.
Duet isn't ready to predict the future just yet, but he is confident that there are bright years ahead for open source.
"Open source has changed so much in the last several years. Looking forward, one thing that we think is going to be a constant is the impact that it has."
Posts
