The use of data-grid technology in service-oriented architectures (SOAs) was the subject of a keynote address at the first annual IT Architect Regional Conference in San Diego, which took place last week. Dave Chappell, Oracle's VP and chief technologist for SOA, spoke on the topic of "Next Generation Grid Enabled SOA" at the
IASA event.
Chappell described the sort of problems that happen when processing large amounts of XML data and trying to ensure reliability and scalability in an SOA.
Oracle's model for grid-enabled SOA stems from technology that the company acquired about seven or eight months ago when it acquired Tangosol. Oracle now offers this technology for mission-critical applications, typically involving extreme transaction processing, through its Coherence product line.
What Is a SOA Grid?
The SOA grid concept is based on the data grid concept, Chappell said. A data grid is a set of cooperating software nodes that know how to work together to manage the ownership of instance data for any given Java object or .NET object that you have in your application.
Applications use familiar data structures, such as a Java hash map or a .NET collection, and then put data into that collection, and that's basically handed off to the grid. Then, the grid automatically elects a primary owner and a backup owner for that piece of instance data for that given application. And it does all of that in memory, so there's no dispersed systems required. This is how the system achieves high availability, Chappell explained.
The nodes on the grid are aware of each other. They are constantly in communication with each other about what instances of data they are owning and the health status, so that if the primary owner of something goes down, the other grid knows automatically to detect that and reelect a new primary and backup owner for that piece of instance data. And that can happen in the middle of updating a transaction.
SOA Solutions, SOA Problems
Chappell explained that much of the technologies associated with SOA have been worked out, but a set of problems still exists. While people speak of SOA as a means for reducing cost and complexity, those aren't the biggest reasons to implement an SOA. Achieving business agility is far more important, and SOA enables it.
"By building flexible business processes, we're able to take advantage of these new services that we are building and recombine them in ways that are better suited to meet the needs of business -- and allow IT to better react to the requests for changes that are coming from the business at ever-increasing rates," he said.
A few noteworthy technologies and concepts have helped enable SOAs, including:
- The use of business process orchestration tools, such as Business Process Execution Language (BPEL) engines;
- Basic SOA patterns for building composite apps that are constructed from service functionalities; and
- Loose coupling and modularity.
However, in the process of using these technologies -- and by choosing to use XML as the means for exchanging data between apps and services -- the size of the data that is being shipped around has been inflated by a factor of five, Chappell said.
With SOA, application silos are separated out and exposed as services. Such an arrangement presents problems in how to share and manage information across these services, Chappell said.
"If you are reusing a given application and Web-enabling it (i.e., via Web services), now the business logic will have a lot more demand on it." He added that IT also must deal with service level agreements on top of that.
Chappell admitted that there's tension between the SOA architects who build these pristine architectures and IT operations personnel. Architects bring the architecture to IT operations saying, "Here, you're responsible for owning this over time. Deploy it, scale it up and make it highly available." There's a bit of a problem there. How does that IT operations person justify how they scale up one process over the other?
SOA requires a too-tight relationship between the architect and those who run it over time. Moreover, IT seeks less complexity, not more.
Aim for Stateless Services
The answer to the scalability conundrum in SOAs is to write your services to be completely stateless in order for them to be highly available, Chappell said.
Being completely stateless means that interactions between the service requestor and the service implementation are stateless. However, as soon as the business logic itself and the service implementation becomes more complex, and also as the longevity of that service increases, then usually the need for statefulness also increases.
Various techniques and patterns have been used over the years to address more complex stateful interactions. One technique is called "state passing by XML payloads." It's largely what's used in BPEL engines today, Chappell said. The state is carried back and forth between the service consumer and the service provider. In the case of the BPEL engine, it packages up the entire context of the business transaction -- the XML table, the purchase order, or whatever else is being manipulated, Chappell said.
Another technique involves "a hardwired shared context database." This approach involves a tightly coupled relationship and it's the hardest to do. The request itself passes nothing or some key into some backend data. It's a relationship or contract between the service implementation and a backend database. Every time that service implementation receives a request, it uses that key to reconstruct the context from the backend database and reestablish the application state and put it back into the database when it's done. And that's how the application can deal with having a request get dispatched in multiple instances and still carry the context of the request from one to the next.
The ideal SOA grid is an infrastructure that provides state-aware continuous availability for service infrastructure, application data and processing logic. It allows you to have predictable scalability that scales out linearly, whether it's deployed on two servers or 2,000 servers, across a heterogeneous environment consisting of any combination of high-end or low-cost hardware.
By reducing the dependency of distributed systems, a SOA grid can provide a marked overall increase in performance and throughput. SOA grids can be thought of as linearly scalable shared memory pools that allow you to offload your data to external nodes. The grid can then grow out horizontally as needed.
Finally, while Chappell referred to Tangosol technology in his depiction of grid-enabled SOA, he said that these concepts could be implemented by any vendor using a variety of techniques.