In-Depth

Services by Design

• By Stephen Swoyer
• November 1, 2005

The Big Idea

Virtual Certainty

• Traditional software development will almost entirely be supplanted by a virtual paradigm in which Web services are orchestrated dynamically to power composite applications.
• Bring in WSDL files from Web servers, and you create pipelines of activity. If you string enough of those routing and transformations together, pretty soon you have an entire enterprise application infrastructure.
• Organizations automate many once-manual business processes; simplify their app dev and maintenance by introducing modularity and effective reuse; and optimize their business processes to ensure they align with IT.

The upkeep of your application infrastructure got you down? Take a number and get in line. Fact is, no organization relishes the beaucoup dollars and harried human resources it pours into maintaining its application infrastructure. Given the choice, most IT shops would gladly do away with production support, feature or function enhancement requests, scheduled maintenance, periodic refactoring and other budget dollar boondoggles.

That's not possible, is it? Well, that depends. Lend an ear to the technology visionaries at BEA Systems, IBM, Oracle or at any of several other SOA religionists, and you might find yourself singing the praises of service enablement-and of a new generation of composite applications. (See related story, "What's a composite app?," below.)

If it takes hold, service enablement will fundamentally alter the enterprise application landscape. And, at this point, anyway, few would bet against service enablement's eventually taking hold. This is in spite of the fact that most organizations are taking baby steps on the way to full-fledged SOA-dom. The trend, SOA proponents claim, is clear-and the viability of the SOA vision is itself beyond dispute. Call it SOAinevitability, of a sort.

And if service enablement truly is inevitable, it will have far-reaching ramifications for enterprise application development. In many cases, technologists argue, traditional applications will go the way of the slide rule, replaced by composite applications that consist of conventional apps working in combination with Web services and other consumable services. In the most radical articulation of this vision, traditional software development will almost entirely be supplanted by a virtual paradigm in which disparate Web services are orchestrated dynamically to power composite applications.

"If you've service enabled things the right way, [composite applications] should be merely a matter of configuration and not coding, and one look at our AquaLogic Service Bus product will tell you how that's possible," says Bill Roth, VPof marketing with BEA. "You bring in various WSDL files from Web servers, and you basically create pipelines of activity...[and] if you string enough of those routing and transformations together, pretty soon you have an entire enterprise [application] infrastructure."

Oh, the places you'll go
BEA isn't an entirely disinterested observer of SOA. Earlier this year, it announced AquaLogic, a technology initiative many industry watchers described as an all-but last-ditch effort to reinvent itself. (BEA has faced withering pressure from IBM and Oracle in the commercial J2EE application server space, while at the same time open-source app servers such as JBoss are also coming on strong.) With AquaLogic, BEA sought to stake a claim to a vanguard position on the emerging virtual application express.

But BEA's AquaLogic vision-which prescribes a heavy dose of service enablement for the legacy application woes that bedevil almost all enterprises-isn't necessarily a technology coup-de-main: The company's competitors have outlined similar visions, and-in the case of IBM especially-have been talking about composite applications for at least 2 years now.

Much of the grunt work of service-enablement involves pretty ordinary stuff: namely, wrapping existing applications in Web services to expose them to other service-enabled applications. Legacy applications aren't going anywhere, says Roth and other composite app enthusiasts: They're instead being abstracted- liquefied in BEA's parlance-by means of standards-compliant services. To the extent that a composite application consumes these services, it is necessarily interfacing with legacy apps, too.

In the end, says Kareem Yusuf, program director for IBM's WebSphere software platform, composite apps aren't such a radical idea. And with SOAs fast taking hold, Yusuf-like Roth and other SOA evangelists-argues that composite applications are themselves inevitable. He's long espoused the idea of service choreography, which essentially describes the managed interaction of services on the basis of business logic and associated rules. The benefits are obvious, he maintains: organizations automate many once-manual business processes; simplify their app development and maintenance by introducing both modularity and effective reuse, and optimize their business processes to ensure optimal alignment with IT.

When you think about it, he argues, it's a no-brainer.

Aha, not so fast
Although vendors and software decision makers embraced service enablement with an uncommon gusto, enterprise codejockeys are more skeptical about SOAs. Their skepticism starts with one of the fundamental assumptions of the composite app vision-the inevitability of service enablement. "SOA...makes the fatal assumption that one can create totally generic services that can be tied together like Legos," argues Ramon Leon, a veteran programmer with an Internet software and e-commerce development house. "In practice, this tends not to work so well for most circumstances. The other flaw is that many of the people writing these services do it backwards. They write the service first and then try and make applications that use them. This predictably leads to services that don't quite fit the application's needs."

This isn't to say that programmers are reflexively anti-SOA, however-just that codejockeys seem to have a much more pragmatic take on service enablement than do their executive masters. As a result, many developers see a future in which the large-scale SOA vision of Joe Enterprise Architect gives way to a kind of hybrid infrastructure in which some but not all applications or resources are service enabled. Call it quasi- or best-effort- service enablement.

The point is that several foundational premises of the virtual app vision aren't yet in place and might never come to pass. There's the inevitability of SOAs, for starters, but there's also the emergence of enabling technologies, such as enterprise service registries, security and policy management technologies, and business process standardization efforts, like the still-gestating Web Services Business Process Execution Language. (See related story, "SOA showstopper?," below.)

As a result, even programmers who have a charitable take on service enablement are skeptical of the virtual application vision touted by the big application server vendors. Take William Pietri, an application developer, author, educator and principal with software development consultancy Scissor, who says he sees lots of similarities between service enablement and many object-oriented design philosophies.

"I like the trend toward loosely coupled systems with clear service interfaces; it seems like object orientation writ large, and I think the benefits are similar," he says, noting that both the SOA paradigm and most OO programming models "aim to encapsulate the technical craft inside a clean interface, and both aim for loosely coupled systems joined by message passing."

Ironically, Pietri suggests, both paradigms also share some problems. "Honestly, quite a lot of shops are still struggling to do OO design right. I'm just not seeing developers naturally producing systems that will be easily integrated with external apps," he argues.

Object oriented all over again Pietri, like many programmers, confesses that he isn't altogether familiar with the composite apps "fad." Based on what he's heard, however, Pietri sees a parallel in the OO triumphalism of yesterday and the SOA euphoria and composite app optimism of today. "A decade or more ago, people were talking up the future of object orientation as one where OO enabled a component marketplace where you could swap in one component for another. For example, if you didn't like one vendor's graphing widget, you could buy another one and just pop it in place. In practice, that never worked out," he points out. "It sounds like virtual applications are a similar pitch, and I'd expect the results will be similar."

He isn't alone. Other codejockeys smell an OO antecedent in the composite application-scape, too. "Isn't that the old, old reuse siren song?" asks Laurent Bossavit, a programmer with app dev consultancy Exoftware. "[You] assemble apps from reusable components instead of writing it all from scratch. Except that components are no longer binaries, but [the] ends of Web wires."

This isn't to dismiss the composite apps vision by dint of association with the exuberance of paradigms past. OO design is an established and vital software development methodology, after all. It's just that some of the most optimistic expectations of the OO design revolution haven't completely come to pass. And if you think of composite or virtual applications as an especially optimistic take on the SOArevolution, it's not unreasonable to suggest that a similar irrational exuberance might be at work here, too.

That's a perspective endorsed by Matthew Fleming, an enterprise architect and freelance developer whose clients include a prominent reseller of automotive parts. Fleming isn't a knee-jerk SOAcritic, either. He cites the example of a successful service-enabled e-commerce application that he helped design for his automotive reseller client, which has a thriving mail order (via catalog) and Web-based retail operation.

Showstopper and starter
Even so, says Fleming, the idea of a marketplace of consumable services, which is in at least one sense a prerequisite for any composite application paradigm, is a showstopper for many businesses.

"[Their] Web site allows third-party vendors to add products to the user's shopping cart via a Web service. However, the business processing is definitely handled by [the automotive reseller]. The issue with a virtual app is that businesses want to control their own business processes and validations. They want to do this because their customers are the ones-and ultimately the business itself-who would suffer if such processes added unnecessary encumbrances to purchasing a product, seeing tailored sports information or whatever."

That's one of the biggest problems with both service enablement and composite apps, says Jeffrey Frederick, a senior development manager with software quality assurance vendor Agitar Software. If a marketplace of consumable services is ever to emerge, and if virtual apps are ever to thrive, businesses will first need unprecedented proof of quality assurance, he argues.

"One of the challenges of the virtual application vision is that of collaboration and of trust at a distance," Fredrick says. "If you are going to base your success on a service provided by someone else, how do you begin that relationship? We think a key part of this relationship will be a transparency into the development process that is now almost unknown, and that quality level agreements will be common."

Not all SOA enthusiasm is premised on the idea of consumable external services, however. And to the extent that businesses embrace service enablement to improve agility and simplify the app dev process, what's not to like about composite apps that are cobbled together from internal services?

Plenty, skeptics say.

Across the great divide
There's a real sense that business leaders and IT are at two removes from one another. The former tend to view IT as a rogue practice that must be brought to heel (made accountable to the business), and the latter has a Dilbert-like take on the intelligence and efficacy of business decision-makers.

Not surprising, some programmers see SOAs and virtual apps as manifestations of this conflict.

Executives are obsessed with controlling risk, and a paradigm in which applications are cobbled together from ganglia of precisely delimited and reusable services helps to mitigate risk and uncertainty. It's enough to make a pointy-headed boss start drooling. The danger, says programmer Leon, is that business leaders can't be trusted to pursue service enablement in an intelligent manner. "I think [the virtual application space is] a pipe dream cooked up by people who have little practical experience actually implementing real code for real people," he argues. "When the business world wakes up and starts listening to developers and letting developers guide them on what is and isn't doable-a huge cultural change not likely to happen anytime soon-then and only then will more generic pluggable applications become a reality."

To the extent that composite or virtual apps are sold on the idea of eliminating a business's dependency on the vagaries of software development and software lifecycle management, Leon and other codejockeys say, they will almost certainly disappoint.

"The issue is that this has been tried before in the software world. At some point in every software engineer's career, he or she will ask themselves: 'Why is it that I cannot automate myself out of a job?'" Fleming says. "It seems that after you make enough tools for a business- tools that get more abstract and robust with each iteration-the business could expand and grow without developer intervention. That never seems to work out in practice," he comments.

Believe in the payoff
There's another wrinkle: IT pros are sometimes nearly as clueless as business leaders about just what service enablement means. And when you combine an SOA mandate from clueless executives with cluelessness on the part of IT worker bees, you've got a recipe for muddling through, says Ken Auer, a principal with agile software development house Role Model Software. Auer speaks from experience. "One client is insisting that everything move toward SOA," he confirms. "There are multiple problems, however. The first being that the folks mandating it don't necessarily know what that means, and the second is that many of the worker bees in IT don't know what it is either."

In Auer's view, this company almost certainly has unrealistic expectations about what service enablement means. On top of that, he says, many of the company's existing applications are poorly architected-in many cases consisting of little more than scripts talking to databases to present screens. So they're far from ideal candidates for service enablement. The atmosphere, he says, is one of general confusion. "We recently wrote an application for this client that we were told required interfaces to nine services that were already in place. After a little more research we found that there was really only one of the nine services in place, and due to expediency, we were encouraged to just make the database calls we needed and write code to fill in the details for the other eight."

Missing services or no, Auer's team delivered a service-enabled app on time. But the company's SOAignorance is still a problem. "We were also encouraged to provide a service for others to plug into. We asked which service and who was going to use it and received nothing but silence."

Not all of Auer's experiences with service enablement and composite application development have been completely Dilbert-esque, however. But even successful composite app dev projects have entailed a setback or two, he says.

"[Another] client had a Web application that we wanted to get a service from, and another service under development that was supposed to provide a lot of things for us," he says. In the first case, Auer and his team had to do a lot of bug-fixing on the first application, and its exposed service, to make it reliable enough to use. And for the second service, Auer requested a simpler interface, which resulted in additional delay.

Eventually, Rolemodel delivered a functional composite application. "[This] application certainly uses two Web services very effectively to provide the application we were chartered to write," Auer says. "We shipped in November of last year and are in the process of our 10th deployment. Both of the Web services we use are certainly useful to other applications, [namely] a scripture server and an online subscription service."

Even so, Auer endorses a kind of healthy skepticism on the subjects of SOAs and virtual applications. "The reality is that business processes change, technologies change and needs of users change. The other reality is that the understanding of all of these are very fuzzy-no matter how well specified people think they are," he concludes. Best practices still apply, says Auer, and many of the bogeymen of traditional software development-poor communication, bureaucratic fiefdoms-will carry over, too.

And that's putting aside, too, the question of whether the majority of enterprise programmers are themselves up to the challenge. "Many existing applications are not factored well, and many of the developers who built them have not learned to factor them well," Auer says. "There are many, many developers who never learned object-oriented software development, which is really the underpinning from any component-oriented anything," he concludes. "SOAs do not require systems built from an object-oriented perspective, but without an object-oriented thought process, the chance of building good services are slim."

Nevertheless, Auer allows, "the payoff can often be worth it."

Composite apps in practice
The appearance of inevitability. Irrational exuberance on the part of technology evangelists. Eager-perhaps uncritical- acceptance by business leaders.

These are some of the factors that have helped fuel the remarkable rise of SOAs and that have encouraged some technologists to wax about a composite or virtual application futurescape. But none of these diminish the basic attractiveness of SOAs. To the extent that service-enablement is done intelligently- responsibly, pragmatically- many enterprise codejockeys say they are onboard. To a lesser extent, programmers are intrigued by the idea of virtual apps, too.

For starters, says Robert Watkins, a senior Java architect with one of Australia's largest financial services institutions, service orientation does make sense for many common applications. "Some technologies are particularly well suited now to SOA. Authentication and authorization is largely solved via LDAP these days, for example. Databases are becoming more SOA-ish through the increased use of abstraction layers built on top of SQL," he notes.

Watkins says that both his current and previous employers are taking substantive steps toward service enablement. "At my current organization, we are working on making our core application expose a true SOA interface for allowing third-party organizations to interact with us- and vice versa. At my previous employer, we were focused mainly on creating SOA-based interfaces to our core systems to allow other internal applications to use them better.

He also thinks composite apps are a good thing, but he's skeptical about whether they will be feasible.

"The main reason is that the services have not become fully generic yet; you still need a lot of glue code in the middle," he argues. "For example, I may have an accounting service and a [point-of sale] system. In order to get transactions raised by the POS service to register in the accounting service, there's a lot of code to be written-you can almost guarantee that I can't simply tell the POS, 'Hey, here's your accounting service- use that.' The reason is that there is no generic accounting interface that a POS system could expect."

Cory Foy, a programmer with a mobile software development company, says his employer is already a practitioner of virtual application development. The company Foy works for designs custom mobile applications that consist mostly of canned Web services.

"We do...most of the [application] processing through Web service calls," Foy explains. "This allows us to separate our UI from the data layer underneath through a Web Service layer. This has worked well for us, though, of course, [it] is slower than a direct database call," he continues. Even in this case, however, the use of Web services processing is a reflection of business conditions, not the result of a top-down technology mandate. Most of his company's customers are using wireless cards for access, so speed isn't a prime concern.

Foy says he's pleased with the quality of these quasi-virtual applications. "Our UI portion of the app is fairly stable. All of the modifications happen dynamically based on the results of Web service calls and config files," he says. He's far from an SOA or virtual app evangelist, however. "I don't think [they're] fringe cases, but I do see them as specific solutions to specific problems."

Looking for the right one
Jonathan House, an IT director with Amirsys, a medical technology company, says he's ready for virtual apps. "We are well positioned to take advantage of virtual applications if we ever find one that would be useful to our organization," he comments. "Our application interactions are segmented into specific internal services of which there is well-defined interfaces that allow us to replace the underlying service by simply creating a new interface implementation for that service." But House, too, says his company's approach hasn't been dictated by a virtual app just for the sake of it but by sound design sense. "Our approach to decoupling our application services is not motivated by a potential move to virtual applications, but rather from good enterprise systems integration practices."

Based on his experiences with one of his company's service-enabled applications, however, House is skeptical about the viability of the virtual application vision. Even if companies like Amirsys take pains to segment their internal application interactions into services, House says, the third-party vendors that must inevitably provide key services- financials, inventory management, customer information management and other key processes-will almost certainly complicate things. They have the oldest reason in the world to do so, he points out: vendor lock-in.

"There is exactly one product that we are using that even comes close to the virtual application concept-Salesforce.com," he says. "We have done some work to integrate this application into our IT environment... with success," he comments. But the promise of service portability-to plug generic, plug-and-play CRM services into the customizations Amirsys has built-remains elusive. "There is no chance of us switching vendors-the interface points are tightly coupled to how Salesforce.com does business."

See Related Story: Codejockeys won’t drink the SODA

On ADTmag.com
Components for SOAs and other projects
By ADT Staff

App integration reflects a new ideal
By Alan Radding

Does service enablement = vendor entitlement?
By Stephen Swoyer