No Rest for the Query -- ADTmag

No Rest for the Query

By Alan R. Earls
January 1, 2006

The Big Idea

EMBEDDING GOOGLE

Data from the Web and apps such as Microsoft Office is making search engines ubiquitous. Orgs want better search capabilities to enhance the value of existing apps and minimize the frustrations of end users accustomed to the simplicity of search engines such as Google.
What most orgs really need is a universal search capability that can extend beyond any one app. Getting there will require a thorough needs assessment, APIs you can work with, good engineering and patience.
Search is a hot research area for suppliers. Emerging search strategies designed to streamline the hunt for information include conceptual searching, indexing, clustering, classification, taxonomy, profiling and summarization.

According to Monica Sharp, product manager at Siemens Medical Solutions, her INVISION health information system customers must provide daily information to non-technical end users within their healthcare organizations to support their workflow and business practices.

"This places a burden on the IT staffs," she explains, who must search for the right information and then build, schedule and distribute these reports. This inefficient process can cause delays and often results in incomplete or redundant information.

Today more than half of those customers use Siemens Express Query to get the info they need. Express Query (with search engine capability provided by Progress Software subsidiary EasyAsk) was developed in 2001 to provide a user-friendly reporting tool.

Using a browser-based user interface, Sharp says, end users quickly, easily and securely access their info. "With numerous navigation options, they may type in business questions in ordinary English, build queries through drop-down boxes and a point-and-click database tree, or run pre-saved reports with the click of their mouse," she says.

Answers are presented in several output options—the most popular is a spreadsheet—and end users can easily save search results on a workstation or distribute reports over a network. End users can teach their language to the EasyAsk dictionary, the heart of the app, further increasing the conversational ability of the search tool.

Sharp says the decision to implement EasyAsk was not difficult. The ability to develop a Siemens offering using EasyAsk, in conjunction with any ODBC compliant data source, was essential, however. Siemens uses many different database management platforms and needs to port Express Query across product lines.

Other software the company evaluated lacked comparable features and functionality. With 90 percent of Siemens' IS customers running Express Query, "it was a matter of designing an implementation of EasyAsk that supported our internal [ASP] requirements," Sharp says.

Discovery or search?

Keyword search is not dead, but its limitations have vendors hotly pursuing alternatives.

The problem with keyword search, according to Delphi’s 2005 report, “New Perspectives on Enterprise Search,” is that it is focused on retrieving a specific reference and not on “discovery,” which is actually finding what users want. As the universe of searchable data expands, keyword searches will become progressively less useful.

Fortunately, several researchers are harnessing ideas about information to produce search alternatives:

Conceptual searching produces results by having the search engine know the meaning of specific phrases, if necessary deriving that meaning from other words or phrases in close proximity. The “knowing” can be provided by human input or through machine-based inferences.
Unified indexing implies that external and internal content has been indexed so well that users can rely on it for finding anything.
Thematic indexing means related items have been carefully linked under a single, manageable topic.
Clustering makes the relationship between documents explicit, using themes, key terms or other shared linguistic patterns to group documents on the basis of semantic attributes.
Classification harnesses semantics in an attempt to accurately categorize the meaning of individual words, sentences and paragraphs.
Taxonomy is a hierarchy that defines “type-of” or “part-of” relationships.
Profiling supports the creation of user- or group-specific profiles that map stated interests described as subjects, sources, keywords or content samples. Profiles can be generated on the basis of rules or through automated inferences.
Summarization focuses on the key elements in a document to simplify searching.

—Alan R. Earls

Global search
Siemens isn't the only organization to add search capability to an app. Everyone these days is looking for search capabilities that can enhance the value of existing apps, power new apps and minimize the frustration of end users accustomed to the simplicity of search engines such as Google. Vendors, some new, some familiar, are obliging with an array of products.

Seybold Group analyst Susan Aldrich believes search engines have become ubiquitous over the last decade, driven in part by the growth in unstructured data generated by the Web and by Microsoft Office applications.

"Suddenly the impetus to get at that data has increased, and the capability to do so has increased, so now application developers have started to ask why they need to have users go elsewhere to invoke a search capability," she says.

Human factors also play a role in the quest for better search engines. "When you watch user behavior on a Web site, you often find people getting lost after a search or running into ambiguities," says Guy Creese, an analyst at Ballardvale Research.

Buy vs. build
Matt Kaiser, product manager at Ceridian, an HR information services company, says his customers wanted more flexibility to slice and access data. The Ceridian IT team looked at all the data users wanted to access and decided to plunge into building its own search and reporting tools.

"At first we thought we would be able to build it, but then we grew up," Kaiser recalls. After realizing that a buy would be preferable to a build, Ceridian began looking at various vendors. Like Siemens, they were drawn to the search capabilities provided by EasyAsk. The search engine is easy to use and can accommodate sophisticated and casual users.

Although Ceridian has not done any formal ROI, Kaiser is certain EasyAsk was the right choice. The company now looks to outsiders as suppliers of code more frequently.

Monster style staffing
Medical device manufacturer Boston Scientific also needed better search capability for an internal app. Personnel recruiters had created a staffing automation environment for job candidates' resumes that spanned the company's 12 domestic hiring locations and 5 international locations. However, admits David Pantano, national recruiting manager, "Everything we did in recruiting was a manual process. It was slow, with duplicated efforts and it was costly."

To streamline the process, Pantano and his colleagues issued an RFP that attracted 22 respondents. The company selected Recruitmax, a workforce management software provider. Akey feature of Recruitmax is the ability to sift through resumes using an embedded search engine called HireReasoning, developed by Engenium.

Recruitmax does not depend on Boolean expressions; it uses its own algorithms and learning. The result is that a recruiter can cut and paste a job description, say for a Java developer, into a search box, and Recruitmax will not only generate a selection of job candidates but also search on its own using related concepts such as JDK, J++ and JCK, in addition to Java.

Google generation
Today embedded search simply needs to be more effective, asserts Whit Andrews, a research VP at Gartner. "What we have here is a world where our employees are now essentially Google addicts," he says. People want to interact with apps in a Google style, especially younger people, who have always had Encarta, Yahoo or Google to help them. Search rather than structure is essential to their view of the world.

Open-source search

When Contact Network wanted to strengthen its flagship application’s search capabilities, the company quickly put aside build-your-own notions in favor of an open-source solution. Today Contact’s app, designed to uncover existing professional relationships between employees and potential clients, gets much of its muscle from Apache Lucene, an open-source, fulltext search app written in Java.

“Our product is built on SQL Server,” CIO Geoff Hyatt explains. “We switched from the built-in search feature in SQL Server to Lucene 3 years after our product’s introduction because it has more power and functionality, and [it] has the ability to deliver more accurate results.”

The open-source search add-on has proven to be a low-maintenance and low-cost option. “We would have paid for it because it is worth it,” Hyatt says.

—Alan R. Earls

Even the industry's iconic apps are feeling the heat. Outlook has had search capability for several years, but Microsoft is now being pressed to improve it and make it more intuitive, says Andrews.

Of course, not everything is built to be searched Google style. Databases, for example, are built to be parameterized, Andrews notes, because that is inherent in the database structure. Orgs must decide whether to conform to an app's inherent search structure or to trump it with new search capability.

Googling the enterprise

The real need in most organizations, whether it is expressed formally or not, usually boils down to a universal search capability that can extend beyond any one app, says Whit Andrews, a research VP at Gartner.

Several suppliers offer enterprise-oriented search apps—FAST, Autonomy, Endeca, IBM, Hummingbird and Convera. “[Apache] Lucene is a well-known open-source offering that is getting a lot of use as an embedded search capability,” says Andrews.

With OEM or embedded search apps, orgs should bear in mind a few key thoughts, Andrews says. Look at the search engine provider’s contractual relations. “Are they going to offer generous terms that will allow you to actually make money off of the search capability?” he asks.

Harder than it looks

Think through the challenges that search add-ons may create. Most important, Andrews says, is getting APIs you can work with. “The ability to integrate into a variety of content repositories—things like ODBC compliance and specific brand relationships such as Informix or Oracle,” he says, are central to getting value from embedded search apps.

Looking at the APIs is crucial, but it isn’t enough, asserts Guy Creese, an analyst at Ballardvale Research. “You need to really see what you are getting and whether you will be able to work at a level that is granular enough to accomplish your search goals,” he says.

Dan Keldsen, an analyst at the Delphi Group, cautions: How easy it is to add a search engine and how well it will work still depend on a thorough needs assessment and a good engineering approach. “The effort and the results can range all over the place,” he warns. If you have different search engines already within apps, portals and so on, and you try to bridge that, it may be hard to get good at searching for any one thing, he adds.

To do search well is still a hefty task. “You can put together a quick-anddirty search solution, but if someone spends more time, you can start to peel back the functionality you really need,” Keldsen says. Eventually, you may get to the search capability that really provides what you and your users are looking for.

—Alan R. Earls

People are no longer willing to accept limitations, and they don't see why they should have to use multiple search points to get at business data, Andrews says. Because Google seems to give them everything they want—even though it is not able to access a great deal of information—people want to have a similar experience with their business searches, "and that's a tall order," he says.

ILLUSTRATION BY ROBERT PASTRANA

On ADTmag.com