Enterprise Search Practice Blog

Enterprise Search: Case Studies and User Communities

2008-08-28T19:25:54Z

While you may be wrapping up your summer vacation or preparing for a ramp up to a busy fourth quarter of business, the Gilbane team is securing the speakers for an exciting conference Dec. 2 - 4 in Boston. Evaluations of past sessions always give high marks to case studies delivered by users. We have several for the search track but would like a few more. If one of your targets for search is documents stored in SharePoint repositories, your experiences are sure to draw interest.

SharePoint is the most popular new collaboration tool for organizations with a large Microsoft application footprint but it usually resides with multiple other repositories that also need to be searched. So, what search products are being used to retrieve SharePoint content plus other content? A majority of search applications provide a connector to index SharePoint documents and they would not be making that available without a demand. We would like to hear what SharePoint adopters are actively using for search. What are you experiencing? If you would like to participate in the Gilbane Conference, and have experiences you to share, I hope you will get in touch and check out the full program.

On a related note, I was surprised, during my recent research, to discover few identifiable user-groups or support communities for search products. Many young companies launch and sponsor "user-group meetings" to share product information, offer training, and facilitate peer-to-peer networking among their customers. It is a sign of confidence when they do help customers communicate with each other. It signals a willingness to open communication paths the might lead to collective product critiques which, if well organized, can benefit users and vendors. It is also a sign of maturity when companies reach out to encourage customers to connect with each other. May-be some are operating in stealth mode but more should be accessible to interested parties in the marketplace.

Organizing functions are difficult to manage by users on their own professional time, so, having a vendor willing to be the facilitator and host for communication mechanisms is valuable. However, they sometimes need to have customers giving them a nudge to open the prospect of such a group. If you would value participating in a network of others using your selected product, I suggest taking the initiative by approaching your customer account representative. Communities for sharing tips about any technology are important but so is mutual guidance to help others become more successful with any product's process management and governance issues. User groups can give valuable feedback to their vendors and spur product usage creativity and efficiency. Finally, as an analyst I would much rather hear straight talk about product experiences from those who are active users, than a filtered version from a company representative. So, please, reach out to your peers and share your story at any opportunity you can. Volunteer to speak at conferences and participate in user groups. The benefits are numerous, the most important being the formation of a strong collective voice.

Researching Search with Intent Firmly in Control

2008-08-14T20:02:04Z

I have hit on intent before and our latest member of the Gilbane blog team, Fred Dalrymple has joined the theme with his entry this week. Welcome Fred! You have given me an opening for an already planned topic, how to conduct research for enterprise search tools, those that go beyond the search box. Actually, this guidance is appropriate for the selection of any technology applications.

Getting intent solidly defined is important for so many reasons, many of them relating to solving a business problem and the expected outcomes. Knowing what these are will give you the framework for isolating likely candidates, efficiently. A second critical reason for having strong intent is to stave off project scope creep. As a former vendor, and now consultant, I see this play out repeatedly as product research ensues. Weak backbones in selection team members or flimsiness of their business case leaves openings for vendors to promote additional features, which often distracts from what is really needed.

So, armed with the right skeleton, a strong framework, a core scaffolding you are ready to approach your research systematically. Four paths are open to a study team; I recommend using all of them, in overlapping passes. Discovery about products, product performance in real-world scenarios, vendor business relationships with their clients, and the user community you will be joining are all targets that need to be exploited. Discovering a user community on-line that might have expressed a potential problem with a vendor or product, should drive you back to do more research to discover potential limitations or why a user might be having a problem that they brought on through inappropriate implementation. Iteration in research for technology requires perseverance and patience. A comment on each path to research might be helpful:

Online research - This requires creativity and the most persistence to verify and validate what you find. I am amazed at how superficially many people read any content. We may be taught that good business writing requires a clear statement in the first paragraph of what follows with a solid summary at the end, but most content does not follow "good" business writing practices. You need to read between the lines, think about what is not being said and ask yourself why, follow every link on the sites of vendors under serious consideration. Look at vendor news notes and press releases to see how much activity is going on with product advances or new installations, and read descriptions of customer implementations to see how closely those deployments match your business need. Finally, search those customer names on the Internet in conjunction with the product name. This may retrieve public content that sheds more light on user experiences.

Professional groups - Professional organizations in which you participate are fertile ground for asking about what others in similar situations to yours are using. As you get closer to a final choice, go back to others you know personally or professionally to get answers to the direct question, "have you had any problems with this product or vendor?" and "what is the benefit of this product for you?"

Societies and academic institutions - These organizations publish content that may have a cost associated. When you consider thousands your organization spends on a selection process (in people time), contracting, licensing, implementation and deployment, it is wise to have a budget of several hundred dollars for reports that give detailed product evaluations. Get recommendations of librarians and peers as to publications' authoritativeness.

User and analyst blogs and industry publications - The same guidance holds for industry publications as for societies and academic publishers but you will also want to pursue blogs of users and analysts. Users are a great source of discovering tidbits about products and vendors but continue beyond what you discover to see if the comments are isolated or follow a pattern.

This is a longer commentary than I intended but the core of my intent needed flesh, so there it is.

Before You Start a List of Vendors: Map Your Course

2008-07-31T11:55:25Z

There is a pattern in how many small to mid-sized enterprises go about researching technology applications, one that does not serve them well. As I interact with colleagues, business affiliates and professional peers, I play into this behavior unwittingly. For example, how many times have you been on the asking or answering side of this question: "My organization is planning to procure a search system this year, what systems should we be looking at?" Too often, I make a quick judgment based on what little I know about the asker and toss out a few potential candidate vendor names.

This scenario plays out frequently and now I am uncomfortable because, as a consultant and search analyst, I know that there is a lot more I need to know before offering off-handed advice to that question. Here are some ideas for questions that you should be asking first so that, when someone like me wants more context, you have ready answers.

You first step is to survey your internal landscape and clearly document the following:

What are the business outcomes you expect to derive from the search product, who will be using it, under what circumstances and for what purpose?

What is the scope of the content that will be indexed for retrieval? Create a content map that explicitly illustrates: What, Where, Who, When. This means capturing what the content is in terms of document types and formats, numbers and size, and topic, and where it is being created, stored and managed. You need to know who created it, owns it, and will have access to it. Finally, it helps to document when it was created and information about retention.

Who will be involved in product selection and evaluation, who needs to sign off at every stage of selection and procurement, who will be involved in installation and deployment, and who will maintain the system on an ongoing basis?

What is your IT infrastructure and who controls it? If a schematic is not in place that depicts at least the portion of the computing infrastructure that will be integral to your search support, it is time to make sure one is prepared. You cannot make an informed decision about appropriate and workable search solutions without this information.

You will also be wasting the time of vendors when you seek product and licensing information if you do not have all of these issues sorted out. Much of the packaging of search products is dependent on numbers of documents or size of the corpus to be indexed, how the software will be installed, and who and how many will be accessing it. Pricing information will be vague until you have concrete content "demographics" to share with prospective vendors. You can't even establish a budget without answering the questions above, and you need a ballpark budget figure to help narrow your choices.

So, I am resolving to be more thoughtful in my responses when queried by friends and colleagues. Before answering I will be asking you for some meaningful data in advance of reeling off a list of products. It is time for you to do some preliminary research in-house before establishing the lineup of suitors. More on the next steps, next time up.

Taking Measure of Search on Vendor Sites

2008-07-24T12:44:31Z

As I was developing concepts put forth in the report Enterprise Search Markets and Applications - Capitalizing on Emerging Demand I bounced around the Internet a lot to verify information I had previously noted about products listed in the vendor directory. As I did so, evidence began to emerge about the ease with which I could resurrect an earlier retrieved bit of content. It mystified me that vendors of products to aid retrieval of content would make it so difficult to find information on their own web site. One assumption of mine has been completely debunked, that vendors would use their own search product to help site visitors discover more about their products and services. It made me wonder why they would not be showcasing the full flavor of their offerings.

The report was not written to evaluate specific products but rather to give a more holistic view of how the markets for products break down and how products themselves can be categorized. In order to do the latter, it required reading about many products with which I had no hands on experience. I wanted to understand how vendors were positioning their products, what markets they felt their products are most suited to satisfy, and what search problems were best solved with their technologies. Coming up with generalizations, trends, and differentiators was one purpose for my research. When I realized how difficult it was to dig out specifics from many vendor Web sites, I moved on, probably leaving stones unturned but time was not on my side.

Now I am going back to learn more about the problem with researching search, something complained about by a number of buyers I interviewed. Vendors are not making it easy for buyers to narrow their search for search, and shame on them. This should be a "no brainer." If you are a vendor pushing a product that is easy to install, implement and deploy, there is no better way than to put it to work on your own site. On the other hand, if you have products that are more sophisticated in terms of offering complex retrieval by leveraging refined ontologies or rules, you had better take the time to make it work well for finding nuggets on a few hundred pages of your Web site.

I am going to be writing more about this because the deeper I dig, the more interesting the results. For starters, of the first 28 vendor on my list, twelve have no site search. Of those that do, several use a third-party search engine, not their own. One major vendor's search result count displayed nearly a hundred records that matched the search while also displaying the breakdown of records by category. The trouble was the category numbers totaled less than 20. Hmmm!

Perhaps the trouble in "searchland" is that no one wants to take the time to implement, deploy and maintain search to satisfy the user. I keep saying, "it's not the technology; it's the thought and skill that goes into the back room implementation." Or is it? Stay tuned.

Search is Not Taking a Summer Break & Call for Papers

2008-07-17T17:38:55Z

Amidst post Gilbane San Francisco business I have been reading what everyone else has been writing about search the past couple of months. While there continues to be much speculation and gossip about the Microsoft acquisition of FAST, and which companies may soon be absorbed into larger entities, there also continues to be interesting activity among the mid-tier and start-up search vendors. Meanwhile, I advise those who aspire to acquire a search solution for "behind the firewall," don't wait for the "big players" to come up with the definitive solution to all your search needs because it will never happen. I'm in good company with other analysts who advise moving on with point search solutions for specific business needs. You will save money, and time because many of the new products are optimized for rapid deployment, in weeks or months, not years.

If you check out my new research report, Enterprise Search Markets and Applications; Capitalizing on Emerging Demand, June, 2008, you will find a directory to companies offering search solutions with choices for what Steve Arnold refers to as "beyond search." Deep test drives of many of these products can be found in his report, as well. Meanwhile, new releases of products listed, and new products both continue to be announced. ISYS, Coveo and Expert System (Cogito) have brought new offerings to market in the last month and Collexis, a relative newcomer, is drawing attention to itself by demonstrating its products at numerous meetings this year.

So, keeping reading and checking out the possibilities. While you are at it, be sure to put the Gilbane Boston Conference on your calendar for December 3 - 4. We are all busy rounding out the program right now.

I am particularly interested in hearing from those of you who have participated in the selection of a search product in the past two years, implementing or deploying a system anywhere within your own enterprise. Please consider sending me a brief proposal for a presentation at the conference. For your effort, you will get to attend all the conference sessions, as well as help the audience with the needed reality checks on what it takes to conduct a selection process and follow through with implementation. I particularly want you to share your learning experiences: the good, the frustrating, and the lessons you have accrued. Professional speaking experience is not required - we want stories. [You'll find my email on the "Contacts" page of the Gilbane site and you should also look at the speakers guidelines for additional information.]

Afterthoughts on the State of Search

2008-06-24T15:29:13Z

Search for the whole enterprise vs. point solutions was the subject of some discussion, especially since our keynote speaker, Stephen Arnold gave strong guidance that you can't think about one search solution ("product") for the entire enterprise and all content. This is something with which I pretty much agree, in most cases.

Just emerging from the Gilbane San Francisco conference, six sessions on search and a workshop I conducted, I want to share a couple of general impressions. Details and expanded reflections will follow in the days and weeks to come.

Search for the whole enterprise vs. point solutions was the subject of some discussion, especially since our keynote speaker, Stephen Arnold gave strong guidance that you can't think about one search solution ("product") for the entire enterprise and all content. This is something with which I pretty much agree, in most cases. However, a question arose in one of the sessions in which a couple of presentations talked about a single search engine for what appeared to be the entire enterprise. A member of the audience asked for clarification in view of Arnold's earlier comments.

I chose to intercede so as not to put our speakers on the defensive about what, for their organizations were very reasonable choices. Both of the cases were for research or professional services organizations with a high incidence of uniformity in the scope and type of content. They are relatively flat in structure with the bulk of the population being researchers: consultants, engineers, scientists. The applications were for intranets that were being leveraged to connect content and experts, so that from either direction (finding an expert and then looking at their content, or finding content to reveal expertise) other professionals could leverage organizational knowledge. It is a safe bet that other search does exist elsewhere in these companies, even if it is in stealth mode or embedded in other applications. Still, in general, large organizations with highly differentiated personnel with functional and disparate content requirements will find value in point search solutions that may only have purpose in a single internal domain.

To that point, if you are a finance professional or business manager you might want to sign up for a webinar this Thursday, June 26th, when I will be laying out a business case for a particular kind of search solution that is targeted at your demographic. This Apps Associates sponsored webinar also describes a solution leveraging Oracle enterprise search, but the ideas in it will give you a sense of what search can provide in your domain.

Judging from the topics presented on search, the reasons and ways in which it is being applied are more diverse than even I imagined. Opinions about what is good/bad, appropriate or not, and how to approach search technology ran the gamut of simple to complex. Two strong points of view were expressed about taxonomy vs. just tagging or letting the search engine categorize. Neither side would give an inch to the other as having an approach that is often "good enough." It is pretty clear that hybrid solutions offering both a structured approach to search where a taxonomy is applied through metadata, and auto-categorization by the search engine without a supporting taxonomy in the background will be applied in many enterprises.

What is Semantic Technology Anyway?

2008-06-16T17:33:10Z

Meaning is a very large concept in every aspect of search technology and dozens of search product sites include either the words "semantic" or "meaning" as a key element of the offered technology. This is not as far fetched as search product claims to "know" what the searcher wants to find, as if "knowing" can be attributable to non-human operations. However, how well a search engine indexes and retrieves content to meet a searcher's intent, is truly in the eyes of the beholder. I can usually understand why, technically speaking, a piece of content turns up in a search result, but that does not mean that it was a valid scrap for my intent. My intent for a search cannot possibly be discernible by a search engine if, as is most often the case, I don't explicitly and eloquently express what, why, and other contextual facts when entering a query.

The session we have set aside at Gilbane San Francisco for a discussion on current activity related to semantic technologies will undoubtedly reveal more meaning about technologies and art of leveraging tools to elicit semantically relevant content. I suspect that someone will also stipulate that what works requires a defined need and clear intent during the implementation process - but what about all those fuzzy situations? I hope to find out.

This is the last posting before the conference this week so I hope you will add this enterprise search session (EST-6: Semantic Technology - Breakdown or Breakthrough) being moderated by Colin Britton to your agenda on June 19th. He will be joined by speakers: Steve Carton, VP Content Technologies, Retrieval Systems Corp., Folksonomies: Just Good Enough for all Kinds of Thing, Prakesh Govindarajulu, President, RealTech Inc, Building Enterprise Taxonomies from the Ground Up, and Jack Jia, Founder & CEO, Baynote.

See you in San Francisco in person or virtually thereafter.

Enterprise Search Expands Beyond the Box

2008-06-12T17:14:02Z

Hustling through my preparation list before the Gilbane San Francisco conference I have come to the fifth session on enterprise search that I'll be moderating, Mining, Analyzing and Delivering Intelligent Content, featuring Amin Negandi, Principal, Echelon Consulting LLC, speaking on Enterprise Search at A.T. Kearney, and Rob Joachim, Information Systems Engineering Lead, MITRE Corporation presenting a case study on the development of An Expertise Finder Application Built on Enterprise Search. In listening to both of them talk about their projects, these are "must-attend" presentations for those seeking to build search-based solutions for their organizations. Both are examples of the practical and real challenges that surround value building projects. Both have positive outcomes but are hardly implementations that will become static legacy deployments; sustaining a value-based system is an ongoing activity.

As the session abstract states, there are as many technologies for finding content as there are types of content and types of enterprises. Locating a pile of links or citations is rarely the end game for those who really seek to leverage content. Both presenters in this session will talk about solutions that serve real and critical needs for one enterprise, in the first case being able to securely search content across a professional services firm in which collaboration is important within defined proprietary boundaries.

The second case also touches on the need for collaboration and sharing, in this case by enabling location of individuals who are experts. Using the context of content and associations to which they are linked for "defining" individual expertise, search filters relevant metadata to reveal those individuals. Connections are made to locate people and their professional work.

Delivering search results intelligently requires not only technology but also the art of the implementation team. Keeping the focus on specific business outcomes is the essence of ensuring that search delivers intelligent content. The stories of what problem was targeted, what tools were deployed, and how search was implemented by savvy search specialists are the most interesting and useful for learning. Finding out that serendipity also plays a role is getting closer to the best solution is always fun to discover in the process. We'll be listening on June 19th.

Enterprise Search Executives to Comment on Trends and Challenges

2008-06-10T12:14:57Z

Executives from four companies with unique products for solving search challenges within organizations will share their thoughts on what is most interesting, promising and problematic in the current market at Gilbane San Francisco, session EST-4 on June 19th. Because it is important to consider markets, this type of session gives the panel and audience a chance to recognize the perspectives of each other, sellers and buyers. I'll be asking the executives to comment on their product strengths, with emphasis on specific value propositions for buyers. There will be an opportunity for the audience to ask questions because what is on the minds of buyers is often the most interesting for everyone to hear. When one listens to questions asked of vendors and hears responses, it reveals two important things about markets:

Alignment that exists (or not) between what buyers want and what sellers have to offer

Challenges both face to create value

Think about it. When individuals are tasked with selecting any technology on behalf of an organization or organizational unit, their ultimate success depends on selecting a product and company that truly supports a value proposition for that enterprise business need. While the buyer has responsibility for implementing or selecting the right implementation team, s/he rightly depends on good guidance and a healthy relationship with the purveyor of product. When a technology company puts its product in the hands of a buyer it must do so with the confidence that its product comes with a total value package. By that I mean not just the technology but also the design, toolkit, and support team to guide its successful implementation.

No one, particularly vendors, likes to talk about negatives. However, given all the things that can go wrong for a buyer trying to manage an implementation team or for sellers who don't anticipate expectations about their products not explored during the selection process, it is important to consider factors that lead to failure or less than satisfactory outcomes. One question I will have for vendors is to share honestly some of the challenges or disappointments about the market that are a particular point of pain for them. Bad-mouthing customers is not the answer but conveying how hard most vendors honestly try to create value and how their best intentions can be derailed through miscommunication may help buyers and sellers smooth the communication flow.

After all, don't we all want to provide value for our internal and external customers? I think you will find the panel a receptive group: a 25+-year veteran of the information access market, a marketing executive with an international search and text analytics firm, a founder of a rapidly growing plug-and-play search solution, and a marketing VP working to position a company with a large scale solution against the "big-three" search solutions. You'll hear straight talk and interesting value propositions.

Speakers: Margie Hlava, President, Accession Innovation, David Haucke, VP Global, Marketing, ISYS, Laurent Simoneau, President & CEO, and Rebecca Thompson, VP Marketing, Vivisimo

What's in a Name: Information Access Software vs. Search?

2008-06-03T13:18:24Z

This one almost slipped right past me but I see we are in another shoot-out in the naming of search market segments. Probably it is because we have too many offerings in the search industry. When any industry reaches a critical mass, players need to find a way to differentiate what they sell. Products have to be positioned as, well, "something else."

In my consulting practice "knowledge management" has been hot (1980s and 90s), dead (late '90s and early 2000s), relevant again (now). In my analyst role for "enterprise search" Gilbane has been told by experts that the term is meaningless and should be replaced with "behind the firewall search," as if that clarifies everything. Of course, marketing directories might struggle with that as a category heading.

For the record, "search" has two definitions in my book. The first is a verb referring to the activity of looking for anything. The second, newer, definition is a noun referring to technologies that support finding "content." Both are sufficiently broad to cover a lot of activities, technologies and stuff. "Enterprises" are organizations of any type in which business, for-profit, non-for-profit, or government, is being conducted. Let us quibble no more.

But I digress; Endeca has broadened its self-classification in any number of press releases to referring to its products that were "search" products last year, as "information access software." This is the major category used by IDC to include "search." That's what we called library systems in the 1970s and 80s. New products still aim for accessing content, albeit with richer functions and features but where are we going to put them in our family of software lists? One could argue that Endeca's products are really a class of "search," search on steroids, a specialized form of search. What are the defining differentiators between "search software" and "information access software?" When does a search product become more than it was or narrower, refined in scope? (This is a rhetorical question but I'm sure each vendor in this new category will break-it out for me in their own terms.)

Having just finished reviewing the market for enterprise search, I believe that many of the products are reaching for the broader scope of functionality defined by IDC as being: search and retrieval, text analytics, and BI. But are they really going to claim to be content management and data warehousing software, as well? Those are included in IDC's definition of "information access software." May-be we are going back to single-vendor platforms with everything bundled and integrated. Sigh... it makes me tired, trying to keep up with all this categorizing and re-redefining.

Thinking about Enterprise Search the Right Way

2008-05-29T11:34:18Z

A major differentiator for search products used within enterprises to enable finding enterprise generated and re-purposed content is intent. For too long the focus has been on search for content based on keywords that are contained in target content. Target content has been determined by what repositories and document formats are explicitly included in the search engine "crawl." This simplistic approach to search for the most appropriate content does not work.

At an upcoming session, EST-3, in the Enterprise Search track at the San Francisco Gilbane Conference, we want to change the discussion about why search is needed for enterprise content and how it should be implemented. This means putting a focus on the intent of a searcher. In an e-commerce Internet experience we assume that the intent of a searcher is to find information with an end goal of selecting or purchasing products. But much of the content that is crawled on the Internet is "discovered" by all kinds of searchers who begin with no particular intent but curiosity, self-education, or with a search for something entirely different. We all know where that lands us - in a pile of stuff that may contain the target of our intent but mostly stuff with little relevance.

Enterprise search has to be thought of as a value-added tool for enriching and improving our work experience and efficiency. If it is installed, implemented and tuned with little thought as to intent, it becomes another white elephant in the basement of legacy IT failures. Intent needs to be constantly explored and examined, which means that search administrators will routinely be talking to representative users, and surveying expectations and experiences.

In our enterprises we search for content for many reasons. It is what we do with that content that creates business value or not. Too often, organizations discover that the content workers need to perform at their highest levels is not found. This may be because search implementation(s) are not delivered to the desktop to fit easily into workflow, or the interface is hard to use. It can also be that required content never gets included as a retrieval option. Search experts can give us guidance to establish search tools in the ways that fit how workers seek information and find actionable content to better their work output.

On June 19th three such experts will talk about cases in which search solutions were designed for a particular audience. If you are in the audience to hear them, please comment through this blog on what you learn. New insights into applying search "the right way" are a refreshing addition to case study library.
Speakers: Jean Bedord, Findability & Search Consultant, Econtent Strategies, Search for the Enterprise: Creating Findability
Mark Bennett, CTO, New Idea Engineering, Protecting Confidential Information within the Corporate Search Box
Mark Morehead, Senior VP, MuseGlobal, UWire: A Case Study in Using Search to Streamline Editorial Processes in the Enterprise

Search your Enterprise Really Needs

2008-05-22T19:00:55Z

In the forthcoming Gilbane research report, Enterprise Search Markets and Applications Capitalizing on Emerging Demand, I describe several distinctly different scenarios for search applications. The variety of search products underscores innovative approaches to applying search and diversity of needs. The Enterprise Search track at the upcoming Gilbane Conference in San Francisco will feature numerous examples of why and how search is being applied across small, medium and very large domains of enterprise content. Hearing from those experienced in implementing and deploying search solutions will inform you when positioning your search "must haves" as you narrow possible options.

Our first group panel will feature two consultants and a solutions provider each with a perspective on aligning the search problem you are trying to solve with a business case and the type of product being offered. As moderator, I will be looking for examples from speakers that will resonate with the audience to provide a connection between what has been demonstrated as valuable and workable, and what conference-goers are seeking. These sessions are about matching experience with investigation and creating an environment for exchanging information and allowing inquiry and research to flourish.

Much has been made of the rise of "social" technologies in the past year, but technology is only a tool. Any meeting gathering with product exhibits facilitates your first-hand viewing of technology and the vendors offering products. But more important, are the professional social connections that give flesh and realism to the application of technology. If you set out to ask just one question of each speaker you meet or fellow attendee, make it one that will help you build a realistic picture around a product you are considering to meet a need. For example, ask not about whether product "A" can perform function "XYZ" but what it took in terms of human resources to deliver that terrific interface that the speaker is showcasing. Social networking gives you that opportunity.

Professional conferences are learning opportunities and, compared to today's college tuition costs, a great bargain. Also, educational institutions are relatively limited, exposing you to controlled scenarios or short-term experiences. What you gain at meetings like the Gilbane conferences is opportunity to benefit from long-term experiences in real business situations by asking those who have been there and done that, how it came about, got built and what the demonstrable outcomes are.

A look at these topics for session EST-2 shows how our speakers will frame their experiences: Venkat Rangan, CTO, Clearwell Systems, Search and Information Retrieval Needs for eDiscovery; Randy Woods, Executive VP, Non-linear Creations, Best Practices for Tuning Enterprise Search; Sam Mefford, Enterprise Search Practice Lead, Avalon Consulting, Beyond Silos: Changing 'Hide and Seek' to 'Index and Find.' I'm always looking for new perspectives on search and ways of helping my clients understand their options. This will enrich my own learning experiences, as well.

Interview with Steve Arnold about "Beyond Search" with Jess Bratcher

2008-05-15T14:08:48Z

Interview with Arnold-04162008.pdf

Enterprise Search and the Conference Season

2008-05-15T13:52:30Z

My blog has been silent for several weeks as I wrapped up a study of the enterprise search marketplace. More information about the report will be forthcoming in the next week or so. In the meantime, the conference season is upon us with the Infonortics Search Engine conference just held in Boston, the Enterprise Search Summit in New York next week, TextAnalytics being held in Boston in mid-June and our own Gilbane San Francisco Conference being held June 18 - 20th. It is a feast for those in the market to buy or just become more familiar with the huge number of options. In my recent research on the marketplace I interviewed a number of people who had recently made a procurement of a search product. To a person there was significant pain expressed about how much time had been spent examining and rejecting options. With well over 100 search and "beyond search" products that are now commercially viable on the market, you need to find ways to winnow your choices efficiently. There is no better way to do this than to acquire publications that give you comprehensive information concentrated in one place PLUS going to conferences to:
a) To meet vendors and assess the type of business relationship you are likely to experience with them
b) Meet other users or potential users of the various technologies to learn, first hand, what their experiences have been buying and using search software

Attending conference sessions where case studies are being given by those deploying or using software is important, but discussions on the side can also be valuable. People who show up at our Gilbane Conferences are a sharing crowd and are easy to network with. As the track chairman for all the enterprise search sessions in San Francisco, I plan to hold at least one and maybe two roundtable discussions, open to anyone who wants to participate in a free flow of ideas about enterprise search. This will likely be in the location of the lunch venue - so we can pick at our food and each others' brains, simultaneously.

Over the next couple of weeks, I plan to showcase the themes for our search sessions in San Francisco, beginning with the Search Keynote. Last year in Boston we had a panel discussion of search executives and analysts; that was a great discussion. This June I am going to thrust Steve Arnold, author of our new publication Beyond Search, into our spotlight with a series of questions about the marketplace to discover things that he thinks buyers should be focused on over the next six months plus soliciting some thoughts on selecting appropriate technologies. He will surely add commentary on the changing vendor landscape and what it means. Once I have had a go at questioning him, the audience will have a chance to seek his guidance. This is a "not to be missed" session so please put it on your calendar - it will not be recorded.

To warm you up to Mr. Arnold's style and range of thoughts on the subject, check out this recent interview he gave to Jess Bratcher of Bratcher & Associates.

Only Humans can Ensure the Value of Search in Your Enterprise

2008-04-14T18:29:48Z

While considering what is most important in selecting the search tools for any given enterprise application, I took a few minutes off to look at the New York Times. This article, He Wrote 200,000 Books (but Computers Did Some of the Work), by Noam Cohen, gave me an idea about how to compare Internet search with enterprise search.

A staple of librarians' reference and research arsenal has been a category of reference material called "bibliographies of bibliographies." These works, specific to a subject domain, are aimed at a usually scholarly audience to bring a vast amount of content into focus for the researcher. Judging from the article, that is what Mr. Parker's artificial intelligence is doing for the average person who needs general information about a topic. According to at least one reader, the results are hardly scholarly.

This article points out several things about computerized searching:

It does a very good job of finding a lot of information easily.

Generalized Internet searching retrieves only publicly accessible, free-for-consumption, content.

Publicly available content is not universally vetted for accuracy, authoritativeness, trustworthiness, or comprehensiveness, even though it may be all of these things.

Vast amounts of accurate, authoritative, trustworthy and comprehensive content does exist in electronic formats that search algorithms used by Mr. Parker or the rest of us on the Internet will never see. That is because it is behind-the-firewall or accessible only through permission (e.g. subscription, need-to-know). None of his published books will serve up that content.

Another concept that librarians and scholars understand is that of primary source material. It is original content, developed (written, recorded) by human beings as a result of thought, new analysis of existing content, bench science, or engineering. It is often judged, vetted, approved or otherwise deemed worthy of the primary source label by peers in the workplace, professional societies or professional publishers of scholarly journals. It is often the substance of what get republished as secondary and tertiary sources (e.g. review articles, bibliographies, books).

We all need secondary and tertiary sources to do our work, learn new things, and understand our work and our world better. However, advances in technology, business operations, and innovation depend on sharing primary source material in thoughtfully constructed domains in our enterprises of business, healthcare, or non-profits. Patient's laboratory or mechanical device test data that spark creation of primary source content need surrounding context to be properly understood and assessed for value and relevancy.

To be valuable enterprise search needs to deliver context, relevance, opportunities for analysis and evaluation, and retrieval modes that give the best results for any user seeking valid content. There is a lot that computerized enterprise search can do to facilitate this type of research but that is not the whole story. There must still be real people who select the most appropriate search product for that enterprise and that defined business case. They must also decide content to be indexed by the search engine based on its value, what can be secured with proper authentication, how it should be categorized appropriately, and so on. To throw a computer search application at any retrieval need without human oversight is a waste of capital. It will result in disappointment, cynicism and skepticism about the value of automating search because the resulting output will be no better than Mr. Parker's books.