Essential SharePoint 2013: Practical Guidance for Meaningful Business Results (2014)
Part II. Optimizing
Chapter 16. Planning Enterprise Search
Enterprise search in SharePoint 2013 has significant changes from previous versions. In this release, Microsoft has unified its prior SharePoint, FAST, Exchange, and Search Server search platforms into a single engine. This search engine is also now available via SharePoint Online as part of Office 365.
In this chapter, we will explore
What’s new in SharePoint 2013
Planning for search
Understanding search from a user perspective
SharePoint 2013 search administration
Search-driven applications
What’s New in SharePoint 2013?
SharePoint 2013 introduces major changes to search that are based on the key concepts and features available in SharePoint 2010 standard search and FAST search for SharePoint, plus new capabilities.
Microsoft acquired FAST Search & Transfer in April 2008. In SharePoint 2010, Microsoft introduced a new version of FAST search built specifically for SharePoint. This version of FAST leveraged the same user interface and Web Parts within SharePoint 2010 but still had separate search services and required separate application servers under the covers. All that has changed in SharePoint 2013; there is now a single search platform.
This new search engine powers all versions of SharePoint 2013: Foundation, Standard, and Enterprise as well as SharePoint Online via Office 365. Depending on the version of SharePoint 2013 you implement, you will see different features and Web Parts available. Microsoft has published a detailed matrix of search features available across the different types of SharePoint 2013 licenses at: http://technet.microsoft.com/en-us/library/jj819291.aspx.
Search has a much more prominent role in SharePoint 2013. Search is used to aggregate, analyze, and recommend sites, people, and documents that SharePoint will suggest for you to follow as well as social tags that are trending within your organization (see Chapter 15, “Planning for Social Computing,” for more discussion on these concepts). There are many search-driven Web Parts, such as the new Content by Search Web Part, which can be configured to display content across multiple site collections.
With the release of SharePoint and Exchange 2013, Microsoft has moved toward using the same enterprise search platform to power both of these products. One area that this benefits is the new eDiscovery features that allow you to find and take legal action on content that may come from SharePoint, Exchange, Lync, or file shares. For example, placing a team site or mailbox on hold will benefit from the search integration. You can find more information on eDiscovery and Site Mailboxes in Chapter 13, “Managing Enterprise Content.”
Another major change is the integration of the Office Web Apps server into search results. Now, if an Office document (Word, Excel, or PowerPoint) is included in the search results, you see a live preview of the document and can jump to specific sections in Word documents or slides within PowerPoint presentations. Office Web Apps are discussed in more depth in Chapter 20, “Integrating Office Applications.”
Planning for Search
Many organizations have stated that enterprise search remains a top priority for their business. Unfortunately, many organizations think that just implementing an enterprise search tool will solve all of their information discovery challenges. At a minimum, organizations should plan to allocate part of a resource to regularly review search reports and to help improve search relevancy by promoting or demoting certain sites and documents. The responsibility for managing search should be considered carefully to ensure that search results are optimized across site collections, represent organizational priorities, and are linguistically sensitive. In some organizations, search may be centrally managed; others may elect to delegate the responsibility for managing search to SharePoint business owners.
In the following sections, we’ll discuss important considerations when planning for search such as:
Why does search matter?
What are some common enterprise search terms and concepts?
How does content management affect search?
Why is configuring and managing SharePoint 2013 search important?
What content should you expose via SharePoint search and how?
Why Does Search Matter?
One of the biggest challenges in most organizations is finding the answers to frequently asked questions. Sometimes what people are searching for is a document or a Web page. Other times it is a video. Many times people are trying to identify a subject matter expert.
In many organizations, trying to find the answer to a query across a variety of sources is challenging. People often need to know what systems exist, and then they need to execute a search in each system. Some solutions just identify results based on metadata attributes such as the document title or author but not necessarily the contents of the document itself.
Finding people is often even more challenging—especially in large global organizations. What if your organization has multiple divisions or subject matter experts (such as engineers or human resource professionals) working in a decentralized manner? Most times people have to rely on talking to the people they know within the organization or performing Internet searches to find experts outside of the organization.
SharePoint search seeks to address these challenges and others by bringing together the ability to search across structured (e.g., line-of-business) data and unstructured (e.g., Word documents) data sources from a single place. The content may be stored inside of SharePoint, or it could exist elsewhere such as on file shares, in other document repositories, or in LOB systems, such as your human resources or sales applications.
What Are Some Common Enterprise Search Terms and Concepts?
A number of different terms and concepts are commonly used in enterprise search. It is helpful to understand the following when planning for search:
Indexing, which is sometimes referred to as crawling, is the technical term for when the search engine processes various types of content and catalogs the key attributes (e.g., Document Name, Author, Modified Date) and contents of files.
Querying is the action of the search engine looking at the index to see if there are any documents or other items that meet the search criteria specified by the user.
Search results are what the search engine returns after a query is passed to it. For example, if the query was “Boston,” the search engine would display all items that contain the word Boston in their metadata or their contents.
Federation is the action when one search engine passes a query on to another search engine to provide search results without having to index the same content source. For example, Wikipedia contains content that you may want to display in your search results. Chances are that you do not want your SharePoint environment to index all of the Wikipedia content. In this case, you would enable search federation between your SharePoint environment and Wikipedia. When a user performs a search in your SharePoint environment, SharePoint can pass the query on to Wikipedia’s search engine and then display the results that Wikipedia returns.
Search Center is typically a central site where individuals can go to perform a search across various sites and systems within an organization. SharePoint contains an Enterprise Search Center site template that includes prebuilt pages for handling user search queries and displaying search results. Like other SharePoint site and page templates, the Enterprise Search Center can be customized to meet the needs of your organization.
Search refiners enable a user to visually filter search results. The search refiners are typically shown on the left side of the search results page. SharePoint automatically displays refiners for common metadata attributes such as the type of result (e.g., Word document, PowerPointpresentation), the name of the author of the item, and when the item was last modified. Users can click on a refiner to display search results for only specific criteria, such as only PowerPoint presentations that have been modified within the past week.
Result sources are used by SharePoint to define different actions to perform against different types of content. For example, SharePoint enables you to handle search results differently for document content and social conversations. You can also define new result sources to specify how to handle searching content from other sources such as Microsoft’s Exchange e-mail platform or content sources with which you are federating. Result sources also incorporate what were formerly known as “search scopes” in prior releases of SharePoint. Out-of-the-box search scope examples include having different views of the search results page to show only “People” or social “Conversations” results.
Result blocks contain a set of related results. For example, results that are PowerPoint documents automatically appear in a result block when the word presentation is part of a search query. You can also define your own custom result blocks.
Promoted results, which were formerly known as best bets, allow you to specify that certain items should always be shown at the top of the search results for specific keywords entered by the user. For example, if a user searches for the word sales, you can have the sales team site show up at the top of the list of search results. Promoted results can be targeted to specific groups of users (known as segments) and can be set to be used only during a specific time frame within a date range.
Crawled properties are metadata that is extracted from items during content indexing. Metadata can be structured content (such as the name of the author of a Word document) or unstructured content (such as the language in which the content is written).
Managed properties have metadata that is either automatically created by SharePoint or created by a search administrator. Crawled properties are mapped to managed properties so that they can be included in the search index. Managed properties can also be used as search refiners and can be used to sort search results.
Entity extraction is the process of SharePoint search parsing content during the crawl process and dynamically building search refiners. One example of this is the built-in “Company name extraction” process where SharePoint can create a refiner based on a list of company names. This enables the search engine to create a company refiner automatically, which allows users to filter search results based on the company names contained within the content that has been crawled—even though the content may not have been explicitly tagged that way.
There are many additional terms and concepts to consider. Microsoft has published documentation to assist with planning and understanding enterprise search. These resources can be found at http://technet.microsoft.com/en-us/library/cc263400.aspx.
How Does Content Management Affect Search?
While SharePoint’s enterprise search engine can perform advanced content processing, such as performing entity extraction to automatically generate search refiners, search results are much more accurate when content is tagged with explicit metadata. There is a tight relationship between effective information classification and taxonomy and search results. For example, SharePoint Content Types and managed metadata can be leveraged to properly classify content within SharePoint, which can then be used by search for displaying and filtering content. By using Content Types, you can classify items as being related to a sales proposal. Search can then be leveraged to find all sales proposals stored across your organization. You can also use managed metadata to apply consistent attributes to content within SharePoint. Perhaps there is an attribute within your organization that tracks the industry of the customer for which a proposal was created. You can then index these attributes and display them as search refiners to be able to filter search results to show sales proposals only for customers that are in the financial services industry.
Why Is Configuring and Managing SharePoint 2013 Search Important?
In SharePoint 2013, the role of search is much more prominent than in previous versions. Search is no longer just a box you type in when you’re looking for something. Search is now leveraged directly by other areas of SharePoint to drive Web content display via page templates and suggest social recommendations for people, sites, tags, and documents that you may want to follow. Search also drives the new usage analytics capabilities in SharePoint as well as the new eDiscovery capabilities.
Many organizations and individuals make the incorrect assumption that enterprise search performs magic and is a totally automated task. They believe that consumer search tools such as Google and Bing operate automatically without human interaction. These are false assumptions. The reality is that human intervention is required to optimize SharePoint search. One example is promoting sites and documents to make sure that the most important content is being shown as a promoted result at the top of the page. For example, when someone searches for “benefits” during the last quarter of a calendar year (October through December), your organization may want to link to a site or page with information on benefits open enrollment. Individuals can also define search synonyms so that when a person searches on a particular term or acronym it can be mapped to display other similar search results. A search administrator can also help to demote documents or sites to push less authoritative content sources to be displayed lower in search results.
Note
SharePoint 2013 does not fix content problems such as incorrect document titles. If you’re not getting very good search results in SharePoint 2007 or SharePoint 2010, an upgrade to SharePoint 2013 won’t fix your problem. In fact, it could make the problem worse. Take the time during the upgrade to educate users on the importance of the Title property, which is used for displaying search results. Search results that look incorrect could simply be reflecting bad data. Remember: garbage in, garbage out.
What Content Should You Expose Via SharePoint Search and How?
When developing your enterprise search strategy, there are a number of key questions to consider, including the following:
What content needs to be indexed via the SharePoint search engine?
Where does that content reside today?
Do you need to crawl content over a wide area network (WAN)?
Are there any time zone considerations? Crawling can impact the source system that you’re crawling—you might be scheduling crawls off-hours where the search server is located, but it could be the middle of the workday for content owners.
Does the content being indexed have adequate security controls in place, or do people practice “security by obscurity” and basically “hide” content by burying files many folders deep? If so, search may now “expose” that “hidden” content, which was never really secured in the first place.
Do you have other search tools already in place that currently index the content?
Do you want to have a single search engine crawl the content so you can have a single source of relevancy for ranking and displaying the results?
Do you have to federate with other search engines internally or externally (e.g., Google, Bing) to also display results for content SharePoint does not crawl? If so, do these other search engines support the OpenSearch federation standard?
Should you feature and promote certain content for certain roles?
What content sources should be indexed as opposed to federated?
What file types do you need to index (e.g., Word, .pdf, CAD, videos, etc.)? Do you need any additional iFilters to crawl the contents of these files (if SharePoint does not already include them), or is indexing the metadata alone sufficient?
What languages do you need to support? Is there any LOB system data (e.g., CRM, ERP) that you want to display via search? If so, are there existing BCS connections already in place, or do you need to create new ones for search purposes?
Will you need to implement a hybrid search architecture to display search results across both SharePoint Online and SharePoint Server?
What are the roles and responsibilities for search administration?
What training is required for managing search?
Will SharePoint 2013 search be the first service implemented on the new platform to perform a “search first” migration where existing content is crawled and displayed in place and gradually migrated or archived?
These are just some of the key questions that we suggest you consider as part of developing your enterprise search strategy. The remainder of this chapter will help you to understand what SharePoint 2013 offers from a user and administrative perspective to guide you in answering these questions and developing new questions of your own.
Understanding Search from a User Perspective
The user search experience has evolved from what was available in SharePoint 2010. There are two primary ways to search: using a Search Center landing page (see Figure 16-1) or directly within a SharePoint site page (see Figure 16-2).
Figure 16-1 SharePoint 2013 Enterprise Search Center
Figure 16-2 SharePoint 2013 site search
Users can access standard result sources (functionality that replaces search scopes in prior versions) from a SharePoint site page. Result sources include the following:
This Site is the default search, which includes content stored in the current site and its child sites.
Everything includes content ranked across a variety of sources such as SharePoint sites, people, file shares, or other document management solutions.
People focuses just on information from other people’s profiles within SharePoint and provides the option to review results by relevance (how likely it is that the person has skills or interests that match your search) or social distance (looking at the people you follow and those that they follow).
Conversations includes results across the public newsfeed, team newsfeeds, and community site discussions based on your permission level. More information on these capabilities can be found in Chapter 15, “Planning for Social Computing.”
SharePoint 2013 search remembers your prior search history and can recommend recently used search terms. It can also suggest search terms (also referred to as queries) based on similar phrases that are trending in popularity across your environment.
SharePoint search result pages provide search refiners to help you filter your results. The refiners vary based on the result source. For example, the Everything result source provides options to refine by the result type (e.g., Word document, .pdf, Web page, etc.), the author of the document, and the date the content was last modified. The Author refiner enables you to also enter a specific name to find results just from a particular person. You can also create your own custom search refiners, such as the Chapter Name refiner shown in Figure 16-3.
Figure 16-3 SharePoint 2013 search refiners
Figure 16.4 shows a sample search result from within a team site. One major change that users coming from FAST search for SharePoint will notice is that by default the search refiners do not show the count for the number of items that are of that type, such as that there are 250 Word documents. While the counts are not shown by default, you should know that SharePoint search does do deep refinement as part of the content-crawling process and does sort some of the result types, such as the author, in descending order so that refiners with the most items will be displayed at the top. The counts can also be displayed in the search results, if desired.
Figure 16-4 SharePoint 2013 search results
There are many changes within the search results as well. As Figure 16-4 shows, when SharePoint 2013 is used with the Office Web Apps server, search results for Word, PowerPoint, and Excel documents include a full live preview of the document that you can scroll through. Word and PowerPoint results provide you with links to jump directly into the document at specific sections or slides.
Even without the Office Web Apps, search results now provide much richer context. Document results give you the direct ability to follow the document from the search results and see important metadata such as who authored the document, when it was last modified, and how many times the document has been viewed.
Conversations search results look across SharePoint community sites and newsfeeds. Community results show how active a given community is, including the number of members, discussions, and replies. You can quickly go to specific discussions within a community (see Figure 16-5) or visit the community site.
Figure 16-5 SharePoint 2013 Conversations search
Search Tips and Syntax
When performing searches, there are a number of options available to help you target search results. Many people will use the built-in search refiners to narrow down the search results to include only specific file types or dates or locations or authors. However, more advanced querying options are available. Some examples of search tips and advanced query options are listed below. Additional tips are summarized in the Appendix.
Put search phrases in quotes to indicate that you’re searching for an exact phrase, for example, “business intelligence.”
When naming content, use whole words and use underscores to separate those words. This is due to word breaking, which splits the file name into real words that are used as search keywords. For example, Sales_Report_Q4_2013 is a better file name than slsrpt-2013Q4. If a user types in “sales” or “report” or “Q4” or “2013” in search, the first document will be found, but the second one would not.
Make sure users update the Title property for document content. This property is what search uses to display results—not the file name. If the Title property is not filled in correctly, SharePoint will either show the bad title or arbitrarily use content from within the document for displaying search results. This can confuse users and make them think search is bad.
Place an asterisk at the end of the search phrase as a wildcard to bring back results that contain the first part of your search phrase. For example, searching on “shar*” will return search results containing words such as SharePoint, sharing, and shared.
Use search operators such as AND, OR, and NOT. For example, you can perform a search such as (“SharePoint” OR “SPS”) AND (title:“search” OR title:“workflow”), which will return search results for items that contain SharePoint or SPS where the title of the item includes the words search or workflow.
SharePoint 2013 also provides an Advanced Search screen that lets any user perform a complex query without having to understand SharePoint search syntax. Figure 16-6 shows one example to find all documents in English created by any author whose name is “Bortlik”; where the document was last modified on or after September 1, 2012; and where the word SharePoint or search is contained within the body of the document or any of its metadata. The Advanced Search page also includes a link to a help resource with additional search tips.
Figure 16-6 SharePoint 2013 Advanced Search page
Additional Search Options
Two additional capabilities are available to users to help them make search work the way they want it:
Search alerts allow you to have a search query run automatically on your behalf and send you the results via e-mail. For example, you could receive an e-mail daily when new items are available for keywords of interest to you. Note that search alerts must be enabled in Central Administration as part of the Search service application for this capability to work.
Search preferences allow users to specify whether they want SharePoint search to suggest search phrases to them and whether the search engine should use their personal search history to suggest previously used search phrases. Users can also specify their default search language (e.g., search results for content in English or Spanish), whether to use the Office Web Apps or Office desktop clients to open search results that are clicked on, and to manage their existing search alerts. Users can access their search preferences at the bottom of the search results page.
SharePoint 2013 Search Administration
The SharePoint 2013 search architecture is different from how search was performed in previous SharePoint and/or FAST versions. This section describes the logical architecture, physical architecture, and options for managing SharePoint 2013 search crawling and results display and includes the following topics:
SharePoint 2013 search logical architecture
SharePoint 2013 search physical architecture
Capacity planning considerations
Upgrading to SharePoint 2013 search
Managing SharePoint 2013 search
Adding new content sources
Adding new result sources
Working with query rules
Customizing and creating SharePoint 2013 search refiners
Exporting and importing search settings
SharePoint Server 2013 and SharePoint Online search capability comparison
SharePoint 2013 Search Logical Architecture
From a logical perspective, content is processed by SharePoint 2013 as shown in Figure 16-7.
Figure 16-7 SharePoint 2013 logical architecture
Content source is the system that SharePoint is crawling. The content source could be SharePoint itself, a file share, a Web site, or some other system.
SharePoint crawler uses a connector to get content and metadata from a content source.
Content processing occurs for each item, such as a document or Web page, that comes from the crawler. The appropriate iFilter is used to analyze the full text of the item, such as a .pdf file. Additional steps are performed to process metadata and extract entities to refine by, such as the built-in Companies refiner.
Analytics processing information is processed to see how often an item is viewed and linked to by other items.
The indexing component uses the output of content processing to create index files. The indexer is also used to provide search results to the query-processing component when a user makes a search request.
The query-processing component performs a number of key tasks after accepting the search request. These include applying query rules, performing linguistics tasks (e.g., stemming, word breaking, spell checking, etc.), and processing result sets before displaying the results to the user.
The search administration component (not shown in Figure 16-7) handles the system processes for search and manages the other search components.
SharePoint 2013 Search Physical Architecture
From a technical implementation perspective, outside of SharePoint Online, you have the option of scaling the various search roles across multiple servers (physical or virtual). Figure 16-8 shows the view in central administration that helps you review which servers in your SharePoint farm are handling the various search roles. In this example, we are running all roles on a single test server. In the real world you would have multiple servers handling different roles for both redundancy and scalability.
Figure 16-8 SharePoint 2013 search application topology
Figure 16.8 also shows the various search databases that are created by the default search configuration wizard. The primary function of each of these databases is as follows:1
1. “Overview of Search in SharePoint 2013.” http://technet.microsoft.com/en-us/library/jj219738(v=office.15).aspx
Administration database stores settings for the Search Service application, such as the topology, crawl rules, query rules, and the mappings between crawled and managed properties.
Analytics reporting database stores the results of usage analysis, such as the number of times an item has been viewed. It also stores statistics from the different analyses. These statistics are used to create usage reports.
Crawl database stores tracking information and details about crawled items such as documents and URLs. It also stores information such as the last crawl time, the last crawl ID, and the type of update (add, update, delete) during the last crawl.
Link database stores unprocessed information that is extracted by the content-processing component and information about search clicks. The analytics-processing component analyzes this information.
Note
Additional information and details on the SharePoint 2013 search architecture can be found on the Microsoft TechNet site at http://technet.microsoft.com/en-us/library/jj219738(v=office.15).aspx.
Capacity Planning Considerations
When planning for search as part of SharePoint Server 2013, there are many factors to consider. Microsoft has published guidance and reference architectures to help you calculate the number of search servers (physical or virtual) you may need as well as what the hardware specifications are, including the amount of memory, storage, and processor power required. This information is published at http://technet.microsoft.com/en-us/library/jj219628(v=office.15).
Some of the key inputs that you will need to identify in your capacity planning include the following:
How many items do you plan to crawl?
How large are the items being crawled (e.g., average file size)?
How much metadata will be indexed?
How many content sources do you plan to index?
How often will you index the content sources (full, incremental, or continuous crawl) and how frequently does the source content change?
What do you expect to be the average number of queries per second that you need to support?
What are your disaster recovery and high-availability requirements for search?
Upgrading to SharePoint 2013 Search
When upgrading to SharePoint 2013 search from SharePoint 2010, there are a few key scenarios and options to consider:
If upgrading from SharePoint 2010 standard search to SharePoint 2013, you have the option to perform a database-attach and upgrade in SharePoint 2013. You will be able to upgrade the SharePoint 2010 Search Service application administration database, which includes existing content sources, crawl rules, and federated locations. Please see Chapter 10, “Planning Your Upgrade,” for more details on upgrading to SharePoint 2013.
If you are using FAST search for SharePoint 2010 or any other prior version of FAST, you will need to do a manual migration to SharePoint 2013 search, although Microsoft may be making some migration tools and resources available at a later date. You will need to inventory any existing content sources and processing changes that you have made and implement them in SharePoint 2013.
Since SharePoint 2013 introduces so many new search changes, you may choose to recreate your content sources and processing rules manually in SharePoint 2013.
Managing SharePoint 2013 Search
Many of the core search administration capabilities are managed in SharePoint central administration (SharePoint Server) or tenant administration (SharePoint Online). There is no longer a separate search administration server as there was with FAST search for SharePoint 2010.
When defining new content sources to crawl in SharePoint Server, one decision that needs to be made is how often the content needs to be indexed. You can also define a schedule for full and incremental crawls. A new capability added for SharePoint Server content is the option to perform a continuous crawl, which enables SharePoint content to be available in the index in near real time, which is important for many of the new search-driven features such as using the Content by Search Web Part to aggregate Web content.
Most organizations establish a single enterprise Search Center where corporate-wide result sources and query rules are defined. In some organizations, however, it is desirable to have departmental search centers that have unique rules and displays available for certain functional areas such as research or legal. If you create different strategies based on the search source, consider how you will communicate this difference to your users.
Adding New Content Sources
When working with SharePoint Server 2013, you can add new content sources to be crawled and indexed by SharePoint search. By default, SharePoint 2013 crawls local SharePoint content (sites, lists, people information, documents, pages, etc.) on the SharePoint farm of which the search server(s) is a member. Typically, organizations set up SharePoint to perform a full crawl once per week (such as Saturday evenings when there typically is lower system load from users) and to have incremental crawls performed throughout the day at scheduled intervals (for example, once every hour).
SharePoint 2013 introduces the concept of continuous crawls to enable specified SharePoint content to be made searchable without going through the full indexing process. This is very important for search-driven Web sites that have dynamic content. You can select to implement either continuous crawling or incremental crawls. We expect that most organizations will choose to use continuous crawls—especially for farms with frequent content updates.
In addition to crawling SharePoint content stored in your SharePoint environment, you can also crawl and index content from other locations. SharePoint 2013 supports the following content source locations out of the box:
Web sites
File shares
Exchange public folders
Line-of-business data via Business Connectivity Services within SharePoint that support content retrieval from external data sources such as a customer relationship management (CRM) or enterprise resource planning (ERP) system.
SharePoint supports the ability to connect to additional content sources if the appropriate custom repository connector is installed and configured. For example, Microsoft offers custom connectors for Lotus Notes and Documentum. You can also purchase connectors from other organizations or build your own.
Adding New Result Sources
A result source is simply a collection of settings that defines what the user will get back when issuing a search query. A result source is defined mainly by three parameters: a source URL (from where do I get results?), a protocol (local SharePoint? OpenSearch?), and a query transform, which allows the user to narrow the results to a subset (just .pdf files? just a particular Content Type?). In prior versions of SharePoint, this concept was referred to as search scopes.
SharePoint 2013 provides 16 preconfigured SharePoint result sources including conversations (social newsfeeds and community sites), documents, people, and video results. More likely than not, you’ll be able to stick with this collection of out-of-the-box result sources. However, if you have a specific scenario that you want to customize, SharePoint provides the capability to create custom result sources if needed.
SharePoint 2013 supports working with the following result source protocols:
Local SharePoint content within the current farm
Remote SharePoint content coming from another SharePoint farm or SharePoint Online
OpenSearch 1.0/1.1 federation with other search engines such as Bing or Google
Exchange server content such as public folders
The remote SharePoint result source is key in a few scenarios. One of the primary use cases is in hybrid environments that include both SharePoint Server on-premises and SharePoint Online, for example, where the organization’s intranet is based on SharePoint on-premises but team and personal sites are provided via SharePoint Online. In this scenario, you will want to enable people to see search results across both environments. Another example may be cases where customers have multiple SharePoint geographically distributed farms and these farms may not be using a single search service. One important aspect to be aware of is that when different result sources are being leveraged, such as when using a remote SharePoint farm as a source, there is no ability to have a single relevancy ranking applied. This means that the local SharePoint results will be indexed and ranked separately from the remote SharePoint farm’s results. If it is important that the relevancy of your search results be factored into how they are displayed, you may want to consider crawling all of the content via a single search engine (if possible) or trying to leverage result blocks (discussed later in this chapter) to highlight and promote certain result sources to have a higher priority over others.
Result sources also allow you to define and apply custom handling and display templates to different results coming from different locations or different types of content within a given source. This is useful since you might want to display product catalog searches differently from people searches.
A final decision in search administration is which result sources should be defined at the farm (SharePoint Server) or tenant (SharePoint Online) level so they can be reused across multiple site collections. Result sources can also be defined at the site collection or site level, which enables greater flexibility, such as for different business units, departments, or team needs.
Working with Query Rules
SharePoint 2013 has somewhat deemphasized the concept of identifying content as a “best bet”—which is now referred to as promoted results. For example, whenever someone searches on a particular keyword (such as SharePoint), a specific result is always shown at the top of the page (seeFigure 16-9). Setting promoted results does not change an authorized user’s ability to access content; it simply changes the item’s position in the search result listing.
Figure 16-9 SharePoint 2013 search with promoted result
Promoted results can be defined as query rules based on specific content sources, such as SharePoint search results. Promoted results can be set to be shown only during specific time periods, for example, only showing benefits open enrollment links between September 1 and December 31 for a given year. Promoted results can also be shown to specific user segments, which allows you to target recommended results at different groups of individuals such as people in sales or research roles.
Query rules also allow you to define result blocks. Result blocks are a powerful tool you can configure to inspect the query results and then take additional actions before displaying the results to the user. For example, if a user searches for a product name, you can display not only documents that match that search criterion but also related product inventory data from an ERP system (such as SAP or PeopleSoft) as part of the search results. SharePoint 2013 ships with many standard result blocks. Figure 16-10 shows an example where the keyword presentation is detected in the search query. Because a result block has been defined for this keyword, SharePoint groups all PowerPoint presentations and displays them at the top of the page.
Figure 16-10 SharePoint 2013 search result block for promoting PowerPoint presentations
Customizing and Creating SharePoint 2013 Search Refiners
SharePoint 2013 search enables you to manage and create search refiners that display as part of the search results. There are a few areas to consider:
Managing the search schema allows you to modify managed properties and map crawled properties to managed properties. Crawled properties are automatically extracted from crawled content. You can use managed properties to refine search results and present the content of the properties in search results. For example, you can specify which properties included in the search crawl should be displayed to the user as part of the “About me” search refiner that is shown in people search results.
Adding and displaying new search refiners may not be obvious to most SharePoint search administrators. Since this is such an important and often overlooked capability, we strongly encourage readers to review this article from Microsoft that describes how to create custom search refiners: http://office.microsoft.com/en-us/office365-sharepoint-online-enterprise-help/manage-the-search-schema-in-sharepoint-online-HA103628856.aspx. Once created, these custom search refiners can be added to your search results page by customizing the built-in Refinement Web Part and using the Choose Refiners feature to select the refiners you want to display.
New in SharePoint 2013 are search dictionaries that are defined and managed via the term store, also known as the Managed Metadata Service (managed metadata is discussed further in Chapter 13, “Managing Enterprise Content”). Search dictionaries allow you to specify the list of values that SharePoint search should include or exclude when it is building the list of a search refinement. For example, in a company name extraction process, search can identify words such as Microsoft, Jornata, or General Electric as names of companies and then let you create a refiner to filter search results to show only items that contain those company names. You can also manage the list of phrases to be included or excluded as “did you mean” search phrases when users search for similar phrases. For example, search could suggest the word SharePoint if someone entered the search term “SharePint.” Out of the box, SharePoint search will also automatically handle suggesting “did you mean” search phrases for commonly used words that may be misspelled in user searches.
Using PowerShell scripts, you can also create your own custom entity extractors that can then be mapped to managed properties within search. For example, search could dynamically identify and build search refiners based on a list of your organization’s product names.
Exporting and Importing Search Settings
SharePoint 2013 search enables you to export and import search settings. This is important when you want to reuse search query rules across SharePoint tenants or site collections. Exporting search settings includes all customized query rules, result sources, result types, ranking models, and site search settings that your organization has customized but not any that shipped with SharePoint.
Comparing SharePoint Server 2013 to SharePoint Online Search
Table 16-1 provides a summary of the key search administration features and indicates what is available within SharePoint Server 2013 compared to SharePoint Online. As noted earlier in this chapter, Microsoft has published a complete licensing matrix that shows what license level is needed to use specific features. A few of the major functional differences between SharePoint Server and SharePoint Online are that the following options are currently available only with SharePoint Server (please see Chapter 11, “Taking SharePoint to the Cloud,” for a more thorough comparison):
Table 16-1 Comparison of Search Administration Options
The new Content by Search Web Part is key to many search-driven intranet site experiences. This Web Part enables aggregation of content across site collections by using the content available in the search index, rather than querying the content databases themselves. Note: This Web Part is part of the Enterprise CAL and as of this writing is not available in Office 365.
Advanced content processing allows you to manipulate some of the internal search relevancy algorithms and weightings via PowerShell scripts.
Custom entity extraction is a capability to define your own metadata attributes for SharePoint 2013 search to dynamically determine during the crawl process and display as search refiners. In SharePoint Online you cannot define your own list of custom entities to derive from the crawled content; however, you can modify the built-in list of companies to include or exclude from the companies refiner.
Search-Driven Applications
One of the emerging trends is to use search as an interface to navigate content. Public-facing Web sites such as bestbuy.com, financialtimes.com, and treasury.gov have made search part of their core user experience. Rather than relying on a relatively static information architecture and site hierarchy, many sites are now leveraging search to drive a more dynamic and personalized experience.
SharePoint 2013 has implemented a number of new capabilities designed to make it easier for companies to implement search-driven Web sites.
Video Search Results
Out of the box, SharePoint 2013 provides a custom search results page showing videos that are tagged to match the keywords you’re searching for. Note that this feature does not index the audio and video—SharePoint bases the search on the metadata and file type.
SharePoint 2013 also provides a new way of viewing and interacting with video search results. This includes the ability to see a thumbnail preview for the video, the ability to play the video in the context of the search results using either a built-in HTML5 or Silverlight viewer, and the ability to perform actions such as following the video or sending a link to others.
New Search-Driven Content Web Parts
SharePoint Server 2013 introduces a number of new search-driven Web Parts. These Web Parts enable dynamic aggregation and viewing of content via crawled search results and are highly configurable without requiring developer assistance. There are 11 Web Parts in SharePoint 2013 in the new Search-Driven Content Web Part category. These include Web Parts to dynamically display popular and recommended items.
For example, you may want to leverage a Search-Driven Web Part to dynamically display a summary of article pages. Inserting the Articles Web Part on a page offers you the option to configure various items for how you want the search results to be displayed and how many items to show.
Changing the query displays a query builder wizard that enables you to perform multiple tasks, including selecting from various search refiners. For example, you can filter the search results to show only the content based on the Article Page template.
Key Points
There are significant changes to enterprise search in SharePoint 2013. Some of the main topics covered in this chapter are:
SharePoint 2013 introduces a brand-new enterprise search engine which is based on the “best” concepts and features that were available in SharePoint 2010 and FAST while adding new capabilities.
There is now a single integrated enterprise search engine that exists across all versions of SharePoint 2013 (Foundation, Standard, Enterprise, Online), and this engine now also powers Exchange 2013 search.
Effectively implementing enterprise search requires up-front planning and review of your search requirements. Some items to plan for include identifying what content sources (e.g., SharePoint, file shares, Documentum, etc.) need to be included, what type of content is being indexed (e.g., Office documents, Web pages, Adobe .pdfs, etc.), understanding how often the content is updated and needs to be made available in search results, and determining what sort of query and result processing you want to perform against search to promote certain search results.
Search is now key to many other areas within SharePoint 2013. Many of the new Web content management features leverage search to dynamically aggregate and display content as well as to recommend content. The new social features leverage search to recommend sites, documents, and people for you to follow. Search is also used to drive the new usage analytics components within SharePoint.