Thursday, April 25, 2013

SharePoint 2013 Search: Removing the Junk from your Search Trunk

Have you ever noticed how much nonsense or irrelevant results come back from various search queries? It's nobody's fault. The content is out there and happens to match the terms we are searching. But do we really need all of this stuff if it's not useful?

I have been recently fine-tuning what comes back in search results. People want content. They don't want containers of content, lists of content, links to content - just plain results they can use. Therefore, I have tweaked my main results to perform the following:

  1. Remove excess external content type links and nonsense
  2. Remove Folders from search results
  3. Remove overall list results (AllItems.aspx)
  4. Remove Link List Items
  5. Remove Link Lists
  6. Remove Explorer Views (web folder results)

I only performed these tweaks on my main results page (a.k.a. Everything) by modifying the query on the Search Results web part:


 
 
External Content Type Nonsense
Even though I created custom Result Sources and Result Types for some of my external content data (aka BDC/BCS stuff), I noticed I would still get crazy bdc3:// results back. This was the first thing I removed:
 
Folders
I noticed folders being returned, which may be correct, but we want content - not the containers of content. I opened one of my folder results and it brought me to a list of --- more folders!! Ahh. We don't need that. Therefore I added another property filter as follows:
 


 

 
Full List Results (AllItems.aspx)
How many search results show the AllItems.aspx view?? In some queries it's every result. We just need the list item itself, not the full list of irrelevant items. Therefore I decided to filter that out too:


Links List Items
We happen to use many lists of links to display items on various department, office, or home pages. They are convienent and provide an easy way to add a new link to a page. However, for search results, we want links to the main content, not links to links to content. For this I needed to add a filter using the contentclass property:

Links Lists
Removing the link items was good but now the entire links list is being returned in results. Might as well remove that too:


Explorer View
Finally I noticed web folders correlating to Explorer Views of lists being returned. There is no need to gunk up the results with various views. Therefore I filtered these out as well using the secret STS_List_850 contentclass:

My final query text?

Here is what I have so far to get all of the junk out of my search results:


Conclusion
I am getting cleaner results with mostly documents, posts, relevant list items, calendar items, etc. I am starting to feel much better with the improvement on our relevancy.
 
Search on!!!
 

NOW AVAILABLE!!!

 

40 comments:

  1. SharePoint 2013's search engine allows search designers to truly improve the search experience for users, provided that sufficient time is assigned to designing and implementing search. Search in SharePoint Server 2013 includes many of the abilities of SharePoint Server 2010.

    ReplyDelete
  2. Good one...

    But for me I am not able to work with Path property. OOB its Complete Matching is true. I set it to false(from Central Admin). Even after changing this it doing the complete matching. I also created new property with same mapping but no luck. Can you please help me with this?

    Note: Full crawl has been done after changing the property

    ReplyDelete
    Replies
    1. I could only get Path to work using contains and providing the start of the Url I want to match. What does your Path filter expression look like?

      Delete
    2. Same for me as well. To solve this issue I recreated new property with same mapping and it worked for me. Initially it did not work somehow but I recreated and now its working.

      Delete
  3. Hi Steve,

    Myself is Rohit,Thanks for your above post.

    Can we have any option to filter sharepoint 2010 enterprise search?

    ReplyDelete
    Replies
    1. You can do something similiar but it would involve adding rules to your All Sites scope to exclude the types of content you don't want to have.

      Delete
  4. Great post. Just a follow on, you can use FileExtension not equal to .aspx to trim all site pages then you will see mainly items and files.

    ReplyDelete
    Replies
    1. Thanks! I was thinking about that too but sometimes we do want to see web page results.

      Delete
  5. Great Post.Really helpful.

    ReplyDelete
  6. Great! Exactly what I needed to remove the dumb default BCS items! Thanks

    ReplyDelete
  7. Great post. There are also library and list settings that site owners can use to exclude marginal stuff from search (e.g. calendars)

    ReplyDelete
  8. Hi Steve, thanks for this post. It is really helpful. I'm trying to display the images only and I have tried to use the pictures result source but I'm getting all the pictures from the site collection in the results. What do I need to change in the query to see the pictures only based on my search terms? Any help would be much appreciated. Thanks again

    ReplyDelete
  9. Make sure you either have {searchboxquery} or {searchterms} in your query. Initial rendering may show all results at first but then entering in a term should produce only the images that match.

    ReplyDelete
  10. Thanks for your help Steve. I have included that and it works now :)

    ReplyDelete
  11. Any suggestion how to influence the Refinement Panel to only show a clean defined set of refiners? We want to have our main refiner “Content Type” showing 5 custom content types (Company News, Policies, Request Forms…) as well as selected out of the box content Types (Newsfeed Post, Blog, Video…) but not other out of the box refiners (like relpy, webpage, task, zip, email) in order to get a very clean refiner showing only important content types. Do you have any suggestion how to achieve this?

    ReplyDelete
    Replies
    1. I don't have any elegant solution at the moment for this. I have been thinking about it though. The quick and dirty would be to make a copy of the refinement templates and modify the logic to only use content types you want. That would be hard-coding but it would be the easiest option. Any other option (new column, Managed Metadata tags) would require tagging each item/document which doesn't seem practical (unless you wrote an automated process).

      Delete
    2. This approach could definitely work for us. The values will be quite static, so hard coding is not a big problem. I am really surprised is was not possible out of the box, it seems such a basic and important scenario for us to get a clean user experience and promote the important content.
      Thanks a lot for your help!

      Delete
  12. Great post, thanks!

    ReplyDelete
  13. THIS IS MY MOST FAVORITE POST EVER. Thank you so much for your help with this. I knew I was close, but I didn't quite have all the property filter names (IsContainer/contentclass/etc...) to get me there! Thanks again.

    ReplyDelete
  14. Hi Steve! First of all thank you for your great posts on Sharepoint search. I've implemented some of your suggestions including removing folders from the search result. Afterwards I found out that IsContainer<>true also removes the welcome page of the sites from the search result. Have you experienced that as well or might it be something which only relates to our search application?

    ReplyDelete
  15. Yes I believe that's why our team sites weren't showing up. Doh! Lately I have been modifying the query in the search results web part under the Ranking (may be in the Sorting tab) and using the dynamic settings at the bottom of that dialog. You can demote the Folder content type there.

    ReplyDelete
    Replies
    1. OK, but doing that will demote the site welcome pages as well, and I would think that's not what you want to achieve?

      Delete
    2. I believe the Folder content type would be different than IsContainer as sites and subsites are their own content types.

      Delete
  16. Very nice post, thanks! I needed to hide allitems.aspx from search results but I didn't have access to Central Admin in order to make a crawl rule. Your post helped me do just that.

    ReplyDelete
  17. Hi Steve - Thank you for your great post. This eliminates the CA access dependency.

    Any suggestion on how to remove custom views. When a power user creates a custom public view on a list say MyCustomView, this shows up in the search result if search term matches filename. Adding an 'exclude' crawl rule to address this issue, excludes content from entire list.

    Do you have any suggestion how to achieve this?

    Thanks !

    ReplyDelete
    Replies
    1. did you try -Path: and use the path to the view aspx?

      Delete
    2. I believe that should work. But I'm kind of looking for one time solution for all custom views that power users may create in the future. *(adding -path might be an on demand solution ?)

      Thanks !

      Delete
    3. Not sure if this is an elegant solution but below filter is removing all custom/default views from results
      -contentclass:STS_List_GenericList

      Delete
    4. nice! I think that would be respectable.

      Delete
    5. nice! I think that would be respectable.

      Delete
  18. I am a big fan of custom search results like this. One way to keep things cleaner moving forward would be to provide a custom list template that prevents the items from being surfaced in search results.

    ReplyDelete
  19. Fantastic, Thanks for this Great Post

    ReplyDelete
  20. Great post!

    Is there a way to remove subsites from results, but keep the pages and other itens? I suspect IsContainer property do that, but I'm not so sure.

    ReplyDelete
    Replies
    1. May need to look at IsContainer and ContentType.

      Delete
  21. Why change the web part instead of create a custom result source?

    ReplyDelete
  22. You are correct. There are several ways to change the queries and results.

    ReplyDelete
  23. I want to trim results based on value of a column. e.g. If approval comments has value as pending,then those values should only appear in search to a particular group and not to all. Can this be achieved?

    ReplyDelete
    Replies
    1. If you use the out-of-the-box approval for items, then only the Owners (and possibly Members - I forget off hand) of the site will see those search results. Anyone in the Visitors or with Read only access will not see pending items.

      Delete
  24. I found this article easy to understand and very helpful. Can’t wait to see the other posts. Thank you for sharing!

    ReplyDelete