I have been telling customers for years to tag their documents with metadata so they can find the document easier using Search.

I have just discovered that a metadata does not rank highly in search results.  It appears that document content ranks much higher than metadata.  

For example we have a metadata called "topic" and a user tags a document with a topic "Water Quality".  A search for "Water Quality" should put these documents at the top of the search result.

Here is the ranking I would like in the search results.  If the search word/phrase is:

1. A library or app
2. A content type
3. In the metadata, title, enterprise keywords
4. In the file name
5. On the page or in the document content

I have had a look at Query Rules but don't understand how they work.  Is what I need easily achievable without code?

Hi,

If I understand you correct, you want container items to show up higher, meaning document libraries and lists, and then the names of the list. Same goes with hits in Content Type, metadata and file names.

The first place to start is to look at the default rank profile and the BM25Main part of it as seen at http://msdn.microsoft.com/en-us/library/office/dn169052(v=office.15).aspx#sp15_rank_features

If you look here you see there are seven managed properties listed, each with different weights:

• body
• title
• author
• filename
• qlogclickedtext
• anchortext
• socialtag

It's not particularly easy to see how much they all relate to each other in terms of weight, but Title is given most and body the least.

If you then head over to the schema page for your managed properties and click "advanced settings" on for example title, you will see which "Weight group" it is mapped to. Also listed here: http://technet.microsoft.com/en-us/library/dn169065(v=office.15).aspx#Ranking_Schema

So... you want your tags to improve on relevancy. If you have a site column named MyTaxColumn, then you will have a managed property automatically created for you named owsTaxIdMyTaxColumn. If you go to the advanced settings page for this one and set the context level to 1, then hits in this field will get the same relevancy as title. For content types, you would change the context level of the managed property SPContentType from 0 to 1,2,5 or 7, in order to give it some more rank points.

Note that a higher context level does not mean higher ranking. This depends entirely of the value the managed property have in the rank profile.

So, by using the same context level as an existing managed property in a rank profile you are getting the same rank weight by association. Think about it a couple of times and let it sink in.

Then after changing context level, do a full crawl of the items affected, and you should see a change in the rank for these items. I recommend using the SharePoint 2013 Query Tool from codeplex or a debug template, in order to see the actual rank score.

You can also create a copy of the default rank profile, and change numbers yourself or add more managed properties to it. But it's not easy.

The other approach is to add boosts to the queries using XRANK. If there's a match in the field, then boost.

Example: {searchTerms} XRANK(cb=1) filename:"{searchTerms}"

In the above query template hits in filename would give 1 extra rank points to the results. The same approach can be applied to any managed property which is queryable, and you can nest the statements.

Hope this helps somewhat... tuning rank is not easy, and you have to check if you want absolute order of results or just nod it upwards or downwards.

Thanks Mikael for the detailed response.  I will look into that in the next few days and get my head around what you have said. 

BTW - I don't want to promote ranking for a specific tag (eg MyTaxColumn in your example), just all tagged items where the tag meets the search criteria.  So I assume we can use owstaxidmetadataalltagsinfo somewhere in all this (which I have done successfully in the refiners)

I actually think a more interesting discussion is around why tagged items aren't weighted/ranked higher by default.  Isn't it the reason we tag content - to make it easily searchable??  I wonder what MS had in mind with the default ranking methodology??

Merry Christmas

Hi mike,

As I understand, you want to build your own search results ranking.

When you search, there is a default ranking model including seven managed properties with different weights, and the order is body, title, author, filename, qlogclickedtext, anchortext, socialtag.

1. If you want to search tagged item directly. You can create a managed property that maps to the crawled property. When you search, you can use the managed property to narrow the search scope.

2. If you want to make tagged item weighted higher. You can create a ranking model, and create a managed property mapping to the crawled property related to the tagged items, and set the managed property with a higher weight.

The article below gives you the details about how to create a custom ranking model with the weighted properties based on your need.

http://msdn.microsoft.com/en-us/library/office/dn169052(v=office.15).aspx

Best regards,

Sara Fan

I agree with Mikael. He is expert in SharePoint search.

Thanks Mikael and Ashish for your time and input.  I have also posted this question on the MS Office 365 Community forum, and have had no satisfaction.  I am starting to think I am a bit thick, because it is such a simple requirement, but I think the actual problem is that I cannot communicate what I need (or no one understands what I need) - which is extremely frustrating. 

Simple concept...tagged content should be ranked higher than non-tagged content in the search results...that's it guys...not too much to ask for??? 

There are dozens of web sites out there that promote the idea of tagging for searchability.  It is just not true.  Tagging documents does not promote/boost those documents in the search results.

I can confirm through extensive testing, that managed metadata terms play very little (or no) part in the ranking.  Documents with only one mention of the search keyword in the body text, often rank much higher than a document tagged with the search keyword. One example I have seen is that 500 non-tagged documents appear before one tagged document in the search results!!.  I have written a 7 page article on my findings if anyone needs the proof.

I cannot believe this is not a BIG deal and a cause for massive problems in the SharePoint world.  It certainly is for some of my customers who have brought it to my attention.

After spending at least 40 hours trying to solve this and reading every available document, I will be looking to pay a consultant to solve this problem for me - any taker

Mike,

I think Mikael explained it quite well, when using a default rank model, tagged content does not play much of a role in the overall rank score as compared to say Title or some other managed properties or events.  To meet your requirement of having tagged content play a major role in rank score calculation, you would have to make sure that context of the associated managed property is high enough or create a custom rank model and add it yourself.

You raise a good question about OOTB usage of Managed Metadata/Enterprise Keywords (or lack of) when it comes to rank calculations. 

To go back for a second, Rank Score in SP2013 is being calculated as follows(simplified version, only looking at Stage1 here):

Rank Score = Static Part + Dynamic Part + Query Properties

Dynamic Part and Static Parts consist of 9 features:  BM25F, FileType, language, ClickDistance, UrlDepth, QLogClicks, QLogSkips, QLogLastClicks, EventRate

Query Property(not configurable) consists of:  AnchorTextCompleteProperty, DateTimeUtcNow, DetectedLanguageRanking, PersonalizationData, RecommendedforQueryProperty

The only Managed Property related to Tags that I could find is called SocialTags, part of the BM25 feature  in Stage1.  I've done these calculations a while ago and things may have changed with new CU's, but if we took Title as 100% Importance, other MP's stacked up like this relative to Title importance and each other:

• Title                           100%
• QLogClickedText         72%
• SocialTag                   59%
• Filename                    52%
• Author                       41%
• AnchorText                18%
• Body                          7%

However, it looks like SocialTags aren't really part of Managed Metadata Service, but a whole different thing.  The below blog makes a point that not all Managed Metadata/Enterprise Keywords are social tags as they are part of User Profile Service and are stored in User Profile databases.

http://www.chrisweldon.net/blog/2012/12/18/sharepoint-2013-tagging-social-tags

I don't have an answer as to why a decision was made not to value Managed Metadata/Enterprise Keywords much in OOTB rank models, but you should have flexibility to add these yourself and meet your requirement.

• Marked as answer by M Fairley Tuesday, March 17, 2015 9:42 AM
• Unmarked as answer by M Fairley Tuesday, March 17, 2015 9:42 AM

If you are using Enterprise Keywords for tagging, then edit the managed property owstaxIdTaxKeywords and change the weight group to context 1 or context 2 on the SSA/tenant level.

Trigger re-indexing of all tagged content and you should see they are being boosted. It would be easy to add this to the rank profile, except if you are in SharePoint Online, as SPO won't allow you to customize a rank profile outside of managed propert boosts (static values). As for defaults, there is no rank profile which will fit all, but including a default profile with weights on tags would be useful i

I did a writeup and provide sample search config files to fix tag boosting in general over at http://techmikael.blogspot.com/2015/01/how-to-boost-metadata-in-sharepoint.html

Thanks Igor for the simplified weight distribution from the rank model :)

try this link:

http://blogs.technet.com/b/tothesharepoint/archive/2013/06/25/changing-the-ranking-of-search-results-in-sharepoint-2013.aspx

http://msdn.microsoft.com/en-us/library/office/dn169052(v=office.15).aspx

http://msdn.microsoft.com/en-us/library/office/jj938031(v=office.15).aspx

http://msdn.microsoft.com/en-us/library/office/dn169052(v=office.15).aspx#sp15_rank_features

If this is SharePoint Online, most of the ranking mods are not available.

Sorry - it's been a while to get back to this issue.  We are indeed talking about SharePoint online.  As are 90% of my customers.  Thanks so much everyone for the input - I am going to return to the issue and see if I can resolve.  Especially thanks to Mikael who has written a blog to try to answer my query.  I will report

Hi Mikael.  Sorry it has taken so long to get back to this. I am struggling through your assistance blindly :) .  I have followed your excellent write-up at http://techmikael.blogspot.com.au/2015/01/how-to-boost-metadata-in-sharepoint.html.

Here is what I have done:

1.  I uploaded search configuration file from https://github.com/wobba/SearchConfiguration. I used TagBoost-Max-SearchConfiguration.xml

2.  I uploaded to my SPO using the link you provided https://tenant-admin.sharepoint.com/_layouts/15/searchadmin/importsearchconfiguration.aspx?level=tenant

3.  When I look in my SPO I notice it states  "SearchBoost" and I see "SPSiteSubscription" and "Imported Successfully" so this seems good.

4.  Looked in Manage Search Schema > Managed Properties and I see TagBoost as per your image in the article.

5.  I performed a re crawl using the code you supplied at http://techmikael.blogspot.com.au/2014/02/how-to-trigger-full-re-index-in.html.  But I am not sure if I am completing this correctly.   I am running this code by simply copying from your web site, changing the credentials and pasting it into SP Online Management Shell. 

Do I need to wait or this an immediate effect?  I have waited 24 hrs.

Am I doing it correctly.  Any more steps?  I am not getting errors, but no boosting either?

Mike in

Hi,

What's the Title and Keyword of the item you are examining the rank log for? If the term "form" is only in the tax field and not title, then you are seeing the correct thing and weight is given (as per the Title property boost listed in the screenshot).

Thanks,
Miklael

The document is an XL file.  It has no content with the word "form".  It has no file name or title containing the word "form".  It is not in a library, list, page or folder named "form".  It has only been tagged with a term "form" from the term store.

And I agree with you that the document appears in the search results (with a score of 11.61157).

However - it is well down the list and the majority of files (100s of them) with higher rankings have the following characteristics:

 - Not been tagged as "form" with metadata

 - Do not have "form" in the title or file name

 - Are not in a library, list, page or folder named "form"

 - Many of the files only have "form" mentioned once in the content.  And yet they get a higher ranking.

So it still not working the way I was hoping. I need the tagged results to be at the top of the search results.  Or at least close to the top :)

I acknowledge the amount of effort you have contributed here and really do appreciate it.  Any more thoughts?

So it's getting a boost as per Title in the screenshot, just not what you expected :) You could for example pull up the rank of some other document and compare why it gets the rank it does.

And the reason it's not higher is because XLS files are given a negative rank on internalfiletype in the default rank model.

Ref: http://techmikael.blogspot.no/2013/04/rank-models-in-2013main-differences.html

And this feature counts more than the BM25F part. Using the rank tuning app you can change the weight for Excel files.

thanks,

Mikael

Thankyou everyone for your input on this issue.  For me - this has been a long drawn out affair over 6 months engaging in many blogs with Microsoft and other world renown experts such as Mikael.  I have finally decided to put a nail in the coffin and bury the issue forever as un-resolved and un-solvable.

The bottom line is this......

Managed metadata taxonomy (Terms from the Term Store) is very useful for ORGANISING CONTENT (ie views and metadata navigation) but do not, and can not, assist CONTENT SEARCHING.

For over 10 years I have advised clients to build their Corporate File Plans around SharePoint taxonomy and search for content.  I now realise this was a big mistake, and will need to re-think this entire approach.  This is a major problem and in my opinion makes the SharePoint Search feature obsolete because the results mean nothing to the normal end user. 

Microsoft need to address this issue.