reddog-io / reddog.search

Management library for Microsoft Azure Search

RedDog.Search

This library interacts with the Microsoft Azure Search REST API. You can use the library to manage indexes, populate indexes and execute queries.

Getting Started

Initialize the ApiConnection with your credentials:

ApiConnection connection = ApiConnection.Create("myservice","mykey");

Index Management

Creating an index:

var client = new IndexManagementClient(connection);
await client.CreateIndexAsync(new Index("records")
    .WithStringField("id", f => f
        .IsKey()
        .IsRetrievable())
    .WithStringField("title", f => f
        .IsSearchable()
        .IsRetrievable())
    .WithDateTimeField("createdOn", f => f
        .IsRetrievable()));

Updating an index:

var client = new IndexManagementClient(connection);
await client.UpdateIndexAsync(new Index("records")
    .WithStringField("id", f => f
        .IsKey()
        .IsRetrievable())
    .WithStringField("author", f => f
        .IsSearchable()
        .IsSortable()
        .IsRetrievable())
    .WithStringField("title", f => f
        .IsSearchable()
        .IsRetrievable())
    .WithDateTimeField("createdOn", f => f
        .IsRetrievable()));

Adding a scoring profile

var index = new Index("products")
// add additional fields to the index.

ScoringProfile scoringProfile = new ScoringProfile();
scoringProfile.Name = "personalized";
scoringProfile.Functions.Add(new ScoringProfileFunction()
{
    Type = ScoringProfileFunctionType.Tag,
    Boost = 2,
    FieldName = "brandTags",
    Tag = new ScoringProfileFunctionTag() { TagsParameter = "brands" }                
}); 
index.ScoringProfiles.Add(scoringProfile);

List all indexes:

var client = new IndexManagementClient(connection);
var indexes = await client.GetIndexesAsync();

Delete an index:

var client = new IndexManagementClient(connection);
var records = await client.DeleteIndexAsync("records");

Get the statistics for an index:

var client = new IndexManagementClient(connection);
var records = await client.GetIndexStatisticsAsync("records");
Console.WriteLine("Total documents: {0}", records.Body.DocumentCount);
Console.WriteLine("Total size: {0} bytes", records.Body.StorageSize);

Upload data to your index:

var client = new IndexManagementClient(connection);
var result = await client.PopulateAsync("records",
    new IndexOperation(IndexOperationType.Upload, "id", "1")
        .WithProperty("title", "My first movie")
        .WithProperty("author", "Sandrino")
        .WithProperty("createdOn", new DateTimeOffset(2014, 8, 1, 0, 0, 0, TimeSpan.Zero)),
    new IndexOperation(IndexOperationType.Upload, "id", "2")
        .WithProperty("title", "My second movie")
        .WithProperty("author", "Sandrino")
        .WithProperty("createdOn", new DateTimeOffset(2014, 8, 2, 0, 0, 0, TimeSpan.Zero)));

Querying

Execute a query:

var client = new IndexQueryClient(connection);
var results = await client.SearchAsync("records", new SearchQuery("movie")
    .OrderBy("author")
    .SearchField("title")
    .Count(true));

Execute a query by using NextLink:

var client = new IndexQueryClient(connection);
var results = await client.SearchAsync("records", new SearchQuery("movie")
    .OrderBy("author")
    .SearchField("title")
    .Count(true));

var nextResults = await client.SearchAsync(results.Body.NextLink);

Execute a query with highlighting:

var client = new IndexQueryClient(connection);
var results = await client.SearchAsync("records", new SearchQuery("movie")
    .OrderBy("author")
    .SearchField("title")
    .Count(true)
	.Highlight("title")
	.HighlightPreTag("<p>")
	.HighlightPostTag("</p>"));

Execute a query with faceting:

var client = new IndexQueryClient(connection);
var results = await client.SearchAsync("records", new SearchQuery("movie")
    .OrderBy("author")
    .SearchField("title")
	.Facet("rating", "values:1|2|3")
    .Count(true));

Execute a query with a score profile:

var client = new IndexQueryClient(connection);
var results = await client.SearchAsync("records", new SearchQuery("products")
    .SearchField("title")
	.ScoringProfile("personalized")
    .ScoringParameter("brands:brandA,brandB")
    .Count(true));

Execute a suggestion:

var client = new IndexQueryClient(connection);
var results = await client.SuggestAsync("records", new SuggestionQuery("mov")
	.Fuzzy(true)
	.Select("author")
	.Select("title")                    
	.SearchField("title")
	.OrderBy("title")
	.Top(10));

Execute a lookup:

var client = new IndexQueryClient(connection);
var results = await client.SuggestAsync("records", new LookupQuery("11ad89b6-9f1b-4380-aa06-8da39df61210")
	.Select("author,title"));

Handling exceptions:

var response = await client.DoSomething();
if (!response.IsSuccess)
{
    Console.WriteLine("{0}: {1}", response.Error.Code, response.Error.Message);
}

reddog.search's Issues

Typo in FunctionAggregation enumeration

There's a typo in FunctionAggregation enumeration. The value should be "Maximum" instead of "Maxium".

Implement Tag Boosting on scoring profile

According to this article:
http://azure.microsoft.com/blog/2015/02/05/personalizing-search-results-announcing-tag-boosting-in-azure-search/

there is a new scroing profile.
Could you please implement this?

Queries with / character causes silent failure

Not sure if this is you, or Azure Search service. If I try to post a query that has a '/' in the key column, I get some really unexpected behavior. The result of the PopulateAsync call comes back as a success, but there is nothing on the index. It turned out that the '/' in my url used as the key field was causing the problem. As soon as I UrlEncoded'd the value, everything started to work again:

var popResult = await client.PopulateAsync(indexName,
    new IndexOperation(IndexOperationType.Upload, "url", HttpUtility.UrlEncode(url))
        .WithProperty("content", articleMetadata.Content)
        .WithProperty("title", articleMetadata.Metadata["pageTitle"])
);

PopulateAsync fails when more the 1000 Index Operations

This is not a bug but rather a missing feature.

As per http://msdn.microsoft.com/en-us/library/azure/dn798930.aspx, a POST for populating an index should not contain more than 1000 document entities.

(up to 1000 documents per batch, or about 16 MB per batch)

Actually, when trying to upload more than 1000 documents, Azure Search REST API returns the following error message:

{"error":{"code":"","message":"The request is invalid.","innererror":{"message":"actions : The items collection must have from 1 to 1000 valid items to be indexed.\r\n","type":"","stacktrace":""}}}

PopulateAsync should handle some kind of batch mode, splitting the IndexOperation param array into parts up to maximum documents per batch.

How to show all the results of the search?

Hi,

Right now we can see only 50 records in the azure search. What if want to implement pagination on the total result set?

Thanks,
Nikita

asciifolding suport

I can't seem to find a way to implement asciifolding support. Since azure search is powered by elastic search, it should be doable?

http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/asciifolding-token-filter.html

sendasync - transfer CancellationToken

Hi.
Can you please implement an override that gets CancellationToken at SearchAsync

don't understand Facet syntax

could you please give some more explanation about the Facet() syntax

in particular facetParameters
thanks

RedDog.Search v.0.5.2.0 - SuggesterName missing

Hi there,
Just find out that SuggesterName property is missing from SuggestionQuery class. That makes suggestion functionality completely unusable. Can that be fixed please.

When Searching the Top parameter is not taken into account.

IndexQueryClient.cs line 48.

Facet search implementation does not respect faceting parameters

As per the api documentation, the facet parameter may contain 4 optional protected parameters 'count', 'sort', 'values' and 'interval'
http://msdn.microsoft.com/en-us/library/azure/dn798927.aspx

For example when I use .Facet("wng_tags,count:100") it should translate to
search=test&facet=wng_tags,count:100 and not treat the special parameter 'count' as a new facet parameter, like it does now: search=test&facet=wng_tags&facet=count:100. This returns an error as we have no field with the name 'count'

An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full

My site went down with this flooding my exception logs, is it leaking socket connections? I've left the app running for 2 weeks since i've implemented reddog.search