C# — query Cognitive Search

In one of previous posts, I’ve shown how to query CosmosDB.

CosmosDB — .NET SKDv3 — work with data

Today I plan to show you, how to query the same data from the same Cosmos database but a Cognitive Search service is in the middle. In this equation the Cosmos database is a data source for the Cognitive Search service and the Cognitive Search service for C# queries.

Assuming we have a CosmosDB container filled with data we can go to Cognitive Service and import the mentioned container content.

Here is a JSON file structure in the Cosmos container.

{
    "County": "Black Hawk",
    "State": "Iowa",
    "Quantity": 131228,
    "id": "566bbd8a-8393-4721-a62f-5dce0da51c4c",
    "_rid": "FiQ7ANIOGHcHAAAAAAAAAA==",
    "_self": "dbs/FiQ7AA==/colls/FiQ7ANIOGHc=/docs/FiQ7ANIOGHcHAAAAAAAAAA==/",
    "_etag": "\"0000f833-0000-0d00-0000-643aad980000\"",
    "_attachments": "attachments/",
    "_ts": 1681567128
}js

Switch over to Cognitive Service, go to an Import data section. Then fill in all required parameters.

In the next step, we customize a target index — this index will be queried from a .NET app. You can set a key field here and a manner of how respective fields should be indexed.

Retrievable — the field is projected in a search result set,

Filterable — accepts OData-style filtering on the field,

Sortable — enables sorting using the field,

Facetable — a field can be dynamically aggregated or grouped,

Searchable — search queries to match terms in the field.

In the last step you can set up a refresh schedule or track deletions.

After the index is created, you can go to an Overview page and check it at an Indexes and an Indexers section.

Now, we can go to the .NET app and try to query some data.

Firstly, we need to install some libraries.

using Azure;
using Azure.Search.Documents;
using Azure.Search.Documents.Indexes;
using Azure.Search.Documents.Models;
using System.Text.Json.Serialization;

In the second step, we create a class to map query results.

public class IowaPopulation
{
    [JsonPropertyName("County")]
    [SearchableField(IsFilterable=true, IsSortable=true)]
    public string ?CountyName { get; set; }
    
    [JsonPropertyName("State")]
    [SearchableField(IsFilterable=true)]
    public string ?StateName { get; set; }

    [JsonPropertyName("Quantity")]
    [SimpleField(IsFilterable=true, IsSortable=true)]
    public int Quantity { get; set; }
}

As you can notice, you can set additional attributes for all the fields like mapping or property behavior in the code.

This part of code sets up the query. We need three elements:

• an Endpoint Uri — available on the Overview page,
• an API key — available at a Keys page,
• an index name — this is the index name we previously created.

string searchEndpoint = AzureVariables.searchEndpoint; // An endpoint Uri
string apiKey = AzureVariables.apiKey; // An API key
string searchIndex = "iowa-population-cosmosdb-index"; // An index name

AzureKeyCredential credential = new AzureKeyCredential(apiKey); // Create a new credential
Uri endpoint = new Uri(searchEndpoint); // Create a new uri
SearchClient client = new SearchClient(endpoint, searchIndex, credential); // Create a new searchclient

SearchOptions options = new SearchOptions()
{
    IncludeTotalCount = true, // A returned data total
    // Filter = "", // All documents returned
    // Filter = SearchFilter.Create($"Quantity eq 13687"),  // A simple query
    Filter = SearchFilter.Create($"(Quantity gt 300000 and County eq 'Polk') or Quantity eq 25062 or County eq 'Cedar' or search.in(County, 'Cass', 'Clarke')  or search.ismatch('*Gordo', 'County')"), // A bit complicated query
    Size = 15, // Records returned (SQL SELECT TOP 15...)
    OrderBy = { "Quantity desc, County asc" } // How data is ordered
};
// Fields returned (SQL SELECT County, State, Quantity...)
options.Select.Add("County");
options.Select.Add("State");
options.Select.Add("Quantity");

In the next step, the app creates a SearchClient object based on provided information in the previous point. Next section set up query properties, like ordering, a result size or filter conditions (OData style). The last section works like a SQL SELECT statement — these fields are projected in the query result.

We can map the result to a class created at the beginning…

// A result mapped to a class, options are applied
SearchResults<IowaPopulation> response00 = await client.SearchAsync<IowaPopulation>("*", options);
var searchResponse = response00.GetResults();

foreach (SearchResult<IowaPopulation> item in searchResponse)
{
    System.Console.WriteLine($"County: {item.Document.CountyName}, Quantity: {item.Document.Quantity}");
}

… or we can assign the result to a dynamic dictionary.

// A result mapped to a dynamic dictionary
SearchResults<SearchDocument> response02 = client.Search<SearchDocument>("*", options);

foreach (SearchResult<SearchDocument> item in response02.GetResults())
{
    System.Console.WriteLine($"County: {item.Document["County"]}, Quantity: {item.Document["Quantity"]}");
}

If we want to search all the fields inside the index without any additional conditions, you can simply use this code.

SearchResults<IowaPopulation> response01 = await client.SearchAsync<IowaPopulation>("Benton");