Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions skills/cosmosdb-best-practices/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ Reference these guidelines when:
- [query-pagination](rules/query-pagination.md) - Use continuation tokens for pagination
- [query-avoid-scans](rules/query-avoid-scans.md) - Avoid full container scans
- [query-parameterize](rules/query-parameterize.md) - Use parameterized queries
- [query-order-filters](rules/query-order-filters.md) - Order filters by selectivity
- [query-order-filters](rules/query-order-filters.md) - Let the query engine order filters; tune indexed predicates rather than their textual order
- [query-top-literal](rules/query-top-literal.md) - Use literal integers for TOP, never parameters
- [query-latest-by-timestamp](rules/query-latest-by-timestamp.md) - Query "latest" documents with explicit ORDER BY and TOP 1
- [query-olap-detection](rules/query-olap-detection.md) - Detect and redirect analytical queries away from transactional containers
Expand Down Expand Up @@ -206,7 +206,7 @@ Reference these guidelines when:
- [fts-index-policy](rules/fts-add-index.md) - Add `fullTextIndexes` entry in the indexing policy to build the inverted index
- [fts-contains-query](rules/fts-keyword-matching.md) - Use `FullTextContains` / `FullTextContainsAll` / `FullTextContainsAny` instead of `CONTAINS(LOWER(...))`
- [fts-score-ranking](rules/fts-relevance-ranking.md) - Use `ORDER BY RANK FullTextScore(path, term)` for BM25 relevance ranking
- [fts-hybrid-query](rules/fts-hybrid-queries.md) - Combine FTS predicates with range/equality filters; put most selective filter first
- [fts-hybrid-query](rules/fts-hybrid-queries.md) - Combine FTS predicates with selective indexed equality/range filters; predicate text order does not control execution

## How to Use

Expand Down
4 changes: 2 additions & 2 deletions skills/cosmosdb-best-practices/rules/fts-hybrid-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ tags:

**Impact: MEDIUM (avoids full-container scans when combined with equality/range filters)**

FTS predicates can be combined with standard SQL predicates. Cosmos DB uses the most selective predicate first. Put the most restrictive filter (e.g., equality on a high-cardinality property) before the FTS predicate to reduce the candidate set.
FTS predicates can be combined with standard SQL predicates. Add selective equality or range filters with appropriate indexes to narrow the matching document set, and include a partition-key equality filter when the query should be scoped to that partition. The query engine determines predicate evaluation order; moving equivalent predicates earlier in the `WHERE` clause does not improve performance.
Comment thread
sevoku marked this conversation as resolved.

**Incorrect (FTS-only query — no range filters, scans all partitions):**

Expand Down Expand Up @@ -57,4 +57,4 @@ return container.queryItems(
- Numeric fields — use range index with `=`, `>`, `<`
- Array elements already indexed with `[]/?` — `CONTAINS(LOWER(t), @q)` via EXISTS is fine

Reference: [Full-text search queries](https://learn.microsoft.com/azure/cosmos-db/gen-ai/full-text-search)
References: [Full-text search queries](https://learn.microsoft.com/azure/cosmos-db/gen-ai/full-text-search), [index usage and filter-clause ordering](https://learn.microsoft.com/azure/cosmos-db/index-overview#composite-indexes)
90 changes: 32 additions & 58 deletions skills/cosmosdb-best-practices/rules/query-order-filters.md
Original file line number Diff line number Diff line change
@@ -1,77 +1,51 @@
---
title: Order Filters by Selectivity
title: Let the Query Engine Order Filters
Comment thread
sevoku marked this conversation as resolved.
impact: MEDIUM
impactDescription: reduces intermediate result sets
impactDescription: avoids ineffective predicate-order tuning and incorrect execution assumptions
tags: query, filters, optimization, performance
---

## Order Filters by Selectivity
## Let the Query Engine Order Filters

Place most selective filters first in WHERE clauses. The query engine processes filters left-to-right, so selective filters early reduce data scanned.
The textual order of equivalent predicates in a `WHERE` clause does not determine their execution order. Cosmos DB's query engine determines which predicates are more selective and how to execute the query. Moving a selective predicate earlier in the SQL text is not a performance optimization.

**Incorrect (least selective filter first):**
**Incorrect (assuming lower RU cost solely from reordering equivalent predicates):**

```csharp
// Status has low selectivity (few unique values)
// Filters 1M items to 300K, then to 100
var query = @"
SELECT * FROM c
WHERE c.status = 'active' -- 30% of items match
AND c.type = 'order' -- 10% of items match
AND c.customerId = @customerId"; -- 0.01% match (highly selective)

// Processes: 1M → 300K → 100K → 100
// More intermediate processing than necessary
var originalQuery = @"
SELECT * FROM c
WHERE c.status = 'active'
AND c.type = 'order'
AND c.customerId = @customerId";

var reorderedQuery = @"
SELECT * FROM c
WHERE c.customerId = @customerId
AND c.type = 'order'
AND c.status = 'active'";
```

**Correct (most selective filter first):**
Both queries are valid and express the same filters. The mistake is claiming that `reorderedQuery` is cheaper merely because `customerId` appears first, or assigning intermediate row counts based on the clauses' textual positions.

```csharp
// CustomerId is highly selective (unique per customer)
var query = @"
SELECT * FROM c
WHERE c.customerId = @customerId -- 0.01% match (filter first!)
AND c.type = 'order' -- Then narrow by type
AND c.status = 'active'"; -- Finally by status

// Processes: 1M → 1K → 100 → 100
// Much less intermediate data
```
**Correct (use readable predicates and optimize actual index usage):**

```csharp
// Selectivity guidelines (from most to least selective):
// 1. Unique identifiers: id, customerId, orderId (highest)
// 2. Foreign keys with many values: productId, userId
// 3. Timestamps (range queries): createdAt, modifiedAt
// 4. Categories with many values: categoryId, departmentId
// 5. Status fields: status, state (low selectivity)
// 6. Boolean flags: isActive, isDeleted (lowest - only 2 values)

// Example: Combining timestamp with status
var query = @"
SELECT * FROM c
var query = new QueryDefinition(@"
SELECT * FROM c
WHERE c.customerId = @customerId
AND c.orderDate >= @startDate
AND c.orderDate < @endDate
AND c.status = 'completed'";

// Even better with composite index
AND c.type = 'order'
AND c.status = 'active'")
.WithParameter("@customerId", customerId);
```

```csharp
// Use BETWEEN with high selectivity values
var query = @"
SELECT * FROM c
WHERE c.orderId >= @startId AND c.orderId <= @endId -- Very selective range
AND c.status = 'active'";
The order above is for readability, not an execution hint. To tune performance, inspect index usage and measured request charges/query metrics rather than assuming that swapping the same predicates reduces work.

// For OR clauses, check if rewriting helps
// Less efficient:
var query1 = "SELECT * FROM c WHERE c.status = 'a' OR c.status = 'b' AND c.customerId = @id";
// Better (explicit grouping):
var query2 = "SELECT * FROM c WHERE (c.status = 'a' OR c.status = 'b') AND c.customerId = @id";
// Best (if possible, use IN):
var query3 = "SELECT * FROM c WHERE c.status IN ('a', 'b') AND c.customerId = @id";
```
**Key points:**

- Selectivity depends on the data distribution, not just the property name or type.
- Add useful filters with appropriate index support; a partition-key equality filter can narrow the query's partition scope regardless of where it appears in the `WHERE` clause.
- Preserve Boolean grouping when rewriting queries. Adding parentheses around `OR` predicates can change which documents match; it is not merely a performance rewrite.

See also: [Avoid cross-partition queries](query-avoid-cross-partition.md), [avoid full scans](query-avoid-scans.md), [combine FTS with indexed filters](fts-hybrid-queries.md).

Reference: [Query optimization tips](https://learn.microsoft.com/azure/cosmos-db/nosql/performance-tips-query-sdk)
Reference: [Index usage and filter-clause ordering](https://learn.microsoft.com/azure/cosmos-db/index-overview#composite-indexes)