Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repo sync for protected branch #2369

Merged
merged 18 commits into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,14 @@ When querying the entire view, the materialized part is combined with the `delta
|`materialized_view_query_optimization_costbased_enabled`| `bool` |If set to `false`, disables summarize/join optimizations in materialized view queries. Uses default strategies. Default is `true`.|
|`materialized_view_shuffle`| `dynamic` |Force shuffling of the materialized view query, and (optionally) provide specific keys to shuffle by. See [examples](#examples) below.|

### `ingestion_time()` function in the context of materialized views

[ingestion_time()](../../query/ingestion-time-function.md) function returns null values, when used in the context of a materialized view, if [querying the entire view](#materialized-views-queries).
When querying the materialized part of the view, the return value depends on the type of materialized view:

* In materialized views which include a single `arg_max()`/`arg_min()`/`take_any()` aggregation, the `ingestion_time()` is equal to the `ingestion_time()` of the corresponding record in the source table.
* In all other materialized views, the value of `ingestion_time()` is approximately the time of materialization (see [how materialized views work](#how-materialized-views-work)).

### Examples

1. Query the entire view. The most recent records in source table are included:
Expand Down
27 changes: 22 additions & 5 deletions data-explorer/kusto/management/update-table-command.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: .update table command
description: Learn how to use the .update table command to perform transactional data updates.
ms.reviewer: vplauzon
ms.topic: reference
ms.date: 08/11/2024
ms.date: 08/22/2024
---
# .update table command

Expand Down Expand Up @@ -69,7 +69,7 @@ The result of the command is a table where each record represents an [extent](ex
| -------- | -------- | -------------------------------------------------------------------------------- |
| Table | `string` | The table in which the extent was created or deleted. |
| Action | `string` | *Create* or *Delete* depending on the action performed on the extent. |
| ExtentId | `guid` | The unique identifier for the extent that was created or deleted by the command. |
| ExtentId | `guid` | The unique identifier for the extent created or deleted by the command. |
| RowCount | `long` | The number of rows created or deleted in the specified extent by the command. |

## Choose between `.update table` and materialized views
Expand All @@ -80,7 +80,7 @@ Use the following guidelines to decide which method to use:

* If your update pattern isn't supported by materialized views, use the update command.
* If the source table has a high ingestion volume, but only few updates, using the update command can be more performant and consume less cache or storage than materialized views. This is because materialized views need to reprocess all ingested data, which is less efficient than identifying the individual records to update based on the append or delete predicates.
* Materialized views is a fully managed solution. The materialized view is [defined once](materialized-views/materialized-view-create-or-alter.md) and materialization happens in the background by the system. The update command requires an orchestrated process (for example, [Azure Data Factory](/azure/data-explorer/data-factory-integration), [Logic Apps](../tools/logicapps.md), [Power Automate](/azure/data-explorer/flow), and others) that explicitly executes the update command every time there are updates. If materialized views work well enough for your use case, using materialized views requires less management and maintenance.
* Materialized views are a fully managed solution. The materialized view is [defined once](materialized-views/materialized-view-create-or-alter.md) and materialization happens in the background by the system. The update command requires an orchestrated process (for example, [Azure Data Factory](/azure/data-explorer/data-factory-integration), [Logic Apps](../tools/logicapps.md), [Power Automate](/azure/data-explorer/flow), and others) that explicitly executes the update command every time there are updates. If materialized views work enough for your use case, using materialized views requires less management and maintenance.

## Performance tips

Expand All @@ -90,7 +90,7 @@ Use the following guidelines to decide which method to use:

## Examples

For the examples, we are going to use the following table:
For the examples, we're going to use the following table:

```kusto
.set-or-replace Employees <|
Expand Down Expand Up @@ -166,7 +166,8 @@ The following example updates multiple columns on all rows with color gray.
| extend Color="";
```

### Update rows using another table

### Update rows using another table (reference values)

In this example, the first step is to create the following mapping table:

Expand All @@ -192,6 +193,22 @@ This mapping table is then used to update some colors in the original table:
| project Id, Code, Color=NewColor
```

### Update rows with a datatable

Sometimes values to update are known without being stored in a table, and the [datatable operator](../query/datatable-operator.md) can be helpful:

```kusto
.update table Employees delete D append A <|
let A = datatable(Id:long, Code:string, Color:string)[
1, "Customer", "Purple",
2, "Customer", "Magenta",
3, "Customer", "Turquoise",
];
let D = Employees
| join kind=leftsemi A on Id
| where true;
```

### Update rows with a staging table

A popular pattern is to first land data in a staging table before updating the main table.
Expand Down
4 changes: 3 additions & 1 deletion data-explorer/kusto/query/ingestion-time-function.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ monikerRange: "microsoft-fabric || azure-data-explorer || azure-monitor || micro

Returns the approximate [datetime](../query/scalar-data-types/datetime.md) in UTC format indicating when the current record was ingested.

This function must be used in context of a table for which the [IngestionTime policy](../management/ingestion-time-policy.md) is enabled. Otherwise, this function produces null values.
This function must be used in the context of a table or a [materialized view](../management/materialized-views/materialized-view-overview.md#ingestion_time-function-in-the-context-of-materialized-views). Otherwise, this function produces null values.

If [IngestionTime policy](../management/ingestion-time-policy.md) was not enabled when the data was ingested, the function returns null values.

::: moniker-end

Expand Down
1 change: 1 addition & 0 deletions data-explorer/managed-identities-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,4 @@ After assigning the managed identity to your cluster and configuring the relevan

* [Configure managed identities for your cluster](configure-managed-identities-cluster.md)
* [Authenticate external tables with managed identities](external-tables-managed-identities.md)
* [Examples of using storage connection string for managed identity](/kusto/api/connection-strings/kusto#examples)