Adding Directlake options for collecting metrics #118
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added 3 different modes for direct lake models so that we do not always force all columns to be transcoded into memory.
I've added this option since the default behaviour at the moment is the same as Full and this can be detrimental to large models. It can take longer than necessary and potentially may still not produce the desired result since as we cycle through all the columns it could force some of the earlier columns to be paged out of memory.
I've set ResidentOnly as the default as it's the cheapest/fastest option.
Note that if an option other than ResidentOnly is chosen I am re-running the DMV collection after the stats collection has been run since the memory usage is usually changed as a result of running the distinctcount queries.
I did consider whether to add the selected Direct Lake analysis mode as a property to the vpax file, but I decided against his for the time being since it can be implied by checking the IsResident property on a column.