24.6.0

Various fixes & improvements

• chore(on-call): Properly handling query validation errors (#6019) by @enochtangg
• update CH version used in CI (#6027) by @volokluev
• code (#6026) by @xurui-c
• code (#6024) by @xurui-c
• doc(rs): some docs for the ClickHouse batch module (#6020) by @onewland
• chore(gocd): Bumping gocd-jsonnet version (#6021) by @IanWoodard
• [CapMan] change default throttle values (#6022) by @xurui-c
• [CapMan] Add throttling to ReferrerGuardRail policy (#6014) by @xurui-c
• feat(issue-search): Add group_first_release_id to GroupAttributes processor (#5987) by @snigdhas
• Fixing the metric increment for cache hit/miss and (#6018) by @nachivrn
• ref(ci): update clickhouse versions (#6016) by @MeredithAnya
• update docs for snuba storage queries (#6013) by @enochtangg
• In the Snuba Admin Production queries UI (#6015) by @nachivrn
• SNS-2737: Making changes for a simple readthrough cache without queuing (#5992) by @nachivrn
• chore(profiles-chunks): Add missing consumer to deployment list (#6001) by @rgibert
• fix(gocd): Updating canary checks to happen after the deploy goes out (#6004) by @IanWoodard
• ref(admin): Add a system query for data skipping indexes (#6008) by @evanh
• feat(issue-search): Add group_first_release_id to the GroupAttributes table (#5986) by @snigdhas
• ref: sentry-kafka-schemas 0.1.90 (#6007) by @lynnagara
• ref(async-inserts): add clickhouse-concurrency cli arg (#5999) by @MeredithAnya
• fix: snuba admin queries check (#6005) by @MeredithAnya
• add tags[environment] to discover low cardinality processor (#5997) by @enochtangg
• meta: Bump new development version (9dac11b)

24.5.1

Various fixes & improvements

• chore(gocd): Bumping gocd-jsonnet version (#6000) by @IanWoodard
• ref(async-inserts): Add system queries (#5998) by @MeredithAnya
• Include metrics for the number of queries resulting in cache hits, misses, and waits associated with datasets. (#5991) by @nachivrn
• fix(meta) Change the view to use LEFT ARRAY JOIN (#5975) by @evanh
• Revert "add option to print full payload on invalid message (#5979)" (#5981) by @john-z-yang
• fix(error): Skip lineno deserialization on failure (#5983) by @john-z-yang
• feat(generic-metrics): Add gauges subscriptions consumers to Snuba deployment (#5980) by @ayirr7
• add option to print full payload on invalid message (#5979) by @john-z-yang
• chore: retire use_new_combine_conditions feature flag (#5977) by @kylemumma
• ref(subscriptions): Add query-based exception handling in scheduler (#5976) by @ayirr7
• SNS 2645 - Surface allocation policy decisions in production queries tool (#5970) by @nachivrn
• remove logging (#5978) by @xurui-c
• fix(admin): Allow LEFT ARRAY JOIN in the admin tool (#5974) by @evanh
• feat(replays): Add migration for distributed materialization (#5951) by @cmanallen
• ref(async-inserts): allow async inserts as an option (#5955) by @MeredithAnya
• allow one replacement project to be skipped (#5965) by @xurui-c
• Debugging bucket timer (#5963) by @xurui-c
• feat(generic-metrics): Add logic in distributions processor to drop percentiles (#5911) by @ayirr7
• feat(generic-metrics): Add migrations for opt-in disabled percentiles (#5910) by @ayirr7
• fix(clickhouse): Remove unecessary logging (#5958) by @nikhars
• logging (#5959) by @xurui-c
• feat(metrics): add metric_stats use case to generic metrics meta tables (#5954) by @shellmayr
• number of projects skipped (#5952) by @xurui-c
• fix(snuba-admin): Add quotes around ColumnSizeOnDisk's table value (#5953) by @phacops

Plus 28 more

24.5.0

Various fixes & improvements

• ref(rust-consumer): join-timeout per-step (#5918) by @untitaker
• ref: Bump arroyo (#5917) by @untitaker
• ref(api): parse pipeline tests, mql after treeify #5894 (#5894) by @kylemumma
• one more auto-offset-reset earliest (#5909) by @lynnagara
• fix: Temporarily remove check for coupled migrations and code changes (#5903) by @ayirr7
• ref: bump sentry-kafka-schemas to 0.1.82 (#5906) by @getsentry-bot
• fix(CI): Make timestamp relative for metrics summaries test (#5904) by @ayirr7
• feat(generic-metrics): Retry adding gauges subscriptions topics + configs (#5892) by @ayirr7
• ref(api): parse pipeline tests, snql pipeline before treeify (#5886) by @kylemumma
• sessions: drop the clickhouse tables (#5882) by @lynnagara
• feat: Add support for DateTime64 column type (#5896) by @phacops
• fix: dlq-consumer should default to auto-offset-reset earliest (#5893) by @lynnagara
• fix(capman): make allocation policies work with joins (#5887) by @volokluev
• ref(devexp): Make entities optional in dataset config (#5879) by @enochtangg
• ref(api): parse pipeline tests, mql pipeline before treeify (#5885) by @kylemumma
• bump snuba-sdk to 2.0.34 (#5884) by @enochtangg
• chore(self-hosted): Remove dependency on snuba-image build step during e2e action (#5891) by @hubertdeng123
• remove skip-write flag (#5846) by @john-z-yang
• feat(devexp): Add required_time_column to storage (#5889) by @volokluev
• feat(spans): Add indexes for tag columns (#5871) by @phacops
• feat(generic-metrics): Add zstd decompression to generic metrics processor (#5845) by @ayirr7
• snuba devserver fix? (#5873) by @MeredithAnya
• update CLICKHOUSE_SERVER_MIN_VERSION and CLICKHOUSE_SERVER_MAX_VERSION (#5853) by @MeredithAnya
• fix: Handle tag columns in low cardinality processor (#5875) by @evanh

Plus 35 more

24.4.2

Various fixes & improvements

• feat(generic-metrics): Add success metric around Base64 message processing (#5830) by @ayirr7
• chore(deps): bump python and node (#5829) by @mdtro
• ref: remove pushing to legacy gcr (#5819) by @asottile-sentry
• Enables Clickhouse Authorization (#5818) by @xurui-c
• remove ConditionSimplifierProcessor (#5820) by @volokluev
• chore(dev-exp): Remove old pipeline code (#5821) by @enochtangg
• Revert "feat(generic-metrics): Add support for subscriptions to gauges (#5736)" (d0c5093) by @getsentry-bot
• feat(generic-metrics): Add Base64 decoding to Snuba processors (#5761) by @ayirr7
• feat(generic-metrics): Add support for subscriptions to gauges (#5736) by @ayirr7
• add a metric for how many queries are waiting for readthrough cache (#5817) by @volokluev
• ref: use artifact registry (#5816) by @asottile-sentry
• feat(meta): Create distributions meta tables (#5748) by @evanh
• chore: Bump Kafka schema version to 0.1.71 (#5814) by @ayirr7
• Remove unnecessary test (#5812) by @ayirr7
• feat(meta): Create gauges meta tables (#5749) by @evanh
• Revert "Revert "ref: dual-write docker image to artifact registry (#5798)"" (#5813) by @volokluev
• fix: Don't log this every time, it clogs the logs up (#5799) by @evanh
• feat(devexp): Split entity and storage processing for composite queries (#5785) by @enochtangg
• feat(meta) Create sets meta tables (#5747) by @evanh
• Emit a metric when replacements are skipped (#5809) by @xurui-c
• Revert "ref: dual-write docker image to artifact registry (#5798)" (b69f85f) by @getsentry-bot
• ref: dual-write docker image to artifact registry (#5798) by @asottile-sentry
• fix(meta) Rename counter tables for consistency (#5797) by @evanh
• fix(22.8) don't use ifnull for cardinality casting (#5807) by @volokluev

Plus 14 more

24.4.1

Various fixes & improvements

• feat(replays): add migration for replay_id in discover (#5790) by @JoshFerge
• feat: replacer defaults to auto.offset.reset=earliest (#5772) by @lynnagara
• feat(migrations): add discover local (#5788) by @dbanda
• fix(spans): Let null domain be null (#5780) by @phacops
• feat(meta): Adjust partitioning/settings of counters meta tables (#5784) by @evanh
• Revert "feat(replays): add replay_id column to merged discover table (#5777)" (2f1509f) by @getsentry-bot
• fix: fix mypy --strict vscode (#5781) by @kylemumma
• feat(replays): add replay_id column to merged discover table (#5777) by @JoshFerge
• ref: Rust consumer should not skip writes by default (#5778) by @lynnagara
• chore(on-call): Add CrossOrgQueryAllocationPolicy to errors (#5774) by @enochtangg
• inc-715: rust consumer can stop processing messages at specific timestamp (#5779) by @lynnagara
• meta: Bump new development version (2248bb5)
• fix(meta): Remove experimental meta tables (#5733) by @evanh
• ref(card-an): Allow the new meta tables in the cardinality analyzer (#5769) by @evanh

24.4.0

Various fixes & improvements

• chore(on-call): Add metric for concurrent queries by referrer that violate policy (#5767) by @enochtangg
• ref: make auto.offset.reset=earliest everywhere (#5765) by @lynnagara
• feat(trace): Add trace id to transactions (#5768) by @wmak
• fix(spans): Move span id above trace id in the prewhere (#5766) by @wmak
• feat(meta): Add record meta column to gauges (#5760) by @evanh
• feat(meta): Add record meta column to distributions (#5759) by @evanh
• feat(meta): Add record_meta column to sets (#5735) by @evanh
• ref: bump sentry-kafka-schemas to 0.1.68 (#5764) by @getsentry-bot
• chore(capman): set default bytes scanned limit for rejecting policy (#5755) by @volokluev
• ref(ch-upgrades): create dist tables functionality (#5737) by @MeredithAnya
• perf(metrics): Use kafka header optimization (#5756) by @nikhars
• feat(meta): Add updated versions of meta tables for counters (#5734) by @evanh
• chore(rust): Update dependencies (#5751) by @nikhars
• chore: update sdk version (#5754) by @kylemumma
• fix: Fix default auto-offset-reset value (#5753) by @lynnagara
• lower max query size to 128KiB (#5750) by @enochtangg
• clean up old simple pipeline (#5732) by @enochtangg
• feat(capman): Long term rejection allocation policy (#5718) by @volokluev
• ref(fetcher): --tables optional now (#5730) by @MeredithAnya
• fix: parser, bug in pushdown filter (#5731) by @kylemumma
• feat(generic-metrics): Add a killswitch to processor (#5617) by @ayirr7
• feat(replays): Replace python processor with a rust-based processor (#5380) by @cmanallen
• feat(replay): add ReplayViewedEvent to replay processor (#5712) by @aliu3ntry
• chore(deps): bump h2 from 0.3.22 to 0.3.26 in /rust_snuba (#5727) by @dependabot

Plus 52 more

24.3.0

Various fixes & improvements

• Unrevert: feat: Remove query splitters from the API (#5581) by @evanh
• feat: Add use_case_id index to generic metrics (#5655) by @evanh
• ref(ci): Remove deleted test file (#5656) by @evanh
• fix vscode debugger (#5652) by @kylemumma
• chore: Upgrade snuba-sdk to 2.0.31 (#5647) by @iambriccardo
• fix(gocd): put snuba cmd into $SNUBA_CMD (#5654) by @MeredithAnya
• enable canary health check (#5649) by @enochtangg
• Revert "fix(CapMan): Allocation Policies causing potentially timeout errors on ST (#4403)" (703042e) by @getsentry-bot
• fix(gocd): add SNUBA_CMD_TYPE (#5648) by @MeredithAnya
• Allows empty trace_id (#5637) by @xurui-c
• fix: Fix bump version for rust (#5643) by @lynnagara
• feat(generic-metrics): Add metrics around encoding format type in processor (#5627) by @ayirr7
• feat: filter by metric_id in select logical query optimizer (#5610) by @kylemumma
• fix(gocd): fix unbound variable (#5641) by @MeredithAnya
• ref: bump sentry-kafka-schemas to 0.1.60 (#5642) by @getsentry-bot
• add canary health check to gocd pipeline (#5638) by @enochtangg
• ref(codecov) Try out the failed test feature in Codecov (#5635) by @evanh
• feat(spans): Enable spans storage in ST and self-hosted (#5629) by @phacops
• fix: Fix a bug in HexIntColumnProcessor that skipped array conditions (#5640) by @evanh
• ref(gocd): use shared script query-fetcher (#5639) by @MeredithAnya
• ref(gocd): add comparer pipeline, consolidate script? (#5636) by @MeredithAnya
• feat(spans): Set the migration group as complete to run migrations everywhere (#5634) by @phacops
• feat(admin): Absolute imports in snuba-admin (#5630) by @volokluev
• the default value of trace_id will be a randomly generated uuid inste… (#5628) by @xurui-c

Plus 72 more

24.2.0

Various fixes & improvements

• Revert "feat(rust): Count schema validation failures (#5515)" (b597627) by @getsentry-bot
• ref(gcs): add list_blobs and blob_exists (#5545) by @MeredithAnya
• add referrer guard rail policy to every storage (#5536) by @volokluev
• ref: Remove PolymorphicMetricsProcessor (#5541) by @untitaker
• fix(rust): Use the same release as in Python (#5543) by @untitaker
• ref: Make ExceptionMechanism nullable (#5542) by @untitaker
• feat: Add team-ops to snuba admin (#5540) by @lynnagara
• ref(rust): Remove concurrency override (#5538) by @lynnagara
• ref: Remove unused topics (#5537) by @lynnagara
• cleanup(capman): remove legacy rate limit configs (#5513) by @volokluev
• fix(spans): Profile id need UUIDColumnProcessor in indexed spans (#5533) by @Zylphrex
• ref: bump sentry-kafka-schemas to 0.1.49 (#5534) by @getsentry-bot
• fix(rust): Nullability around headers and frames (#5532) by @untitaker
• fix(gocd): remove explicit context flag from snuba-stable (#5529) by @bmckerry
• fix(rust): Fix a few more nullability issues (#5531) by @untitaker
• fix(rust-errors): Make stacktrace nullable as well (#5527) by @untitaker
• change: Move curl from buildDep to runtimeDep (#5528) by @dmajere
• fix: xfail optimize test again :( (#5506) by @MeredithAnya
• feat(querylog): Add query to get project scanning most bytes (#5530) by @nikhars
• fix(metrics): Update gauges to handle avgIf functions (#5523) by @evanh
• fix(querylog): Use _dist table (#5526) by @nikhars
• build(deps): bump sentry-arroyo from 2.16.0 to 2.16.1 (#5516) by @dependabot
• feat(rust): Count schema validation failures (#5515) by @phacops
• feat(mql): Parse the entity from the MRI (#5501) by @evanh

Plus 4 more

24.1.2

Various fixes & improvements

• feat(generic-metrics): Add DLQ back to generic metrics storages (#4964) by @ayirr7
• feat(rust): Generic metrics compat processor (#5469) by @lynnagara
• ref(rust): Add better error messages (#5508) by @untitaker
• Reapply "ref: Python 3.11.6 (#5476)" (#5505) by @getsentry-bot
• fix(py311): Explicitly import importlib.abc (#5507) by @untitaker
• Revert "ref: Python 3.11.6 (#5476)" (#5505) by @volokluev
• feat(capman): Referrer guard rail policy (#5481) by @volokluev
• ref: Python 3.11.6 (#5476) by @untitaker
• Bumping GoCD jsonnet to include de (#5493) by @IanWoodard
• fix(rust): Work around poor error messages by serde (#5503) by @untitaker
• Tags can be submitted as null (#5504) by @cmanallen
• fix(capman): Fix invalid tenant ids exception in allocation policy (#5489) by @volokluev
• admin nav bug fix (#5499) by @kylemumma
• feat: Deploy release health rust consumer (#5500) by @lynnagara
• ref: Add rust-errors-consumer to deploys, remove a bunch of already-removed deployments (#5478) by @untitaker
• fix(cardinality): Fix the query run for cardinality (#5498) by @nikhars
• ref(ddm): Disable metrics in Snuba (#5497) by @evanh
• fix: Use the working link to the MQL repository (#5495) by @olksdr
• Take 2: Port PolymorphicMetricsProcessor to Rust (#5492) by @ayirr7
• Revert "Revert "Revert "feat(rust): Port PolymorphicMetricsProcessor to Rust (#5419)""" (#5491) by @john-z-yang
• Revert "Revert "feat(rust): Port PolymorphicMetricsProcessor to Rust (#5419)"" (7446c1a) by @john-z-yang
• Revert "feat(rust): Port PolymorphicMetricsProcessor to Rust (#5419)" (871e509) by @getsentry-bot
• feat(rust): Port PolymorphicMetricsProcessor to Rust (#5419) by @ayirr7
• fix(ddm): Fix config bug (#5488) by @evanh

Plus 28 more

24.1.1

Various fixes & improvements

• fix(inc-602): make migration connection checks readiness-state aware (#5414) by @volokluev
• chore(entity): Remove runtime config for illegal aggregation validator (#5443) by @enochtangg
• fix(gcb): Remove build steps from GCB (#5450) by @untitaker
• build(deps): bump sentry-sdk from 1.26.0 to 1.39.2 (#5366) by @dependabot
• fix: Bump sentry-kafka-schemas in right project (#5451) by @untitaker
• ref(optimize): Use time-machine instead of freezegun (#5447) by @nikhars
• chore(deps): bump sentry-relay from 0.8.39 to 0.8.44 (#5442) by @dependabot
• fix(devserver): Enforce schemas again (#5449) by @untitaker
• fix: Upgrading to Clickhouse 22.8 (#5445) by @evanh
• fix(devserver): Do not enforce schema (#5446) by @untitaker
• fix(cardinality): Add filter to cardinality report (#5439) by @nikhars
• fix(optimize): Add parallel optimize job cutoff time (#5444) by @nikhars
• feat(spans): Add a schema test (#5440) by @phacops
• feat(profiles): Add test to see if the struct matches the schema (#5438) by @phacops
• ref: bump sentry-kafka-schemas to 0.1.46 (#5441) by @getsentry-bot
• feat(on-call): Add illegal aggregate function in conditions entity validator (#5435) by @enochtangg
• ref(docs) Add some MQL documentation to Snuba (#5432) by @evanh
• feat(metrics-summaries): Handle incomplete metrics summaries (#5428) by @phacops
• fix config validation for cross org policy (#5426) by @volokluev
• fix(rust): Deal with empty batches correctly (#5433) by @untitaker
• ref(replays): Re-add e2e latency metric [SNS-2606] (#5420) by @untitaker
• fix(rust): Fix broken DLQ produer in Rust (#5431) by @untitaker
• fix(cardinality): Increase time duration for finding all modules (#5430) by @nikhars
• feat(gcs): Allow snuba to write to gcs for tooling (#5410) by @nikhars

Plus 32 more