Update Flyte components (#5330)

* Update Flyte Components

Signed-off-by: Flyte-Bot <[email protected]>

* Bump version and add changelog

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Flyte-Bot <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: eapolinario <[email protected]>

CHANGELOG/CHANGELOG-v1.12.0.md

@@ -0,0 +1,56 @@
+# Flyte 1.12.0 Release Notes
+
+Flyte 1.12.0 brings a host of new features, optimizations, and fixes, enhancing the platform's functionality and user experience. This release also welcomes several new contributors to the Flyte community. Below are the highlights of this release.
+
+## 🚀 New Features & Improvements
+
+1. **Admin & Core Enhancements**  
+   - [Implemented the `GetProject` endpoint](https://github.com/flyteorg/flyte/pull/4825) in FlyteAdmin.
+   - [Added tracking for active node and task execution counts](https://github.com/flyteorg/flyte/pull/4986) in Propeller.
+   - [Set FlyteAdmin gRPC port correctly in config](https://github.com/flyteorg/flyte/pull/5013) and included [separate gRPC Ingress flag](https://github.com/flyteorg/flyte/pull/4946).
+
+2. **Helm Charts & Manifests**  
+   - [CI workflow enhancements](https://github.com/flyteorg/flyte/pull/5027) for Helm charts and manifests.
+   - [Fixed syntax issues](https://github.com/flyteorg/flyte/pull/5056) and [rendering errors](https://github.com/flyteorg/flyte/pull/5048) in Helm charts.
+   - [Updated Spark-on-K8s-operator address in Helm charts](https://github.com/flyteorg/flyte/pull/5198).
+
+3. **Documentation Improvements**  
+   - Enhanced documentation across various modules, including [a guide for running the newest Flyteconsole in Flyte sandbox](https://github.com/flyteorg/flyte/pull/5100) and [a troubleshooting guide for Docker errors](https://github.com/flyteorg/flyte/pull/4972).
+   - [Updated Propeller architecture documentation](https://github.com/flyteorg/flyte/pull/5117) and [added a guide for enabling/disabling local caching](https://github.com/flyteorg/flyte/pull/5242).
+
+4. **Performance & Bug Fixes**  
+   - [Improved lint error detection](https://github.com/flyteorg/flyte/pull/5072), fixing multiple issues across components.
+   - [Optimized Flyte components](https://github.com/flyteorg/flyte/pull/5097), including Golang, Protobuf, and GRPC versions.
+   - Fixed several bugs related to execution phases, [GPU resource overrides](https://github.com/flyteorg/flyte/pull/4925), and [Databricks errors](https://github.com/flyteorg/flyte/pull/5226).
+
+## 🔧 Housekeeping & Deprecations
+
+1. **Deprecated Configuration**  
+   - [Deprecated `MaxDatasetSizeBytes` propeller config](https://github.com/flyteorg/flyte/pull/4852) in favor of `GetLimitMegabytes` storage config.
+   - [Removed obsolete Flyte config files](https://github.com/flyteorg/flyte/pull/5196).
+
+2. **Miscellaneous**  
+   - [Simplified boilerplate](https://github.com/flyteorg/flyte/pull/5134).
+   - [Regenerated Ray pflags](https://github.com/flyteorg/flyte/pull/5149).
+   - Upgraded various dependencies, including [cloudevents](https://github.com/flyteorg/flyte/pull/5142), [logrus](https://github.com/flyteorg/flyte/pull/5139), and [go-restful](https://github.com/flyteorg/flyte/pull/5140).
+
+## 🆕 New Contributors
+
+- **@RRap0so** for [implementing the `GetProject` endpoint](https://github.com/flyteorg/flyte/pull/4825).
+- **@cjidboon94** for [adding the GKE starter values file](https://github.com/flyteorg/flyte/pull/5026).
+- **@ddl-rliu** for [improving audience mismatch debugging](https://github.com/flyteorg/flyte/pull/5078).
+- **@pbrogan12** for [fixing Helm chart rendering](https://github.com/flyteorg/flyte/pull/5048).
+- **@austin362667** for [showing diff structure when re-registering tasks](https://github.com/flyteorg/flyte/pull/4924).
+- **@ssen85** for [updating environment setup documentation](https://github.com/flyteorg/flyte/pull/4963).
+- **@dansola** for [changing retry error types](https://github.com/flyteorg/flyte/pull/5128).
+- **@noahjax** for [adding identity to task execution metadata](https://github.com/flyteorg/flyte/pull/5105).
+- **@ongkong** for [fixing ID bigint conversion issues](https://github.com/flyteorg/flyte/pull/5157).
+- **@sshardool** for [tracking active node and task execution counts](https://github.com/flyteorg/flyte/pull/4986).
+- **@Jeinhaus** for [adding a missing key in the auth guide](https://github.com/flyteorg/flyte/pull/5169).
+- **@yini7777** for [fixing secret mounting issues](https://github.com/flyteorg/flyte/pull/5063).
+- **@Sovietaced** for [fixing grammatical errors in the documentation](https://github.com/flyteorg/flyte/pull/5227).
+- **@agiron123** for [configuring the RunLLM widget](https://github.com/flyteorg/flyte/pull/5266).
+- **@mark-thm** for [removing the upper bound on FlyteIDL's protobuf dependency](https://github.com/flyteorg/flyte/pull/5285).
+
+**[Full Changelog](https://github.com/flyteorg/flyte/compare/v1.11.0...v1.12.0)**
+

charts/flyte-binary/README.md

@@ -42,7 +42,7 @@ Chart for basic single Flyte executable deployment
 | configuration.auth.oidc.clientId | string | `""` |  |
 | configuration.auth.oidc.clientSecret | string | `""` |  |
 | configuration.co-pilot.image.repository | string | `"cr.flyte.org/flyteorg/flytecopilot"` |  |
-| configuration.co-pilot.image.tag | string | `"v1.12.0-b0"` |  |
+| configuration.co-pilot.image.tag | string | `"v1.12.0"` |  |
 | configuration.database.dbname | string | `"flyte"` |  |
 | configuration.database.host | string | `"127.0.0.1"` |  |
 | configuration.database.options | string | `"sslmode=disable"` |  |

charts/flyte-binary/values.yaml

@@ -159,7 +159,7 @@ configuration:
       # repository CoPilot sidecar image repository
       repository: cr.flyte.org/flyteorg/flytecopilot  # FLYTECOPILOT_IMAGE
       # tag CoPilot sidecar image tag
-      tag: v1.12.0-b0  # FLYTECOPILOT_TAG
+      tag: v1.12.0  # FLYTECOPILOT_TAG
   # agentService Flyte Agent configuration
   agentService:
     defaultAgent:

charts/flyte-core/README.md

@@ -95,8 +95,8 @@ helm install gateway bitnami/contour -n flyte
 | configmap.clusters.clusterConfigs | list | `[]` |  |
 | configmap.clusters.labelClusterMap | object | `{}` |  |
 | configmap.console | object | `{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"}` | Configuration for Flyte console UI |
-| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.12.0-b0","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration |
-| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.12.0-b0","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/[email protected]/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) |
+| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.12.0","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration |
+| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.12.0","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/[email protected]/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) |
 | configmap.core | object | `{"manager":{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"},"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}}` | Core propeller configuration |
 | configmap.core.manager | object | `{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/manager/config#Config). |
 | configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). |
@@ -130,7 +130,7 @@ helm install gateway bitnami/contour -n flyte
 | datacatalog.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
 | datacatalog.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
 | datacatalog.image.repository | string | `"cr.flyte.org/flyteorg/datacatalog"` | Docker image for Datacatalog deployment |
-| datacatalog.image.tag | string | `"v1.12.0-b0"` | Docker image tag |
+| datacatalog.image.tag | string | `"v1.12.0"` | Docker image tag |
 | datacatalog.nodeSelector | object | `{}` | nodeSelector for Datacatalog deployment |
 | datacatalog.podAnnotations | object | `{}` | Annotations for Datacatalog pods |
 | datacatalog.podEnv | object | `{}` | Additional Datacatalog container environment variables |
@@ -166,7 +166,7 @@ helm install gateway bitnami/contour -n flyte
 | flyteadmin.extraArgs | object | `{}` | Appends extra command line arguments to the serve command |
 | flyteadmin.image.pullPolicy | string | `"IfNotPresent"` |  |
 | flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment |
-| flyteadmin.image.tag | string | `"v1.12.0-b0"` |  |
+| flyteadmin.image.tag | string | `"v1.12.0"` |  |
 | flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create |
 | flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment |
 | flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods |
@@ -228,7 +228,7 @@ helm install gateway bitnami/contour -n flyte
 | flytepropeller.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
 | flytepropeller.image.pullPolicy | string | `"IfNotPresent"` |  |
 | flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment |
-| flytepropeller.image.tag | string | `"v1.12.0-b0"` |  |
+| flytepropeller.image.tag | string | `"v1.12.0"` |  |
 | flytepropeller.manager | bool | `false` |  |
 | flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment |
 | flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods |
@@ -259,7 +259,7 @@ helm install gateway bitnami/contour -n flyte
 | flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files |
 | flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
 | flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment |
-| flytescheduler.image.tag | string | `"v1.12.0-b0"` | Docker image tag |
+| flytescheduler.image.tag | string | `"v1.12.0"` | Docker image tag |
 | flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment |
 | flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods |
 | flytescheduler.podEnv | object | `{}` | Additional Flytescheduler container environment variables |

charts/flyte-core/values.yaml

@@ -16,7 +16,7 @@ flyteadmin:
   image:
     # -- Docker image for Flyteadmin deployment
     repository: cr.flyte.org/flyteorg/flyteadmin  # FLYTEADMIN_IMAGE
-    tag: v1.12.0-b0  # FLYTEADMIN_TAG
+    tag: v1.12.0  # FLYTEADMIN_TAG
     pullPolicy: IfNotPresent
   # -- Additional flyteadmin container environment variables
   #
@@ -144,7 +144,7 @@ flytescheduler:
     # -- Docker image for Flytescheduler deployment
     repository: cr.flyte.org/flyteorg/flytescheduler  # FLYTESCHEDULER_IMAGE
     # -- Docker image tag
-    tag: v1.12.0-b0  # FLYTESCHEDULER_TAG
+    tag: v1.12.0  # FLYTESCHEDULER_TAG
     # -- Docker image pull policy
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Flytescheduler deployment
@@ -210,7 +210,7 @@ datacatalog:
     # -- Docker image for Datacatalog deployment
     repository: cr.flyte.org/flyteorg/datacatalog  # DATACATALOG_IMAGE
     # -- Docker image tag
-    tag: v1.12.0-b0  # DATACATALOG_TAG
+    tag: v1.12.0  # DATACATALOG_TAG
     # -- Docker image pull policy
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Datacatalog deployment
@@ -300,7 +300,7 @@ flytepropeller:
   image:
     # -- Docker image for Flytepropeller deployment
     repository: cr.flyte.org/flyteorg/flytepropeller  # FLYTEPROPELLER_IMAGE
-    tag: v1.12.0-b0  # FLYTEPROPELLER_TAG
+    tag: v1.12.0  # FLYTEPROPELLER_TAG
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Flytepropeller deployment
   resources:
@@ -743,7 +743,7 @@ configmap:
         # -- Structure documented [here](https://pkg.go.dev/github.com/lyft/[email protected]/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig)
         co-pilot:
           name: flyte-copilot-
-          image: cr.flyte.org/flyteorg/flytecopilot:v1.12.0-b0  # FLYTECOPILOT_IMAGE
+          image: cr.flyte.org/flyteorg/flytecopilot:v1.12.0  # FLYTECOPILOT_IMAGE
           start-timeout: 30s
 
   # -- Core propeller configuration