How do I remove/rebuild a Prefect Agent?

[kcpevey]

I've enabled prefect and added the appropriate prefect to the env var. The deployment successfully created a Prefect Kubernetes Agent.

Now I'm transferring ownership over another prefect account so I created a new prefect API key and replaced the appropriate prefect env var for qhub. I redeployed, but my prefect agent is still running under the old account.

Do I need to manually kill the old agent before qhub will regenerate a new one (i.e. in k9s)?

[aktech]

so I created a new prefect API key and replaced the appropriate prefect env var for qhub. I redeployed, but my prefect agent is still running under the old account.

This seems like the right way to do it. Can you try this again to see if this was done properly, one way to check if this was done properly is exec into prefect agent (press "s" in k9s on prefect agent) and type:

export | grep PREFECT__CLOUD__AGENT__AUTH_TOKEN

This will show you the prefect token you set, if this doesn't looks correct, then the first step didn't happen properly.

[brl0]

@kcpevey,

@iameskild and I spent some time battling with Prefect yesterday, although we did not resolve all of the issues we were encountering. Here are some thoughts that may be helpful, but honestly some of this is probably just guess work from trying to stumble through troubleshooting these issues.

Anyway, here is what I think might be happening. The prefect token seems to be stored as a kubernetes secret which is mounted by the prefect agent pod. Since terraform only redeploys cluster resources that are changed, if all you changed is the secret, the stored secret gets changed, but the prefect agent pod may appear to be unaffected. In this can, you might try simply killing that pod, and when the deployment restarts, I think it should pick up on the new token.

If that doesn't do it, you can uninstall the prefect agent using helm and then simply redeploy afterwards. To do this, you would run a command like helm uninstall prefect --namespace dev. When you run the deploy, terraform will see the need to install the helm chart again and the agent will be redeployed.

@aktech, I would appreciate any feedback, of course, if any of this is wrong or a bad idea.

But hopefully this is helpful.

[aktech]

Ah, I see. That makes sense. Did killing the pod not help as you mentioned?

[Adam-D-Lewis]

I'm closing old discussions in a repo cleanup effort. Feel free to re-open if needed.

[kcpevey]

Prefect is no longer integrated into Nebari. Makes sense to close!