Simplify Image tag caching

[tarunwadhwa13]

First of all, thanks for this great product. I really like the vision and would like to contribute to make it even more awesome !!!!

I wanted to propose the idea of simplifying image tag to variant mapping.
Current design requires pulling all tags, matching if the tag name is present in s3 config and then saving in DB

Proposal -

1. Fetch the config from S3
2. Attempt pulling from docker hub directly
3. If pull successful, inspect the image and get all details and save in DB

Benefits -

1. No need for getting all tag lists from dockerhub (saves API call and processing)
2. We can make sure the image exists (any pull failures can be detected earlier)
3. Reduced need for caching all image tags info in redis

I can raise a PR to make this change if everyone else. Or in case there was a reason for this flow which I missed or mis-understood

Code ref - https://github.com/Agenta-AI/agenta/blob/main/agenta-backend/agenta_backend/services/templates_manager.py#L39

[jp-agenta]

Hi @tarunwadhwa13,

Thank you for your interest and energy !

We looked into your suggestion and we do think that it would be great to make the change. However, as part of our roadmap, we are planning to make several changes, in the next few weeks, that might heavily impact many sections of the current codebase, including the ones that would be affected by your proposal.

I would suggest that you keep an eye on our changes and that we discuss this again in a few weeks in case your proposal is still applicable. What do you think ?

Thank you again and don't hesitate to suggest more improvements !

Cheers,