What would you like to be added:
Right now, we need to apply the envoy ai gateway manually following the https://github.com/InftyAI/llmaz/tree/main/docs/examples/envoy-ai-gateway, however, we need to create all these stuff automatically by introducing a controller.
Also the configurations are static, once the service is down, the ai gateway is unaware of this, this is not right.
Why is this needed:
- Easy to setup
- Dynamically change the envoy configurations based on the status of the services.
Completion requirements:
This enhancement requires the following artifacts:
The artifacts should be linked in subsequent comments.
What would you like to be added:
Right now, we need to apply the envoy ai gateway manually following the https://github.com/InftyAI/llmaz/tree/main/docs/examples/envoy-ai-gateway, however, we need to create all these stuff automatically by introducing a controller.
Also the configurations are static, once the service is down, the ai gateway is unaware of this, this is not right.
Why is this needed:
Completion requirements:
This enhancement requires the following artifacts:
The artifacts should be linked in subsequent comments.