Web API architecture
AI Hub Web API is a scalable, low-latency, process execution framework, which consists of multiple components inside the AI Hub stack and offers execution of processes via REST:
- Web API Agents,
- Web API Gateway,
- AI Hub Server / Web API Registry.
Web API Agents
Web API Agents are the execution components of the Web API Architecture. They store Web API Deployments, the managed counterpart to the Real Time Scoring Agent deployments. Multiple Web API Agents form a redundant Web API Group, which share all permissions and Web API Deployments, meaning fail safety can be guaranteed if more than one Web API Agent is inside a Web API Group. When a Web API Agent is no longer able to serve requests anymore, e.g. the VM did shut down, all requests are routed to other Web API Agents in that Web API Group.
Guarantees
The following guarantees can only be given when the Web API Group has more than one Web API Agent. One Web API Agent can always serve requests when:
- A Web API Deployment of the Web API Group is updated
- A new extension has been uploaded with the web interface
Its important to note, that when an error-nous Web API Deployment was tried to be deployed, the Web API Agent on which the Web API Deployment was tried to be deployed on will stay Out Of Service untill the error in the Web API Deployment has been fixed. Until the error has been fixed, all other Web API Agents and Web API Deployments are in an unknown state, though in most cases, they are still working fine.
Web API Gateway
The Web API Gateway is the central entry point for requests of any invocation of a Web API endpoint. When a request is sent to the Web API Gateway, the load balancer chooses the next Web API Agent to handle this request. More information about the Web API Gateway can be found on the Web API Gateway overview.
Web API Registry
The Web API Registry is the central place for keeping information about Web API Agents. AI Hub Server orchestrates the distribution of the Web API Deployments, making sure that, when more than one Web API Agent is connected to a Web API Group, always one Web API Agent is up and can serve requests, even if maintenance work, like updating Web API Deployments or storing new extensions, is done.
Deployment overview
These three components can be deployed inside one Docker network, forming a central architecture with a proxy:
When external Web API Agents should be used, inside the Web API Agents docker container a proxy can be installed
