1. What LiteLLM Is LiteLLM is an open-source LLM adapter: call 100+ APIs (OpenAI, Anthropic, Gemini, Bedrock, Azure, Ollama, vLLM, Qwen, and more) using the standard OpenAI format. Two ways to use it: Python SDK — called directly from your code AI Gateway — a team-level proxy with authentication, billing,

1. Common Object Configuration 1.1 Gateway selector Selects the Envoy on which the rules take effect servers The domains to match Port Protocol TLS certificate 1.2 VirtualService gateways Specifies the gateway on which this takes effect; the default value mesh is for east-west traffic, while specifying a Gateway object makes it north-south traffic http Layer 7 routing Redirect Rewrite Retry Condition rules Timeout Fault injection CORS policy tcp Layer 7 routing tls Routing with certificates TLS certificate 1.

1. About Traffic Distribution Traffic governance is divided into north-south and east-west. In the typical drawing convention for Network Diagrams, core network components are drawn at the top and clients at the bottom, with different service levels drawn in between — hence the names north-south and east-west traffic. North-south traffic refers to traffic initiated by clients, while east-west traffic refers to traffic between services.

1. Comparing a Few Common Gateways Nginx, a reverse proxy with a modular design, written in C OpenResty, a web development platform built around Nginx that can parse and execute Lua scripts Kong, an application on top of OpenResty, an API gateway with API management and request proxying, using PostgreSQL for storage APISIX, which replaces Kong’s PostgreSQL with Etcd and is built on Nginx’s core libraries APISIX’s advantage lies in the API management and extensibility it provides, so the gateway no longer merely forwards traffic to services but can be configured and customized.