A load balancer is a system (hardware or software) that distributes incoming network traffic (or workload) across multiple servers (or resources) to ensure no single server becomes overwhelmed, which improves responsiveness and availability of an application or website.

Here’s a simple breakdown:

Example scenario

You have a website with heavy traffic:

Types of load balancing

Type Description
Layer 4 (Transport) Balances based on IP address, TCP/UDP port (low-level networking).
Layer 7 (Application) Balances based on content of requests (like HTTP headers, URLs, cookies). More advanced.

Algorithms used by load balancers (to decide where to send traffic)

Example load balancers:

First: Does Apigee use a load balancer?

Yes — but it depends on which Apigee deployment model you’re using:

Apigee Deployment Model Load Balancer Details
Apigee SaaS (Apigee Edge / Apigee X - Google managed) Google Cloud manages all load balancing for you (you don’t see/manage it). It uses Google’s internal global load balancers.
Apigee Hybrid (you manage infra) You manage load balancer (typically an external load balancer like GCP Load Balancer, NGINX, F5, etc.), but Apigee components (like the runtime pods) also have built-in traffic distribution (via Kubernetes services).

Now — Does Apigee have a built-in load balancer?

Not exactly a full-featured load balancer like NGINX or F5 — but:

In simple terms:

Scope How load balancing works
Externally (client to Apigee) You use an external load balancer (managed by Google in SaaS, or your own in Hybrid)
Internally (within Apigee components) Apigee uses internal load distribution (Kubernetes Services, or Apigee Routers distribute traffic to Message Processors)

Quick summary:

Apigee does not provide a full standalone load balancer product like NGINX/F5. It internally distributes load between components but for exposing APIs to clients, an external load balancer is used — either managed by Google (SaaS) or managed by you (Hybrid).

Short answer

An API Gateway and a Load Balancer are two different things, but:

Some tools (like NGINX) can act as both — depending on how you configure them.

Let’s clear this up.

API Gateway vs Load Balancer — Concept

  API Gateway Load Balancer
Main job Manage APIs (route to services, apply policies like auth, rate limit, etc.) Distribute traffic across multiple backend servers to spread load
Layer Application Layer (Layer 7) Can be Layer 4 (TCP) or Layer 7 (HTTP)
Examples Apigee, Kong, AWS API Gateway, NGINX (configured as gateway) NGINX (configured as LB), HAProxy, AWS ELB, Google Cloud LB
Knows about APIs? Yes — works with API paths, methods, etc. No — just balances traffic, unaware of API methods or policies

Does an API Gateway internally use a load balancer?

Not exactly. But — under the hood, an API Gateway does distribute requests to backend services — this behavior is similar to a load balancer, but with API-awareness.

You can think of it like:

API Gateway = Smart load balancer + API management features

Now let’s take NGINX as an example (since you asked)

Use Case How NGINX behaves
As Load Balancer only You configure NGINX to distribute HTTP/TCP traffic across backend servers → classic load balancing
As API Gateway You configure NGINX to do API routing, apply JWT validation, rate limiting, etc., and ALSO distribute to backends → behaves like API Gateway + LB

So:

NGINX can do both — it depends on how you configure it.

Real-world usage pattern (typical architecture)

Client
   |
   v
[ External Load Balancer (Optional - Google/AWS LB) ]
   |
   v
[ API Gateway (like Apigee or NGINX configured as gateway) ]
   |
   v
[ Backend microservices (Service A, B, C...) ]

Sometimes, companies even combine them:

External LB → routes to multiple API Gateway instances → which route to backends.

Interview-friendly answer to your question

“An API Gateway doesn’t technically use a load balancer internally, but it does distribute traffic across backend services in a similar way — with extra API management features like security and rate limiting. Tools like NGINX can act as both, depending on how they’re configured.”

API Gateway decides which service to call (it does API-level routing).

So then…

Why do we need an External Load Balancer in front of the API Gateway?

The external load balancer’s job is not to route APIs to microservices

→ Its job is to distribute incoming client traffic to multiple instances of the API Gateway itself.

Here’s why — imagine this situation:

You’re running a big application → Thousands of users are calling APIs at the same time → 1 API Gateway instance (1 server) can get overloaded easily → You deploy multiple API Gateway instances (on multiple servers or containers)

Example:

Client
   |
   v
External Load Balancer (GCP LB / AWS ELB / NGINX LB)
   |-----------------------|
   |                       |
API Gateway #1       API Gateway #2
   |                       |
   v                       v
Backend services   Backend services

In short

External LB = balances load across multiple API Gateway instances API Gateway = handles API-level routing and policies

Interview-friendly explanation you can say

“An external load balancer sits in front of multiple API Gateway instances to distribute client traffic evenly across them, ensuring high availability and scalability. The API Gateway then handles API-level routing and policies to backend services. The LB’s role is to keep the API Gateways healthy and balanced — not to route to microservices directly.”

Why is this important in real systems?

Bottom line

The external load balancer helps scale and protect the API Gateway itself. The API Gateway helps route and secure the backend APIs.

The full clean sequence

Client → External LB → API Gateway → Backend Services → Response to Client

Step by step:

  1. Client sends a request (API call)

  2. External Load Balancer → Distributes the request across multiple API Gateway instances → Its job: spread load, improve availability (nothing to do with microservices yet)

  3. API Gateway (like Apigee) → Does API-level responsibilities:

    • Authentication (JWT/API key validation)
    • Rate limiting
    • Routing: Decides which backend service to call
    • Request/response transformations (optional)

  4. API Gateway forwards the request to the correct backend service

    • Now here’s the key → If the backend service itself is deployed in multiple instances (for scaling), API Gateway can:

      • Either call the service registry (like Kubernetes service, or Consul, or an internal LB)
      • Or directly call an internal load balancer (optional — depends on infra)

  5. Backend service processes the request and returns the response

  6. API Gateway receives the response → applies optional policies (like response transformation, logging)

  7. API Gateway sends response back to the client (via external LB → but LB doesn’t touch response, just forwards)

Important clarification (because you wrote this)

Internally API Gateway will use another load balancer to equally distribute the load → Yes and No (it depends)

Clean, crisp interview answer (memorize this)

“The external load balancer distributes client requests across API Gateway instances to ensure high availability and scalability. The API Gateway then handles API-level concerns like authentication and routing. If backend services have multiple replicas, the API Gateway either directly balances across them, or forwards requests via a service discovery mechanism or internal load balancer. The response flows back through the same path to the client.”

Final architecture (modern cloud)

Client
   |
[ External LB (Cloud LB) ]
   |
[ API Gateway (multiple instances) ]
   |
[ Kubernetes Service / Service Mesh / Internal LB ]
   |
[ Backend services (multiple replicas) ]

Bottom line:

You don’t always need an explicit “internal LB” between API Gateway and services — because things like Kubernetes Service or Service Mesh can already balance traffic across backend replicas.

1. Apigee + Kubernetes (Modern Cloud Native Architecture)

Client
   |
[ External Load Balancer (GCP Cloud Load Balancer) ]
   |
[ Apigee API Gateway (multiple Apigee runtime instances) ]
   |
[ Kubernetes Service (Service discovery + Internal load balancing) ]
   |
[ Backend Microservices (Service A, B, C — each has multiple pods) ]

Step-by-step flow

  1. Client sends API request
  2. External LB distributes it to one of the Apigee runtime instances

  3. Apigee does:

    • Authentication (OAuth, API key)
    • Rate limiting
    • Routing (decides which microservice to call)
  4. Apigee forwards request to Kubernetes Service (for that microservice)
  5. Kubernetes Service load-balances request to one of the microservice’s pods
  6. Microservice processes request → sends response back
  7. Response flows back through Apigee → External LB → Client

Why Kubernetes here?

2. NGINX API Gateway + No Kubernetes (Traditional VMs/Servers)

Client
   |
[ External Load Balancer (NGINX or AWS ELB) ]
   |
[ NGINX API Gateway (multiple NGINX servers) ]
   |
[ Internal Load Balancer (NGINX configured as internal LB OR AWS ALB) ]
   |
[ Backend Services (deployed on VMs — multiple replicas per service) ]

Step-by-step flow

  1. Client sends API request
  2. External LB distributes it to one of the NGINX API Gateway servers

  3. NGINX API Gateway does:

    • Authentication (JWT validation)
    • Rate limiting, request transformation
    • Routing: Decides which backend service to call
  4. NGINX Gateway forwards request to Internal LB for that service
  5. Internal LB distributes request to one of the backend service replicas (VMs or containers)
  6. Backend Service processes request → sends response back
  7. Response flows back through NGINX Gateway → External LB → Client

Why Internal LB here?

Super clean takeaway (interview gold)

With Kubernetes Without Kubernetes
K8s Service handles load balancing to pods Internal LB explicitly balances across replicas
Apigee integrates with K8s Service NGINX integrates with Internal LB
Less infra setup More manual setup

→ “How does Kubernetes Service distribute requests to backend Pods? Is it equal? Round robin? Random?”

Short answer

Kubernetes Service uses kube-proxy and iptables (or IPVS) to distribute traffic to Pods in a randomized round-robin fashion — roughly equal distribution.

Step-by-step process — super clean

Let’s say:

Service A (Kubernetes Service)
 ↳ Pod 1 (10.0.0.1)
 ↳ Pod 2 (10.0.0.2)
 ↳ Pod 3 (10.0.0.3)

  1. API Gateway / Client sends request to Service A (ClusterIP)

  2. Kube-proxy (running on every node) intercepts requests going to Service A

  3. kube-proxy has programmed iptables rules (or IPVS) to map:

    Service IP → Pod 1 OR Pod 2 OR Pod 3

  4. The load balancing strategy used is randomized round robin

    • For each new connection/request, kube-proxy randomly selects a backend Pod from the list
    • Over time, this results in roughly equal distribution of requests across Pods
    • It is NOT strict round robin — randomness makes it more resilient

  5. The selected Pod processes the request and sends back the response

Key facts to say in interview

Bonus (Advanced understanding)

apiVersion: v1
kind: Service
spec:
  sessionAffinity: None # or ClientIP

Clean interview answer

“Kubernetes Service distributes requests to backend Pods using kube-proxy, which programs iptables (or IPVS) rules. The default strategy is randomized round robin — requests are randomly but roughly evenly spread across healthy Pods. Kubernetes automatically excludes unhealthy Pods and updates the rules dynamically. For higher performance, IPVS mode can be used. Session affinity is optional.”

Bottom line

Yes — K8s Service distributes load roughly equally (randomized round robin) to healthy Pods automatically. You don’t have to configure an explicit load balancer — it’s built into kube-proxy.

NGINX load balancing — Do we have to configure it?

Yes, in NGINX you have to explicitly configure how requests should be distributed to backend servers.

NGINX does not magically discover services like Kubernetes does. You have to tell it:

Step-by-step: How NGINX load balances

Let’s say:

You have 3 backend servers:
Backend 1 — 10.0.0.1
Backend 2 — 10.0.0.2
Backend 3 — 10.0.0.3

Your NGINX config (nginx.conf)

http {
  upstream backend_servers {
    server 10.0.0.1;
    server 10.0.0.2;
    server 10.0.0.3;
  }

  server {
    listen 80;

    location / {
      proxy_pass http://backend_servers;
    }
  }
}

This says:

“NGINX, here are 3 servers — distribute requests among them”

**By default — NGINX uses round robin

You can customize the load balancing algorithm

NGINX supports multiple strategies:

Strategy Config When to use?
Round robin (default) Just list servers When all backends are equally powerful
Least connections least_conn; When backends handle long requests (like DB)
IP Hash (sticky) ip_hash; When session stickiness is needed
Weight-based server 10.0.0.1 weight=3; When servers have different capacity

Example — Least connections

upstream backend_servers {
  least_conn;
  server 10.0.0.1;
  server 10.0.0.2;
  server 10.0.0.3;
}

Key difference vs Kubernetes

Kubernetes Service NGINX
Automatically discovers Pods You manually define servers
Automatically balances (random RR) You configure algorithm explicitly
Auto removes dead Pods You need health checks + max_fails manually
Infra-level load balancing App-level reverse proxy and balancing

Clean interview answer

“In NGINX, we have to explicitly configure backend servers and choose the load balancing strategy in the config file. By default, NGINX uses round robin but supports least connections, IP hash (sticky sessions), and weighted balancing. Unlike Kubernetes, NGINX does not auto-discover services or auto-exclude dead servers — we configure health checks and servers manually.”

Bottom line

Yes — in NGINX you MUST configure backend servers and load balancing strategy yourself Kubernetes does it automatically for you via kube-proxy and Service.

First — What is Service Discovery? (Why do we even need it?)

When you build microservices, the number of services and their instances (replicas) can dynamically change:

This means:

We can’t hardcode IP addresses of services because they are dynamic.

So we need a Service Discovery system — A central registry where all services register themselves and other services can discover them dynamically.

Zuul + Eureka (Spring Boot world)

In traditional Spring Boot microservices (without Kubernetes), we use:

Picture

Client
   |
[ Zuul API Gateway ]
   |
( Zuul asks Eureka — "Where is Service A?" )
   |
[ Eureka Registry ]
   ↳ Service A — 10.0.0.1
   ↳ Service A — 10.0.0.2
   ↳ Service B — 10.0.0.3

BUT… when you use Kubernetes → You DON’T NEED Eureka or Zuul

Because Kubernetes already provides built-in service discovery + load balancing via Kubernetes Service + DNS.

Here’s how:

So:

Eureka + Zuul → Not needed because K8s Service + DNS + Ingress / API Gateway replace them

(Apigee + Kubernetes)

Old (Spring Boot with Zuul/Eureka) New (K8s + Apigee)
Zuul (API Gateway) Apigee (API Gateway)
Eureka (Service discovery registry) Kubernetes DNS + K8s Service (built-in discovery + balancing)
Manual config, brittle Auto-discovery, resilient, scalable

Clean interview answer (Gold)

“In traditional microservices, we use Eureka (service registry) and Zuul (API Gateway) to handle service discovery and dynamic routing. Each service registers itself with Eureka, and Zuul queries Eureka to route requests dynamically. However, when using Kubernetes, we don’t need Eureka or Zuul because Kubernetes provides built-in service discovery via Services and DNS. API Gateways like Apigee or NGINX can directly route to Kubernetes services, and Kubernetes handles internal load balancing and discovery automatically.”

Bottom line

Short answer: → Kubernetes + API Gateway (like Apigee / NGINX / Kong / Istio Gateway) is what modern systems use today. Zuul + Eureka is legacy now (only in old Spring Boot apps without K8s).

Current industry trend (2025)

Architecture Usage today? Why?
Spring Boot + Eureka + Zuul Rare / Legacy Used when companies don’t have Kubernetes or are stuck on old systems
Kubernetes + Apigee / Istio / NGINX / Kong Very common (modern) Built-in discovery + auto-scaling + cloud native ready
Kubernetes + Service Mesh (Istio) Growing fast (advanced setups) For fine-grained routing, mTLS, advanced traffic control

Why is Eureka + Zuul fading away?

Clean answer to say in interview

“In modern microservices architectures, Kubernetes has replaced the need for tools like Eureka and Zuul by providing built-in service discovery and load balancing through its Service abstraction and DNS. Today, we commonly use Kubernetes along with cloud-native API Gateways like Apigee, Istio Gateway, or Kong to handle external traffic, authentication, and routing.”

If you’re preparing for product-based companies

→ Focus on K8s + API Gateway knowledge (not Zuul/Eureka)