1. What is Elasticsearch?

• Elasticsearch is a distributed search and analytics engine.
• Built on Apache Lucene.
• It allows you to store, search, and analyze huge volumes of data in near real-time.

👉 Think of it as a super-fast search engine you can use in your apps (like Google Search, but for your own data).

---

2. Why do we use Elasticsearch?

• Normal databases (SQL/NoSQL) are not optimized for text search (e.g., find “laptop” in millions of product descriptions).

• Elasticsearch is made for:

  • Full-text search (supports relevance, ranking, fuzzy search, autocomplete).
  • Analytics (aggregations, dashboards).
  • Speed → near real-time search (latency in ms).
  • Scalability → can handle billions of records across many servers.

👉 Use cases: E-commerce product search, log analytics, monitoring (ELK stack), recommendation systems.

---

3. How is data pushed into Elasticsearch?

There are 2 main ways:

1. From Backend (API → ES directly):

   • Your application can write data directly into Elasticsearch using its REST APIs.
   • Example: When a new product is added in your e-commerce app → backend pushes product data into ES.

2. From Database (via ETL or sync tools):

   • Data is stored in a relational/NoSQL DB → then synced into ES for fast searching.

   • Tools used:

     • Logstash (part of ELK stack).
     • Beats (lightweight shippers).
     • Kafka connectors (real-time pipelines).
     • Custom batch jobs (cron jobs to push data).

👉 Typically: DB = source of truth, Elasticsearch = for search/querying.

---

4. How is Elasticsearch so fast?

Key reasons:

• Inverted Index: Instead of storing documents normally, it stores an index of words → like a dictionary (word → list of docs containing it).

  • Example: Search “laptop” → ES directly knows which docs contain “laptop”.
• Distributed Architecture: Data split into shards across nodes. Searches happen in parallel.
• Caching: Uses caching for repeated queries.
• Lucene engine: Highly optimized for text search and scoring relevance.

👉 That’s why queries return in milliseconds, even with millions of records.

---

5. How does Elasticsearch stay updated with latest data?

• Near Real-time updates: When you push new data, ES indexes it almost instantly.

• If using DB + ES:

  • Tools like Logstash/Kafka keep syncing changes from DB → ES.

• If backend pushes data directly:

  • Whenever data changes (create/update/delete), your app calls ES API to keep index updated.

👉 Elasticsearch is not always the “system of record” → DB is the source of truth, ES is optimized for search.

---

6. What happens if Elasticsearch is down?

• If ES is your only data store → you lose both search + data (bad design).
• If ES is used only for search (common) → your app cannot perform searches, but core system still works (because DB is intact).

👉 Downtime impact = search features fail (e.g., product search, logs, analytics dashboards stop working).

---

7. How to handle Elasticsearch downtime (precautions)?

• Cluster setup:

  • Use multiple nodes with replication → if one node is down, others still serve queries.
  • Use replica shards for high availability.

• Failover strategy:

  • If ES is down, fallback to DB search (slower but still works).
  • Or display limited cached results.

• Monitoring & Alerts:

  • Monitor ES cluster health with tools (Kibana, Grafana, AWS OpenSearch monitoring).

• Backups:

  • Regularly snapshot data to S3 or other storage for disaster recovery.

👉 In production: Always deploy Elasticsearch as a cluster with replicas, never as a single node.

---

✅ Summary

• Elasticsearch = distributed, fast, full-text search engine.
• Why: DBs are not good at full-text search → ES is built for speed, relevance, and scale.
• Data push: From backend APIs or synced from DB (via Logstash/Kafka).
• Speed: Uses inverted index, distributed architecture, caching.
• Updates: Data synced in near real-time.
• Downtime: Search features break → mitigate using replicas, failover, monitoring.

---