A powerful mini-implementation of a search engine built with Java and Spring Boot that uses web crawling, the Depth-First Search (DFS) algorithm, Redis, Kafka, and Elasticsearch to extract and store relevant web addresses.
- Web Crawler: Utilizes a web crawler to fetch web pages and extract relevant addresses.
- Depth-First Search (DFS): Implements the DFS algorithm for efficient traversal of web pages.
- Redis Integration: Stores extracted web addresses using Redis for fast and scalable data storage.
- Kafka: Uses Kafka for task distribution, improving the efficiency of the system.
- Elasticsearch: Employs Elasticsearch for indexing and searching web addresses