Jobs AI Search

AI-powered job search platform with real-time preference filtering.

Project Overview

This proof-of-concept application helps users find relevant job listings through an intelligent search interface:

• The Worker scrapes job listings from LinkedIn using Selenium and BeautifulSoup
• Job descriptions and titles are embedded using transformers and stored in PostgreSQL with pg_vector
• The NLU component uses a RASA DIET model trained on thousands of annotated job listings to extract structured information from user queries
• The API leverages the NLU model to process search queries and builds hybrid search (semantic + keyword) to query the database
• The Frontend provides a real-time, interactive search experience with preference bubbles

User Experience

Users can search for jobs by typing natural language queries into the search field. The system:

1. Dynamically extracts job preferences as the user types (skill bubbles, location, work type, etc.)
2. Updates search results in real-time based on these preferences
3. Presents semantically relevant job listings that match the user's requirements

Preview

Project Components

This mono-repo contains four main components:

• API: Backend FastAPI service for job search and NLU processing
• Worker: Job scraper and data processor
• Frontend: Svelte-based user interface
• NLU: RASA-based natural language understanding service

Technology Stack

• Frontend: Svelte
• Backend API:
  • Python FastAPI
  • Pydantic for data validation
  • AsyncPG for fast database interactions
• Worker:
  • Selenium, BeautifulSoup
  • Celery for task queue management
  • Transformers for text embedding
• NLU: RASA (DIET model)
• Database: PostgreSQL with pg_vector for semantic search

System Architecture

Click to expand

Quick Start

1. Clone the repository:

git clone https://github.com/amitpdev/jobs-ai-search.git
cd jobs-ai-search

2. Start infrastructure services using Docker:

# Run the infrastructure services
docker-compose up -d db redis selenium

3. Choose your preferred setup method for application components:

   • Setup with Docker (recommended for quick start)
   • Setup with Python Environment (recommended for development)

4. See individual component documentation for more details:

   • API Documentation
   • Worker Documentation
   • Frontend Documentation
   • NLU Documentation

Installation Options

Pre-requisites

• Python 3.11 or higher (for Python environment setup)
• Docker and Docker Compose
• Node.js and npm (for frontend development)

Setup with Docker

After starting the infrastructure services, you can run the application components with Docker as well:

# Build and run all application components
docker-compose up -d api worker frontend nlu beat

To shut down all services:

docker-compose down

Setup with Python Environment (for development)

For detailed instructions on setting up Python environments for each component, refer to:

• API Setup
• Worker Setup
• NLU Setup

For the frontend development environment, see:

• Frontend Setup

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contact

Amit Palomo
Email: apalgm@gmail.com

If you have any questions or suggestions about this project, feel free to reach out!