Climate Academy RAG Chatbot — Backend

A production-grade Retrieval-Augmented Generation (RAG) API for the Climate Academy student book. Users ask questions and receive answers grounded strictly in the book's content, with inline section citations (§x.y.z).

---

How It Works

1. The HTML book is parsed into sections, chunked, and embedded into a vector database (one-time ingestion)
2. A user sends a question via the frontend or API
3. Flask dispatches the question to a Celery background worker
4. The worker embeds the question, finds the most relevant book passages via ChromaDB, and sends them to AWS Bedrock (Llama 3.1 8B)
5. The LLM generates a grounded answer citing section numbers
6. The result is returned to the client via polling

---

Architecture

Internet
    │  HTTPS :443
    ▼
  Nginx              reverse proxy, SSL, rate limiting
    │  Unix socket
    ▼
  Gunicorn           production WSGI server (4 async workers)
    │
  Flask              REST API — session management, task dispatch
    │
  Celery             async RAG pipeline execution
    │
  ┌─────────────────────────────────────────────┐
  │  Redis       message broker + session store │
  │  ChromaDB    persistent vector database     │
  │  MiniLM      local embedding model          │
  │  Bedrock     managed LLM (Llama 3.1 8B)     │
  └─────────────────────────────────────────────┘

---

Tech Stack

Layer	Technology	Purpose
Web framework	Flask + Gunicorn	REST API server
Async tasks	Celery	RAG pipeline execution
Message broker	Redis	Flask ↔ Celery communication
Session store	Redis	Conversation history per user
Vector database	ChromaDB	Semantic chunk retrieval
Embedding model	all-MiniLM-L6-v2	Local text → vector (384-dim)
LLM runtime	AWS Bedrock — Llama 3.1 8B	Answer generation via managed inference
Reverse proxy	Nginx	SSL, rate limiting, routing
Book parser	BeautifulSoup4	HTML → section records → chunks
Package manager	uv	Dependency management + venv

---

Project Structure

semantic_RAG/
├── app/
│   ├── __init__.py        Flask app factory + global error handlers
│   ├── routes.py          API endpoint definitions
│   ├── tasks.py           Celery task — RAG pipeline orchestration
│   ├── retriever.py       ChromaDB semantic search
│   ├── embedder.py        MiniLM embedding wrapper (singleton)
│   ├── llm.py             LLM call wrapper (Bedrock / Ollama / Grok)
│   ├── session.py         Redis session management
│   └── logger.py          Shared structured logging
├── deploy/
│   ├── nginx.conf.example          Reference Nginx config
│   ├── gunicorn.service.example    Reference systemd service
│   ├── celery.service.example      Reference systemd service
│   └── DEPLOY.md                   Full server deployment guide
├── input/
│   └── climate_academy.html         Source HTML book
├── html_sectioning.py     HTML parser → IndexedChunk objects
├── ingest.py              One-time ingestion pipeline script
├── config.py              All environment variable configuration
├── run.py                 Development server entry point
├── run_production.sh      Production startup script (used by systemd)
├── gunicorn.conf.py       Gunicorn production configuration
├── pyproject.toml         Project dependencies
├── uv.lock                Locked dependency versions
└── .env.example           Environment variable template

---

API Endpoints

Method	Endpoint	Description	Response
GET	/health	Health check	{"status":"ok"}
POST	/session	Create chat session	{"session_id":"..."}
DELETE	/session/<id>	Clear session + history	{"message":"..."}
POST	/chat	Send message	{"task_id":"..."}
GET	/result/<task_id>	Poll for answer	{"status":"done","answer":"...","sources":[...]}

Full Request Flow

# 1. Create session
curl -X POST http://localhost:5000/session
# → {"session_id": "abc-123"}

# 2. Send message
curl -X POST http://localhost:5000/chat \
  -H "Content-Type: application/json" \
  -d '{"session_id": "abc-123", "message": "What is the greenhouse effect?"}'
# → {"task_id": "xyz-456"}

# 3. Poll until done (status changes from "pending" to "done")
curl http://localhost:5000/result/xyz-456
# → {"status": "done", "answer": "The greenhouse effect...", "sources": [...]}

---

Local Development Setup

Prerequisites

Tool	Install
Python 3.11+	sudo apt install python3 / brew install python
uv	curl -LsSf https://astral.sh/uv/install.sh | sh
Redis	sudo apt install redis-server / brew install redis
Ollama	curl -fsSL https://ollama.com/install.sh | sh

Steps

# 1. Clone
git clone https://github.com/semanticClimate/semantic_RAG.git
cd semantic_RAG

# 2. Install all dependencies
uv sync

# 3. Configure environment
cp .env.example .env
# defaults work for local development — no changes needed

# 4. Start Redis
sudo systemctl start redis-server
redis-cli ping   # should return PONG

# 5. Pull the LLM model and start Ollama
ollama pull llama3.1:8b
ollama serve &

# 6. Run ingestion (once, or whenever the book changes)
uv run python ingest.py

# 7. Start services (four separate terminals)
uv run python run.py                                              # Terminal 1 — Flask
uv run celery -A app.tasks worker --concurrency=2 --loglevel=info # Terminal 2 — Celery

# 8. Test
curl http://localhost:5000/health
# → {"status": "ok"}

---

Using a Remote Ollama Server (GPU)

If Ollama runs on a separate GPU server, use SSH port forwarding:

# Stop local Ollama to free the port
sudo systemctl stop ollama

# Open the tunnel — keep this terminal open
ssh -L 11434:localhost:11434 user@gpu-server-ip -N

# Verify the tunnel hits the GPU server (not local)
curl http://localhost:11434/api/tags
# Must show installed models — not {"models":[]}

No .env changes needed. The app calls localhost:11434 which the tunnel forwards transparently.

Important: Always start the SSH tunnel before starting Celery workers.

---

Environment Variables

Variable	Default	Description
FLASK_ENV	development	Flask environment
FLASK_DEBUG	1	Debug mode (set to 0 in production)
SECRET_KEY	—	Flask secret key (change in production)
REDIS_URL	redis://localhost:6379/0	Redis connection URL
CHROMA_PATH	./chroma_db	ChromaDB storage path
CHROMA_COLLECTION	climate_academy	Collection name
EMBEDDING_MODEL	all-MiniLM-L6-v2	Sentence transformer model
LLM_PROVIDER	auto	auto, ollama, bedrock, or grok; bedrock uses AWS Bedrock with Llama 3.1 8B
AWS_REGION	us-east-1	AWS region for Bedrock
BEDROCK_MODEL_ID	meta.llama3-1-8b-instruct-v1:0	Bedrock model identifier
BEDROCK_MAX_TOKENS	1024	Maximum tokens generated per response
BEDROCK_TEMPERATURE	0.3	Sampling temperature for Bedrock
OLLAMA_BASE_URL	http://localhost:11434	Ollama server URL
OLLAMA_MODEL	llama3.1:8b	Model name (must match ollama list)
GROK_API_KEY / XAI_API_KEY	—	xAI API key for Grok fallback or LLM_PROVIDER=grok
GROK_BASE_URL	https://api.x.ai/v1	xAI OpenAI-compatible API base URL
GROK_MODEL	grok-4.3	Grok chat model
GROK_TIMEOUT_SECONDS	120	Grok request timeout
CHUNK_SIZE	150	Words per chunk
CHUNK_OVERLAP	30	Overlap words between chunks
TOP_K	5	Chunks retrieved per query
DISTANCE_THRESHOLD	0.7	Max cosine distance for relevant chunks
SESSION_TTL_SECONDS	86400	Session expiry (24 hours)
LOG_LEVEL	INFO	Logging level (DEBUG / INFO / WARNING)

---

Re-running Ingestion

Run whenever the source book changes:

uv run python ingest.py
sudo systemctl restart celery-climate   # restart workers to reload collection

Check indexed chunks:

uv run python -c "
import chromadb; from config import Config
c = chromadb.PersistentClient(path=Config.CHROMA_PATH)
print('Chunks:', c.get_collection(Config.CHROMA_COLLECTION).count())
"

---

Logging

Development: all logs print to stdout with format:

2026-05-21 15:30:01 | INFO     | app.retriever | Retrieved 5 chunks — distance range: 0.31–0.58

Production: Gunicorn writes to /var/log/climate-rag/. View live:

sudo journalctl -u gunicorn-climate -f
sudo journalctl -u celery-climate -f

---

Production Deployment

See deploy/DEPLOY.md for the complete step-by-step server deployment guide.

---

Known Constraints

Constraint	Detail
LLM concurrency	Ollama handles one inference at a time — concurrent users queue via Celery
Vector DB scale	ChromaDB is single-node — suitable for ≤50 concurrent users
SSH tunnel	Remote Ollama requires an active SSH tunnel — automated via systemd in production
Chunk size	MiniLM max input is ~256 tokens ≈ 180 words — CHUNK_SIZE=150 stays safely within limit

Contributors:

Renu Kumari : Program Manager

Udita Agarwal: AI/RAG Engineer and Technical Program Manager

Ali Mehdi Naqvi : Backend Developer

Aleena Harold Peter : Frontend Developer

Saarthak Patwal : Documentation and Chatbot testing

Parth Kadam, Uddhav Mishra : Video Content Creation

Samyukta Nair, Bui Doan Khanh Linh and Misha Ali: Chatbot Testing

Acknowledgements

We gratefully acknowledge the guidance, support, and contributions of the following individuals and organizations:

Prof. Peter Murray-Rust for his vision, guidance, and continuous support throughout the project.

Dr. Gitanjali Yadav for her valuable guidance, feedback, and support.

The Climate Academy for providing the educational materials that form the foundation of the chatbot's knowledge base.

Open Knowledge Foundation (OKFN), AI Learning Labs for their collaboration and support.