AudioMind-AI

AudioMind-AI is a production-grade Audio RAG backend that transforms recorded voice notes into an AI-searchable knowledge base.

The Problem

Ideas often come when we're walking, commuting, or busy with other tasks. Recording a voice note is easy, but replaying long recordings to find a specific thought is time-consuming, so many valuable ideas are never revisited.

The Solution Overview

AudioMind-AI lets users upload recorded audio, automatically transcribes it using Deepgram, generates vector embeddings, and stores them in a vector database. Instead of listening to an entire recording again, users can ask natural language questions such as "What business idea did I mention?" or "What action items did I discuss?" and receive context-aware answers grounded in the uploaded audio.

Built with FastAPI, Temporal, PostgreSQL, Redis, AWS S3, and Pinecone, the platform uses asynchronous workflows for reliable transcription, embedding generation, indexing, and retrieval, making it suitable for production-scale audio processing.

Features

• Audio Ingestion Pipeline — Upload .wav files, stored in MinIO (S3-compatible), transcribed via Deepgram (Nova-2), chunked, and embedded
• Temporal Workflow Orchestration — Reliable, retryable async processing with progress tracking
• Hybrid Semantic Search — Pinecone vector similarity + BM25 keyword scoring with reranking
• RAG Pipeline (LangGraph) — Retrieve → Rerank → Generate (Groq/Cerebras) → Validate (hallucination guard)
• JWT Authentication — Register/login flow with bcrypt password hashing
• Async Everything — FastAPI + async SQLAlchemy + async HTTP clients

Tech Stack

Component	Technology
API	FastAPI + Uvicorn
Database	PostgreSQL 15 (asyncpg, SQLAlchemy 2.0)
Migrations	Alembic
Vector DB	Pinecone
Object Storage	MinIO (S3-compatible, boto3)
Workflow Engine	Temporal.io
Speech-to-Text	Deepgram (Nova-2)
LLM (Primary)	Groq (llama-3.3-70b)
LLM (Fallback)	Cerebras (llama-4-scout-17b)
Embeddings	Jina AI (jina-embeddings-v2-base-en)
RAG Orchestration	LangGraph
Auth	JWT (python-jose) + bcrypt (passlib)
Serialization	Pydantic v2 + Pydantic-Settings
Testing	pytest + pytest-asyncio

Architecture

Client ──► FastAPI ──┬──► /auth ──► JWT
                     ├──► /upload ──► MinIO ──► Temporal Worker ──┬──► Deepgram (STT)
                     │                                            ├──► Jina AI (Embed)
                     │                                            └──► Pinecone (Upsert)
                     └──► /query ──► LangGraph ──┬──► Pinecone (Retrieve)
                                                  ├──► Reranker
                                                  ├──► Groq/Cerebras (Generate)
                                                  └──► Validator

Prerequisites

• Python 3.11+
• Docker & Docker Compose
• API keys for: Groq, Pinecone, Deepgram, Cerebras, Jina AI

Quick Start

# 1. Clone and set up Python environment
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# 2. Configure environment
cp .env.example .env
# Edit .env with your API keys

# 3. Start infrastructure (PostgreSQL, MinIO, Temporal)
cd docker && docker compose up -d && cd ..

# 4. Create MinIO bucket
# Open http://localhost:9000 (minio / minio123), create bucket "audio-files"

# 5. Create Pinecone index
python -m scripts.create_pinecone_index

# 6. Run database migrations
alembic upgrade head

# 7. Start Temporal worker (separate terminal)
python -m app.workflows.worker

# 8. Start FastAPI
uvicorn app.main:app --reload

Or use the all-in-one script:

python -m scripts.start

Open http://localhost:8000/docs for Swagger UI.

Configuration

All config is via .env at the project root. Key variables:

Variable	Description
DATABASE_URL	PostgreSQL connection string (postgresql+asyncpg://...)
JWT_SECRET	Secret key for JWT signing
PINECONE_API_KEY / PINECONE_INDEX	Pinecone vector DB credentials
S3_ENDPOINT / S3_ACCESS_KEY / S3_SECRET_KEY / S3_BUCKET	MinIO / S3 configuration
GROQ_API_KEY / CEREBRAS_API_KEY	LLM provider keys
DEEPGRAM_API_KEY	Speech-to-text API key
JINA_API_KEY	Embedding model API key
TEMPORAL_HOST / TEMPORAL_NAMESPACE / TASK_QUEUE	Temporal server config

API Endpoints

Auth (/auth)

Method	Path	Auth	Description
POST	/auth/register	No	Register with email + password
POST	/auth/login	No	Login, returns JWT
GET	/auth/me	JWT	Current user info

Upload (/upload)

Method	Path	Auth	Description
POST	/upload	JWT	Upload .wav file, triggers ingestion workflow
GET	/audio/{audio_id}/status	JWT	Check processing status & progress

Query (/query)

Method	Path	Auth	Description
POST	/query	JWT	Ask a natural language question about uploaded audio

Health

Method	Path	Auth	Description
GET	/health	No	Health check

Ingestion Pipeline

Upload ──► MinIO ──► Temporal Workflow
                         │
                     ├── Download from MinIO
                     ├── Transcribe (Deepgram)
                     ├── Clean transcript
                     ├── Chunk text (300 words, 50 overlap)
                     ├── Embed (Jina AI)
                     ├── Upsert to Pinecone
                     └── Update status → completed

Progress is tracked in PostgreSQL at each stage (10%, 25%, 45%, 65%, 85%, 100%).

RAG Pipeline

Query ──► 1. Retrieve (Pinecone vector + BM25 hybrid search)
       ──► 2. Rerank (semantic score + text overlap, top-3)
       ──► 3. Generate (Groq → Cerebras fallback)
       ──► 4. Validate (≥30% word overlap with context to prevent hallucination)
       ──► Answer + Citations

Project Structure

AudioMind-AI/
├── app/
│   ├── main.py                     # FastAPI app entry point
│   ├── api/                        # Route handlers (auth, upload, query, health)
│   ├── core/                       # Config, security, exceptions, logging
│   ├── db/                         # Models, session, repositories
│   ├── ingestion/                  # Chunking, text cleaning
│   ├── rag/                        # LangGraph pipeline (retriever, reranker, generator, validator)
│   ├── services/                   # External service integrations (Deepgram, Pinecone, Jina, LLM, S3)
│   ├── schemas/                    # Pydantic request/response models
│   ├── workflows/                  # Temporal workflow definitions & worker
│   ├── utils/                      # Retry, IDs, time helpers
│   └── tests/                      # pytest test suite
├── alembic/                        # Database migrations
├── docker/                         # Docker Compose (PostgreSQL, MinIO, Temporal)
├── scripts/                        # Utility scripts (start, seed, create index)
├── .env.example
└── requirements.txt

Testing

pytest

Infrastructure

The docker/docker-compose.yml launches:

• PostgreSQL 15 — primary database (port 5432)
• MinIO — S3-compatible object storage (ports 9000, 9001)
• Temporal Server — workflow engine (port 7233)
• Temporal UI — workflow visibility (port 8233)

License

This project is private / proprietary.