chatpack

Token-efficient chat export processing for LLM and RAG pipelines.

Website |

---

Table of Contents

• Overview
• CLI Tool
• Web Version
• Library
• Documentation
• Feature Flags
• Contributing
• License

---

Overview

Chatpack parses chat exports from Telegram, WhatsApp, Instagram, and Discord, converting them into token-efficient formats for LLM analysis and RAG ingestion.

Raw chat exports waste 80%+ of tokens on JSON structure, metadata, and formatting. Chatpack removes this noise, achieving 13x compression (92% token reduction) with CSV output.

┌─────────────────┐                   ┌─────────────────┐
│ Telegram JSON   │                   │ Clean CSV       │
│ WhatsApp TXT    │ ──▶ chatpack ──▶ │ 13x compression │
│ Instagram JSON  │                   │ LLM-ready       │
│ Discord Export  │                   │ RAG-optimized   │
└─────────────────┘                   └─────────────────┘

Token Compression Results

Format	Input	Output	Compression
CSV	11.2M tokens	850K tokens	13x (92%)
JSONL	11.2M tokens	1.0M tokens	11x (91%)
JSON	11.2M tokens	1.3M tokens	8x (88%)

---

CLI Tool

Command-line interface for chatpack. Separate repository: chatpack-cli

Installation

From crates.io:

cargo install chatpack-cli

Pre-built binaries: Download from GitHub Releases

Platform	Architecture	Download
Linux	x86_64	chatpack-linux-x86_64.tar.gz
Linux	ARM64	chatpack-linux-aarch64.tar.gz
macOS	Intel	chatpack-macos-x86_64.tar.gz
macOS	Apple Silicon	chatpack-macos-aarch64.tar.gz
Windows	x86_64	chatpack-windows-x86_64.zip

Usage

chatpack tg result.json           # Telegram
chatpack wa chat.txt              # WhatsApp
chatpack ig message_1.json        # Instagram
chatpack dc export.json           # Discord

Output: optimized_chat.csv — ready for LLM analysis.

Options

chatpack tg chat.json -f json              # Output format: csv, json, jsonl
chatpack tg chat.json -t                   # Include timestamps
chatpack tg chat.json --after 2024-01-01   # Filter by date
chatpack tg chat.json --from "Alice"       # Filter by sender
chatpack tg chat.json --no-merge           # Disable message merging

Full documentation: chatpack-cli README

---

Web Version

Browser-based interface — no installation required. Separate repository: chatpack-web

🌐 chatpack.berektassuly.com

• 100% Private — All processing happens locally via WebAssembly
• Files never leave your device — No server uploads
• Fast — Rust-powered WASM, 100K+ messages/sec

How to Use

1. Drag & drop your export file
2. Select source platform and output format
3. Click Convert
4. Download the result

---

Library

Python library for integration into your own projects.

Installation

pip install chatpack

Quick Start

from chatpack import TelegramParser

parser = TelegramParser()
messages = parser.parse("export.json")

# Merge consecutive messages from same sender
merged = merge_consecutive(messages)

# Write as CSV (13x compression)
write_csv(merged, "output.csv")

print(f"Processed {len(merged)} messages")

Feature Flags

Feature	Description	Default
full	All parsers + outputs + streaming	Yes
telegram	Telegram JSON parser	Yes
whatsapp	WhatsApp TXT parser	Yes
instagram	Instagram JSON parser	Yes
discord	Discord multi-format parser	Yes
csv-output	CSV output writer	Yes
json-output	JSON/JSONL output writers	Yes
streaming	O(1) memory streaming parsers	Yes

---

Contributing

Contributions welcome! Please:

1. Check existing issues
2. Open an issue before large changes
3. Run pytest before submitting PRs

---

License

MIT License. See LICENSE for details.

---

Related Repositories

Repository	Description
chatpack	Core library (this repo)
chatpack-cli	Command-line tool
chatpack-web	Web interface (WASM)
chatpack-python	Python wrapper