Exact token counting (optional tiktoken or a custom tokenizer)

[TheCoder30ec4]

Motivation

estimate_tokens assumes about 4 characters per token. That undercounts code, CJK text and emoji, so a prompt can pass the context check and still overflow at the provider.

Proposal

• Router(..., count_tokens=callable) lets users plug in their own counter
• Optional extra: pip install model-router-python[tiktoken]; use tiktoken when it's installed, otherwise fall back to the heuristic
• The core package keeps zero required dependencies

Done when

• The custom counter is used for both the context and the cost checks
• Tests with a fake counter