Multi-Query

Query LLMs from the command line. Send the same query to several different models at a time, responses being printed as they are received. Currently supports text-based models from OpenAI and Gemini.

Disclaimer: Token usage varies between models. No checks regarding token usage or token availability are performed before sending queries.

Prerequisites (Last updated June 2025)

• Python 3.12 or later + optional packages: google-generativeai, openai.

Setup

1. Clone this repository:

git clone https://github.com/GordonRamsay-689/Multi-Query.git

2. Install at least one of the following libraries (you only need the ones you plan to use):

Gemini API

1. Install Gemini API:

pip3 install google-generativeai

2. Generate a free API key at https://aistudio.google.com/apikey.

3. Set the following environment variable:

export GOOGLE_API_KEY=*

OpenAI API

1. Install OpenAI API

pip3 install openai

2. Create a project and generate an API key at https://platform.openai.com/docs/api-reference/authentication. You may need to purchase tokens at least once to avoid RateLimitErrors.

3. Set the following environment variable:

export OPENAI_API_KEY=*

Usage

Tip: Create an alias for easy access:

alias mq="python3 ../Multi-Query/master.py"

Multi-Query can run in Conversation Mode or One-Shot Mode. If a query is passed the program will start in One-Shot Mode. If no query is passed or if the conversation flag (-c) is passed the program will start in Conversation Mode.

• One-Shot Mode: Terminates after all responses to the query passed as argument have been received.
• Conversation Mode: An interface will be presented for submitting queries. To send a message in Conversation Mode, enter three newline characters in a row (press enter four times). Pasted content is supported, but if the pasted content contains three consecutive newline characters the query will be sent immediately and anything after that point will not be included in the query.

Running the script with no arguments will provide a list of model aliases. Both aliases and full names are supported when specifying models.

To submit a query:

mq "QUERY" [MODEL NAME | MODEL ALIAS] [--FLAG | -FLAG SHORTHAND]

The query is delimited by double quotes, and must be the first argument passed if provided. Other arguments are not positional, and can be mixed freely. Any number of clients can be added:

mq "QUERY" --flag CLIENT_A --flag CLIENT_B ...

Examples:

Submit a query, add three models, start Conversation Mode and print raw responses (no markdown formatting):

mq "Write a haiku about README.md" g1.5f -c g2.5f gpt4 --no-format

Start Conversation Mode with two models:

mq g2.5f gpt4

Flags

Runtime Flags

Runtime flags are passed as arguments on runtime.

Available runtime flags:

Flag	Shorthand	Function
--models	-m	Refresh model cache and print a table of supported models and their aliases.
--chat	-c	Start in Conversation Mode.
--help	-h	Detailed usage information.
--no-format	-f	Disables output formatting.
--stream	-s	Enables streamed responses for supported model. Only one model can stream at a time.
--sys-message	-S	Provide a system message to supported models. Must be followed by a message.

Prompt Flags

Prompt flags are used in Conversation Mode, prefixed with two dashes, followed by a colon and any necessary argument.

--flag:[ARGUMENT]

Flags that do not require an argument will not be recognized if one is passed.

Flags will be extracted from the query (and executed) before it is sent:

This is a --add:g2.5f --rm:gpt4 query.

The model (Gemini 2.5 Flash in this case) would receive the query: "This is a query."

Available prompt flags:

Flag	Function
--add:	Add a model to the conversation. Must be passed a valid model name or alias.
--rm:	Remove a model from the conversation. Must be passed a valid model name or alias. Tip: Pass "all" to remove all active sessions.
--clear:	Restart all active sessions.
--commands:	Prints a list of prompt flags and their usage strings.
--dump:	Dumps current conversation to file at ./dumps/dump_MM_DD_YYYY_HH_MM_SS.txt.
--format:	Toggle response markdown formatting.
--help:	Print help string and usage for flag name. If no valid model name or alias is passed print general program help.
--stream:	Toggle streamed responses for a specific model. *
--aliases:	Display a table of available models and their aliases.
--sys:	Specifies a new system message. Must be passed a message enclosed in double quotes. **

* Only one client can have streaming enabled at any given time. Enabling streaming for a new client will disable streaming for all other clients.

** Currently, only OpenAI models support receiving system messages mid-conversation.

Tips and Tricks:

Upload file contents with your initial prompt:

mq "Extract key ideas from this article: $(cat article.txt) and insert into this essay: $(cat essay.txt)."