Streaming APIs (phase 2): stream()/start() terminal methods on the fluent WinRMClient

[bertysentry]

Note (2026-07-27): reframed as phase 2 of the API modernization. Streaming is now designed as additional terminal methods on the fluent WinRMClient operation builders introduced by #131, rather than new methods on WindowsRemoteExecutor. The technical requirements below (lazy Pull, incremental decoding, lifecycle, cleanup) are unchanged.

Context

The public API currently materializes complete responses in memory:

• WQL execution returns all rows after collecting the initial Enumerate response and every subsequent Pull response.
• Command execution returns a result only after collecting all stdout and stderr into strings.

This is inconvenient and memory-intensive for large WQL result sets and long-running or verbose commands. Callers should be able to process rows and command output incrementally as WinRM responses arrive — the wire protocol is already streaming (WS-Enumeration Pull loop, Receive loop); only the client buffers.

This issue targets the lightweight implementation only: LightWinRMService / WsmanClient.

Depends on

• Modern fluent client API: WinRMClient builder with per-operation builders #131 — the fluent WinRMClient API. Modern fluent client API: WinRMClient builder with per-operation builders #131 deliberately builds its internal seams stream-first (the Pull and Receive loops expose iterator/callback-shaped primitives, and the blocking execute() is implemented as "drain the stream"), so this issue only adds new terminal methods on the existing operation builders.

Proposed API

The choice between "materialize everything" and "stream" is a different terminal method on the same operation builder — everything upstream (auth, TLS, namespace, working directory, timeout) is shared:

Operation	Materialized (#131)	Streaming (this issue)
WQL	execute() → WqlResult	stream() → Stream<WqlRow> (closeable)
Command	execute() → CommandResult	start() → RemoteProcess (closeable)

WQL: lazy Stream<WqlRow>

try (Stream<WqlRow> rows = client.wql("SELECT * FROM Win32_NTLogEvent").stream()) {
    rows.filter(r -> "Error".equals(r.string("Type")))
        .limit(100)
        .forEach(this::process);
}

Requirements:

• yields each row as soon as it is parsed;
• performs WS-Enumeration Pull requests only as iteration advances (backpressure is free — it is a pull protocol);
• does not retain all rows or DOM nodes from previous pages: memory is bounded by the active page;
• preserves property ordering;
• the stream is AutoCloseable and documented as such (same contract as Files.lines());
• closing early (or exhaustion, timeout, cancellation, failure) sends a WS-Enumeration Release so the server frees the enumeration context — a nicety the client does not send today, which would also help with server-side operation quotas (e.g. 2008R2 MaxConcurrentOperationsPerUser);
• optional pageSize(int) on the WQL builder for tuning (today's hardcoded MaxElements=32000 becomes the default) — overlaps with Allow configuration of batch size (MaxElements) and pull timeout (MaxTime) for WQL/WinRM queries #86/WQL queries should support setting max elements #35;
• exception propagation during lazy consumption is defined and documented (unchecked hierarchy from Modern fluent client API: WinRMClient builder with per-operation builders #131).

Command: a Process-like handle

A Stream<String> of lines is too lossy for commands (two output channels, an exit code, a lifecycle). The shape Java developers already know is java.lang.Process:

try (RemoteProcess p = client.command("wevtutil qe System /f:text").start()) {
    try (BufferedReader out = p.stdout()) {        // Reader, decoded incrementally
        out.lines().forEach(this::process);
    }
    int exit = p.waitFor();                        // or waitFor(Duration)
}

Plus a callback middle ground for "tail the output but block until done", keeping execute() as the terminal:

client.command("longRunningThing.exe")
      .onStdout(chunk -> log.info(chunk))
      .onStderr(chunk -> log.warn(chunk))
      .execute();   // still returns CommandResult at the end

Requirements:

• distinguish stdout from stderr and preserve ordering within each channel;
• expose the eventual exit code and execution metadata;
• support cancellation/close and reliably send the WinRM terminate Signal;
• avoid deadlock or unbounded buffering if the caller consumes one channel more slowly than the other;
• incremental decoding: the current receiveLoop deliberately accumulates raw bytes and decodes once at completion, because a multibyte character can be split across Stream chunks / Receive responses. Streaming output must use a stateful CharsetDecoder that carries partial-character state between chunks — not per-chunk new String(bytes);
• flush chunks promptly as each Receive response is processed.

Semantics that must be defined up front

• Timeout: for execute(), timeout = overall deadline (as today). For stream()/start(), an overall deadline makes long tails impossible — the builder timeout should mean inactivity timeout (max silence between chunks/pages), with waitFor(Duration) for callers who want a hard deadline. Whichever is chosen, it is documented in the Javadoc from day one; retrofitting timeout semantics is painful.
• Connection pinning: a WinRMClient is one serial channel (one socket, stateful RC4, operations serialized). An open stream/handle pins the connection; other operations on that client block until it is closed. Do not fight this — document it (same contract as a JDBC ResultSet on its connection).

Backward compatibility

• Purely additive on the Modern fluent client API: WinRMClient builder with per-operation builders #131 builders.
• The legacy WindowsRemoteExecutor.executeWql(...) / executeCommand(...) and the static helpers remain source- and behavior-compatible. Where practical they become convenience collectors over the streaming primitives (drain-the-stream), so the two paths cannot drift apart.

Acceptance criteria

• WQL rows can be consumed before the complete enumeration has been retrieved.
• Command stdout and stderr can be consumed before the remote command exits.
• Memory usage is bounded by the active WSMan page/chunks rather than total result size.
• Early close, cancellation, timeout, parse failure, and consumer failure clean up the enumeration/command (WS-Enumeration Release / terminate Signal) and release locks/connections.
• Timeout semantics (inactivity vs deadline) are documented and enforced throughout stream consumption.
• LightWinRMService / WsmanClient implements the streaming contract; the blocking execute() path is implemented on top of it.
• Tests (FakeWsmanServer / WsmanProtocolTest style) cover multi-page enumeration, early termination, large output, interleaved stdout/stderr, split multibyte characters, timeout, cancellation, and resource cleanup.
• Javadocs include try-with-resources examples for both streaming operations; README documents the streaming API.

[anuragpaul602-netizen]

Hi! I'd like to take this on, but given the scope described in the acceptance criteria (WQL streaming, command streaming, timeout semantics, cancellation/cleanup, split-multibyte decoding, and a full test matrix), I think it's safer to split it into two PRs rather than one large one — happy to hear if you'd rather have it as a single PR instead.

Proposed first slice: WQL stream() only

• Add a lazy iterator/callback seam to WsmanClient's Pull loop so rows are yielded as they're parsed, instead of the loop collecting everything before returning.
• WqlOperationBuilder.stream() returns a closeable Stream<WqlRow> built on that seam.
• Pull requests are issued only as the stream is consumed (real backpressure, not read-ahead).
• Memory bounded by the active page, not the total result size.
• Close (early termination, exhaustion, timeout, or consumer exception) sends a WS-Enumeration Release so the server frees the enumeration context.
• execute() stays implemented as "drain the stream" per Modern fluent client API: WinRMClient builder with per-operation builders #131's design note, so the two paths can't drift.
• Tests against FakeWsmanServer/WsmanProtocolTest: multi-page enumeration, early close before exhaustion, timeout mid-stream, and Release-on-close verification.
• README gets a stream() example with try-with-resources.

Deferred to a second PR: command start()/RemoteProcess, the onStdout/onStderr callback middle-ground, and the incremental CharsetDecoder work for split-multibyte output — that's a distinct chunk of surface area (process lifecycle, terminate Signal, dual-channel buffering) and deserves its own review pass.

One thing I'd like to confirm before starting: for stream(), should the builder's timeout() mean an inactivity timeout (max silence between Pull responses), as the issue suggests, or should there also be a separate hard-deadline option? I want to lock that down in the Javadoc from the first commit rather than retrofit it.

Let me know if this split and scope make sense and I'll get started.

[bertysentry]

@anuragpaul602-netizen Sorry this issue is already in process of being implemented!

[bertysentry]

@anuragpaul602-netizen I'm about to release officially version 2.0.00 with the new API, no dependencies, etc.
Then I'll draft a list of possible new features and enhancements. You'll be able to work on these in a couple days.
Thank you for the help! 😊