Add support for JSONL

[danroth27]

JSON Lines (JSONL) is a text-based data format where each line represents a valid JSON object. This structure allows for easy parsing and processing of data on a line-by-line basis, making it suitable for handling large datasets. efficiently, and streaming content. JSONL is used by A2UI (Agent to UI), a declarative UI protocol for agent-driven interfaces.

Consider adding framework support for producing and consuming JSONL documents including a serialization and deserialization model. This support could then be used to build A2UI support for .NET.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json
See info in area-owners.md if you want to be subscribed.

[MihuBot]

I'm a bot. Here is a possible related and/or duplicate issue (I may be wrong):

• Let Utf8JsonReader process input with one complete JSON document per line #33030

[eiriktsarpalis]

.NET 9 shipped comprehensive JSONL deserialization support (#33030). At the low level, JsonReaderOptions.AllowMultipleValues permits the Utf8JsonReader to consume multiple top-level JSON values separated by whitespace. At the high level, JsonSerializer.DeserializeAsyncEnumerable<T> with topLevelValues: true streams JSONL documents from a Stream or PipeReader:

await foreach (LogEntry entry in JsonSerializer.DeserializeAsyncEnumerable<LogEntry>(stream, topLevelValues: true))
{
    ProcessEntry(entry);
}

Without topLevelValues (or when set to false), the same method reads from a root-level JSON array instead. Note that the reader is lenient: it accepts any whitespace (spaces, tabs, \r\n, \n) between documents, which is a superset of strict JSONL. This is fine — Postel's law applies.

Serialization

There is no dedicated JSONL serialization API today, but the same result can be achieved by calling SerializeAsync in a loop and writing newline separators manually:

await foreach (LogEntry entry in GetLogEntries())
{
    await JsonSerializer.SerializeAsync(stream, entry);
    await stream.WriteAsync("\n"u8);
}

Or, for better performance, using Utf8JsonWriter with Reset() to reuse writer state across values:

using var bufferWriter = new ArrayBufferWriter<byte>();
using var writer = new Utf8JsonWriter(bufferWriter);

await foreach (LogEntry entry in GetLogEntries())
{
    JsonSerializer.Serialize(writer, entry);
    writer.Flush();

    await stream.WriteAsync(bufferWriter.WrittenMemory);
    await stream.WriteAsync("\n"u8);
    bufferWriter.Clear();
    writer.Reset();
}

Both approaches work but require boilerplate that is easy to get wrong (forgetting the newline, forgetting to reset, not flushing). The proposed API below encapsulates this pattern and mirrors the symmetry of the existing DeserializeAsyncEnumerable API.

API Proposal

namespace System.Text.Json;

public static partial class JsonSerializer
{
    // EXISTING (deserialization, shipped in .NET 9)
    // public static IAsyncEnumerable<TValue?> DeserializeAsyncEnumerable<TValue>(Stream utf8Json, bool topLevelValues, JsonSerializerOptions? options = null, CancellationToken cancellationToken = default);
    // public static IAsyncEnumerable<TValue?> DeserializeAsyncEnumerable<TValue>(Stream utf8Json, JsonTypeInfo<TValue> jsonTypeInfo, bool topLevelValues, CancellationToken cancellationToken = default);
    // public static IAsyncEnumerable<TValue?> DeserializeAsyncEnumerable<TValue>(PipeReader utf8Json, bool topLevelValues, JsonSerializerOptions? options = null, CancellationToken cancellationToken = default);
    // public static IAsyncEnumerable<TValue?> DeserializeAsyncEnumerable<TValue>(PipeReader utf8Json, JsonTypeInfo<TValue> jsonTypeInfo, bool topLevelValues, CancellationToken cancellationToken = default);

    // Stream overloads
    public static Task SerializeAsyncEnumerable<TValue>(
        Stream utf8Json,
        IAsyncEnumerable<TValue> value,
        bool topLevelValues = false,
        JsonSerializerOptions? options = null,
        CancellationToken cancellationToken = default);

    public static Task SerializeAsyncEnumerable<TValue>(
        Stream utf8Json,
        IAsyncEnumerable<TValue> value,
        JsonTypeInfo<TValue> jsonTypeInfo,
        bool topLevelValues = false,
        CancellationToken cancellationToken = default);

    // PipeWriter overloads
    public static Task SerializeAsyncEnumerable<TValue>(
        PipeWriter utf8Json,
        IAsyncEnumerable<TValue> value,
        bool topLevelValues = false,
        JsonSerializerOptions? options = null,
        CancellationToken cancellationToken = default);

    public static Task SerializeAsyncEnumerable<TValue>(
        PipeWriter utf8Json,
        IAsyncEnumerable<TValue> value,
        JsonTypeInfo<TValue> jsonTypeInfo,
        bool topLevelValues = false,
        CancellationToken cancellationToken = default);
}

When topLevelValues is false (the default), the output is a root-level JSON array [v1,v2,...]. When topLevelValues is true, the output is JSON Lines (one value per line). This mirrors the existing DeserializeAsyncEnumerable API exactly.

The newline separator emitted in JSONL mode respects JsonSerializerOptions.NewLine for consistency with Utf8JsonWriter indentation behavior. The JSONL spec accepts both \n and \r\n.

API Usage

// JSON array (default)
await JsonSerializer.SerializeAsyncEnumerable(stream, GetLogEntries());
// Output: [{"timestamp":"..."},{"timestamp":"..."}]

// JSON Lines
await JsonSerializer.SerializeAsyncEnumerable(stream, GetLogEntries(), topLevelValues: true);
// Output: {"timestamp":"..."}\n{"timestamp":"..."}\n

// Source-generated, JSON Lines
await JsonSerializer.SerializeAsyncEnumerable(stream, GetLogEntries(), MyContext.Default.LogEntry, topLevelValues: true);

// Round-trips with existing deserialization API
stream.Position = 0;
await foreach (var entry in JsonSerializer.DeserializeAsyncEnumerable<LogEntry>(stream, topLevelValues: true))
{
    Console.WriteLine(entry.Message);
}

Alternative Designs

• Utf8JsonWriter.AllowMultipleValues with automatic newline insertion: Considered and prototyped, but rejected. Adding state tracking to the writer's hot path for an uncommon scenario is not justified when Reset() already provides a clean mechanism.

Risks

• No changes to Utf8JsonWriter internals — zero hot-path impact.
• The JSONL implementation uses writer.Reset() between values, which is an existing public API with well-defined semantics.

Prototype: af783d6c224

[svick]

Do I understand it correctly that this makes it easy to produce JSONL, but it does not enforce the JSONL spec? So if I set JsonSerializerOptions.WriteIndented to true or if I serialize arrays or primitive types (JSONL mandates only objects at the top level), then I will get invalid JSONL?

[eiriktsarpalis]

Do I understand it correctly that this makes it easy to produce JSONL, but it does not enforce the JSONL spec?

Only on the deserialization side, following Postel's law.

So if I set JsonSerializerOptions.WriteIndented to true or if I serialize arrays or primitive types (JSONL mandates only objects at the top level), then I will get invalid JSONL?

Indentation settings should be ignored when used by these methods specifically. When it comes to primitive types or arrays, yes, individual top level values may not get serialized as objects, but the same holds for regular serialization which may or may not produce valid JSON documents for the same reason.

[bartonjs]

Video

Looks good as proposed

namespace System.Text.Json;

public static partial class JsonSerializer
{
    // Stream overloads
    public static Task SerializeAsyncEnumerable<TValue>(
        Stream utf8Json,
        IAsyncEnumerable<TValue> value,
        bool topLevelValues = false,
        JsonSerializerOptions? options = null,
        CancellationToken cancellationToken = default);

    public static Task SerializeAsyncEnumerable<TValue>(
        Stream utf8Json,
        IAsyncEnumerable<TValue> value,
        JsonTypeInfo<TValue> jsonTypeInfo,
        bool topLevelValues = false,
        CancellationToken cancellationToken = default);

    // PipeWriter overloads
    public static Task SerializeAsyncEnumerable<TValue>(
        PipeWriter utf8Json,
        IAsyncEnumerable<TValue> value,
        bool topLevelValues = false,
        JsonSerializerOptions? options = null,
        CancellationToken cancellationToken = default);

    public static Task SerializeAsyncEnumerable<TValue>(
        PipeWriter utf8Json,
        IAsyncEnumerable<TValue> value,
        JsonTypeInfo<TValue> jsonTypeInfo,
        bool topLevelValues = false,
        CancellationToken cancellationToken = default);
}