Auto-inhibition of the attribute is too inconvenient on trivial semantic actions

[saki7]

Motivation

When matching a brace-wrapped text, it is very common that we expect to match the corresponding closing brace character after the opening brace matches. The naive approach seems to look like this:

constexpr auto opening_brace = 
    (char_(U'「') | char_(U'『')).on_match([](auto&& ctx) {
        _local_var(ctx) = get_closing_brace(_attr(ctx));
    });

constexpr auto closing_brace = char_.on_match([](auto&& ctx) {
    return _attr(ctx) == _local_var(ctx);
});

constexpr auto dialogue = x4::with_local<char32_t>[ // save closing brace to this local variable
    opening_brace >>
    +(char_ - closing_brace) >>
    closing_brace
];

However, this approach has several defects:

1. It does not work with x4::rule grammar. The surrounding rule grammar (dialogue) must wrap its parser body with x4::as<Dialogue>(...), otherwise the content is silently discarded. This is because opening_brace and closing_brace have semantic action attached, so the rule parser would treat them as "no attribute" by default.
   • See also: Mixing non-unused and unused parsers in a parent parser with a non-unused exposed attribute leads to incomplete parse result at runtime #48
2. The semantic action on opening_brace is inevitable because we literally require "semantic action" here. (i.e. get_closing_brace)
3. The semantic action on closing_brace can be reduced if and only if X4 provides some builtin way to invoke the lazy predicate on arbitrary value-based parser (in this case, we need char_(lazy_pred) here).

These are three independent issues with very significant impact on our library's convenience.

1. Dealing with auto-inhibition of the attribute on x4::rule parser + semantic action

This corresponds to the first item of the "Motivation" section above.

The primary purpose of semantic action is to do something intrusive with the attribute, thus the original attribute normally does not preserve its data-oriented meaning. So I think the auto-inhibition by default is still fine, we don't need to change that, because we would obviously going to get an extraordinarily unintuitive outcome if we keep the semantically-modified attribute by default.

However, I think we should provide some official facility to re-enable the inhibited attribute. An important note is that we already have the rule parser's operator%= since the Spirit.Qi era specifically for this purpose. However, I find that feature wrongly designed:

• The operator%= only acts against the entire right-hand-side operand of the rule %= something syntax, whereas the "real" inhibited attribute is each component of the combined parser on the right hand side. So the operator%= is more like an abuse of the assignment statement; it does not correctly express the intent of the reactivation of the inhibited component.
• The operator%= almost never works in practical grammar because a real-world grammar often uses IRIS_X4_DECLARE and IRIS_X4_DEFINE, which does not allow the definition's left hand side to be the concrete x4::rule type.
• We really need more fine-grained diagnostics based on the existence (or inhibition) of each of the components in the first place.
  • See: Using optional directive in string-like parser results in null ('\0') character being encoded in the result #79
• In fact, the x4::as parser already "lies" the existence of semantic action to provide the natural behavior.
  • I think this ::has_action API should be renamed as it does not reflect the name anymore.

  • x4/include/iris/x4/attribute/as.hpp

    Line 60 in c9c23bc

    static constexpr bool has_action = false; // Explicitly re-enable attribute detection in `x4::rule`

For these reasons, I think we need some dedicated attribute modifier facility to re-enable the inhibited attribute.

Naming candidates:

• x4::no_omit[p] does not work, because it technically can't reactivate the truly omitted attribute.
• x4::auto_(p) -- another option, but this looks more like C++23's auto(x) syntax, which falsely suggests decay-copy semantics.
• x4::force_attribute(p) -- similar to x4::as, this seems to fit inside the brand new "attribute/" category, thus the parenthesis would be preferred instead of the square brackets.
• Other candidates needed

2. Reactivating the inhibited attribute on some semantic-action-applied parser that legitimately requires "semantic action"

This corresponds to the second item of the "Motivation" section above.

I think we should add an another member like .on_match_with_attribute(f) alongside with the current .on_match(f). The only consideration required here is the naming. I generally don't like the verbose names seen in recent C++ standard codebase, but I haven't come up with some good candidate yet.

3. Value-based parsers like x4::char_ should accept a lazy predicate

Value-based parsers like x4::char_ should accept x4::char_(lazy_pred) or something similar. If we implement that in core, the semantic action for closing_brace can be dropped entirely.

However this is not a straightforward addition, because we can't constrain an arbitrary (generic) functor object to model "a predicate that takes T const& and returns bool", because the auto&& ctx type passed to the predicate cannot be determined until the runtime.

There seems to be two solutions:

• Embrace the unconstrained nature and hold the LazyPredF directly inside the parser.
  • This seems to be a horrible idea because an unconstrained overload virtually destroys and infects the legitimate overloads and we really shouldn't introduce them in post C++20 era.
• Take the context ID as the parameter, and don't take the actual functor.
  • For example: x4::char_(x4::lazy_value<my_context_id>). This kind of interface is somewhat becoming increasingly common in modern C++, so it should be acceptable (std::in_place_type<T>, std::from_range, etc.)
    • I think we still need a predicate here (in addition to the x4::lazy_value), but not a truly unconstrained one; we can expect some transparent comparators like std::equal_to<void>.