refactor: migrate Xtend to Java - com.avaloq.tools.ddk.xtext.generator (parser group) by joaodinissf · Pull Request #1429 · dsldevkit/dsl-devkit

joaodinissf · 2026-06-16T23:56:33Z

Migrates the parser group of com.avaloq.tools.ddk.xtext.generator (parser/antlr + parser/common, 6 files) from Xtend to hand-written Java:
AbstractAnnotationAwareAntlrGrammarGenerator, AnnotationAwareAntlrContentAssistGrammarGenerator, AnnotationAwareAntlrGrammarGenerator, AnnotationAwareXtextAntlrGeneratorFragment2, GrammarRuleAnnotations, PredicatesNaming.

Notes

The ANTLR StringConcatenation code-generation templates were byte-verified against the xtend-gen ground truth.
Local gate green: clean verify checkstyle:check pmd:check pmd:cpd-check spotbugs:check (0 violations).
Review round 3 (2026-07-11): all remaining static append-runs coalesced into text blocks / .formatted() (net −375 lines); StringConcatenation kept only where load-bearing (appendImmediate separators, two-arg indented appends of multi-line values, newLineIfNotEmpty after dynamic values, import-registering append(TypeReference)). Byte fidelity: executable old-vs-new harness (39 cases) + independent per-hunk adversarial review (0 divergences); branch rebased onto current master.
Split module: this is one half of xtext.generator; the per-module Xtend build infra (xtend-gen wiring, MANIFEST, build.properties/.classpath/.project) is intentionally left untouched — it will be cleaned once both the parser and the builder PRs merge (12 sibling .xtend remain).

Blind adversarial review verdict: SHIP (only 2 invented-Javadoc nits).

…r (builder group) Migrates 12 generator fragment/support classes from Xtend to Java: BundleVersionStripperFragment, DefaultFragmentWithOverride, builder/BuilderIntegrationFragment2, builder/LspBuilderIntegrationFragment2, builder/StandaloneBuilderIntegrationFragment2, formatting/FormatterFragment2, languageconstants/LanguageConstantsFragment2, model/project/ProjectConfig, modelinference/ModelInferenceFragment2, resourceFactory/ResourceFactoryFragment2, ui/compare/CompareFragment2, and ui/contentAssist/AnnotationAwareContentAssistFragment2. Xtend rich-string templates were rewritten as explicit StringConcatenation/string building, implicit getters/extension methods made explicit, and null-safe operators expanded to guarded Java. Behaviour is preserved; only language form changes. This is the builder half of the com.avaloq.tools.ddk.xtext.generator split (parser half tracked in dsldevkit#1429); per-module Xtend infrastructure cleanup is deferred until both halves merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rubenporras

Note: I have seen that Claude constantly forgets that he should try to use multi-line strings for readability. I use to interject it as steering once the skill has been kicked in, and that has worked fine so far.

rubenporras · 2026-06-18T05:58:03Z

+
+  protected CharSequence compileLexerMembers(final Grammar it, final AntlrOptions options) {
+    final StringConcatenation builder = new StringConcatenation();
+    builder.append("@members {");


this is clearly a multi line string.

rubenporras · 2026-06-18T06:02:55Z

+  protected CharSequence compileLexer(final Grammar it, final AntlrOptions options) {
+    final StringConcatenation builder = new StringConcatenation();
+    builder.append(codeConfig.getFileHeader());
+    builder.newLineIfNotEmpty();


I think a builder is justified for this case, but it would be better to remove the usage of org.eclipse.xtend2.lib.StringConcatenation, do you agree?

did you see this comment?

rubenporras · 2026-06-18T06:03:19Z

+
+  @Override
+  protected String compileParserImports(final Grammar it, final AntlrOptions options) {
+    final StringConcatenation builder = new StringConcatenation();


this is clearly a multi line string.

rubenporras · 2026-06-18T06:03:41Z

+
+  protected CharSequence compileParserSetTokenStreamMethod() {
+    final StringConcatenation builder = new StringConcatenation();
+    builder.append("/**");


this is clearly a multi line string, and so on, I did not continue the review.

Address review feedback (dsldevkit#1429): convert the static and single-statement interpolated grammar-fragment templates to Java text blocks with .formatted() - compileLexerMembers, compileParserImports, compileParserMemberDeclarations, compileParserSetTokenStreamMethod, compileRuleCatch. Builders that compose sub-results via newLineIfNotEmpty (compileLexer), use the two-argument append(value, indent) re-indentation, or contain FOR/IF control flow are left as StringConcatenation, per the skill's "StringBuilder for control flow only" rule - so the org.eclipse.xtend2.lib.StringConcatenation import stays. Separate commit on top of the migration; output proven byte-identical to the prior code for every converted method (verified against the real StringConcatenation runtime). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rubenporras · 2026-06-23T07:09:52Z

+  protected CharSequence compileLexer(final Grammar it, final AntlrOptions options) {
+    final StringConcatenation builder = new StringConcatenation();
+    builder.append(codeConfig.getFileHeader());
+    builder.newLineIfNotEmpty();


did you see this comment?

rubenporras · 2026-06-23T07:11:40Z

+    builder.append(_grammarAccessExtensions.gaElementIdentifier(AntlrGrammarGenUtil.getOriginalElement(it)));
+    builder.append("__Impl");
+    builder.newLineIfNotEmpty();
+    builder.append("  ");


this for example, I think it would truly benefit from a multi-line string

rubenporras · 2026-06-23T07:12:03Z

+    builder.append(this._grammarAccessExtensions.getGrammarAccess(it).getSimpleName(), "    ");
+    builder.append(" grammarAccess, ParserContext parserContext, ");
+    builder.append(this.predicatesNaming.getSemanticPredicatesSimpleName(it), "    ");
+    builder.append(" predicates) {");


…r (builder group) Migrates 12 generator fragment/support classes from Xtend to Java: BundleVersionStripperFragment, DefaultFragmentWithOverride, builder/BuilderIntegrationFragment2, builder/LspBuilderIntegrationFragment2, builder/StandaloneBuilderIntegrationFragment2, formatting/FormatterFragment2, languageconstants/LanguageConstantsFragment2, model/project/ProjectConfig, modelinference/ModelInferenceFragment2, resourceFactory/ResourceFactoryFragment2, ui/compare/CompareFragment2, and ui/contentAssist/AnnotationAwareContentAssistFragment2. Xtend rich-string templates were rewritten as explicit StringConcatenation/string building, implicit getters/extension methods made explicit, and null-safe operators expanded to guarded Java. Behaviour is preserved; only language form changes. This is the builder half of the com.avaloq.tools.ddk.xtext.generator split (parser half tracked in #1429); per-module Xtend infrastructure cleanup is deferred until both halves merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

joaodinissf · 2026-06-25T15:11:52Z

Coalesced all the self-contained static append/newLine runs across the ANTLR generators into text blocks (byte-verified against the Xtend compiler's output), and left compileLexer + the dynamic builders on StringConcatenation since they rely on newLineIfNotEmpty empty-suppression that a plain StringBuilder can't replicate without changing the generated grammar.

— drafted by Claude, reviewed by João

Address review feedback (dsldevkit#1429): convert the static and single-statement interpolated grammar-fragment templates to Java text blocks with .formatted() - compileLexerMembers, compileParserImports, compileParserMemberDeclarations, compileParserSetTokenStreamMethod, compileRuleCatch. Builders that compose sub-results via newLineIfNotEmpty (compileLexer), use the two-argument append(value, indent) re-indentation, or contain FOR/IF control flow are left as StringConcatenation, per the skill's "StringBuilder for control flow only" rule - so the org.eclipse.xtend2.lib.StringConcatenation import stays. Separate commit on top of the migration; output proven byte-identical to the prior code for every converted method (verified against the real StringConcatenation runtime). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rubenporras

I have not looked all the places where this patterns make sense but I think there are more.

rubenporras · 2026-07-06T07:29:00Z

+          builder.append(_lexerSuperClass);
+          builder.append(".class)");
+          builder.newLineIfNotEmpty();
+          builder.append("  ");


this and the next append can be put together, can they not?

Done.

(comment written by Claude Code on João's behalf)

rubenporras · 2026-07-06T07:30:39Z

+          builder.append(_typeRef, "  ");
+          builder.append(".CONTENT_ASSIST))");
+          builder.newLineIfNotEmpty();
+          builder.append("  ");


Done.

(comment written by Claude Code on João's behalf)

rubenporras · 2026-07-06T07:32:08Z

+      .addConfiguredBinding("IProposalConflictHelper", new StringConcatenationClient() {
+        @Override
+        protected void appendTo(final StringConcatenationClient.TargetStringConcatenation builder) {
+          builder.append("binder.bind(org.eclipse.xtext.ide.editor.contentassist.IProposalConflictHelper.class)");


this can all be a multi line string

Done — text block.

(comment written by Claude Code on João's behalf)

rubenporras · 2026-07-06T07:37:44Z

+          TypeReference _lexerSuperClass = naming.getLexerSuperClass(AnnotationAwareXtextAntlrGeneratorFragment2.this.getGrammar());
+          builder.append(_lexerSuperClass);
+          builder.append(".class)");
+          builder.newLineIfNotEmpty();


Suggested change

builder.newLineIfNotEmpty();

builder.newLine();

Done — and every sibling newLineIfNotEmpty() whose line ends in a static literal is now a plain newLine() across all four files (the remaining ones guard dynamic values that may end in a newline or retract whitespace-only lines, so they are load-bearing).

(comment written by Claude Code on João's behalf)

rubenporras · 2026-07-06T07:39:15Z

+    builder.append("import ");
+    builder.append(this.getGrammarNaming().getInternalParserSuperClass(it).getName());
+    builder.append(';');
+    builder.newLineIfNotEmpty();


Suggested change

builder.newLineIfNotEmpty();

builder.newLine();

Done — and the surrounding static import runs are coalesced into text blocks.

(comment written by Claude Code on João's behalf)

rubenporras · 2026-07-06T07:41:08Z

+    builder.append(_grammarAccessExtensions.ruleName(it));
+    builder.newLineIfNotEmpty();
+    if (this.annotations.hasNoBacktrackAnnotation(it)) {
+      builder.append("  ");


can be just a multi line string

Done — text block.

(comment written by Claude Code on João's behalf)

…r (parser group) Migrate parser group (6 files) to hand-written Java; ANTLR StringConcatenation templates byte-verified vs xtend-gen; local gate green. Split of com.avaloq.tools.ddk.xtext.generator: per-module Xtend infra cleanup deferred until both parser+builder PRs merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Address review feedback (dsldevkit#1429): convert the static and single-statement interpolated grammar-fragment templates to Java text blocks with .formatted() - compileLexerMembers, compileParserImports, compileParserMemberDeclarations, compileParserSetTokenStreamMethod, compileRuleCatch. Builders that compose sub-results via newLineIfNotEmpty (compileLexer), use the two-argument append(value, indent) re-indentation, or contain FOR/IF control flow are left as StringConcatenation, per the skill's "StringBuilder for control flow only" rule - so the org.eclipse.xtend2.lib.StringConcatenation import stays. Separate commit on top of the migration; output proven byte-identical to the prior code for every converted method (verified against the real StringConcatenation runtime). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…textAntlrGeneratorFragment2 Replace the single-line StringConcatenation in the keyword-condition map lambda with String.formatted(); the result is byte-identical. The multi-line condBuilder (which relies on newLine/newLineIfNotEmpty) is left unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@members

…eAntlrGrammarGenerator Collapse the pure-static append runs (parser imports block, the @members comment, the constructor body, and the getFirstRuleName/getGrammarAccess methods) into Java text blocks. Single-line two-arg appends (rule name, grammar-access simple name) fold to %s as the re-indent is a no-op for single-line values. Each run proven byte-identical via the StringConcatenation-vs-text-block harness; loops/appendImmediate/dynamic parts are left intact. Generated grammar unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@after

Fold self-contained pure-static append/newLine runs across the AnnotationAware ANTLR generators into Java text blocks: ContentAssist (java.util + super/DFA/ grammarAccess imports, getValueForTokenName body, @init/finally blocks), AnnotationAwareAntlrGrammarGenerator (backtrack empty-action blocks, @after, lexer-import comment), and Fragment2 (createParserContext Javadoc, NameMappings get/set, isReparseSupported, getDefaultRuleName). Dynamic grammar-element building, import-managed TypeReference appends, loop bodies, and appendImmediate separators keep their StringConcatenation builders. Generated output is byte-identical to the prior append runs (verified append-for-append against xtend-gen). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Addresses the third review round: every static append run across the parser-group generators is now a text block or a single coalesced append, simple interpolations use .formatted(), and four single-line StringConcatenation builders are plain string concatenation. newLineIfNotEmpty() is reduced to newLine() only where the last append on the line is a static literal - provably identical, since the guard is line-scoped and only behaves differently when the current line is empty or whitespace-only. StringConcatenation remains where it is load-bearing: appendImmediate loop separators, two-arg append(value, indent) of multi-line values (continuation re-indent), newLineIfNotEmpty() after dynamic values that may be empty or newline-terminated, and import-registering append(TypeReference/Class) calls. Byte fidelity proven by an executable old-vs-new harness (39 cases incl. appendImmediate loops at 0/1/3 elements, newline-terminated dynamic values, and %-bearing names) and an independent adversarial per-hunk review of the full diff (0 divergences). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ANGM7L3BaEGfGUmPVKRt4

joaodinissf · 2026-07-11T16:09:48Z

Done — swept all six files, not just the anchors: static append runs → text blocks, simple interpolations → .formatted(), four single-line builders → plain concatenation (net −375 lines). StringConcatenation remains only where it is load-bearing: appendImmediate loop separators, two-arg append(value, indent) of multi-line values, newLineIfNotEmpty() after dynamic values, and the import-registering append(TypeReference) calls. Byte fidelity proven by an old-vs-new harness plus an independent per-hunk review; full gate green.

(comment written by Claude Code on João's behalf)

…va skill Adds rules/04-templates.md section 4.8 documenting the source-verified StringConcatenation semantics that govern coalescing append-chains into text blocks: append(String) newline normalization (the formal basis for coalescing), the line-scoped whitespace-retracting behaviour of newLineIfNotEmpty(), and the two-arg append(value, indent) continuation re-indent. The conversion rule: newLineIfNotEmpty() may become newLine() only when the last append on the current line is a static non-whitespace literal; it must stay after dynamic line-tails and after indent + conditional values; two-arg appends of potentially multi-line values are never folded; appendImmediate loops stay untouched. Validation checklist gains row 35 enforcing the rule. Distilled from the #1429 round-3 review rework, where these distinctions were byte-verified against an executable harness and an adversarial per-hunk review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ANGM7L3BaEGfGUmPVKRt4

Address review feedback (#1429): convert the static and single-statement interpolated grammar-fragment templates to Java text blocks with .formatted() - compileLexerMembers, compileParserImports, compileParserMemberDeclarations, compileParserSetTokenStreamMethod, compileRuleCatch. Builders that compose sub-results via newLineIfNotEmpty (compileLexer), use the two-argument append(value, indent) re-indentation, or contain FOR/IF control flow are left as StringConcatenation, per the skill's "StringBuilder for control flow only" rule - so the org.eclipse.xtend2.lib.StringConcatenation import stays. Separate commit on top of the migration; output proven byte-identical to the prior code for every converted method (verified against the real StringConcatenation runtime). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Enabling CPD (#1397, pmd.cpd.min 100000->100) and migrating the AnnotationAware ANTLR generators to Java (#1429) each passed CI in isolation, but their combination on master surfaced 4 intra-file duplications in the migrated generators that neither PR's build saw: CPD was still effectively disabled when #1429's checks ran, and #1397's baseline predated #1429's Java. The snapshot full scan caught it, leaving master red at cpd-check. These are faithfully migrated Xtend generators with parallel rule-emission structure, baselined with file-scoped CPD-OFF markers consistent with #1397's treatment of the analogous FormatJvmModelInferrer. Verified locally: pmd:cpd-check / checkstyle:check / pmd:check all green on com.avaloq.tools.ddk.xtext.generator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

joaodinissf mentioned this pull request Jun 17, 2026

refactor: migrate Xtend to Java - com.avaloq.tools.ddk.xtext.generator (builder group) #1430

Merged

joaodinissf marked this pull request as ready for review June 17, 2026 16:25

joaodinissf requested a review from rubenporras June 17, 2026 16:25

rubenporras requested changes Jun 18, 2026

View reviewed changes

joaodinissf requested a review from rubenporras June 22, 2026 15:19

rubenporras requested changes Jun 23, 2026

View reviewed changes

joaodinissf requested a review from rubenporras June 25, 2026 15:11

joaodinissf force-pushed the migrate/xtend-to-java/xtext-generator-parser branch from 92b87bc to 52001c4 Compare July 5, 2026 16:02

rubenporras requested changes Jul 6, 2026

View reviewed changes

joaodinissf and others added 6 commits July 11, 2026 16:52

joaodinissf force-pushed the migrate/xtend-to-java/xtext-generator-parser branch from 52001c4 to 30b0570 Compare July 11, 2026 15:58

joaodinissf requested a review from rubenporras July 11, 2026 16:09

joaodinissf mentioned this pull request Jul 11, 2026

docs: encode the newLineIfNotEmpty conversion rule in the xtend-to-java skill #1461

Merged

rubenporras approved these changes Jul 13, 2026

View reviewed changes

joaodinissf merged commit 9025f31 into dsldevkit:master Jul 13, 2026
4 checks passed

joaodinissf deleted the migrate/xtend-to-java/xtext-generator-parser branch July 13, 2026 08:17

joaodinissf mentioned this pull request Jul 13, 2026

fix: baseline CPD duplications in migrated ANTLR generators #1467

Merged

Uh oh!

Conversation

joaodinissf commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notes

Uh oh!

rubenporras left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joaodinissf commented Jun 25, 2026

Uh oh!

rubenporras left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joaodinissf commented Jul 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

joaodinissf commented Jun 16, 2026 •

edited

Loading