File out and file in Topaz .gs files - #555
Conversation
File Out on every GemStone Explorer level the issue names, plus the two nice-to-haves, and a File In that reads the result back. File Out writes the same Topaz artifact Jadeite writes -- a `fileformat utf8` header naming the image, then the chunks -- to a destination chosen through VS Code's save dialog, so the folder is on the user's own machine rather than the stone's host. Dictionary, class category, class (Classes and Hierarchy panes), method category, and a multi-selection of methods. Method- and protocol-level file-outs carry no class definition: reading one back must not redefine the class and drop every method it has. A dictionary file-out gets a preamble that creates the dictionary. GemStone's `fileOutClassesAndMethodsInDictionary:on:` writes none, while every class definition in it says `inDictionary: <Name>` -- so without one the file fails on its first chunk on a stone that lacks the dictionary. The preamble is composed client-side: building it in the doit needs a `$'` literal, which 3.6.x's compiler fails on with an internal ComStrmSetCursor error. File In runs a file chunk by chunk over the GCI rather than handing it to `GsFileIn`, which reads a path on the stone's host and cannot see a local file. Running it here is what makes the report possible: a chunk GemStone refuses is named with its file and line while the rest of the file still goes in. Class definitions, comments, `removeAllMethods`, categories, `set compile_env:` and `input` are all honoured. A hand-written topaz script files in too. Its preamble addresses the topaz program rather than the image -- `set gemstone`, `login`, `output push`, `commit`, `logout` -- and none of it is run: Jasper files in over the session the user chose, and does not commit because a file said to. Those lines are reported rather than dropped, `exit` ends the file-in where topaz stops reading, and a `run` chunk ending in `^ something` is handled (the chunk wrapper uses `ensure:` so a non-local return cannot skip past it). Reachable from a File In button on each connected session in Logins & Sessions -- which files into that session without asking which -- from the Command Palette, from a .gs/.tpz in VS Code's file Explorer, and from an open one's editor title bar and context menu. Nothing is committed; the summary says so. Tests include live-stone round trips: a class filed out, deleted and filed back returns with its methods in their protocols; a dictionary removed entirely comes back with its classes; two concurrent sessions prove a file-in lands only in the session it was handed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
File Out sits on the GemStone Explorer's rows; File In was in another view entirely, so having just filed a class out there was no way back in from the same place. Two entry points close that. The Dictionaries pane gets a File In button on its toolbar, and a dictionary row gets one in its context menu -- in a group of its own below "File Out Dictionary...", not beside it. A file names its own dictionaries, so a file-in is not scoped to the row it was started from, and sharing the group would say it was. Both file into the session the Explorer is already showing, so neither asks which session; with nothing connected the target is undefined and the usual prompt applies, as on every other route with no row. The command stays out of the Command Palette, where it would duplicate the existing picker under the same title. An open .gs or .tpz now carries a "File In to GemStone" link on its first line. The command was otherwise an icon in the title bar, an entry in a right-click menu, or palette wording the user had to know already. The lens is built with its command set rather than filled in by resolveCodeLens, so it cannot shove the source down after first paint, and it names its own document rather than relying on the active editor. Also corrects a comment that stopped being true when File Out began using globalState for the remembered directory. Manually verified against a live stone: filing the fixture in, a class round trip returning every selector to its protocol, class-category ordering, and a method-category file-out carrying no class definition. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…-filein CHANGELOG conflicted where both sides appended under "### Added" -- this branch's File In and File Out entries against main's whole-database start/stop. Additive on both sides, so all three are kept. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lens carries its own command and is deliberately absent from codeLensData, so resolveCodeLens returns it by the `!data` path. Nothing tested that: remove the guard and the lens quietly picks up the senders/implementors treatment -- "No session" -- while every other test still passes. Verified by mutation: with the guard disabled this test fails and nothing else does. The second was meant to check that the lens wins by insertion order where a method lens shares line 0, and found instead that it cannot happen. A method lens anchors on the SELECTOR, which the `method:` directive must precede, so even a file opening straight into a method chunk puts its first method lens on line 1. The lens has line 0 to itself and the ordering is positional, so the test now pins that rather than an insertion-order claim that was never load- bearing. Full suite run against a live 3.6.2 stone: 463 files, 7475 passed. The File In and File Out integration round trips pass for the first time against this work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Eight of these failed on every windows-latest job and passed on every ubuntu
one. The fixture paths were built with `path.resolve('/src/Animal.gs')`, which
on Windows takes the cwd's drive letter as-is -- `D:\src\Animal.gs` -- while
the code under test receives `Uri.fsPath`, which lowercases it to
`d:\src\Animal.gs` (vscode-uri's `uriToFsPath`). The mock filesystem is a plain
object, so the fixture was simply invisible to any file-in that arrived through
a Uri: every Uri-driven test failed, and the ones handing `fileInFile` a raw
path passed, which is exactly the split CI showed.
Routing both through `Uri.file` puts them on one key. A no-op off Windows.
The sibling file-out tests do not have this: they use `Uri.file('/out/A.gs')`
with no drive letter and compare against `path.normalize`, so nothing ever
introduces one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous fix took the windows jobs from eight failures to two, both the
same drive-letter mechanism in places the first pass missed. CI printed one
verbatim:
expected 'c:\Users\runneradmin\Animal.gs'
to be 'C:\Users\runneradmin\Animal.gs'
They are inverses of each other. The file-out test compared `Uri.fsPath`
against `path.join(os.homedir(), ...)`, where homedir keeps the system's drive
case and fsPath lowercases it. The file-in test had the opposite shape: it
restated the remembered folder as `path.normalize('/src')`, which carries no
drive at all, while the value under test is the dirname of a drive-qualified
path.
Both now derive from the same expression production uses rather than restating
it -- `at(path.join(os.homedir(), ...))` against fileOut.ts's own
`Uri.file(path.join(rememberedDirectory(store), name))`, and `path.dirname(A_GS)`
against fileIn.ts's `path.dirname(uris[0].fsPath)` -- so they match by
construction instead of by coincidence.
Swept the rest of the branch for the two constructs that can introduce a drive
letter, `os.homedir()` and `path.resolve`. These were the only remaining sites;
the explorer file-out tests use rooted literals like `/out/File.gs`, which carry
no drive and match `path.normalize` on every platform.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| defaultFileName: fileOutFileName(node.fullPath), | ||
| label: node.fullPath, | ||
| build: () => { | ||
| const ordered = queries |
There was a problem hiding this comment.
💭 getDictionaryClassFileOutOrder answers [] when the dictionary doesn't resolve (dict ifNil: [^ '']), and this then filters an already-empty list — so composeFileOut(header, []) writes a header-only .gs and the user gets "Filed out Fauna" for a file with no code in it. The inCategory.size === 0 guard above catches the empty category, but not a state.dictIndex that has gone stale (a dictionary removed in another session, a tree not yet refreshed).
The same thing happens more quietly for a partial mismatch: any class in inCategory that isn't in the order query's answer is dropped without a word, so the file is short a class and nothing says so.
Everywhere else this PR is careful about exactly this — classFileOutBody turns the not-found sentinel into a raise precisely so a .gs can't be written with the wrong contents, and fileOutDictionary raises rather than "writing an empty file". Should this path do the same, e.g. throw from build when ordered.length !== inCategory.size (or at least when it's 0), naming the classes that didn't resolve?
| /** Save-dialog file types. `.gs` first — a Topaz file-out is what these commands write. */ | ||
| export const FILE_OUT_FILTERS: Record<string, string[]> = { | ||
| 'GemStone Files': ['gs'], | ||
| 'Smalltalk Files': ['st'], |
There was a problem hiding this comment.
💭 Offering .st here breaks the round trip this PR is built around. What gets written is Topaz chunk format either way, but .st maps to gemstone-tonel in the manifest, so a file-out saved through this filter comes back as: wrong syntax highlighting, no "File In to GemStone" lens (isFileInTarget gates on gemstone-topaz), no entry in the editor title bar or context menu (both resourceLangId == gemstone-topaz), no entry in VS Code's Explorer menu (.gs/.tpz only), and not even listed by the File In open dialog, whose FILE_IN_FILTERS are gs/tpz.
So the one filter that isn't .gs is the one that makes the file unreachable from every route the other half of the PR adds. Should this be ['gs', 'tpz'] to match FILE_IN_FILTERS, and drop .st entirely?
| async (uri?: vscode.Uri, selected?: vscode.Uri[]) => { | ||
| // VS Code hands an Explorer context command the clicked resource AND the whole | ||
| // selection; the editor title bar passes only the one resource. Falling back to | ||
| // the active editor covers the palette-shaped call with no argument at all. |
There was a problem hiding this comment.
💭 The comment says the active-editor fallback "covers the palette-shaped call with no argument at all", but gemstone.fileInFile is "when": "false" in commandPalette — and fileIn.manifest.test.ts asserts that on purpose ("it has no file to act on there"). Every remaining route passes a URI: the Explorer menu passes (uri, selected), the title bar and editor menu pass the resource, and the code lens passes document.uri explicitly.
So as far as I can tell the fallback and the 'Open or select a .gs file to file in.' warning are both unreachable today. No objection to keeping them as defence for a future keybinding or an executeCommand from elsewhere — but should the comment say that rather than naming the palette, so the next reader doesn't go looking for a palette entry that the manifest deliberately doesn't have?
| // exactly one session. Without it the user is asked, as any other write is. | ||
| session?: ActiveSession, | ||
| ): Promise<void> { | ||
| const uris = await vscode.window.showOpenDialog({ |
There was a problem hiding this comment.
💭 Should the session be resolved before the open dialog rather than after it? As written, with nothing connected the user browses, picks their files, and only then gets "No GemStone sessions are active. Please log in first." from resolveSession — the picking was wasted.
File Out goes the other way round on purpose: fileOutSession() warns first, and the dialog only opens once there's a session (the ordering comment in saveFileOut is about the opposite hazard — not paying for expensive queries on a cancel — which doesn't apply here, since resolving a session is free). Doing the same here would also mean fileInCommand and fileInUris agree on when the question gets asked.
| // section headings — nothing to run. | ||
| if (trimmed.length === 0 || trimmed.startsWith('!')) return; | ||
|
|
||
| const cat = trimmed.match(/^category:\s*'(.*)'\s*$/i); |
There was a problem hiding this comment.
💭 This is now the third category: scanner in this file, and the only one that un-doubles the quotes: fileInClass and parseFileStructure both use /^category:\s*'([^']*)'/i, which stops at the first inner quote and would file Bob''s methods into a category called Bob.
Those two are outside this PR's scope, but they got a sibling that behaves differently, which is the kind of divergence that's hard to spot later. Worth pulling the match into one small helper here (parseCategoryDirective(line): string | undefined) and pointing the older two at it? The '' handling you added is the correct one, and the class mirror would quietly get it too.
| expect(outcome.skipped[0].message).toContain('wibble'); | ||
| }); | ||
|
|
||
| it('runs a topaz script s chunks and leaves its topaz commands alone', () => { |
There was a problem hiding this comment.
nit: the apostrophes have been stripped out of a handful of test titles, which reads oddly in the runner output — "a topaz script s chunks" here, plus "the caller s session" (:337), "topaz s own" (topazScript.test.ts:158), "a binary selector s punctuation" (fileOut.test.ts:29), "the organizer s" / "the category s" (explorerFileOut.test.ts:117, :139), and "GemStone s own" / "GemStone s error" (fileIn.integration.test.ts:126, :464). Double-quoting the title is enough (it("runs a topaz script's chunks…")), and there are already tests in these files doing that.
| @@ -0,0 +1,176 @@ | |||
| import { describe, it, expect } from 'vitest'; | |||
There was a problem hiding this comment.
💭 Not sure what would be a good folder name for grouping this "file in"/"file out" logic, but if you could find a concept to group all the related code into its own folder, that would help to keep the codebase more manageable/easier to follow
What this does
Jasper can now write GemStone code out to a
.gsfile on your own machine, and read one back in. Neither half existed before: Jasper could compile a method and define a class, but it could not produce the Topaz file that GemStone code is shipped in, and it could not read one.Both halves work on your machine, not the stone's host. File Out uses VS Code's save dialog, so the folder you pick is the one you're sitting at. File In reads the file here and runs it chunk by chunk over the connection, rather than handing a path to GemStone's
GsFileIn— which reads a path on the stone's host and so cannot see a file on your laptop at all.Running it on this side is also what makes the report possible. Every chunk's outcome is known, so a method GemStone refuses is named with its file and line while the rest of the file still files in — which is what you want when one method out of twenty is stale. Nothing is ever committed; the summary says so.
Where to find it
.gs/.tpzin VS Code's Explorer; an open Topaz file's title bar, context menu, and a link on its first lineDetails
File Out
Writes the same artifact Jadeite writes — a
fileformat utf8header naming the image, then the chunks — via GemStone's ownClass>>fileOutClass, so a file-out from Jasper and one from Jadeite read back in identically. The folder you chose last time is where the next one opens, shared with File In.Two cases needed care:
fileOutClassesAndMethodsInDictionary:on:writes none, while every class definition in it saysinDictionary: <Name>— so without one, the file fails on its very first chunk on a stone that lacks the dictionary and takes the rest with it. The preamble is composed client-side: building it in the doit needs a$'literal, which 3.6.x's compiler rejects with an internalComStrmSetCursorerror.A class category brings its sub-categories' classes with it, matching what the Classes pane shows, and stops there — a superclass filed under a different category is not swept in behind your back. A dictionary comes out superclass-first, so it reads back in as-is.
File In
Honours class definitions, class comments,
removeAllMethods, method categories,set compile_env:andinput, so a class filed out and filed back in returns with its methods in their original protocols, and a loader naming sibling files pulls them in beside it.A hand-written
.tpzscript files in too. Its preamble addresses the topaz program rather than the image —set gemstone,login,output push,commit,logout— and none of it runs: Jasper files in over the session you chose, not one the file names, and does not commit because a file said to. Those lines are reported rather than silently dropped, since they change what the file means;exitends the file-in where topaz stops reading; and arunchunk ending in^ somethingis handled (the chunk wrapper usesensure:so a non-local return can't skip past it).Session targeting
Every route that already names a session files straight into it with no "which session?" prompt — the session row knows its own, and the Explorer is showing exactly one. Routes with no such context (Command Palette, a file in VS Code's Explorer) ask, as any other write does.
The Explorer's File In sits in a group of its own below "File Out Dictionary…", not beside it: a file names its own dictionaries, so a file-in is not scoped to the row it was started from, and sharing the group would imply it was.
Testing
Full suite against a live GemStone 3.6.2 stone: 463 files, 7475 passed, 0 failures.
Known limitations
SecurityErrorper refused chunk. This is GemStone authorization working correctly — theGsRefactoringclasses, for instance, are installed over a transient SystemUser session, so a DataCurator session cannot modify them. It is reported honestly but not explained: sixteen near-identical authorization errors is a poor way to say "you don't own this class." Worth its own issue.%— the chunk terminator — is not covered by a test, since such a method can't be authored in a hand-written chunk file. It needs acompileMethod:fixture built from parts.Fixes #539
🤖 Generated with Claude Code