add docs and AI skills

2026-08-17 18:23:10 +02:00
parent c3d5d3541b
commit 09dcfede46
13 changed files with 506 additions and 0 deletions
@@ -0,0 +1,67 @@
+# Project architecture and evolution notes
+
+## Runtime flow
+
+```text
+.eftl document
+  -> VS Code language client (`src/extension.ts`)
+  -> LSP server (`src/server.ts`)
+  -> tokenizer (`src/tokenizer.ts`)
+  -> stack parser (`src/parser.ts`)
+  -> diagnostics and per-document variable definitions
+```
+
+Separately, VS Code loads the TextMate grammar, language configuration, and snippets directly from `package.json` contributions.
+
+## Current responsibilities
+
+- `src/extension.ts`: starts `out/server.js` over IPC for file-backed EFTL documents.
+- `src/tokenizer.ts`: recognizes a fixed, mostly uppercase set of tags, expressions, comments, and plain text; stores one-based line and column plus a character length.
+- `src/parser.ts`: checks stack-balanced open and close token kinds. It declares an `AstNode` interface but does not currently build an AST or enforce IF/WHILE child ordering.
+- `src/server.ts`: tokenizes and parses on document changes, validates VAR `type`, collects variable declarations, publishes diagnostics, and implements same-document go-to-definition by word matching.
+- `syntaxes/eftl.tmLanguage.json`: highlights a broader syntax independently of the TypeScript parser.
+- `language-configuration.json`: defines comment, bracket, auto-close, surrounding-pair, and folding behavior.
+- `snippets/eftl.json` and `examples/sample.eftl`: provide authoring examples and useful regression inputs.
+
+## Baseline gaps to re-check
+
+These observations describe the repository when this skill was created and are not permanent requirements:
+
+- Documented `LOG`, `FOR`, `IS_EMPTY`, and `IS_NOT_EMPTY` constructs are not tokenized or structurally parsed.
+- The specification describes case-insensitive tags; tokenization currently uses exact uppercase matches.
+- An unrecognized `[` currently makes `scanText()` return without consuming input, so lowercase, unknown, or incomplete tags can trap tokenization in an infinite loop.
+- Prefix checks such as `[VAR` need a tag-name boundary; otherwise longer unknown names can be misclassified.
+- WHILE attributes and the attribute form of TRIM are not recognized by the current exact scanners.
+- Parser validation is balance-only; it does not enforce root scope, IF branch ordering/cardinality, WHILE shape, or allowed bodies.
+- Attribute parsing uses regular expressions in the server and validates only VAR type.
+- Source positions combine one-based line/column with token string length. Multiline diagnostic end ranges and variable-definition selections require care.
+- `eftl.maxNumberOfProblems` is declared but not consumed by the server.
+- No automated test script is declared.
+- `npm run compile` passes at the baseline. `npm run lint` fails before linting because ESLint 9 cannot find a flat `eslint.config.*` file.
+
+## Preferred direction
+
+Evolve incrementally toward:
+
+1. a source-span model based on offsets with reliable LSP conversion;
+2. a context-aware scanner that can report malformed input and recover;
+3. structured attribute parsing with ranges;
+4. an AST or equivalent parse structure that represents documented relationships;
+5. separate semantic passes for declarations, references, types, and TAG payloads;
+6. table-driven language metadata shared where doing so reduces drift;
+7. core unit tests plus a small number of LSP integration tests.
+
+Do not perform this redesign wholesale for an unrelated small fix. Introduce seams that make the next supported construct easier and safer.
+
+## Completion criteria for a language construct
+
+A construct is complete only when applicable layers agree on:
+
+- accepted spellings and delimiters;
+- attribute names and body form;
+- nesting and ordering;
+- malformed-input recovery;
+- diagnostic message, severity, and exact range;
+- highlighting and folding;
+- snippet/example syntax;
+- focused positive, negative, boundary, and regression tests.