68 lines
3.8 KiB
Markdown
68 lines
3.8 KiB
Markdown
# Project architecture and evolution notes
|
|
|
|
## Runtime flow
|
|
|
|
```text
|
|
.eftl document
|
|
-> VS Code language client (`src/extension.ts`)
|
|
-> LSP server (`src/server.ts`)
|
|
-> tokenizer (`src/tokenizer.ts`)
|
|
-> stack parser (`src/parser.ts`)
|
|
-> diagnostics and per-document variable definitions
|
|
```
|
|
|
|
Separately, VS Code loads the TextMate grammar, language configuration, and snippets directly from `package.json` contributions.
|
|
|
|
## Current responsibilities
|
|
|
|
- `src/extension.ts`: starts `out/server.js` over IPC for file-backed EFTL documents.
|
|
- `src/tokenizer.ts`: recognizes a fixed, mostly uppercase set of tags, expressions, comments, and plain text; stores one-based line and column plus a character length.
|
|
- `src/parser.ts`: checks stack-balanced open and close token kinds. It declares an `AstNode` interface but does not currently build an AST or enforce IF/WHILE child ordering.
|
|
- `src/server.ts`: tokenizes and parses on document changes, validates VAR `type`, collects variable declarations, publishes diagnostics, and implements same-document go-to-definition by word matching.
|
|
- `syntaxes/eftl.tmLanguage.json`: highlights a broader syntax independently of the TypeScript parser.
|
|
- `language-configuration.json`: defines comment, bracket, auto-close, surrounding-pair, and folding behavior.
|
|
- `snippets/eftl.json` and `examples/sample.eftl`: provide authoring examples and useful regression inputs.
|
|
|
|
## Baseline gaps to re-check
|
|
|
|
These observations describe the repository when this skill was created and are not permanent requirements:
|
|
|
|
- Documented `LOG`, `FOR`, `IS_EMPTY`, and `IS_NOT_EMPTY` constructs are not tokenized or structurally parsed.
|
|
- The specification describes case-insensitive tags; tokenization currently uses exact uppercase matches.
|
|
- An unrecognized `[` currently makes `scanText()` return without consuming input, so lowercase, unknown, or incomplete tags can trap tokenization in an infinite loop.
|
|
- Prefix checks such as `[VAR` need a tag-name boundary; otherwise longer unknown names can be misclassified.
|
|
- WHILE attributes and the attribute form of TRIM are not recognized by the current exact scanners.
|
|
- Parser validation is balance-only; it does not enforce root scope, IF branch ordering/cardinality, WHILE shape, or allowed bodies.
|
|
- Attribute parsing uses regular expressions in the server and validates only VAR type.
|
|
- Source positions combine one-based line/column with token string length. Multiline diagnostic end ranges and variable-definition selections require care.
|
|
- `eftl.maxNumberOfProblems` is declared but not consumed by the server.
|
|
- No automated test script is declared.
|
|
- `npm run compile` passes at the baseline. `npm run lint` fails before linting because ESLint 9 cannot find a flat `eslint.config.*` file.
|
|
|
|
## Preferred direction
|
|
|
|
Evolve incrementally toward:
|
|
|
|
1. a source-span model based on offsets with reliable LSP conversion;
|
|
2. a context-aware scanner that can report malformed input and recover;
|
|
3. structured attribute parsing with ranges;
|
|
4. an AST or equivalent parse structure that represents documented relationships;
|
|
5. separate semantic passes for declarations, references, types, and TAG payloads;
|
|
6. table-driven language metadata shared where doing so reduces drift;
|
|
7. core unit tests plus a small number of LSP integration tests.
|
|
|
|
Do not perform this redesign wholesale for an unrelated small fix. Introduce seams that make the next supported construct easier and safer.
|
|
|
|
## Completion criteria for a language construct
|
|
|
|
A construct is complete only when applicable layers agree on:
|
|
|
|
- accepted spellings and delimiters;
|
|
- attribute names and body form;
|
|
- nesting and ordering;
|
|
- malformed-input recovery;
|
|
- diagnostic message, severity, and exact range;
|
|
- highlighting and folding;
|
|
- snippet/example syntax;
|
|
- focused positive, negative, boundary, and regression tests.
|