refine tag management

add threshold attribute to WHILE tag
fix unclosed block detection
2026-08-19 14:12:48 +02:00 · 2026-08-19 14:12:26 +02:00 · 2026-08-18 12:49:31 +02:00 · 2026-08-18 12:48:55 +02:00 · 2026-08-18 12:48:20 +02:00 · 2026-08-17 18:23:10 +02:00
25 changed files with 2244 additions and 206 deletions
@@ -0,0 +1,46 @@
+---
+name: evolve-eftl-linter
+description: Evolve the EFTL VS Code linter and language server while keeping tokenizer, parser, LSP diagnostics, navigation, TextMate grammar, editor configuration, snippets, examples, and documentation consistent. Use for EFTL feature implementation, bug fixes, refactors, new diagnostics, language-server capabilities, or support for a newly documented EFTL construct.
+---
+
+# Evolve EFTL Linter
+
+Implement the smallest coherent change across all affected language-support layers. Read [references/project-architecture.md](references/project-architecture.md) before a non-trivial change.
+
+## Workflow
+
+1. Inspect the worktree and preserve unrelated user changes.
+2. Convert the request into a language rule card. Use `$interpret-eftl-language` for new or disputed EFTL behavior.
+3. Search all support surfaces for the construct and record the required files.
+4. Add or update focused tests and fixtures before broad refactoring. If test infrastructure is absent, introduce the smallest repository-level harness that exercises tokenizer and parser without VS Code.
+5. Change recognition in `src/tokenizer.ts`. Preserve exact source spans and make a deliberate decision about case-insensitivity and malformed delimiters.
+6. Change structure in `src/parser.ts`. Validate grammar relationships and ordering, not only matching close tags. Recover after an error so one defect does not flood the document with misleading diagnostics.
+7. Change semantic analysis and LSP behavior in `src/server.ts`. Keep internal positions unambiguous and convert to zero-based LSP ranges only at the boundary.
+8. Synchronize editor assets when user-visible syntax changes: TextMate grammar, language configuration, snippets, examples, and README.
+9. Run `$verify-eftl-linter`; inspect every failure and any new diagnostic range manually.
+
+## Design constraints
+
+- Do not encode the same tag metadata independently in several new switch statements. Prefer a shared declarative definition when a change spans tokenization, parsing, and validation.
+- Keep parsing separate from LSP transport so core behavior is testable without starting a language client.
+- Do not claim full expression validation until the grammar supports precedence, literals, operators, and recovery defined by the runtime language.
+- Avoid static errors for values known only at runtime. Use warnings only when they are actionable and low-noise.
+- Cap published diagnostics with `eftl.maxNumberOfProblems` and make truncation deterministic when implementing server work.
+- Preserve valid user text and ElixForms TAG payloads; brackets inside comments, expressions, quoted attributes, or TAG bodies need context-aware handling.
+- Guarantee scanner progress: every tokenization step must consume input or terminate with an error. Include unknown and incomplete `[` constructs in regression tests.
+- Treat diagnostic ranges as first-class behavior. Test single-line and multiline tokens, CRLF and LF, and non-ASCII text before the error.
+
+## Change-surface checklist
+
+| Concern | Primary files |
+| --- | --- |
+| Token kinds, scanning, locations | `src/tokenizer.ts` |
+| Nesting, ordering, recovery, AST | `src/parser.ts` |
+| Diagnostics, config, symbols, definitions | `src/server.ts` |
+| Client activation | `src/extension.ts`, `package.json` |
+| Highlighting | `syntaxes/eftl.tmLanguage.json` |
+| Brackets, comments, folding | `language-configuration.json` |
+| Authoring examples | `snippets/eftl.json`, `examples/`, `README.md` |
+| Build and tests | `package.json`, `tsconfig.json`, test files |
+
+Update only surfaces affected by the rule card, but explicitly state why an apparently related surface is unchanged.
@@ -0,0 +1,4 @@
+interface:
+  display_name: "Evolve EFTL Linter"
+  short_description: "Implement coordinated EFTL linter changes"
+  default_prompt: "Use $evolve-eftl-linter to implement this EFTL language-support change safely."
@@ -0,0 +1,67 @@
+# Project architecture and evolution notes
+
+## Runtime flow
+
+```text
+.eftl document
+  -> VS Code language client (`src/extension.ts`)
+  -> LSP server (`src/server.ts`)
+  -> tokenizer (`src/tokenizer.ts`)
+  -> stack parser (`src/parser.ts`)
+  -> diagnostics and per-document variable definitions
+```
+
+Separately, VS Code loads the TextMate grammar, language configuration, and snippets directly from `package.json` contributions.
+
+## Current responsibilities
+
+- `src/extension.ts`: starts `out/server.js` over IPC for file-backed EFTL documents.
+- `src/tokenizer.ts`: recognizes a fixed, mostly uppercase set of tags, expressions, comments, and plain text; stores one-based line and column plus a character length.
+- `src/parser.ts`: checks stack-balanced open and close token kinds. It declares an `AstNode` interface but does not currently build an AST or enforce IF/WHILE child ordering.
+- `src/server.ts`: tokenizes and parses on document changes, validates VAR `type`, collects variable declarations, publishes diagnostics, and implements same-document go-to-definition by word matching.
+- `syntaxes/eftl.tmLanguage.json`: highlights a broader syntax independently of the TypeScript parser.
+- `language-configuration.json`: defines comment, bracket, auto-close, surrounding-pair, and folding behavior.
+- `snippets/eftl.json` and `examples/sample.eftl`: provide authoring examples and useful regression inputs.
+
+## Baseline gaps to re-check
+
+These observations describe the repository when this skill was created and are not permanent requirements:
+
+- Documented `LOG`, `FOR`, `IS_EMPTY`, and `IS_NOT_EMPTY` constructs are not tokenized or structurally parsed.
+- The specification describes case-insensitive tags; tokenization currently uses exact uppercase matches.
+- An unrecognized `[` currently makes `scanText()` return without consuming input, so lowercase, unknown, or incomplete tags can trap tokenization in an infinite loop.
+- Prefix checks such as `[VAR` need a tag-name boundary; otherwise longer unknown names can be misclassified.
+- WHILE attributes and the attribute form of TRIM are not recognized by the current exact scanners.
+- Parser validation is balance-only; it does not enforce root scope, IF branch ordering/cardinality, WHILE shape, or allowed bodies.
+- Attribute parsing uses regular expressions in the server and validates only VAR type.
+- Source positions combine one-based line/column with token string length. Multiline diagnostic end ranges and variable-definition selections require care.
+- `eftl.maxNumberOfProblems` is declared but not consumed by the server.
+- No automated test script is declared.
+- `npm run compile` passes at the baseline. `npm run lint` fails before linting because ESLint 9 cannot find a flat `eslint.config.*` file.
+
+## Preferred direction
+
+Evolve incrementally toward:
+
+1. a source-span model based on offsets with reliable LSP conversion;
+2. a context-aware scanner that can report malformed input and recover;
+3. structured attribute parsing with ranges;
+4. an AST or equivalent parse structure that represents documented relationships;
+5. separate semantic passes for declarations, references, types, and TAG payloads;
+6. table-driven language metadata shared where doing so reduces drift;
+7. core unit tests plus a small number of LSP integration tests.
+
+Do not perform this redesign wholesale for an unrelated small fix. Introduce seams that make the next supported construct easier and safer.
+
+## Completion criteria for a language construct
+
+A construct is complete only when applicable layers agree on:
+
+- accepted spellings and delimiters;
+- attribute names and body form;
+- nesting and ordering;
+- malformed-input recovery;
+- diagnostic message, severity, and exact range;
+- highlighting and folding;
+- snippet/example syntax;
+- focused positive, negative, boundary, and regression tests.
@@ -0,0 +1,53 @@
+---
+name: interpret-eftl-language
+description: Interpret the EFTL language specification and turn the repository PDF documentation into explicit lexical, syntactic, structural, attribute, and semantic rules. Use when adding or reviewing EFTL tags, expressions, nesting constraints, ElixForms TAG payloads, examples, diagnostics, or when the documentation and current TypeScript implementation disagree.
+---
+
+# Interpret EFTL Language
+
+Derive a small, testable language contract before changing implementation code.
+
+## Establish the source of truth
+
+Use the sources in this order:
+
+1. Read the relevant section of `docs/elixForms_Doc_EFTL.pdf` for EFTL behavior and examples.
+2. Read `docs/elixForms_EFTLParser_syntax_rels.pdf` for parser relationships and allowed composition.
+3. Read `docs/elixForms_TAG_Sintassi.pdf` for positional `SCHEMAID` and `GETVALUEBYTAG` payloads.
+4. Use [references/eftl-language-reference.md](references/eftl-language-reference.md) as a navigation aid and concise baseline, never as a replacement for a disputed PDF passage.
+5. Treat `src/`, `syntaxes/`, `snippets/`, `examples/`, and `README.md` as the current implementation, not as normative language documentation.
+
+If the PDFs contradict one another or leave a rule unclear, report the ambiguity and preserve it in tests or design notes. Do not silently infer a restrictive diagnostic.
+
+## Produce a rule card
+
+For every construct being changed, record:
+
+- spelling and case-sensitivity;
+- block, bodyless, expression, or comment form;
+- required and optional attributes, value types, defaults, and mutual exclusions;
+- allowed body and parent/child relationships;
+- variable scope, result type, and runtime behavior relevant to static analysis;
+- valid, invalid, and boundary examples;
+- the safest diagnostic when a property cannot be proven statically.
+
+Distinguish syntax errors from semantic warnings. Runtime-only facts such as external TAG values must not become false-positive syntax errors.
+
+## Reconcile with the implementation
+
+Search every support surface before proposing a change:
+
+```powershell
+rg -n "CONSTRUCT|TokenType" src syntaxes snippets examples README.md language-configuration.json
+```
+
+Compare the rule card with:
+
+- `src/tokenizer.ts` for recognition, source positions, and case handling;
+- `src/parser.ts` for nesting and ordering;
+- `src/server.ts` for semantic diagnostics and symbols;
+- `syntaxes/eftl.tmLanguage.json` for highlighting;
+- `language-configuration.json` for brackets, folding, and comments;
+- `snippets/eftl.json`, `examples/`, and `README.md` for user-facing syntax.
+
+Return a change-surface list and explicit acceptance cases. Hand implementation work to `$evolve-eftl-linter` and validation work to `$verify-eftl-linter` when those skills are available.
@@ -0,0 +1,4 @@
+interface:
+  display_name: "Interpret EFTL Language"
+  short_description: "Interpret EFTL syntax and semantic rules"
+  default_prompt: "Use $interpret-eftl-language to derive the EFTL rules needed for this change."
@@ -0,0 +1,101 @@
+# EFTL language reference
+
+This is a compact index derived from the documents currently under `docs/`. Re-check the PDFs when exact wording, diagrams, or edge behavior matters.
+
+## Documentation map
+
+- `elixForms_Doc_EFTL.pdf`: EFTL specification through the documented 3.1.0 changes, language behavior, attributes, control flow, and examples.
+- `elixForms_EFTLParser_syntax_rels.pdf`: generated lexer/parser relationship diagrams.
+- `elixForms_TAG_Sintassi.pdf`: positional syntax for the ElixForms `SCHEMAID` and `GETVALUEBYTAG` plugins.
+
+## Common lexical and structural rules
+
+- EFTL uses square-bracket tags and requires properly nested, non-overlapping elements.
+- A bodyless element ends with `/]`; a block has matching opening and closing tags.
+- Attribute values are quoted. Attribute order is not semantically significant.
+- The main specification says uppercase and lowercase tag spellings are interpreted equally and recommends lowercase style. The current implementation is largely uppercase-only.
+- Text outside `[EFTL]...[/EFTL]` is not processed. Text inside an EFTL root but outside an executable tag is emitted as user text where the grammar permits it.
+- Comments use `[!-- ... --]` and may span lines.
+- The generated relationships include user text, ignored whitespace, CDATA, headers, comments, statements, code blocks, and output blocks. Consult the diagram PDF before enforcing a new parent/child restriction.
+
+## Root, directives, and context
+
+| Construct | Form | Key rules |
+| --- | --- | --- |
+| EFTL | `[EFTL] ... [/EFTL]` | Root block; no documented attributes; a document needs at least one root to be processed. |
+| HEADER | `[HEADER name="..." value="..." type="..." /]` | Declares a directive. The documented directive is `trimDocument`, with boolean `true` or `false`. |
+| LOG | `[LOG] ... [/LOG]` | Writes evaluated content to the server log; no documented attributes. |
+
+Predefined execution-context names include `currentDateTime`, `defaultLocale`, `currentLocale`, and `requestId`. Other values may be supplied by the calling service.
+
+## Variables and value functions
+
+| Construct | Form | Key rules |
+| --- | --- | --- |
+| VAR | `[VAR name="..." type="..." unique="..." ] ... [/VAR]` | `name` identifies a context variable. Documented types: `string`, `boolean`, `number`, `date`, `object`, `iterable`. `unique` defaults to `false` and is meaningful only for `iterable`. |
+| VALUE_OF | `[VALUE_OF varname="..." index="..." /]` | Emits a variable value. `index` identifies the position for an iterable and may name another variable. |
+| SIZE_OF | `[SIZE_OF varname="..." /]` | Returns iterable size, otherwise `0`. |
+| IS_EMPTY | `[IS_EMPTY varname="..." /]` | Tests missing, null, blank string, empty iterable, or empty map according to the documented rules. |
+| IS_NOT_EMPTY | `[IS_NOT_EMPTY varname="..." /]` | Logical negative of `IS_EMPTY`. |
+| CONTAINS | `[CONTAINS varname="..." value="..." /]` or `[CONTAINS varname="..."] ... [/CONTAINS]` | Supports string or iterable input. Attribute `value` and body are alternatives; `value` has priority when both exist. |
+
+Do not require `type` merely because current examples commonly include it: some examples omit it. Resolve requiredness from the parser relations or runtime contract before emitting an error.
+
+## Transformations
+
+| Construct | Form | Key rules |
+| --- | --- | --- |
+| FORMAT | `[FORMAT varname="..." type="..." /]` or `[FORMAT type="..." pattern="..."] ... [/FORMAT]` | `varname` takes precedence over the body. Documented formatting covers number and date families; number modes include currency, integer, double, percent, and generic. |
+| SPLIT | `[SPLIT regex="..." emptyIfBlank="true|false"] ... [/SPLIT]` | Applies a valid regular expression to a string result. `emptyIfBlank` defaults to `false`. An invalid regex is a runtime parsing error and can be checked statically only for a literal. |
+| TRIM | `[TRIM varname="..." /]` or `[TRIM] ... [/TRIM]` | Trims a named variable or evaluated body. A found variable takes precedence; otherwise the body is evaluated. |
+
+## Control structures
+
+- IF shape: `[IF] [CONDITION] ... [/CONDITION] [THEN] ... [/THEN] { [ELSE IF] ... [/ELSE IF] } [ELSE] ... [/ELSE] [/IF]`.
+- `CONDITION` must evaluate to boolean. There may be multiple `ELSE IF` blocks and at most one final `ELSE`; both are optional.
+- WHILE shape: `[WHILE ...] [CONDITION] ... [/CONDITION] [DO] ... [/DO] [/WHILE]`.
+- The WHILE section documents an optional `threshold` with default and maximum `32766`, but also contains a sentence saying the tag has no attributes. Treat this as an explicit documentation inconsistency.
+- FOR shape: `[FOR varName="item" iterable="items"] ... [/FOR]`. The loop variable exists during the loop and is removed afterward.
+- The FOR prose depends on both `varName` and `iterable`, but the generated relationship only shows a generic attribute node. Treat formal requiredness, attribute-name casing, duplicate/unknown attributes, null or non-iterable input, and shadowing as unresolved until confirmed against the runtime grammar.
+
+## Code blocks
+
+| Kind | Form | Meaning |
+| --- | --- | --- |
+| Assignment | `[% target = expression; ... %]` | Mutates variables; documented statements end with `;`. |
+| Evaluation | `[% expression %]` | Evaluates and returns a result, often boolean in `CONDITION`. |
+| Output | `[%= variable %]` | Writes a context variable; a missing variable is a runtime error. |
+
+The expression language examples use assignment, equality/comparison, arithmetic, boolean operators, strings, numbers, booleans, and `null`. The PDFs do not provide a complete operator-precedence grammar; avoid inventing one without a runtime grammar source.
+
+## ElixForms TAG payloads
+
+`SCHEMAID` is positional:
+
+- position 0: plugin name `SCHEMAID`;
+- position 1: schema ID;
+- position 2: column reference;
+- position 3: `IUQOID`;
+- position 4: line terminator, with a space selecting the default;
+- position 5: column separator;
+- position 6: default value;
+- position 7: date/time formatter;
+- position 8: language (documented as currently unused).
+
+Positions 0 through 4 must be present. Preserve commas for omitted intermediate optional positions.
+
+`GETVALUEBYTAG` is positional:
+
+- position 0: plugin name `GETVALUEBYTAG`;
+- position 1: tag name;
+- position 2: lookup type: `REQUEST`, `MODULE`, or `USER_PROFILE`;
+- position 3: `IUQOID`;
+- position 4: weight/order option;
+- position 5: column separator;
+- position 6: format string.
+
+The weight list in the prose and the final syntax example differ in spelling and membership (`UPDATED_FIST`/`UPDATED_FIRST`, `UPDATE_LAST`/`UPDATED_LAST`, and `CONCAT`). Treat exact validation as ambiguous until confirmed against the TAG runtime.
+
+## Known implementation delta at skill creation
+
+The current TypeScript tokenizer/parser does not yet model all documented constructs. Notably absent or incomplete are `LOG`, `FOR`, `IS_EMPTY`, `IS_NOT_EMPTY`, case-insensitive spellings, structural ordering inside IF/WHILE, attribute validation beyond VAR type, and full expression parsing. Re-check source before relying on this list because it is expected to shrink as the linter evolves.
@@ -0,0 +1,56 @@
+---
+name: verify-eftl-linter
+description: Verify the EFTL linter and VS Code language support with repository checks, tokenizer/parser cases, diagnostic range review, TextMate grammar checks, snippets, examples, and LSP behavior. Use after EFTL implementation changes, while diagnosing regressions, when adding tests, or before declaring a linter feature complete.
+---
+
+# Verify EFTL Linter
+
+Validate the core language implementation first, then the LSP and editor surfaces affected by the change. Read [references/verification-matrix.md](references/verification-matrix.md) when selecting cases.
+
+## Run the baseline check
+
+From the repository root, run:
+
+```powershell
+node .agents/skills/verify-eftl-linter/scripts/check-repository.mjs
+```
+
+The script parses contributed JSON, checks referenced files, compiles TypeScript, runs an existing test script when present, and smoke-tests the compiled tokenizer/parser. Pass `--lint` only when the repository has a working ESLint configuration.
+
+Treat this script as a floor, not as feature proof.
+
+## Build a focused test set
+
+For each changed rule, include:
+
+- the smallest valid form;
+- every optional form or bodyless/block alternative;
+- a representative nested form;
+- missing, duplicated, invalid, and out-of-order attributes or children;
+- mismatched, missing, and unexpected closing delimiters;
+- incomplete input at end of file for live-edit recovery;
+- lowercase and mixed-case spellings when the runtime is case-insensitive;
+- LF, CRLF, multiline, and non-ASCII range cases;
+- a regression case from `examples/` or the reported bug.
+
+Assert token types and spans separately from parser errors. Assert diagnostic code, severity, message, and exact LSP range; do not rely only on snapshot text.
+
+## Verify by layer
+
+1. Run tokenizer/parser tests without VS Code.
+2. Run server-level tests for config, diagnostic caps, declaration/reference scope, and range conversion.
+3. Parse all JSON contribution files and inspect TextMate captures for the changed construct.
+4. Open a representative `.eftl` file in an Extension Development Host when highlighting, folding, snippets, activation, or navigation changed.
+5. Confirm valid examples produce no diagnostics and each invalid example produces the intended minimal set.
+
+When no automated harness exists for a required layer, add one if it is stable and proportionate; otherwise document the exact manual check and result.
+
+## Interpret failures
+
+- A compilation failure blocks all further verification.
+- A failure caused by a known baseline issue must be reproduced on the pre-change baseline before being labeled unrelated.
+- Diagnostic cascades usually indicate parser recovery problems; fix recovery instead of weakening assertions.
+- TextMate highlighting success does not prove tokenizer/parser support, and parser success does not prove editor integration.
+- Never declare `npm run lint` successful in the current baseline: ESLint 9 has no flat configuration until the repository adds one.
+
+Report commands, pass/fail counts, skipped layers, and residual risks.
@@ -0,0 +1,4 @@
+interface:
+  display_name: "Verify EFTL Linter"
+  short_description: "Test EFTL diagnostics and editor support"
+  default_prompt: "Use $verify-eftl-linter to validate this EFTL linter change end to end."
@@ -0,0 +1,54 @@
+# EFTL verification matrix
+
+## Core matrix
+
+| Area | Positive cases | Negative and recovery cases |
+| --- | --- | --- |
+| Roots | one and multiple EFTL fragments with surrounding user text | close without open, missing close, nested roots if forbidden by confirmed grammar |
+| Blocks | correctly nested VAR, IF, WHILE, functions, expressions, comments | crossing closes, incomplete close, unexpected close, unfinished or unknown `[` construct; assert termination |
+| Attributes | documented required and optional attributes in varied order | missing required, duplicate, unknown, unquoted, invalid enum/boolean/number |
+| IF | condition + then, repeated else-if, optional final else | missing condition/then, else before then, duplicate else, branch after else |
+| WHILE | condition + do, confirmed threshold boundaries | missing child, reversed order, invalid threshold, incomplete body |
+| FOR | iterable variable and scoped loop variable | missing attributes, unknown iterable when statically knowable, reference after scope |
+| Expressions | assignment, evaluation, output, supported literals/operators | unterminated block, malformed operator, missing semicolon where required |
+| Functions | bodyless and block variants, precedence rules | mutually exclusive inputs, wrong body shape, invalid literal regex |
+| TAG payload | minimal and full positional forms | missing mandatory position, invalid literal enum, preserved empty positions |
+| Variables | declaration, reference, shadowing policy, iterable index | invalid type, duplicate declaration policy, undefined reference, invalid index type |
+
+For every tag keyword, test the exact boundary after its name so a longer unknown name such as `[VARIABLE]` is not classified as `[VAR ...]`.
+
+Only assert rules confirmed by the PDFs or runtime grammar. Keep ambiguous rules as pending cases rather than opinionated errors.
+
+## Range matrix
+
+Exercise each diagnostic at:
+
+- first character and end of file;
+- after ASCII and non-ASCII text;
+- on a later line with LF and CRLF;
+- inside a multiline token or comment;
+- after an incremental edit that changes preceding line lengths.
+
+LSP positions are zero-based UTF-16 line/character pairs. Prefer storing source offsets internally and converting with the current `TextDocument` at publication time.
+
+## Editor matrix
+
+When syntax changes, inspect:
+
+- language activation and file association;
+- TextMate scope on open name, close name, attributes, strings, and delimiters;
+- bracket matching and auto-closing;
+- folding start/end markers;
+- snippet insertion and tab stops;
+- go-to-definition selection range;
+- consistency of examples and README syntax.
+
+## Release evidence
+
+Record:
+
+1. exact commands executed;
+2. automated pass/fail totals;
+3. Extension Development Host scenarios checked;
+4. known baseline failures reproduced independently;
+5. unverified runtime semantics or ambiguous documentation.
@@ -0,0 +1,117 @@
+#!/usr/bin/env node
+
+import { spawnSync } from 'node:child_process';
+import { createRequire } from 'node:module';
+import { existsSync, readFileSync } from 'node:fs';
+import { resolve } from 'node:path';
+
+const args = process.argv.slice(2);
+const includeLint = args.includes('--lint');
+const positional = args.filter(arg => !arg.startsWith('--'));
+const root = resolve(positional[0] ?? process.cwd());
+const require = createRequire(import.meta.url);
+
+function fail(message) {
+    throw new Error(message);
+}
+
+function readJson(relativePath) {
+    const absolutePath = resolve(root, relativePath);
+    if (!existsSync(absolutePath)) {
+        fail(`Missing ${relativePath}`);
+    }
+    try {
+        return JSON.parse(readFileSync(absolutePath, 'utf8'));
+    } catch (error) {
+        fail(`Invalid JSON in ${relativePath}: ${error.message}`);
+    }
+}
+
+function runNpm(script) {
+    const windows = process.platform === 'win32';
+    const command = windows ? (process.env.ComSpec ?? 'cmd.exe') : 'npm';
+    const commandArgs = windows ? ['/d', '/s', '/c', `npm run ${script}`] : ['run', script];
+    const result = spawnSync(command, commandArgs, {
+        cwd: root,
+        stdio: 'inherit'
+    });
+    if (result.error) {
+        fail(`Could not run npm script ${script}: ${result.error.message}`);
+    }
+    if (result.status !== 0) {
+        fail(`npm run ${script} exited with ${result.status}`);
+    }
+}
+
+function assertContributionPaths(packageJson) {
+    const paths = [packageJson.main];
+    for (const language of packageJson.contributes?.languages ?? []) {
+        paths.push(language.configuration);
+    }
+    for (const grammar of packageJson.contributes?.grammars ?? []) {
+        paths.push(grammar.path);
+    }
+    for (const snippet of packageJson.contributes?.snippets ?? []) {
+        paths.push(snippet.path);
+    }
+    for (const relativePath of paths.filter(Boolean)) {
+        if (!existsSync(resolve(root, relativePath))) {
+            fail(`package.json references missing path ${relativePath}`);
+        }
+    }
+}
+
+function smokeTestCore() {
+    const tokenizerPath = resolve(root, 'out/tokenizer.js');
+    const parserPath = resolve(root, 'out/parser.js');
+    if (!existsSync(tokenizerPath) || !existsSync(parserPath)) {
+        fail('Compiled tokenizer/parser output is missing');
+    }
+
+    const { EftlTokenizer } = require(tokenizerPath);
+    const { EftlParser } = require(parserPath);
+    const sample = readFileSync(resolve(root, 'examples/sample.eftl'), 'utf8');
+    const sampleResult = new EftlTokenizer(sample).tokenize();
+    const sampleParse = new EftlParser(sampleResult.tokens).parse();
+    if (sampleResult.errors.length || sampleParse.errors.length) {
+        fail(`sample.eftl is not accepted: ${sampleResult.errors.length} tokenizer and ${sampleParse.errors.length} parser errors`);
+    }
+
+    const mismatched = new EftlTokenizer('[EFTL][IF][/EFTL]').tokenize();
+    if (new EftlParser(mismatched.tokens).parse().errors.length === 0) {
+        fail('Parser smoke case did not reject mismatched nesting');
+    }
+
+    const comment = new EftlTokenizer('[EFTL][!-- unfinished').tokenize();
+    if (comment.errors.length === 0) {
+        fail('Tokenizer smoke case did not reject an unterminated comment');
+    }
+}
+
+try {
+    const packageJson = readJson('package.json');
+    readJson('language-configuration.json');
+    readJson('syntaxes/eftl.tmLanguage.json');
+    readJson('snippets/eftl.json');
+
+    runNpm('compile');
+    assertContributionPaths(packageJson);
+    smokeTestCore();
+
+    if (packageJson.scripts?.test) {
+        runNpm('test');
+    } else {
+        console.warn('WARN: package.json has no test script; only smoke cases were run.');
+    }
+
+    if (includeLint) {
+        runNpm('lint');
+    } else if (packageJson.scripts?.lint) {
+        console.warn('WARN: lint was skipped; pass --lint after configuring ESLint.');
+    }
+
+    console.log('EFTL repository baseline checks passed.');
+} catch (error) {
+    console.error(`FAIL: ${error.message}`);
+    process.exitCode = 1;
+}
@@ -16,7 +16,7 @@ Syntax highlighting and language support for **EFTL (ElixForms Template Language
 - `[EFTL]...[/EFTL]` - Main EFTL block
 - `[VAR name="..." type="..."]...[/VAR]` - Variable declarations
 - `[IF]...[/IF]` - Conditionals
- `[WHILE]...[/WHILE]` - Loops
+- `[WHILE threshold="..."]...[/WHILE]` - Loops with an optional string threshold

 ### Expressions

@@ -40,7 +40,7 @@ Syntax highlighting and language support for **EFTL (ElixForms Template Language
 ### Control Structures

 - `[IF]`, `[CONDITION]`, `[THEN]`, `[ELSE]`, `[ELSE IF]`
- `[WHILE]`, `[DO]`
+- `[WHILE]`, `[WHILE threshold="..."]`, `[DO]`

 ### Comments

@@ -78,7 +78,7 @@ To make the extension available in your VS Code without packaging it as a VSIX:

    ```powershell
    # PowerShell
-    New-Item -ItemType Junction -Path "$env:USERPROFILE\.vscode\extensions\eftl-language" -Value "D:\__Git\vscode-eftl-language"
+    New-Item -ItemType Junction -Path "$env:USERPROFILE\.vscode\extensions\eftl-language" -Value "$(pwd)"
    ```

    *Note: Replace the value path with the actual absolute path to your project folder if it differs.*
@@ -105,25 +105,26 @@ If you want to debug the extension or the language server:

 ## Snippets

-| Prefix      | Description            |
-| ----------- | ---------------------- |
-| `eftl`      | EFTL block with header |
-| `var`       | Variable declaration   |
-| `vartag`    | Variable from TAG      |
-| `varschema` | Variable from SCHEMAID |
-| `if`        | IF statement           |
-| `ifelse`    | IF-ELSE statement      |
-| `while`     | WHILE loop             |
-| `split`     | Split function         |
-| `trim`      | Trim function          |
-| `valueof`   | VALUE_OF function      |
-| `sizeof`    | SIZE_OF function       |
-| `contains`  | CONTAINS function      |
-| `formatnum` | Format number          |
-| `tag`       | GETVALUEBYTAG tag      |
-| `schema`    | SCHEMAID tag           |
-| `out`       | Output expression      |
-| `comment`   | Comment block          |
+| Prefix           | Description               |
+| ---------------- | ------------------------- |
+| `eftl`           | EFTL block with header    |
+| `var`            | Variable declaration      |
+| `vartag`         | Variable from TAG         |
+| `varschema`      | Variable from SCHEMAID    |
+| `if`             | IF statement              |
+| `ifelse`         | IF-ELSE statement         |
+| `while`          | WHILE loop                |
+| `whilethreshold` | WHILE loop with threshold |
+| `split`          | Split function            |
+| `trim`           | Trim function             |
+| `valueof`        | VALUE_OF function         |
+| `sizeof`         | SIZE_OF function          |
+| `contains`       | CONTAINS function         |
+| `formatnum`      | Format number             |
+| `tag`            | GETVALUEBYTAG tag         |
+| `schema`         | SCHEMAID tag              |
+| `out`            | Output expression         |
+| `comment`        | Comment block             |

 ## Example

@@ -0,0 +1,20 @@
+[!-- Example EFTL file for testing syntax highlighting --]
+[EFTL][HEADER name="trimDocument" value="true" type="boolean" /]
+[VAR name="isRichiedenteInPartecipanti" type="string"][% isRichiedenteInPartecipanti = "Utente non ammesso!"; %][/VAR]
+[VAR name="partecipanti" type="string"][TAG]GETVALUEBYTAG,CONTRATTO_PARTECIPANTI,REQUEST,IUQOID[/TAG][/VAR]
+[VAR name="nominativoUtente" type="string"][TAG]GETVALUEBYTAG,RICHIEDENTE_NOMINATIVO,REQUEST,IUQOID[/TAG][/VAR]
+
+[!-- Check if user is in participants list --]
+[% nominativoUtente = " " + nominativoUtente + ", CF: "; %]
+
+[IF]
+    [CONDITION][CONTAINS varname="partecipanti"][VALUE_OF varname="nominativoUtente" /][/CONTAINS][/CONDITION]
+    [THEN][% isRichiedenteInPartecipanti = ""; %][/THEN]
+    [ELSE IF]
+        [CONDITION][% partecipanti == "" %][/CONDITION]
+    [THEN][% isRichiedenteInPartecipanti = "Nessun partecipante trovato"; %][/THEN]
+    [/ELSE IF]
+[/IF]
+
+[%= isRichiedenteInPartecipanti %]
+[/EFTL]
@@ -1,27 +1,4 @@
-[!-- Example EFTL file for testing syntax highlighting --]
-
-[EFTL][HEADER name="trimDocument" value="true" type="boolean" /]
-[VAR name="isRichiedenteInPartecipanti" type="string"][% isRichiedenteInPartecipanti = "Utente non ammesso!"; %][/VAR]
-[VAR name="partecipanti" type="string"][TAG]GETVALUEBYTAG,CONTRATTO_PARTECIPANTI,REQUEST,IUQOID[/TAG][/VAR]
-[VAR name="nominativoUtente" type="string"][TAG]GETVALUEBYTAG,RICHIEDENTE_NOMINATIVO,REQUEST,IUQOID[/TAG][/VAR]
-
-[!-- Check if user is in participants list --]
-[% nominativoUtente = " " + nominativoUtente + ", CF: "; %]
-
-[IF]
-    [CONDITION][CONTAINS varname="partecipanti"][VALUE_OF varname="nominativoUtente" /][/CONTAINS][/CONDITION]
-    [THEN][% isRichiedenteInPartecipanti = ""; %][/THEN]
-    [ELSE IF]
-        [CONDITION][% partecipanti == "" %][/CONDITION]
-    [THEN][% isRichiedenteInPartecipanti = "Nessun partecipante trovato"; %][/THEN]
-    [/ELSE IF]
-[/IF]
-
-[%= isRichiedenteInPartecipanti %]
-[/EFTL]
-
 [!-- Another example with loops and calculations --]
-
 [EFTL][HEADER name="trimDocument" value="true" type="boolean" /]
 [VAR name="IDX_NOME" type="number"][% IDX_NOME = 0; %][/VAR]
 [VAR name="IDX_SEDE" type="number"][% IDX_SEDE = 2; %][/VAR]
@@ -31,7 +8,7 @@
 [VAR name="CONTRAENTI" type="iterable"][SPLIT regex="\n"][TRIM][TAG]SCHEMAID,296,COL0004,IUQOID, , [/TAG][/TRIM][/SPLIT][/VAR]
 [VAR name="CONTRAENTI_NUM" type="number"][SIZE_OF varname="CONTRAENTI" /][/VAR]

-[WHILE]
+[WHILE threshold="99"]
    [CONDITION][% IDX < CONTRAENTI_NUM %][/CONDITION]
    [DO]
        [VAR name="CONTRAENTE" type="string"][VALUE_OF varname="CONTRAENTI" index="IDX" /][/VAR]
@@ -6,7 +6,7 @@
    ["[EFTL]", "[/EFTL]"],
    ["[VAR", "[/VAR]"],
    ["[IF]", "[/IF]"],
-    ["[WHILE]", "[/WHILE]"],
+    ["[WHILE", "[/WHILE]"],
    ["[FORMAT", "[/FORMAT]"],
    ["[CONDITION]", "[/CONDITION]"],
    ["[THEN]", "[/THEN]"],
@@ -10,10 +10,10 @@
  "categories": [
    "Programming Languages"
  ],
+  "main": "./out/extension.js",
  "activationEvents": [
    "onLanguage:eftl"
  ],
-  "main": "./out/extension.js",
  "contributes": {
    "languages": [
      {
@@ -57,6 +57,7 @@
  "scripts": {
    "vscode:prepublish": "npm run compile",
    "compile": "tsc -p ./",
+    "test": "npm run compile && node --test test/*.test.js",
    "watch": "tsc -watch -p ./",
    "lint": "eslint src --ext ts"
  },
@@ -68,7 +69,10 @@
  "devDependencies": {
    "@types/node": "^20.10.0",
    "@types/vscode": "^1.75.0",
-    "typescript": "^5.3.0"
+    "eslint": "^9.39.2",
+    "typescript": "^5.3.0",
+    "vscode-oniguruma": "^2.0.1",
+    "vscode-textmate": "^9.3.2"
  },
  "repository": {
    "type": "git",
@@ -79,6 +79,18 @@
    ],
    "description": "Create a WHILE loop"
  },
+  "While Loop with Threshold": {
+    "prefix": "whilethreshold",
+    "body": [
+      "[WHILE threshold=\"${1:99}\"]",
+      "    [CONDITION][% ${2:condition} %][/CONDITION]",
+      "    [DO]",
+      "        $0",
+      "    [/DO]",
+      "[/WHILE]"
+    ],
+    "description": "Create a WHILE loop with a threshold"
+  },
  "Split": {
    "prefix": "split",
    "body": [
@@ -17,6 +17,13 @@ export interface ParserError {
    length: number;
 }

+type IfPhase = 'condition' | 'then' | 'branches' | 'afterElse';
+
+interface IfContext {
+    token: Token;
+    phase: IfPhase;
+}
+
 /**
 * EFTL Parser - validates structure and reports errors
 */
@@ -25,6 +32,8 @@ export class EftlParser {
    private pos: number = 0;
    private errors: ParserError[] = [];
    private blockStack: { type: TokenType; token: Token }[] = [];
+    private ifContexts: IfContext[] = [];
+    private eftlRootCount: number = 0;

    constructor(tokens: Token[]) {
        this.tokens = tokens;
@@ -34,6 +43,8 @@ export class EftlParser {
        this.pos = 0;
        this.errors = [];
        this.blockStack = [];
+        this.ifContexts = [];
+        this.eftlRootCount = 0;

        while (!this.isAtEnd()) {
            this.parseTopLevel();
@@ -41,22 +52,41 @@ export class EftlParser {

        // Check for unclosed blocks
        for (const block of this.blockStack) {
+            const expectedClose = this.getMatchingClose(block.type);
            this.errors.push({
-                message: `Unclosed block: ${block.token.value}`,
+                message: `Unclosed tag: ${block.token.value} (expected ${expectedClose})`,
                line: block.token.line,
                column: block.token.column,
                length: block.token.length
            });
        }

+        if (this.eftlRootCount === 0) {
+            const eof = this.current();
+            this.errors.push({
+                message: 'Missing required [EFTL] root block',
+                line: eof.line,
+                column: eof.column,
+                length: eof.length
+            });
+        }
+
        return { errors: this.errors };
    }

    private parseTopLevel(): void {
        const token = this.current();
+        this.validateOutsideEftl(token);
+        this.validateDirectIfContent(token);

        switch (token.type) {
            case TokenType.EFTL_OPEN:
+                if (this.blockStack.some(block => block.type === TokenType.EFTL_OPEN)) {
+                    this.addError(token, 'EFTL root blocks cannot be nested');
+                } else if (this.eftlRootCount > 0) {
+                    this.addError(token, 'Only one [EFTL] root block is allowed');
+                }
+                this.eftlRootCount++;
                this.pushBlock(token);
                this.advance();
                break;
@@ -67,6 +97,14 @@ export class EftlParser {
                break;

            case TokenType.VAR_OPEN:
+                if (this.blockStack.some(block => block.type === TokenType.VAR_OPEN)) {
+                    this.errors.push({
+                        message: 'VAR tags cannot be nested inside another VAR tag',
+                        line: token.line,
+                        column: token.column,
+                        length: token.length
+                    });
+                }
                this.pushBlock(token);
                this.advance();
                break;
@@ -77,12 +115,17 @@ export class EftlParser {
                break;

            case TokenType.IF_OPEN:
+                this.ifContexts.push({ token, phase: 'condition' });
                this.pushBlock(token);
                this.advance();
                break;

            case TokenType.IF_CLOSE:
+                const closesIf = this.blockStack[this.blockStack.length - 1]?.type === TokenType.IF_OPEN;
                this.popBlock(TokenType.IF_OPEN, token);
+                if (closesIf) {
+                    this.ifContexts.pop();
+                }
                this.advance();
                break;

@@ -127,6 +170,9 @@ export class EftlParser {
                break;

            case TokenType.WHILE_OPEN:
+                if (!/^\[WHILE(?:\s+threshold\s*=\s*"[^"]*")?\s*\]$/.test(token.value)) {
+                    this.addError(token, 'WHILE accepts only the optional string attribute threshold="..."');
+                }
                this.pushBlock(token);
                this.advance();
                break;
@@ -233,6 +279,9 @@ export class EftlParser {
            case TokenType.VALUE_OF:
            case TokenType.SIZE_OF:
            case TokenType.HEADER:
+                if (!token.value.endsWith('/]')) {
+                    this.pushBlock(token);
+                }
                this.advance();
                break;

@@ -248,7 +297,91 @@ export class EftlParser {
        }
    }

+    private validateOutsideEftl(token: Token): void {
+        if (this.blockStack.some(block => block.type === TokenType.EFTL_OPEN)) {
+            return;
+        }
+
+        if (token.type === TokenType.EFTL_OPEN || token.type === TokenType.EFTL_CLOSE ||
+            token.type === TokenType.COMMENT_OPEN || token.type === TokenType.COMMENT_CLOSE ||
+            token.type === TokenType.EOF || (token.type === TokenType.TEXT && token.value.trim() === '')) {
+            return;
+        }
+
+        this.addError(token, 'Only comments and whitespace are allowed outside the [EFTL] root block');
+    }
+
+    private validateDirectIfContent(token: Token): void {
+        const parent = this.blockStack[this.blockStack.length - 1];
+        if (!parent || parent.type !== TokenType.IF_OPEN) {
+            return;
+        }
+
+        const context = this.ifContexts[this.ifContexts.length - 1];
+        if (!context || context.token !== parent.token) {
+            return;
+        }
+
+        if (token.type === TokenType.TEXT && token.value.trim() === '') {
+            return;
+        }
+
+        if (token.type === TokenType.IF_OPEN) {
+            this.addError(token, 'IF cannot be nested directly inside another IF; place it inside a branch block');
+            return;
+        }
+
+        if (context.phase === 'condition' && token.type === TokenType.CONDITION_OPEN) {
+            context.phase = 'then';
+            return;
+        }
+
+        if (context.phase === 'then' && token.type === TokenType.THEN_OPEN) {
+            context.phase = 'branches';
+            return;
+        }
+
+        if (context.phase === 'branches' && token.type === TokenType.ELSE_IF_OPEN) {
+            return;
+        }
+
+        if (context.phase === 'branches' && token.type === TokenType.ELSE_OPEN) {
+            context.phase = 'afterElse';
+            return;
+        }
+
+        if (token.type === TokenType.IF_CLOSE) {
+            if (context.phase === 'condition') {
+                this.addError(token, 'Expected [CONDITION] immediately after [IF]');
+            } else if (context.phase === 'then') {
+                this.addError(token, 'Expected [THEN] immediately after [/CONDITION]');
+            }
+            return;
+        }
+
+        const expected = context.phase === 'condition'
+            ? '[CONDITION]'
+            : context.phase === 'then'
+                ? '[THEN]'
+                : context.phase === 'branches'
+                    ? '[ELSE IF], [ELSE], or [/IF]'
+                    : '[/IF]';
+        this.addError(token, `Expected ${expected} as the next direct child of [IF], got ${token.value}`);
+    }
+
+    private addError(token: Token, message: string): void {
+        this.errors.push({
+            message,
+            line: token.line,
+            column: token.column,
+            length: token.length
+        });
+    }
+
    private pushBlock(token: Token): void {
+        if (token.value.endsWith('/]')) {
+            return; // It's self-closing, don't push to stack
+        }
        this.blockStack.push({ type: token.type, token });
    }

@@ -295,6 +428,9 @@ export class EftlParser {
            [TokenType.TRIM_OPEN]: '[/TRIM]',
            [TokenType.CONTAINS_OPEN]: '[/CONTAINS]',
            [TokenType.FORMAT_OPEN]: '[/FORMAT]',
+            [TokenType.VALUE_OF]: '[/VALUE_OF]',
+            [TokenType.SIZE_OF]: '[/SIZE_OF]',
+            [TokenType.HEADER]: '[/HEADER]',
            [TokenType.EXPR_OPEN]: '%]',
            [TokenType.EXPR_OUTPUT_OPEN]: '%]',
            [TokenType.COMMENT_OPEN]: '--]'
@@ -21,7 +21,7 @@ export enum TokenType {
    ELSE_CLOSE = 'ELSE_CLOSE',         // [/ELSE]
    ELSE_IF_OPEN = 'ELSE_IF_OPEN',     // [ELSE IF]
    ELSE_IF_CLOSE = 'ELSE_IF_CLOSE',   // [/ELSE IF]
-    WHILE_OPEN = 'WHILE_OPEN',         // [WHILE]
+    WHILE_OPEN = 'WHILE_OPEN',         // [WHILE ...]
    WHILE_CLOSE = 'WHILE_CLOSE',       // [/WHILE]
    DO_OPEN = 'DO_OPEN',               // [DO]
    DO_CLOSE = 'DO_CLOSE',             // [/DO]
@@ -192,6 +192,11 @@ export class EftlTokenizer {
                this.addToken(TokenType.WHILE_OPEN, '[WHILE]', startLine, startColumn);
                return;
            }
+            if (this.source.startsWith('[WHILE', this.pos) && /\s/.test(this.source[this.pos + '[WHILE'.length] || '')) {
+                this.match('[WHILE');
+                this.scanTagWithAttributes(TokenType.WHILE_OPEN, 'WHILE', startLine, startColumn);
+                return;
+            }
            if (this.match('[/WHILE]')) {
                this.addToken(TokenType.WHILE_CLOSE, '[/WHILE]', startLine, startColumn);
                return;
@@ -298,7 +303,7 @@ export class EftlTokenizer {

    private scanVarOpen(startLine: number, startColumn: number): void {
        let value = '[VAR';
-        // Scan until we find ] or ]
+        // Scan until we find ]
        while (this.pos < this.source.length && this.peek() !== ']') {
            value += this.advance();
        }
@@ -328,13 +333,7 @@ export class EftlTokenizer {
                break;
            }
            if (ch === ']') {
-                // Not self-closing, error
-                this.errors.push({
-                    message: `Expected self-closing tag: [${tagName} ... /]`,
-                    line: startLine,
-                    column: startColumn,
-                    length: value.length
-                });
+                // Could be block start, let parser handle it if it's supposed to be self-closing
                break;
            }
        }
@@ -94,7 +94,7 @@
        },
        {
          "name": "keyword.operator.arithmetic.eftl",
-          "match": "[+\\-*/%]"
+          "match": "([+\\-*%]|/(?!\\]))"
        },
        {
          "name": "keyword.operator.comparison.eftl",
@@ -129,45 +129,85 @@
        }
      ]
    },
-    "tags": {
+    "attributes": {
      "patterns": [
        {
-          "name": "meta.tag.data.eftl",
-          "begin": "(\\[TAG\\])",
-          "end": "(\\[/TAG\\])",
-          "beginCaptures": {
-            "1": { "name": "entity.name.tag.eftl" }
-          },
-          "endCaptures": {
-            "1": { "name": "entity.name.tag.eftl" }
-          },
-          "patterns": [
-            { "include": "#tag-content" }
-          ]
+          "comment": "name attribute",
+          "match": "\\b(name)\\b\\s*(=)\\s*(\"(.*?)\")",
+          "captures": {
+            "1": { "name": "entity.other.attribute-name.eftl" },
+            "2": { "name": "punctuation.separator.key-value.eftl" },
+            "3": { "name": "string.quoted.double.eftl" },
+            "4": { "name": "variable.other.eftl" }
+          }
+        },
+        {
+          "comment": "type attribute",
+          "match": "\\b(type)\\b\\s*(=)\\s*(\"(string|boolean|number|date|object|iterable)\")",
+          "captures": {
+            "1": { "name": "entity.other.attribute-name.eftl" },
+            "2": { "name": "punctuation.separator.key-value.eftl" },
+            "3": { "name": "string.quoted.double.eftl" },
+            "4": { "name": "support.type.primitive.eftl" }
+          }
+        },
+        {
+          "comment": "generic attribute",
+          "match": "\\b([a-zA-Z_][a-zA-Z0-9_-]*)\\b\\s*(=)\\s*(\"(.*?)\")",
+          "captures": {
+            "1": { "name": "entity.other.attribute-name.eftl" },
+            "2": { "name": "punctuation.separator.key-value.eftl" },
+            "3": { "name": "string.quoted.double.eftl" }
+          }
        }
      ]
    },
-    "tag-content": {
+    "tag-open-context": {
+      "patterns": [
+        { "include": "#attributes" },
+        { "include": "#expressions" },
+        { "include": "#functions" },
+        { "include": "#tags" }
+      ]
+    },
+    "tags": {
      "patterns": [
        {
-          "name": "support.function.builtin.eftl",
-          "match": "\\b(GETVALUEBYTAG|SCHEMAID|PLUGIN_DOM_PARAM)\\b"
+          "name": "meta.tag.block.eftl",
+          "begin": "(\\[)(TAG)(?:\\s+[^\\]]*?)?(?<!/)(\\])",
+          "end": "(\\[/)(TAG)(\\])",
+          "beginCaptures": {
+            "1": { "name": "entity.name.tag.eftl" },
+            "2": { "name": "entity.name.tag.eftl" },
+            "3": { "name": "entity.name.tag.eftl" }
+          },
+          "endCaptures": {
+            "1": { "name": "entity.name.tag.eftl" },
+            "2": { "name": "entity.name.tag.eftl" },
+            "3": { "name": "entity.name.tag.eftl" }
+          },
+          "patterns": [
+            { "include": "#expressions" },
+            { "include": "#tags" },
+            { "include": "#functions" },
+            { "include": "#control-structures" },
+            { "include": "#variables" }
+          ]
        },
        {
-          "name": "variable.parameter.eftl",
-          "match": "\\b(REQUEST|MODULE|IUQOID)\\b"
-        },
-        {
-          "name": "constant.numeric.eftl",
-          "match": "\\b[0-9]+\\b"
-        },
-        {
-          "name": "variable.other.eftl",
-          "match": "\\b(COL[0-9]+)\\b"
-        },
-        {
-          "name": "punctuation.separator.eftl",
-          "match": ","
+          "name": "meta.tag.eftl",
+          "begin": "(\\[)(TAG)\\b",
+          "end": "(/\\s*\\]|(?<!/)\\])",
+          "beginCaptures": {
+            "1": { "name": "entity.name.tag.eftl" },
+            "2": { "name": "entity.name.tag.eftl" }
+          },
+          "endCaptures": {
+            "0": { "name": "entity.name.tag.eftl" }
+          },
+          "patterns": [
+            { "include": "#tag-open-context" }
+          ]
        }
      ]
    },
@@ -175,14 +215,18 @@
      "patterns": [
        {
          "name": "meta.header.eftl",
-          "match": "(\\[HEADER)\\s+(name=\"[^\"]*\")\\s+(value=\"[^\"]*\")\\s+(type=\"[^\"]*\")\\s*(/\\])",
-          "captures": {
+          "begin": "(\\[)(HEADER)\\b",
+          "end": "(/\\s*\\]|(?<!/)\\])",
+          "beginCaptures": {
            "1": { "name": "keyword.other.header.eftl" },
-            "2": { "name": "entity.other.attribute-name.eftl" },
-            "3": { "name": "string.quoted.double.eftl" },
-            "4": { "name": "entity.other.attribute-name.eftl" },
-            "5": { "name": "punctuation.definition.tag.end.eftl" }
-          }
+            "2": { "name": "keyword.other.header.eftl" }
+          },
+          "endCaptures": {
+            "0": { "name": "keyword.other.header.eftl" }
+          },
+          "patterns": [
+            { "include": "#tag-open-context" }
+          ]
        }
      ]
    },
@@ -192,155 +236,103 @@
          "name": "keyword.control.conditional.eftl",
          "match": "\\[(IF|ELSE IF|ELSE|/IF|CONDITION|/CONDITION|THEN|/THEN|/ELSE IF|/ELSE)\\]"
        },
+        {
+          "name": "meta.control.loop.eftl",
+          "begin": "(\\[)(WHILE)\\b",
+          "end": "(\\])",
+          "beginCaptures": {
+            "1": { "name": "keyword.control.loop.eftl" },
+            "2": { "name": "keyword.control.loop.eftl" }
+          },
+          "endCaptures": {
+            "1": { "name": "keyword.control.loop.eftl" }
+          },
+          "patterns": [
+            { "include": "#attributes" }
+          ]
+        },
        {
          "name": "keyword.control.loop.eftl",
-          "match": "\\[(WHILE|/WHILE|DO|/DO)\\]"
+          "match": "\\[(/WHILE|DO|/DO)\\]"
        }
      ]
    },
    "variables": {
      "patterns": [
        {
-          "name": "meta.variable.declaration.eftl",
-          "begin": "(\\[VAR)",
-          "end": "(\\[/VAR\\])",
+          "name": "meta.variable.block.eftl",
+          "begin": "(\\[)(VAR)(?:\\s+[^\\]]*?)?(?<!/)(\\])",
+          "end": "(\\[/)(VAR)(\\])",
          "beginCaptures": {
-            "1": { "name": "storage.type.variable.eftl" }
+            "1": { "name": "storage.type.variable.eftl" },
+            "2": { "name": "storage.type.variable.eftl" },
+            "3": { "name": "storage.type.variable.eftl" }
          },
          "endCaptures": {
-            "1": { "name": "storage.type.variable.eftl" }
+            "1": { "name": "storage.type.variable.eftl" },
+            "2": { "name": "storage.type.variable.eftl" },
+            "3": { "name": "storage.type.variable.eftl" }
          },
          "patterns": [
-            { "include": "#var-attributes" },
            { "include": "#expressions" },
            { "include": "#functions" },
-            { "include": "#tags" }
+            { "include": "#tags" },
+            { "include": "#control-structures" }
          ]
-        }
-      ]
-    },
-    "var-attributes": {
-      "patterns": [
-        {
-          "comment": "name attribute - variable name gets special color",
-          "match": "(name)(=)(\"([^\"]*)\")",
-          "captures": {
-            "1": { "name": "entity.other.attribute-name.name.eftl" },
-            "2": { "name": "punctuation.separator.key-value.eftl" },
-            "3": { "name": "string.quoted.double.eftl" },
-            "4": { "name": "variable.other.declaration.eftl" }
-          }
        },
        {
-          "comment": "type attribute with valid type values",
-          "match": "(type)(=)(\"(string|boolean|number|date|object|iterable)\")",
-          "captures": {
-            "1": { "name": "entity.other.attribute-name.type.eftl" },
-            "2": { "name": "punctuation.separator.key-value.eftl" },
-            "3": { "name": "string.quoted.double.eftl" },
-            "4": { "name": "support.type.primitive.eftl" }
-          }
-        },
-        {
-          "comment": "unique attribute - boolean value",
-          "match": "(unique)(=)(\"(true|false)\")",
-          "captures": {
-            "1": { "name": "entity.other.attribute-name.unique.eftl" },
-            "2": { "name": "punctuation.separator.key-value.eftl" },
-            "3": { "name": "string.quoted.double.eftl" },
-            "4": { "name": "constant.language.boolean.eftl" }
-          }
-        },
-        {
-          "comment": "closing bracket of VAR open tag",
-          "match": "\\]",
-          "name": "punctuation.definition.tag.end.eftl"
+          "name": "meta.variable.declaration.eftl",
+          "begin": "(\\[)(VAR)\\b",
+          "end": "(/\\s*\\]|(?<!/)\\])",
+          "beginCaptures": {
+            "1": { "name": "storage.type.variable.eftl" },
+            "2": { "name": "storage.type.variable.eftl" }
+          },
+          "endCaptures": {
+            "0": { "name": "storage.type.variable.eftl" }
+          },
+          "patterns": [
+            { "include": "#tag-open-context" }
+          ]
        }
      ]
    },
    "functions": {
      "patterns": [
        {
-          "name": "meta.function.split.eftl",
-          "begin": "(\\[SPLIT)\\s+(regex=\"[^\"]*\")(\\])?",
-          "end": "(\\[/SPLIT\\])",
+          "name": "meta.function.block.eftl",
+          "begin": "(\\[)(VALUE_OF|SIZE_OF|SPLIT|FORMAT|CONTAINS|TRIM)(?:\\s+[^\\]]*?)?(?<!/)(\\])",
+          "end": "(\\[/)(VALUE_OF|SIZE_OF|SPLIT|FORMAT|CONTAINS|TRIM)(\\])",
          "beginCaptures": {
            "1": { "name": "support.function.eftl" },
-            "2": { "name": "string.regexp.eftl" }
+            "2": { "name": "support.function.eftl" },
+            "3": { "name": "support.function.eftl" }
          },
          "endCaptures": {
-            "1": { "name": "support.function.eftl" }
+            "1": { "name": "support.function.eftl" },
+            "2": { "name": "support.function.eftl" },
+            "3": { "name": "support.function.eftl" }
          },
          "patterns": [
            { "include": "#expressions" },
+            { "include": "#functions" },
            { "include": "#tags" },
-            { "include": "#functions" }
+            { "include": "#control-structures" }
          ]
        },
        {
-          "name": "meta.function.trim.eftl",
-          "begin": "(\\[TRIM\\])",
-          "end": "(\\[/TRIM\\])",
-          "beginCaptures": {
-            "1": { "name": "support.function.eftl" }
-          },
-          "endCaptures": {
-            "1": { "name": "support.function.eftl" }
-          },
-          "patterns": [
-            { "include": "#expressions" },
-            { "include": "#tags" },
-            { "include": "#functions" }
-          ]
-        },
-        {
-          "name": "meta.function.contains.eftl",
-          "begin": "(\\[CONTAINS)\\s+(varname=\"[^\"]*\")(\\])?",
-          "end": "(\\[/CONTAINS\\])",
+          "name": "meta.function.eftl",
+          "begin": "(\\[)(VALUE_OF|SIZE_OF|SPLIT|FORMAT|CONTAINS|TRIM)\\b",
+          "end": "(/\\s*\\]|(?<!/)\\])",
          "beginCaptures": {
            "1": { "name": "support.function.eftl" },
-            "2": { "name": "entity.other.attribute-name.eftl" }
+            "2": { "name": "support.function.eftl" }
          },
          "endCaptures": {
-            "1": { "name": "support.function.eftl" }
+            "0": { "name": "support.function.eftl" }
          },
          "patterns": [
-            { "include": "#functions" }
-          ]
-        },
-        {
-          "name": "meta.function.value-of.eftl",
-          "match": "(\\[VALUE_OF)\\s+(varname=\"[^\"]*\")\\s*(index=\"?[^\"]*\"?)?\\s*(/\\])",
-          "captures": {
-            "1": { "name": "support.function.eftl" },
-            "2": { "name": "entity.other.attribute-name.eftl" },
-            "3": { "name": "entity.other.attribute-name.eftl" },
-            "4": { "name": "punctuation.definition.tag.end.eftl" }
-          }
-        },
-        {
-          "name": "meta.function.size-of.eftl",
-          "match": "(\\[SIZE_OF)\\s+(varname=\"[^\"]*\")\\s*(/\\])",
-          "captures": {
-            "1": { "name": "support.function.eftl" },
-            "2": { "name": "entity.other.attribute-name.eftl" },
-            "3": { "name": "punctuation.definition.tag.end.eftl" }
-          }
-        },
-        {
-          "name": "meta.function.format.eftl",
-          "begin": "(\\[FORMAT)\\s+(type=\"[^\"]*\")\\s*(pattern=\"[^\"]*\")?(\\])?",
-          "end": "(\\[/FORMAT\\])",
-          "beginCaptures": {
-            "1": { "name": "support.function.eftl" },
-            "2": { "name": "entity.other.attribute-name.eftl" },
-            "3": { "name": "string.other.pattern.eftl" }
-          },
-          "endCaptures": {
-            "1": { "name": "support.function.eftl" }
-          },
-          "patterns": [
-            { "include": "#expressions" }
+            { "include": "#tag-open-context" }
          ]
        }
      ]
@@ -0,0 +1,77 @@
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const test = require('node:test');
+
+const oniguruma = require('vscode-oniguruma');
+const textmate = require('vscode-textmate');
+
+async function loadGrammar() {
+    await oniguruma.loadWASM(fs.readFileSync(require.resolve('vscode-oniguruma/release/onig.wasm')).buffer);
+
+    const registry = new textmate.Registry({
+        onigLib: Promise.resolve({
+            createOnigScanner: patterns => new oniguruma.OnigScanner(patterns),
+            createOnigString: value => new oniguruma.OnigString(value)
+        }),
+        loadGrammar: async scopeName => scopeName === 'source.eftl'
+            ? textmate.parseRawGrammar(
+                fs.readFileSync('syntaxes/eftl.tmLanguage.json', 'utf8'),
+                'eftl.tmLanguage.json'
+            )
+            : null
+    });
+
+    return registry.loadGrammar('source.eftl');
+}
+
+test('highlights nested VAR blocks, self-closing functions, and WHILE attributes', async () => {
+    const grammar = await loadGrammar();
+    const source = [
+        '[EFTL]',
+        '[VAR name="count" type="number"][SIZE_OF varname="items" /][/VAR]',
+        '[WHILE threshold="99"][CONDITION][% count > 0 %][/CONDITION][DO]',
+        '[VAR name="current" type="string"][VALUE_OF varname="items" index="count" /][/VAR]',
+        '[/DO][/WHILE]',
+        '[IF][CONDITION][% count == 0 %][/CONDITION][THEN]',
+        '[VAR name="empty" type="boolean"][% empty = true; %][/VAR]',
+        '[/THEN][/IF]',
+        '[/EFTL]'
+    ];
+    let ruleStack = textmate.INITIAL;
+    const variableTokens = [];
+    const selfClosingFunctionTokens = [];
+    const thresholdTokens = [];
+    const thresholdValueTokens = [];
+    const whileTokens = [];
+
+    for (const line of source) {
+        const result = grammar.tokenizeLine(line, ruleStack);
+        ruleStack = result.ruleStack;
+
+        for (const token of result.tokens) {
+            const text = line.slice(token.startIndex, token.endIndex);
+            if (text === 'VAR' && line[token.startIndex - 1] !== '/') variableTokens.push(token);
+            if (text === 'SIZE_OF' || text === 'VALUE_OF') selfClosingFunctionTokens.push(token);
+            if (text === 'threshold') thresholdTokens.push(token);
+            if (text === '"99"') thresholdValueTokens.push(token);
+            if (text === 'WHILE') whileTokens.push(token);
+        }
+    }
+
+    assert.equal(variableTokens.length, 3);
+    for (const token of variableTokens) {
+        assert.ok(token.scopes.includes('storage.type.variable.eftl'));
+    }
+
+    for (const token of selfClosingFunctionTokens) {
+        assert.ok(token.scopes.includes('meta.function.eftl'));
+        assert.ok(!token.scopes.includes('meta.function.block.eftl'));
+    }
+
+    assert.equal(thresholdTokens.length, 1);
+    assert.ok(thresholdTokens[0].scopes.includes('entity.other.attribute-name.eftl'));
+    assert.equal(thresholdValueTokens.length, 1);
+    assert.ok(thresholdValueTokens[0].scopes.includes('string.quoted.double.eftl'));
+    assert.equal(whileTokens.length, 1);
+    assert.ok(whileTokens[0].scopes.includes('keyword.control.loop.eftl'));
+});
@@ -0,0 +1,197 @@
+const assert = require('node:assert/strict');
+const test = require('node:test');
+
+const { EftlParser } = require('../out/parser');
+const { EftlTokenizer } = require('../out/tokenizer');
+
+function parse(source) {
+    const { tokens, errors: tokenizerErrors } = new EftlTokenizer(source).tokenize();
+    assert.deepEqual(tokenizerErrors, []);
+    return new EftlParser(tokens).parse().errors;
+}
+
+function messages(source) {
+    return parse(source).map(error => error.message);
+}
+
+test('accepts a correctly closed VAR tag', () => {
+    assert.deepEqual(parse('[EFTL][VAR][/VAR][/EFTL]'), []);
+});
+
+test('accepts sibling VAR tags', () => {
+    assert.deepEqual(parse('[EFTL][VAR][/VAR][VAR][/VAR][/EFTL]'), []);
+});
+
+test('reports a VAR nested directly inside another VAR', () => {
+    assert.deepEqual(parse('[EFTL][VAR][VAR][/VAR][/VAR][/EFTL]'), [{
+        message: 'VAR tags cannot be nested inside another VAR tag',
+        line: 1,
+        column: 12,
+        length: 5
+    }]);
+});
+
+test('reports a VAR nested indirectly inside another VAR', () => {
+    assert.deepEqual(parse('[EFTL][VAR][SPLIT][VAR][/VAR][/SPLIT][/VAR][/EFTL]'), [{
+        message: 'VAR tags cannot be nested inside another VAR tag',
+        line: 1,
+        column: 19,
+        length: 5
+    }]);
+});
+
+test('reports a VAR tag without its closing tag', () => {
+    assert.ok(messages('[EFTL][VAR]').includes('Unclosed tag: [VAR] (expected [/VAR])'));
+});
+
+test('reports a mismatched closing tag', () => {
+    const errors = parse('[EFTL][/VAR][/EFTL]');
+
+    assert.equal(errors.length, 1);
+    assert.equal(errors[0].message, 'Mismatched closing tag: expected [/EFTL], got [/VAR]');
+});
+
+test('reports every unclosed nested tag at its opening token', () => {
+    assert.deepEqual(parse('[EFTL]\n[VAR]'), [
+        {
+            message: 'Unclosed tag: [EFTL] (expected [/EFTL])',
+            line: 1,
+            column: 1,
+            length: 6
+        },
+        {
+            message: 'Unclosed tag: [VAR] (expected [/VAR])',
+            line: 2,
+            column: 1,
+            length: 5
+        }
+    ]);
+});
+
+test('requires exactly one EFTL root block', () => {
+    assert.deepEqual(parse(''), [{
+        message: 'Missing required [EFTL] root block',
+        line: 1,
+        column: 1,
+        length: 0
+    }]);
+
+    assert.deepEqual(parse('[EFTL][/EFTL]\n[EFTL][/EFTL]'), [{
+        message: 'Only one [EFTL] root block is allowed',
+        line: 2,
+        column: 1,
+        length: 6
+    }]);
+});
+
+test('allows comments and multiline whitespace around the EFTL root', () => {
+    assert.deepEqual(parse(' \n[!-- before --]\n[EFTL][/EFTL]\n[!-- after --]\n '), []);
+});
+
+test('rejects instructions outside the EFTL root', () => {
+    assert.deepEqual(messages('[VAR][/VAR][EFTL][/EFTL]'), [
+        'Only comments and whitespace are allowed outside the [EFTL] root block',
+        'Only comments and whitespace are allowed outside the [EFTL] root block'
+    ]);
+});
+
+test('rejects a nested EFTL root', () => {
+    assert.deepEqual(parse('[EFTL][EFTL][/EFTL][/EFTL]'), [{
+        message: 'EFTL root blocks cannot be nested',
+        line: 1,
+        column: 7,
+        length: 6
+    }]);
+});
+
+test('accepts CONDITION, THEN, repeated ELSE IF, and one final ELSE in an IF', () => {
+    const source = [
+        '[EFTL][IF]',
+        '  [CONDITION][% true %][/CONDITION]',
+        '  [THEN][/THEN]',
+        '  [ELSE IF][CONDITION][% false %][/CONDITION][THEN][/THEN][/ELSE IF]',
+        '  [ELSE IF][CONDITION][% false %][/CONDITION][THEN][/THEN][/ELSE IF]',
+        '  [ELSE][/ELSE]',
+        '[/IF][/EFTL]'
+    ].join('\n');
+
+    assert.deepEqual(parse(source), []);
+});
+
+test('allows an IF nested inside a THEN block', () => {
+    const source = '[EFTL][IF][CONDITION][/CONDITION][THEN]'
+        + '[IF][CONDITION][/CONDITION][THEN][/THEN][/IF]'
+        + '[/THEN][/IF][/EFTL]';
+
+    assert.deepEqual(parse(source), []);
+});
+
+test('rejects an IF nested directly inside another IF', () => {
+    const source = '[EFTL][IF]'
+        + '[IF][CONDITION][/CONDITION][THEN][/THEN][/IF]'
+        + '[CONDITION][/CONDITION][THEN][/THEN][/IF][/EFTL]';
+
+    assert.deepEqual(messages(source), [
+        'IF cannot be nested directly inside another IF; place it inside a branch block'
+    ]);
+});
+
+test('requires CONDITION followed by THEN as the first direct IF children', () => {
+    assert.deepEqual(messages('[EFTL][IF][/IF][/EFTL]'), [
+        'Expected [CONDITION] immediately after [IF]'
+    ]);
+
+    assert.deepEqual(messages('[EFTL][IF][THEN][/THEN][CONDITION][/CONDITION][THEN][/THEN][/IF][/EFTL]'), [
+        'Expected [CONDITION] as the next direct child of [IF], got [THEN]'
+    ]);
+
+    assert.deepEqual(messages('[EFTL][IF][CONDITION][/CONDITION][/IF][/EFTL]'), [
+        'Expected [THEN] immediately after [/CONDITION]'
+    ]);
+});
+
+test('allows only whitespace between direct IF child blocks', () => {
+    const source = '[EFTL][IF][CONDITION][/CONDITION]'
+        + '[VAR][/VAR][!-- comment --]text'
+        + '[THEN][/THEN][/IF][/EFTL]';
+
+    assert.deepEqual(messages(source), [
+        'Expected [THEN] as the next direct child of [IF], got [VAR]',
+        'Expected [THEN] as the next direct child of [IF], got [!--',
+        'Expected [THEN] as the next direct child of [IF], got text'
+    ]);
+});
+
+test('allows ELSE IF only before the optional ELSE', () => {
+    const source = '[EFTL][IF][CONDITION][/CONDITION][THEN][/THEN]'
+        + '[ELSE][/ELSE][ELSE IF][/ELSE IF][/IF][/EFTL]';
+
+    assert.deepEqual(messages(source), [
+        'Expected [/IF] as the next direct child of [IF], got [ELSE IF]'
+    ]);
+
+    assert.deepEqual(messages(
+        '[EFTL][IF][CONDITION][/CONDITION][THEN][/THEN][ELSE][/ELSE][ELSE][/ELSE][/IF][/EFTL]'
+    ), [
+        'Expected [/IF] as the next direct child of [IF], got [ELSE]'
+    ]);
+});
+
+test('accepts WHILE with an optional string threshold attribute', () => {
+    assert.deepEqual(parse('[EFTL][WHILE][/WHILE][/EFTL]'), []);
+    assert.deepEqual(parse('[EFTL][WHILE threshold="99"][/WHILE][/EFTL]'), []);
+
+    const { tokens } = new EftlTokenizer('[WHILE threshold="99"][/WHILE]').tokenize();
+    assert.equal(tokens[0].type, 'WHILE_OPEN');
+    assert.equal(tokens[0].value, '[WHILE threshold="99"]');
+});
+
+test('rejects invalid WHILE attributes', () => {
+    const expectedMessage = 'WHILE accepts only the optional string attribute threshold="..."';
+
+    assert.deepEqual(messages('[EFTL][WHILE threshold=99][/WHILE][/EFTL]'), [expectedMessage]);
+    assert.deepEqual(messages('[EFTL][WHILE limit="99"][/WHILE][/EFTL]'), [expectedMessage]);
+    assert.deepEqual(messages(
+        '[EFTL][WHILE threshold="10" threshold="20"][/WHILE][/EFTL]'
+    ), [expectedMessage]);
+});
Author	SHA1	Message	Date
pierpaolo.mammi	3c0695f8f2	refine tag management	2026-08-19 14:12:48 +02:00
pierpaolo.mammi	300864a15e	add threshold attribute to WHILE tag	2026-08-19 14:12:26 +02:00
pierpaolo.mammi	15422cca1f	fix unclosed block detection add parser tests (codex)	2026-08-18 12:49:31 +02:00
pierpaolo.mammi	c34d2864aa	update installation instructions in readme	2026-08-18 12:48:55 +02:00
pierpaolo.mammi	d43f6ac434	split samples for correct parsing	2026-08-18 12:48:20 +02:00
pierpaolo.mammi	09dcfede46	add docs and AI skills	2026-08-17 18:23:10 +02:00
pierpaolo.mammi	c3d5d3541b	refine blocks detection	2026-08-17 18:22:58 +02:00
pierpaolo.mammi	d963472729	update node modules configuration	2026-08-17 18:22:49 +02:00