Implement language server protocol (LSP) and VS Code extension client

AGENTS.md

@@ -55,3 +55,9 @@ If an implementation plan artifact is created, also print the full plan directly
 When modifying code, comments, or documentation, use formal terminology based on Programming Language Theory, Theory of Computation, and Type Theory for internal logic. Prefer terms such as `Symbol` and `Production` internally.
 
 For user-facing Bison-inspired syntax, keep familiar Bison terminology such as `%token` and `%tokentype`.
+
+## 9. Keep LSP Synchronized with Grammar Changes
+
+Whenever changes are made to the grammar syntax, directives, patterns, or variables:
+- Update the LSP implementation in `rusty_lr_lsp` to fully support and recognize the updated grammar.
+- Ensure that semantic tokens, hover information, completions, inlay hints, and diagnostic handling are kept aligned with the new grammar specifications.

Cargo.toml

@@ -6,8 +6,10 @@ members = [
   "rusty_lr_parser",
   "rusty_lr_buildscript",
   "rusty_lr_executable",
+  "rusty_lr_lsp",
   "example/calculator",
   "example/calculator_u8",
   "example/glr",
   "example/json",
 ]
+

README.md

@@ -258,6 +258,13 @@ println!("{}", context);   // Formats the state tree (requires 'tree' feature)
 
 ---
 
+## Editor Support
+
+An experimental RustyLR language server is under development in [`rusty_lr_lsp`](rusty_lr_lsp), with a temporary VSCode client in [`editors/vscode-rustylr`](editors/vscode-rustylr).
+It currently targets `*.rustylr` files and files named `rustylr.rs`.
+
+---
+
 ## Examples
 
 - [Calculator (enum tokens)](https://github.com/ehwan/RustyLR/blob/main/example/calculator/src/parser.rustylr): A numeric expression parser using custom token enums.

SYNTAX.md

@@ -10,7 +10,7 @@ This document provides a comprehensive guide to the grammar definition syntax us
 - [Token Definition (`%token`)](#token-definition-must-defined)
 - [Production Rules](#production-rules)
 - [Patterns](#patterns)
-- [ProductionType (Non-Terminal Types)](#ruletype-optional)
+- [ProductionType (Non-Terminal Types)](#productiontype-optional)
 - [Reduce Actions](#reduceaction-optional)
 - [Accessing Data in Reduce Actions](#accessing-token-data-in-reduceaction)
 - [Exclamation Mark (`!`) Value Discard](#exclamation-mark-)
@@ -511,7 +511,7 @@ You can use variables prefixed with `$` inside any RustCode block in the grammar
 - `$location` -> Evaluates to the type defined by `%location` (defaults to `::rusty_lr::DefaultLocation`).
 - `$userdata` -> Evaluates to the type defined by `%userdata` (defaults to `()`).
 - `$error` or `$errortype` -> Evaluates to the type defined by `%errortype` / `%error` (defaults to `::rusty_lr::DefaultReduceActionError`).
-- `$NonTerminalName` -> Evaluates to the `ruletype` defined for `NonTerminalName`.
+- `$NonTerminalName` -> Evaluates to the `ProductionType` defined for `NonTerminalName`.
 - `$terminal_name` -> Evaluates to the match pattern/definition of `<terminal_name>`.
 
 ### Substitution Errors

editors/vscode-rustylr/.gitignore

@@ -0,0 +1,2 @@
+node_modules/
+*.vsix

editors/vscode-rustylr/CHANGELOG.md

@@ -0,0 +1,17 @@
+# Changelog
+
+All notable changes to the "RustyLR" extension will be documented in this file.
+
+## 0.1.0
+
+- First public release of RustyLR language support!
+- Fully integrated with the `rusty_lr_lsp` server:
+  - **Syntax Highlighting (Semantic Tokens):** Distinct syntax coloring for terminals, non-terminals, directives, bindings, location bindings, and variables.
+  - **Diagnostics:** Inline warning and error reporting directly in the editor.
+  - **Code Actions:** Quick-fix actions to suppress warnings with `%allow` directives.
+  - **Formatting:** Code formatting and indentation support for rule definitions and reduce actions.
+  - **Go to Definition:** Jump directly to token declarations, production definitions, and precedence definitions.
+  - **Find References:** Find all usages of terminals, non-terminals, and precedence symbols across the grammar document.
+  - **Hover Tooltips:** Interactive documentation tooltips for keywords, patterns, and variables.
+  - **Inlay Hints:** Inline type hints for grammar patterns and reduce actions.
+  - **Auto-Completion:** Intelligent suggestions for symbols, directives, variables, and locations.

editors/vscode-rustylr/README.md

@@ -0,0 +1,34 @@
+# RustyLR Language Support
+
+This extension provides rich language support for the [RustyLR](https://github.com/ehwan/RustyLR) parser generator grammar files (`*.rustylr` and `rustylr.rs`).
+
+## Features
+
+- **Diagnostics & Error Reporting:** Real-time diagnostics for grammar syntax errors, unused symbols, conflict resolutions, and more.
+- **Go to Definition:** Quickly navigate to rule definitions, terminal declarations, and precedence rules.
+- **Find References:** Find all occurrences and usages of terminals, non-terminals, and precedence symbols.
+- **Syntax Highlighting (Semantic Tokens):** Distinct, theme-aligned colors for terminal names, non-terminal rules, directives, bindings, location bindings (`@loc`), and variables (`$var`).
+- **Formatting:** Automatic document formatter that standardizes directives, separates rules, and indents rule lines and reduce-action bodies.
+- **Code Actions (Quick Fixes):** Fast diagnostic suppression actions using the `%allow` directive.
+- **Hover tooltips:** Documented explanations and types for terminal tokens, non-terminal rules, keywords, and patterns.
+- **Inlay Hints:** Inline type annotations and reduce action indicators.
+- **Auto-Completion:** Intelligent suggestions for directives, symbols, locations, variables, and diagnostics.
+
+## Extension Settings
+
+This extension contributes the following settings to control the language server behavior:
+
+* `rustylr.server.command`: Path to the `rusty_lr_lsp` server binary. Leave empty to automatically detect or run from Cargo.
+* `rustylr.server.args`: Arguments passed to the language server command.
+* `rustylr.server.cwd`: Working directory for the language server.
+* `rustylr.semanticTokens.enabled`: Toggle semantic token syntax highlighting.
+
+## Requirements
+
+The language features require the `rusty_lr_lsp` server, which is part of the RustyLR cargo workspace. You can build it from the repository root:
+
+```bash
+cargo build -p rusty_lr_lsp
+```
+
+By default, the extension will attempt to auto-detect the built binary in your workspace target folder or run it dynamically using Cargo.

editors/vscode-rustylr/extension.js

@@ -0,0 +1,201 @@
+const fs = require("fs");
+const path = require("path");
+const vscode = require("vscode");
+const { LanguageClient, TransportKind } = require("vscode-languageclient/node");
+
+let client;
+let outputChannel;
+let startingClient;
+
+async function activate(context) {
+  outputChannel = vscode.window.createOutputChannel("RustyLR LSP");
+  context.subscriptions.push(outputChannel);
+
+  context.subscriptions.push(
+    vscode.commands.registerCommand("rustylr.restartServer", async () => {
+      await stopClient();
+      try {
+        await startClient(context);
+        vscode.window.showInformationMessage("RustyLR language server restarted.");
+      } catch (error) {
+        reportStartError(error);
+      }
+    })
+  );
+
+  startClient(context).catch(reportStartError);
+}
+
+async function deactivate() {
+  await stopClient();
+}
+
+async function startClient(context) {
+  if (startingClient) {
+    return startingClient;
+  }
+  if (client) {
+    return;
+  }
+
+  startingClient = doStartClient(context);
+  try {
+    await startingClient;
+  } finally {
+    startingClient = undefined;
+  }
+}
+
+async function doStartClient(context) {
+  const config = vscode.workspace.getConfiguration("rustylr.server");
+  const workspaceFolder =
+    vscode.workspace.workspaceFolders && vscode.workspace.workspaceFolders.length > 0
+      ? vscode.workspace.workspaceFolders[0].uri.fsPath
+      : undefined;
+  const repoRoot = findRustyLrRoot(workspaceFolder) || findRustyLrRoot(context.extensionPath);
+
+  const configuredCwd = config.get("cwd", "");
+  const cwd = configuredCwd
+    ? expandPath(configuredCwd, { workspaceFolder, extensionPath: context.extensionPath, repoRoot })
+    : repoRoot || workspaceFolder || context.extensionPath;
+
+  const configuredCommand = config.get("command", "");
+  const configuredArgs = config.get("args", []);
+  const server = resolveServerCommand(configuredCommand, configuredArgs, {
+    workspaceFolder,
+    extensionPath: context.extensionPath,
+    repoRoot,
+    cwd,
+  });
+
+  const patterns = config.get("documentPatterns", [
+    "**/*.rustylr",
+    "**/rustylr.rs",
+  ]);
+
+  const documentSelector = [
+    { scheme: "file", language: "rustylr" },
+    ...patterns.map((pattern) => ({ scheme: "file", pattern })),
+  ];
+
+  outputChannel.appendLine(`Starting RustyLR LSP: ${server.command} ${server.args.join(" ")}`);
+  outputChannel.appendLine(`RustyLR LSP cwd: ${cwd}`);
+
+  client = new LanguageClient(
+    "rustylr",
+    "RustyLR Language Server",
+    {
+      command: server.command,
+      args: server.args,
+      options: { cwd },
+      transport: TransportKind.stdio,
+    },
+    {
+      documentSelector,
+      outputChannel,
+      synchronize: {
+        configurationSection: "rustylr",
+      },
+    }
+  );
+
+  await client.start();
+}
+
+async function stopClient() {
+  if (startingClient) {
+    try {
+      await startingClient;
+    } catch (_error) {
+      // The start failure will already be reported by the original caller.
+    }
+  }
+
+  if (!client) {
+    return;
+  }
+
+  const activeClient = client;
+  client = undefined;
+  try {
+    await activeClient.stop();
+  } catch (error) {
+    const message = error && error.message ? error.message : String(error);
+    if (outputChannel) {
+      outputChannel.appendLine(`Ignoring RustyLR LSP stop error: ${message}`);
+    }
+  }
+}
+
+function expandPath(value, vars) {
+  return value
+    .split("${workspaceFolder}")
+    .join(vars.workspaceFolder || "")
+    .split("${extensionPath}")
+    .join(vars.extensionPath || "")
+    .split("${repoRoot}")
+    .join(vars.repoRoot || "");
+}
+
+function resolveServerCommand(configuredCommand, configuredArgs, vars) {
+  if (configuredCommand) {
+    return {
+      command: expandPath(configuredCommand, vars),
+      args: configuredArgs.map((arg) => expandPath(arg, vars)),
+    };
+  }
+
+  const binaryName = process.platform === "win32" ? "rusty_lr_lsp.exe" : "rusty_lr_lsp";
+  const candidates = [
+    vars.repoRoot && path.join(vars.repoRoot, "target", "debug", binaryName),
+    vars.repoRoot && path.join(vars.repoRoot, "target", "release", binaryName),
+  ].filter(Boolean);
+
+  for (const candidate of candidates) {
+    if (fs.existsSync(candidate)) {
+      return { command: candidate, args: [] };
+    }
+  }
+
+  return {
+    command: "cargo",
+    args: ["run", "--quiet", "--package", "rusty_lr_lsp"],
+  };
+}
+
+function findRustyLrRoot(startPath) {
+  if (!startPath) {
+    return undefined;
+  }
+
+  let current = fs.statSync(startPath).isDirectory() ? startPath : path.dirname(startPath);
+  while (true) {
+    if (
+      fs.existsSync(path.join(current, "Cargo.toml")) &&
+      fs.existsSync(path.join(current, "rusty_lr_lsp", "Cargo.toml"))
+    ) {
+      return current;
+    }
+
+    const parent = path.dirname(current);
+    if (parent === current) {
+      return undefined;
+    }
+    current = parent;
+  }
+}
+
+function reportStartError(error) {
+  const message = error && error.stack ? error.stack : String(error);
+  if (outputChannel) {
+    outputChannel.appendLine("Failed to start RustyLR LSP.");
+    outputChannel.appendLine(message);
+    outputChannel.show(true);
+  }
+  vscode.window.showErrorMessage("Failed to start RustyLR language server. See Output: RustyLR LSP.");
+}
+
+module.exports = {
+  activate,
+  deactivate,
+};

editors/vscode-rustylr/language-configuration.json

@@ -0,0 +1,74 @@
+{
+  "comments": {
+    "lineComment": "//",
+    "blockComment": [
+      "/*",
+      "*/"
+    ]
+  },
+  "brackets": [
+    [
+      "{",
+      "}"
+    ],
+    [
+      "[",
+      "]"
+    ],
+    [
+      "(",
+      ")"
+    ]
+  ],
+  "autoClosingPairs": [
+    {
+      "open": "{",
+      "close": "}"
+    },
+    {
+      "open": "[",
+      "close": "]"
+    },
+    {
+      "open": "(",
+      "close": ")"
+    },
+    {
+      "open": "\"",
+      "close": "\"",
+      "notIn": [
+        "string"
+      ]
+    },
+    {
+      "open": "'",
+      "close": "'",
+      "notIn": [
+        "string",
+        "comment"
+      ]
+    }
+  ],
+  "surroundingPairs": [
+    [
+      "{",
+      "}"
+    ],
+    [
+      "[",
+      "]"
+    ],
+    [
+      "(",
+      ")"
+    ],
+    [
+      "\"",
+      "\""
+    ],
+    [
+      "'",
+      "'"
+    ]
+  ]
+}