Merge pull request #13718 from quarto-dev/feature/engine-extension-3

feature: engine extension

Author: gordonwoodhull

dev-docs/subtree-extensions.md

@@ -0,0 +1,41 @@
+# Subtree extensions
+
+Subtree extensions live in `src/resources/subtree-extensions`.
+
+Each is the complete git repository of a Quarto extension, e.g. the directory tree for the Julia engine looks like
+
+```
+src/resources/extension-subtrees/
+  julia-engine/
+    _extensions/
+      julia-engine/
+        _extension.yml
+        julia-engine.ts
+        ...
+```
+
+The command to add or update a subtree is
+
+```
+quarto dev-call pull-git-subtree subtree-name
+```
+
+Omit _subtree-name_ to add/update all.
+
+The code in `src/command/dev-call/pull-git-subtree/cmd.ts` contains a table of subtree
+
+- `name`
+- `prefix` (subdirectory)
+- `remoteUrl`
+- `remoteBranch`
+
+If the command is successful, it will add two commits, one the squashed changes from the remote repo and one a merge commit.
+
+The commits have subtree status information in the message and metadata, so don't change them.
+
+The commits can't be rebased -- you'll get weird errors indicating it tried to merge changes at the root of the repo.
+
+So you must either
+
+- run the command when ready to merge to main, or
+- remove the commits when rebasing, and run the `dev-call` command again

llm-docs/error-messages.md

@@ -0,0 +1,103 @@
+# Writing Error Messages in Quarto
+
+## The Rule
+
+Each `error()`, `warning()`, or `info()` call should be **exactly one line**.
+
+- ✅ End messages with `\n` to add blank lines after
+- ❌ Never start messages with `\n`
+- ❌ Never use empty `error("")` calls
+
+## Why This Matters
+
+Quarto's logging prefixes each call with `ERROR:` / `WARNING:` / `INFO:`. Starting a message with `\n` or using empty calls creates confusing output:
+
+```
+ERROR: Multiple files found
+ERROR:
+Or specify entry point:    ← Empty "ERROR:" line from \n at start
+```
+
+## Adding Blank Lines
+
+To add a blank line between sections, end the **previous** message with `\n`:
+
+### ✅ Good
+
+```typescript
+error("Multiple .ts files found in src/\n");  // \n at END
+error("Specify entry point as argument:");
+error("  quarto call build-ts-extension src/my-engine.ts");
+```
+
+Output:
+```
+ERROR: Multiple .ts files found in src/
+ERROR:
+ERROR: Specify entry point as argument:
+ERROR:   quarto call build-ts-extension src/my-engine.ts
+```
+
+### ❌ Bad
+
+```typescript
+error("Multiple .ts files found in src/");
+error("\nSpecify entry point as argument:");  // \n at START
+error("  quarto call build-ts-extension src/my-engine.ts");
+```
+
+Output:
+```
+ERROR: Multiple .ts files found in src/
+ERROR:
+ERROR: Specify entry point as argument:     ← Blank "ERROR:" line before
+ERROR:   quarto call build-ts-extension src/my-engine.ts
+```
+
+### ❌ Also Bad
+
+```typescript
+error("Multiple .ts files found in src/");
+error("");  // Empty call to add spacing
+error("Specify entry point as argument:");
+```
+
+Output:
+```
+ERROR: Multiple .ts files found in src/
+ERROR:                                      ← Empty "ERROR:" line
+ERROR: Specify entry point as argument:
+```
+
+## Complete Example
+
+Here's a real example from `build-ts-extension` showing proper formatting:
+
+### ✅ Good
+
+```typescript
+error("No src/ directory found.\n");
+error("Create a TypeScript file in src/:");
+error("  mkdir -p src");
+error("  touch src/my-engine.ts\n");
+error("Or specify entry point as argument:");
+error("  quarto call build-ts-extension src/my-engine.ts");
+```
+
+Output:
+```
+ERROR: No src/ directory found.
+ERROR:
+ERROR: Create a TypeScript file in src/:
+ERROR:   mkdir -p src
+ERROR:   touch src/my-engine.ts
+ERROR:
+ERROR: Or specify entry point as argument:
+ERROR:   quarto call build-ts-extension src/my-engine.ts
+```
+
+Notice:
+- Each `error()` call is one complete line
+- Blank lines are created by ending the previous message with `\n`
+- Indentation (with spaces) is preserved within each message
+- Message flow is clear and readable

llm-docs/quarto-api.md

@@ -0,0 +1,188 @@
+# Quarto API and @quarto/types
+
+## Building @quarto/types
+
+To build the @quarto/types package:
+
+```
+cd packages/quarto-types
+npm run build
+```
+
+This runs typecheck and then bundles all type definitions into `dist/index.d.ts`.
+
+---
+
+## Updating the Quarto API
+
+The Quarto API is how external execution engines access Quarto's core functionality. The API exists in two places:
+
+1. **Type definitions** in `packages/quarto-types/` - consumed by external engines (TypeScript)
+2. **Implementation** in `src/core/quarto-api.ts` - used within quarto-cli
+
+### Step-by-step: Adding to the Quarto API
+
+Follow these steps in order when adding new functionality to the API:
+
+#### 1. Update quarto-types type definitions
+
+**Add auxiliary types** (if needed):
+
+- Types belong in `packages/quarto-types/src/`
+- Follow the existing file organization:
+  - `system.ts` - System/process types (ProcessResult, TempContext, etc.)
+  - `console.ts` - Console/UI types (SpinnerOptions, etc.)
+  - `jupyter.ts` - Jupyter-specific types
+  - `check.ts` - Check command types
+  - `execution.ts` - Execution engine types
+  - etc.
+- Create new files if needed for logical grouping
+
+**Export types from index.ts:**
+
+```typescript
+// In packages/quarto-types/src/index.ts
+export type * from "./your-new-file.ts";
+```
+
+**Add to QuartoAPI interface:**
+
+```typescript
+// In packages/quarto-types/src/quarto-api.ts
+
+// 1. Import any new types at the top
+import type { YourNewType } from "./your-file.ts";
+
+// 2. Add to the QuartoAPI interface
+export interface QuartoAPI {
+  // ... existing namespaces
+
+  yourNamespace: {
+    yourMethod: (param: YourNewType) => ReturnType;
+  };
+}
+```
+
+#### 2. Test the type definitions
+
+```bash
+cd packages/quarto-types
+npm run build
+```
+
+This will:
+
+- Run `tsc --noEmit` to typecheck
+- Bundle types into `dist/index.d.ts`
+- Show any type errors
+
+Fix any errors before proceeding.
+
+#### 3. Update the internal QuartoAPI interface
+
+The file `src/core/quarto-api.ts` contains a **duplicate** QuartoAPI interface definition used for the internal implementation. Update it to match:
+
+```typescript
+// In src/core/quarto-api.ts (near top of file)
+export interface QuartoAPI {
+  // ... existing namespaces
+
+  yourNamespace: {
+    yourMethod: (param: YourNewType) => ReturnType;
+  };
+}
+```
+
+**Note:** This interface must match the one in quarto-types, but uses internal types.
+
+#### 4. Wire up the implementation
+
+Still in `src/core/quarto-api.ts`:
+
+**Add imports** (near top):
+
+```typescript
+import { yourMethod } from "./your-module.ts";
+```
+
+**Add to quartoAPI object** (at bottom):
+
+```typescript
+export const quartoAPI: QuartoAPI = {
+  // ... existing namespaces
+
+  yourNamespace: {
+    yourMethod,
+  },
+};
+```
+
+#### 5. Verify with typecheck
+
+Run the quarto typecheck:
+
+```bash
+package/dist/bin/quarto
+```
+
+No output means success! Fix any type errors.
+
+#### 6. Commit with built artifact
+
+**Always commit the built `dist/index.d.ts` file** along with source changes:
+
+```bash
+git add packages/quarto-types/src/your-file.ts \
+        packages/quarto-types/src/index.ts \
+        packages/quarto-types/src/quarto-api.ts \
+        packages/quarto-types/dist/index.d.ts \
+        src/core/quarto-api.ts
+
+git commit -m "Add yourNamespace to Quarto API"
+```
+
+### Using the Quarto API in source files
+
+#### Inside quarto-cli (internal modules)
+
+```typescript
+// Import the quartoAPI instance
+import { quartoAPI as quarto } from "../../core/quarto-api.ts";
+
+// Use it
+const caps = await quarto.jupyter.capabilities();
+await quarto.console.withSpinner({ message: "Working..." }, async () => {
+  // do work
+});
+```
+
+#### External engines
+
+External engines receive the API via their `init()` method:
+
+```typescript
+let quarto: QuartoAPI;
+
+export const myEngineDiscovery: ExecutionEngineDiscovery = {
+  init: (quartoAPI: QuartoAPI) => {
+    quarto = quartoAPI; // Store for later use
+  },
+
+  // ... other methods can now use quarto
+};
+```
+
+#### Removing old imports
+
+When moving functionality to the API, **remove direct imports** from internal modules:
+
+```typescript
+// ❌ OLD - direct import
+import { withSpinner } from "../../core/console.ts";
+
+// ✅ NEW - use API
+import { quartoAPI as quarto } from "../../core/quarto-api.ts";
+const result = await quarto.console.withSpinner(...);
+```
+
+This ensures external engines and internal code use the same interface.

package/src/common/import-report/deno-info.ts

@@ -7,6 +7,8 @@
 *
 */
 
+import { architectureToolsPath } from "../../../../src/core/resources.ts";
+
 ////////////////////////////////////////////////////////////////////////////////
 
 export interface DenoInfoDependency {
@@ -61,8 +63,9 @@ export interface Edge {
 ////////////////////////////////////////////////////////////////////////////////
 
 export async function getDenoInfo(_root: string): Promise<DenoInfoJSON> {
+  const denoBinary = Deno.env.get("QUARTO_DENO") || architectureToolsPath("deno");
   const process = Deno.run({
-    cmd: ["deno", "info", Deno.args[0], "--json"],
+    cmd: [denoBinary, "info", Deno.args[0], "--json"],
     stdout: "piped",
   });
   const rawOutput = await process.output();

package/src/common/import-report/explain-all-cycles.ts

@@ -156,15 +156,19 @@ if (import.meta.main) {
 between all files reachable from some source file.
 
 Usage:
-  $ quarto run explain-all-cycles.ts <entry-point.ts> 
+  $ quarto run --dev explain-all-cycles.ts <entry-point.ts> [--simplify <prefixes...>] [--graph|--toon [filename]]
 
-Examples:
-
-  From ./src:
+Options:
+  --simplify <prefixes...>  Collapse paths with given prefixes (must be first if used)
+  --graph [filename]        Output .dot specification (default: graph.dot)
+  --toon [filename]         Output edges in TOON format (default: cycles.toon)
 
-  $ quarto run quarto.ts
+Examples:
+  $ quarto run --dev package/src/common/import-report/explain-all-cycles.ts src/quarto.ts
+  $ quarto run --dev package/src/common/import-report/explain-all-cycles.ts src/quarto.ts --simplify core/ command/ --toon
+  $ quarto run --dev package/src/common/import-report/explain-all-cycles.ts src/quarto.ts --graph cycles.dot
 
-If the second parameter is "--graph", then this program outputs the .dot specification to the file given by the third parameter, rather opening a full report.
+If no output option is given, opens an interactive preview.
 `,
     );
     Deno.exit(1);
@@ -187,7 +191,17 @@ If the second parameter is "--graph", then this program outputs the .dot specifi
 
   result = dropTypesFiles(result);
 
-  if (args[1] === "--graph") {
+  if (args[1] === "--toon") {
+    // Output in TOON format
+    const lines = [`edges[${result.length}]{from,to}:`];
+    for (const { from, to } of result) {
+      lines.push(`  ${from},${to}`);
+    }
+    Deno.writeTextFileSync(
+      args[2] ?? "cycles.toon",
+      lines.join("\n") + "\n",
+    );
+  } else if (args[1] === "--graph") {
     Deno.writeTextFileSync(
       args[2] ?? "graph.dot",
       generateGraph(result),