diff --git a/README.md b/README.md index 4f4526d..4fa478c 100644 --- a/README.md +++ b/README.md @@ -23,258 +23,64 @@ It is implemented using [mq](https://github.com/harehare/mq), a jq-like command- ## Features -- Execute code blocks from specific sections in Markdown files -- Task [dependencies](https://task.mqlang.org/book/guide/dependencies.html) with automatic execution ordering, including calls that pass arguments -- [Parallel execution](https://task.mqlang.org/book/guide/parallel.html) of independent dependencies, joined before the dependent task runs -- [Loops](https://task.mqlang.org/book/guide/loops.html) (`for` over a list, a glob, a matrix or another task's output) -- [`defer`](https://task.mqlang.org/book/guide/defer.html) cleanup tasks that run even after a failure -- [Conditions](https://task.mqlang.org/book/guide/conditions.html) (`if`), [retries and timeouts](https://task.mqlang.org/book/guide/retries-timeouts.html) (plus `ignore_error`), [confirmation prompts](https://task.mqlang.org/book/guide/prompts.html), and [passing a task's output](https://task.mqlang.org/book/guide/output.html) to its dependents -- [Importing tasks](https://task.mqlang.org/book/guide/imports.html) from other Markdown files -- Skip [up-to-date tasks](https://task.mqlang.org/book/guide/cache-watch.html) (`sources`/`outputs` or custom `status` commands) and `--watch` mode -- [Dependency graph](https://task.mqlang.org/book/guide/graph.html) output (`graph`) and [shell completions](https://task.mqlang.org/book/guide/completions.html) -- Finds `tasks.md` (or `README.md`) from any subdirectory, [completes task names](https://task.mqlang.org/book/guide/completions.html) in your shell, and catches typos in `meta` and task names with a suggestion -- [`mq-task check`](https://task.mqlang.org/book/guide/check.html) finds unknown keys, missing dependencies, cycles and missing runtimes without running anything -- [CLI options](https://task.mqlang.org/book/reference/cli.html) for scripts and CI: several tasks at once (`mq-task lint test`), `--no-deps`, `--quiet`, `--timings`, `--status`, `--summary`, `list --json`, `--output group` (folds per task on GitHub Actions), `--yes` -- README-friendly: `no-run` code blocks and data-language blocks (`json`, `text`, ...) are not commands, and `!pattern` excludes files from `sources` -- [Interactive TUI](https://task.mqlang.org/book/guide/tui.html) with search, details, argument prompts, per-task progress and a log pane -- [Named task parameters](https://task.mqlang.org/book/guide/params-env.html) with defaults, plus positional args -- Per-run `--env`/`--dir` overrides, plus `meta`-declared defaults for environment variables, `.env` files and working directory ([details](https://task.mqlang.org/book/guide/params-env.html)) -- Shared [variables](https://task.mqlang.org/book/guide/vars.html), fixed or computed by a command (`vars = { VERSION = { sh = "git describe" } }`) -- Default task, [aliases, and private tasks](https://task.mqlang.org/book/guide/aliases-private.html) -- Configurable [runtimes](https://task.mqlang.org/book/start/configuration.html) for different programming languages, with real exit codes and interactive stdin -- Support for custom heading levels -- TOML-based configuration -- Built on top of the mq query language - -See the [documentation](https://task.mqlang.org/book/) for the full guide and reference, or [how the concepts map](https://task.mqlang.org/book/start/migrating.html) if you come from `just` or Task. +- Run the code blocks under any heading, in any language with a configured [runtime](https://task.mqlang.org/book/start/configuration.html) +- [Dependencies](https://task.mqlang.org/book/guide/dependencies.html), [parallel execution](https://task.mqlang.org/book/guide/parallel.html), [loops](https://task.mqlang.org/book/guide/loops.html), [`defer`](https://task.mqlang.org/book/guide/defer.html) cleanup and shared [variables](https://task.mqlang.org/book/guide/vars.html) +- [Conditions](https://task.mqlang.org/book/guide/conditions.html), [retries and timeouts](https://task.mqlang.org/book/guide/retries-timeouts.html), [confirmation prompts](https://task.mqlang.org/book/guide/prompts.html) and [output passing](https://task.mqlang.org/book/guide/output.html) +- [Named parameters, env and working directory](https://task.mqlang.org/book/guide/params-env.html), [imports](https://task.mqlang.org/book/guide/imports.html), [aliases and private tasks](https://task.mqlang.org/book/guide/aliases-private.html) +- Skip [up-to-date tasks](https://task.mqlang.org/book/guide/cache-watch.html) and `--watch` mode +- [`mq-task check`](https://task.mqlang.org/book/guide/check.html), [dependency graph](https://task.mqlang.org/book/guide/graph.html), [interactive TUI](https://task.mqlang.org/book/guide/tui.html) and [shell completions](https://task.mqlang.org/book/guide/completions.html) +- Options for scripts and CI: see the [CLI reference](https://task.mqlang.org/book/reference/cli.html) ## Installation -### Quick Install - ```bash curl -sSL https://task.mqlang.org/install.sh | bash ``` -The installer will: -- Download the latest mq-task binary for your platform and verify its checksum -- Install it to `~/.local/bin/` -- Update your shell profile to add `~/.local/bin` to your PATH - -### Cargo +or with Cargo: ```sh -$ cargo install --git https://github.com/harehare/mq-task.git +cargo install --git https://github.com/harehare/mq-task.git ``` ## Usage -### Run a task (shorthand) - -```bash -# Run from tasks.md, else README.md (searched up to the repository root) -mq-task "Task Name" - -# Run from a specific file -mq-task -f tasks.md "Task Name" -``` - -### Run a task (explicit) - -```bash -mq-task run "Task Name" -mq-task run --file tasks.md "Task Name" -``` - -### Pass arguments to a task - -You can pass arguments to your task using `--` separator: - -```bash -# Pass arguments to a task -mq-task "Task Name" -- arg1 arg2 arg3 - -# With explicit run command -mq-task run "Task Name" -- arg1 arg2 arg3 - -# From a specific file -mq-task -f tasks.md "Task Name" -- arg1 arg2 -``` - -Arguments are accessible via environment variables: -- `MX_ARGS`: All arguments joined by space (e.g., "arg1 arg2 arg3") -- `MX_ARG_0`, `MX_ARG_1`, ...: Individual arguments - -Example in a Markdown task: - -````markdown -## My Task - -```bash -echo "All args: $MX_ARGS" -echo "First arg: $MX_ARG_0" -echo "Second arg: $MX_ARG_1" -``` -```` - -### Environment variables and working directory - -Like `just`, you can set environment variables and a working directory for a task at invocation time, with no declaration needed in the Markdown file: - -```bash -# Set one or more environment variables (repeatable) -mq-task deploy --env REGION=eu --env DEBUG=1 - -# Run the task's commands in a specific working directory -mq-task deploy --dir ../other-project - -# Combine both -mq-task run deploy --env REGION=eu --dir ../other-project -``` - -`--env KEY=VALUE` variables are available in the task's shell alongside `MX_ARGS`/`MX_PARAM_*`. `--dir` applies to every command in the task (and its dependencies) for that run. Both flags are available on the shorthand form, `run`, and `tui`. - -You can also declare default environment variables directly in a task's `meta` block: +A task is a heading followed by code blocks. A `meta` block (TOML) adds options such as dependencies, parameters or conditions: ````markdown -## deploy - -```meta -env = ["REGION=staging", "LOG_LEVEL=info"] -``` - -```bash -echo "deploying to $REGION ($LOG_LEVEL)" -``` -```` - -A CLI `--env` flag of the same name overrides the `meta` default for that run: - -```bash -mq-task deploy # REGION=staging -mq-task deploy --env REGION=prod # REGION=prod -``` - -A working directory can be declared the same way with `dir` (relative paths resolve against the directory `mq-task` was invoked from): - -````markdown -## deploy - -```meta -dir = "services/api" -``` - -```bash -pwd # services/api -``` -```` - -The CLI `--dir` flag overrides a `meta`-declared `dir` for that run: - -```bash -mq-task deploy # runs in services/api -mq-task deploy --dir services/worker # runs in services/worker instead -``` - -#### Document-wide defaults - -A `meta` block placed **before the first heading** in the file applies to every task, not just one. It's useful for defaults you'd otherwise have to repeat in each task's own `meta` block: - -````markdown -```meta -env = ["REGION=staging", "LOG_LEVEL=info"] -dir = "services/api" -``` - -## deploy - -```bash -echo "deploying to $REGION ($LOG_LEVEL) from $(pwd)" -``` - ## build -```bash -echo "building in $(pwd)" -``` -```` - -Both tasks inherit `REGION`, `LOG_LEVEL`, and `dir` from the document-wide block. Precedence, from lowest to highest, is: document-wide `meta` → task's own `meta` → CLI `--env`/`--dir`. A task's `meta` only needs to declare the keys it wants to change: - -````markdown -```meta -env = ["REGION=staging"] -``` - -## deploy - -```meta -env = ["REGION=prod"] -``` - -```bash -echo "$REGION" # prod (the task's own meta wins) -``` - -## build - -```bash -echo "$REGION" # staging (falls back to the document-wide default) -``` -```` - -### Task dependencies - -You can declare dependencies between tasks using a `meta` code block (TOML format). When a task is run, its dependencies are automatically executed first in the correct order. - -````markdown -## format +Compile the project. ```bash -cargo fmt +cargo build --release ``` -## lint +## test ```meta -depends = ["format"] +depends = ["build"] +sources = ["src/**/*.rs", "tests/**/*.rs"] # skipped when unchanged +retries = 2 ``` ```bash -cargo clippy +cargo test ``` -## test +## deploy ```meta -depends = ["lint"] +params = ["env=staging", "region"] # name=default; a bare name is required +depends = ["test"] +prompt = "Deploy?" +if = 'flag("CI") || which("kubectl")' ``` ```bash -cargo test -``` -```` - -Running `mq-task test` will automatically execute `format → lint → test` in order: - +./deploy.sh "$MX_PARAM_ENV" "$MX_PARAM_REGION" ``` -▶ test -↳ format (dependency) -... -↳ lint (dependency) -... -(test output) -``` - -- Multiple dependencies are supported: `depends = ["format", "lint"]` -- Shared dependencies are executed only once even if multiple tasks depend on them -- Circular dependencies are detected and reported as an error -### Parallel execution - -By default a task's dependencies run one after another in the order they are declared. Set `parallel = true` to run them concurrently; the task itself starts only after **all** of them have finished (a fan-out / fan-in): - -````markdown ## ci ```meta @@ -285,320 +91,31 @@ parallel = true ```bash echo "all checks passed" ``` -```` -- Only the real dependency edges constrain the order, so a task starts as soon as *its own* dependencies are done. Tasks that depend on each other (`test` depends on `build`) still run in order -- Output lines are prefixed with the task name (`lint │ ...`) and each task reports `✔`/`✘` with its duration -- By default the first failure aborts the run: running tasks are stopped (including processes they started) and nothing new begins. Use `--keep-going`/`-k` to let independent tasks finish; tasks that depend on a failed task are not run -- `--parallel` runs every dependency graph concurrently regardless of `parallel` in the file, and `-j N`/`--jobs N` limits how many tasks run at once (`-j 1` forces sequential execution). The default limit is the number of CPUs -- In parallel runs stdin is closed for tasks; interactive tasks should run sequentially. Dry-run is always sequential +## report -To run several tasks at once, define an aggregate task that depends on them (as `ci` above). +Code blocks can use any language with a runtime. -### Conditions - -`if` skips a task when its condition is false. A skipped task does **not** block the tasks that depend on it. - -````markdown -## deploy - -```meta -if = 'flag("CI") && os() == "linux"' -``` - -```bash -./deploy.sh +```python +print("report generated") ``` ```` -| Expression | True when | -|---|---| -| `flag("NAME")` | the variable is set and not empty, `0` or `false` | -| `getenv("NAME") == "x"`, `getenv("NAME") != "x"` | the variable equals / differs from `x` | -| `os() == "linux"` | the OS is `linux`, `macos` or `windows` | -| `exists("path")` | the path exists (relative to the task's working directory) | -| `which("cmd")` | `cmd` is on `PATH` | -| `!expr`, `a && b`, `a \|\| b` | boolean combinations; parentheses group | - -Variables are looked up in `--env`, the task's `meta`/`params`, then the process environment. - -A condition is an [mq](https://mqlang.org) expression that must evaluate to `true` or `false`, so anything else mq offers (string functions, `contains`, ...) works too. Use `getenv`/`flag` rather than mq's built-in `env()`, which only sees the process environment. - -### Retries and timeouts - -````markdown -## flaky-integration - -```meta -retries = 3 # up to 4 attempts in total -retry_delay = "2s" # default: 1s -timeout = "5m" # per attempt; exit code 124 when exceeded -``` - -```bash -./run-integration-tests.sh -``` -```` - -Durations accept `ms`, `s`, `m` and `h` (a bare number is seconds). Only runtime failures are retried; missing runtimes or parameters fail immediately. A task with a `timeout` runs in its own process group so the whole process tree is stopped; for that reason it cannot read from the terminal. - -### Passing output between tasks - -Set `capture = true` to expose a task's stdout (trailing newline removed) to the tasks that depend on it as `MX_OUTPUT_` (the title upper-cased, other characters replaced with `_`). The output is still printed. - -````markdown -## version - -```meta -capture = true -``` - -```bash -git describe --tags -``` - -## release - -```meta -depends = ["version"] -``` - -```bash -echo "releasing $MX_OUTPUT_VERSION" -``` -```` - -Captured output is available to the task's direct and transitive dependents. When a task captures its output, stdout is piped, so programs may stop colouring it. - -### Importing tasks from other files - -List files in the document-wide `meta` block (before the first heading). Their tasks become available as `:`: - -````markdown -```meta -imports = ["ci/checks.md", "deploy.md"] -``` - -## release - -```meta -depends = ["checks:lint", "deploy:staging"] -``` -```` - -- Paths are relative to the importing file; imports can be nested (`a:b:task`) and circular imports are rejected -- Inside an imported file, `depends` refer to that file's own tasks by their short names -- The imported file's document-wide `env`/`dir` defaults apply to its tasks. Relative `dir` values resolve against the directory `mq-task` was invoked from, as usual - -### Skipping up-to-date tasks and watch mode - -Declare the files a task reads with `sources` (globs). The task is skipped when its sources, code, arguments and environment are unchanged since its last successful run. With `outputs`, the task also runs if any listed output is missing. - -````markdown -## build - -```meta -sources = ["src/**/*.rs", "Cargo.toml"] -outputs = ["target/release/app"] -``` - -```bash -cargo build --release -``` -```` - -Fingerprints are stored in `.mq-task/cache.json` in the directory `mq-task` runs from (consider git-ignoring `.mq-task/`). `--no-cache` ignores them. - -`--watch`/`-w` re-runs the task whenever any `sources` of the task or its dependencies, or the task file itself, change: - -```bash -mq-task test --watch -``` - -### Dependency graph - -```bash -mq-task graph ci # tree -mq-task graph --format mermaid # Mermaid diagram of all public tasks -``` - -``` -ci [parallel] -├── lint -└── test - └── build -``` - -### Interactive TUI - -`mq-task tui` shows a searchable task list with a details pane (description, dependencies, params, options, code). - -| Key | Action | -|---|---| -| `↑`/`↓`, `j`/`k` | move | -| `/` | search (matches titles, aliases, descriptions) | -| `Enter` | run inside the TUI (prompts for arguments when a param is required) | -| `a` | run with arguments (`name=value`, positional; quote values containing spaces) | -| `x` | leave the TUI and run in the terminal (interactive stdin works) | -| `q`/`Esc` | quit | - -While a task runs, the top pane draws the dependency graph as a left-to-right workflow (one stage per column, each task showing its state `○` pending, spinner running, `✔`, `✘`, `⊘`, `↻` retry and its elapsed time) and the bottom pane the log. Arrow keys (or `hjkl`) move the selection, `v` switches to a flat task list, `Tab` switches between the combined log and the selected task's log, `PgUp`/`PgDn` scroll, `q`/`Ctrl+C` aborts, and `r` re-runs. The execution flags (`--parallel`, `-j`, `-k`, `--no-cache`) apply to the TUI too. - -### Shell completions - ```bash -mq-task completions zsh > ~/.zfunc/_mq-task -mq-task completions bash > /etc/bash_completion.d/mq-task # also completes task names +mq-task # list tasks from tasks.md, else README.md +mq-task build # run a task (and its dependencies first) +mq-task deploy -- region=eu # named parameters; positional args also work +mq-task ci -j 4 # run dependencies in parallel +mq-task test --watch # re-run when sources change +mq-task -f docs/dev.md # use another file +mq-task init # write a starter tasks.md ``` -### Named parameters - -Declare parameters in the `meta` block. A bare name is required; `name=value` sets a default. - -````markdown -## deploy - -```meta -params = ["env=staging", "region"] -``` - -```bash -echo "deploying to $MX_PARAM_ENV / $MX_PARAM_REGION" -``` -```` - -```bash -mq-task deploy -- region=eu # env falls back to "staging" -mq-task deploy -- prod eu # positional, filled in declaration order -mq-task deploy -- env=prod region=eu # named -``` - -Parameters are exposed as `MX_PARAM_` environment variables. A required parameter with no value bound is an error, and so is a `name=value` argument whose name the task does not declare (`regoin=eu` is reported with a `did you mean` hint instead of becoming a value). - -### Aliases and the default task - -Give a task alternate names with `alias` in its `meta` block: - -````markdown -## build - -```meta -alias = ["b"] -``` - -```bash -cargo build -``` -```` - -`mq-task b` now runs `build`. Set `default_task` in `mq-task.toml` to run a task when `mq-task` is invoked with no task name: - -```toml -default_task = "build" -``` - -### Private tasks - -A task whose title starts with `_`, or whose `meta` block has `private = true`, is hidden from `list`/`tui` output but can still be run directly or used as a dependency: - -````markdown -## _cleanup - -```bash -rm -rf tmp/ -``` -```` - -### List available tasks - -```bash -# List tasks from tasks.md, else README.md -mq-task - -# List tasks from a specific file -mq-task -f tasks.md -mq-task list --file tasks.md - -# Include private tasks -mq-task list --all -``` - -### Get started - -```bash -mq-task init # writes a starter tasks.md -mq-task init --config # writes mq-task.toml with the default runtimes -``` - -`mq-task.toml` is loaded automatically from the directory of the task file. - -## Configuration - -Create an `mq-task.toml` file to customize runtime behavior: - -```toml -# Runtimes configuration -# Simple format: language = "command", execution mode defaults to "file" -[runtimes] -bash = "bash" -sh = "sh" -python = "python3" -ruby = "ruby" -node = "node" -javascript = "node" -js = "node" -php = "php" -perl = "perl" - -# Detailed format with execution mode -# Execution modes: "file" (default), "stdin", or "arg" -# - file: write code to a temp file and run it as an argument (keeps stdin interactive) -# - stdin: pipe code into the command's standard input (stdin unavailable to the script itself) -# - arg: pass code as a command-line argument - -[runtimes.go] -command = "go run" -execution_mode = "file" - -[runtimes.jq] -command = "jq" -execution_mode = "arg" # jq's filter is a CLI argument, not read from stdin - -[runtimes.mq] -command = "mq" -execution_mode = "arg" # mq uses query as argument -``` - -Use `execution_mode = "stdin"` only when a command needs the code piped in literally (e.g. `psql`, `redis-cli`). The task script then can't also read from the terminal. - -You can also mix both formats: - -```toml -[runtimes] -python = "python3" # Simple format, uses default file mode - -[runtimes.go] # Detailed format with custom execution mode -command = "go run" -execution_mode = "file" -``` - -```bash -# Using shorthand (from tasks.md by default) -mq-task Build - -# From a specific file -mq-task -f tasks.md Build - -# Using explicit run command -mq-task run Build -mq-task run --file tasks.md Build -``` - -`default_task` (top-level, outside `[runtimes]`) sets which task runs when `mq-task` is invoked with no task name. See [Aliases and the default task](#aliases-and-the-default-task). - -## Exit codes +## Documentation -`mq-task` exits with the same code the task's process exited with, so it composes correctly with `&&`, CI, and git hooks. A task killed by its `timeout` exits with `124`. When several tasks fail under `--keep-going`, the first failure's code is used. +- [Quick Start](https://task.mqlang.org/book/start/quickstart.html) and [Configuration](https://task.mqlang.org/book/start/configuration.html) (`mq-task.toml`, runtimes) +- [Guide](https://task.mqlang.org/book/guide/tasks.html), [meta keys](https://task.mqlang.org/book/reference/meta.html), [CLI](https://task.mqlang.org/book/reference/cli.html) and [exit codes](https://task.mqlang.org/book/reference/exit-codes.html) +- [Coming from `just` or Task](https://task.mqlang.org/book/start/migrating.html) ## License diff --git a/docs/llms.txt b/docs/llms.txt index 6726281..04e2817 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -22,9 +22,11 @@ - [Importing Tasks](https://task.mqlang.org/book/guide/imports.html): imports - [Caching and Watch Mode](https://task.mqlang.org/book/guide/cache-watch.html): sources, outputs, --watch - [Parameters, Env and Working Directory](https://task.mqlang.org/book/guide/params-env.html): params, env, dir +- [Aliases, Default and Private Tasks](https://task.mqlang.org/book/guide/aliases-private.html): alias, private, default_task - [Interactive TUI](https://task.mqlang.org/book/guide/tui.html): mq-task tui - [Checking a Task File](https://task.mqlang.org/book/guide/check.html): check, --strict, --json - [Dependency Graph](https://task.mqlang.org/book/guide/graph.html): mq-task graph +- [Shell Completions](https://task.mqlang.org/book/guide/completions.html): mq-task completions - [Meta keys](https://task.mqlang.org/book/reference/meta.html): All meta keys - [CLI](https://task.mqlang.org/book/reference/cli.html): Commands and options - [Exit codes](https://task.mqlang.org/book/reference/exit-codes.html): Exit code behaviour diff --git a/src/cache.rs b/src/cache.rs index 7cc2aa4..5a4cff3 100644 --- a/src/cache.rs +++ b/src/cache.rs @@ -6,7 +6,7 @@ use std::fs; use std::path::{Path, PathBuf}; use std::sync::Mutex; -use crate::runner::Section; +use crate::section::Section; use crate::util::Fnv; const CACHE_DIR: &str = ".mq-task"; diff --git a/src/check.rs b/src/check.rs index bb32120..019489f 100644 --- a/src/check.rs +++ b/src/check.rs @@ -12,8 +12,9 @@ use serde::Serialize; use crate::condition::{self, CondContext}; use crate::config::Config; use crate::error::Error; -use crate::runner::{LoopSpec, Runner, Section}; +use crate::runner::Runner; use crate::schedule::{build_plan, resolve_call}; +use crate::section::{LoopSpec, Section}; use crate::util::closest; #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)] @@ -565,7 +566,7 @@ fn check_conditions(sections: &[Section], out: &mut Vec) { #[cfg(test)] mod tests { use super::*; - use crate::runner::{CodeBlock, ParamDef}; + use crate::section::{CodeBlock, ParamDef}; fn task(title: &str, depends: &[&str]) -> Section { Section { @@ -603,11 +604,11 @@ mod tests { fn a_script_reading_an_undeclared_param_is_warned_about() { let mut deploy = with_code(task("deploy", &[]), "echo $MX_PARAM_REGOIN ${MX_PARAM_ENV}"); deploy.params = vec![ - crate::runner::ParamDef { + crate::section::ParamDef { name: "region".into(), ..Default::default() }, - crate::runner::ParamDef { + crate::section::ParamDef { name: "env".into(), ..Default::default() }, diff --git a/src/exec.rs b/src/exec.rs index 15ebbb6..024d4e9 100644 --- a/src/exec.rs +++ b/src/exec.rs @@ -18,8 +18,8 @@ use crate::condition::{self, CondContext}; use crate::config::{Config, ExecutionMode}; use crate::error::{Error, Result}; use crate::events::{NullObserver, RunObserver, Stream, TaskStatus}; -use crate::runner::{ParamDef, Section}; use crate::schedule::{Deferred, loop_item_vars}; +use crate::section::{ParamDef, Section}; use crate::util::closest; use crate::vars::{self, ShellCache, VarDef}; diff --git a/src/graph.rs b/src/graph.rs index bfe9ecd..8579086 100644 --- a/src/graph.rs +++ b/src/graph.rs @@ -3,8 +3,8 @@ use std::collections::{HashMap, HashSet}; use std::fmt::Write; -use crate::runner::Section; use crate::schedule::{find_section, resolve_call}; +use crate::section::Section; /// Output format for [`render`]. #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -238,7 +238,7 @@ mod tests { #[test] fn badges_describe_the_notable_settings() { - use crate::runner::LoopSpec; + use crate::section::LoopSpec; let s = Section { title: "t".into(), for_each: Some(LoopSpec::List(vec!["a".into(), "b".into()])), diff --git a/src/lib.rs b/src/lib.rs index 75597ac..c901a31 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -14,6 +14,7 @@ pub mod exec; pub mod graph; pub mod runner; pub mod schedule; +pub mod section; pub mod summary; pub mod template; pub mod tui; diff --git a/src/runner.rs b/src/runner.rs index 103bbdc..fdcd6f5 100644 --- a/src/runner.rs +++ b/src/runner.rs @@ -1,604 +1,24 @@ use std::collections::HashSet; -use std::fs; use std::path::{Path, PathBuf}; use std::sync::Arc; use std::sync::atomic::AtomicBool; -use std::time::{Duration, Instant}; +use std::time::Instant; use colored::Colorize; -use mq_lang::{DictMap, Engine, Ident, RuntimeValue, parse_markdown_input}; -use serde::{Deserialize, Serialize}; +use mq_lang::Engine; -use crate::config::{Config, RuntimeConfig}; +mod loader; + +use crate::config::Config; use crate::error::{Error, Result}; use crate::events::{ConsoleObserver, FailureTracker, OutputMode, RunObserver}; use crate::exec::Executor; use crate::graph::{self, GraphFormat}; -use crate::schedule::{self, PlanContext, ScheduleOptions, build_plan_in, split_words}; -use crate::util::parse_duration; -use crate::vars::{self, VarDef}; +use crate::schedule::{self, PlanContext, ScheduleOptions, build_plan_in}; +use crate::section::{ImportSpec, Section}; const SECTIONS_QUERY: &str = include_str!("../sections.mq"); -/// Info-string word that keeps a code block from running: -/// ` ```bash no-run `. -const NO_RUN: &str = "no-run"; - -/// Languages that name data or output rather than something to run. A block -/// in one of them is skipped, unless the configuration gives the language a -/// runtime, so README examples (` ```json `, ` ```text `) do not break a task. -const DATA_LANGUAGES: [&str; 22] = [ - "text", - "txt", - "plain", - "plaintext", - "console", - "output", - "log", - "json", - "jsonc", - "yaml", - "yml", - "toml", - "xml", - "html", - "css", - "csv", - "diff", - "ini", - "markdown", - "md", - "mermaid", - "dockerfile", -]; - -/// Represents a code block in a section -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -pub struct CodeBlock { - /// Language of the code block - pub lang: String, - /// Code content - pub code: String, -} - -/// What a `for` loop iterates over. -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -pub enum LoopSpec { - /// Literal items: `for = ["a", "b"]` - List(Vec), - /// Paths matching these globs, relative to the task's directory: - /// `for = { glob = "src/*.md" }` - Glob(Vec), - /// Every combination of the axes' values: - /// `for = { matrix = { os = ["linux", "macos"], arch = ["x64"] } }` - /// Axes are kept sorted by name. - Matrix(Vec<(String, Vec)>), - /// Lines of a dependency's captured output (`capture = true`): - /// `for = { from = "list-files" }`. Items run sequentially in the task. - From(String), -} - -impl LoopSpec { - /// Short description for graph output, e.g. `a, b, c` or `glob src/*.md`. - pub fn summary(&self) -> String { - match self { - LoopSpec::List(items) => { - let shown: Vec<&str> = items.iter().take(3).map(String::as_str).collect(); - let more = if items.len() > 3 { ", ..." } else { "" }; - format!("{}{}", shown.join(", "), more) - } - LoopSpec::Glob(patterns) => format!("glob {}", patterns.join(", ")), - LoopSpec::From(task) => format!("output of {}", task), - LoopSpec::Matrix(axes) => format!( - "matrix {}", - axes.iter() - .map(|(k, _)| k.as_str()) - .collect::>() - .join(" x ") - ), - } - } - - /// Interpret the TOML value of a `for` key. - fn from_toml(title: &str, value: &toml::Value) -> Result { - let invalid = |msg: &str| Error::Invalid(format!("task '{}': `for`: {}", title, msg)); - let scalar = |v: &toml::Value| -> Result { - match v { - toml::Value::String(s) => Ok(s.clone()), - toml::Value::Integer(i) => Ok(i.to_string()), - toml::Value::Float(f) => Ok(f.to_string()), - toml::Value::Boolean(b) => Ok(b.to_string()), - _ => Err(invalid("items must be strings, numbers or booleans")), - } - }; - let list = |v: &toml::Value| -> Result> { - match v { - toml::Value::Array(items) => items.iter().map(scalar).collect(), - _ => Err(invalid("expected an array")), - } - }; - match value { - toml::Value::Array(_) => Ok(LoopSpec::List(list(value)?)), - toml::Value::Table(table) => { - if table.len() != 1 { - return Err(invalid( - "expected exactly one of `glob`, `matrix` or `from`", - )); - } - let (key, v) = table.iter().next().expect("one entry"); - match key.as_str() { - "glob" => match v { - toml::Value::String(s) => Ok(LoopSpec::Glob(vec![s.clone()])), - other => Ok(LoopSpec::Glob(list(other)?)), - }, - "from" => match v { - toml::Value::String(task) if !task.trim().is_empty() => { - Ok(LoopSpec::From(task.trim().to_string())) - } - _ => Err(invalid("`from` must name a task")), - }, - "matrix" => { - let toml::Value::Table(axes) = v else { - return Err(invalid("`matrix` must be a table of arrays")); - }; - let mut out = Vec::new(); - for (axis, values) in axes { - out.push((axis.clone(), list(values)?)); - } - out.sort(); - Ok(LoopSpec::Matrix(out)) - } - other => Err(invalid(&format!( - "unknown key '{}' (expected `glob`, `matrix` or `from`)", - other - ))), - } - } - _ => Err(invalid( - "expected an array, `{ glob = ... }`, `{ matrix = ... }` or `{ from = ... }`", - )), - } - } -} - -/// Document-wide defaults parsed from a `meta` code block that appears -/// before the first heading in the file. Applies to every task; a task's -/// own `meta` block overrides same-named keys, and CLI flags override both. -#[derive(Debug, Deserialize, Default)] -#[serde(deny_unknown_fields)] -struct GlobalMeta { - /// Default environment variables applied to every task - #[serde(default)] - env: Vec, - /// Default working directory applied to every task without its own `dir` - #[serde(default)] - dir: Option, - /// Other task files whose tasks are made available as `:`: - /// paths, globs or `{ path, as, optional, flatten }` tables - #[serde(default)] - imports: Vec, - /// `.env` files loaded for every task - #[serde(default)] - dotenv: Vec, - /// Variables available to every task - #[serde(default)] - vars: Option, - /// Runtimes for this file's languages, in the form of `[runtimes]` in - /// `mq-task.toml` - #[serde(default)] - runtimes: Option, - /// Task run when no task is named - #[serde(default)] - default_task: Option, -} - -/// Task metadata parsed from a `meta` code block within a section -#[derive(Debug, Deserialize, Default)] -#[serde(deny_unknown_fields)] -struct TaskMeta { - #[serde(default)] - depends: Vec, - #[serde(default)] - params: Vec, - #[serde(default)] - alias: Vec, - #[serde(default)] - private: bool, - /// Default environment variables, e.g. `env = ["REGION=eu", "DEBUG=1"]` - #[serde(default)] - env: Vec, - /// Working directory to run this task's commands in, e.g. `dir = "services/api"` - #[serde(default)] - dir: Option, - /// Run this task's dependencies concurrently - #[serde(default)] - parallel: bool, - /// Condition that must hold for the task to run - #[serde(default, rename = "if")] - condition: Option, - /// How many times to retry after a failure - #[serde(default)] - retries: u32, - /// Delay between retries, e.g. `"2s"` - #[serde(default)] - retry_delay: Option, - /// Kill the task if it runs longer than this, e.g. `"30s"` - #[serde(default)] - timeout: Option, - /// Expose this task's stdout to dependents as `MX_OUTPUT_` - #[serde(default)] - capture: bool, - /// Globs of input files; unchanged inputs let the task be skipped - #[serde(default)] - sources: Vec, - /// Globs of files the task produces; must exist for it to be skipped - #[serde(default)] - outputs: Vec, - /// Run the task once per item: an array, `{ glob = ... }` or `{ matrix = ... }` - #[serde(default, rename = "for")] - for_each: Option, - /// Extra environment variable name for the loop item, e.g. `as = "file"` - #[serde(default, rename = "as")] - loop_var: Option, - /// Tasks run after this one finishes, even if it failed - #[serde(default)] - defer: Vec, - /// Treat a failed run (after retries) as success - #[serde(default)] - ignore_error: bool, - /// Ask for confirmation before running, e.g. `prompt = "Deploy to production?"` - #[serde(default)] - prompt: Option, - /// `.env` files loaded for this task, e.g. `dotenv = [".env", ".env.local"]` - #[serde(default)] - dotenv: Vec, - /// Shell commands; the task is up to date when all of them exit 0 - #[serde(default)] - status: Vec, - /// Variables: fixed values or `{ sh = "command" }` - #[serde(default)] - vars: Option, - /// Run only once per run, however it is called - #[serde(default)] - once: bool, - /// Variables that must be set (and not empty), e.g. `["API_TOKEN"]` - #[serde(default)] - requires: Vec, -} - -/// A named task parameter, e.g. `params = ["env=staging", "verbose"]`. -/// A bare name is required; `name=value` supplies a default. -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Default)] -pub struct ParamDef { - pub name: String, - pub default: Option, - /// The values the parameter accepts; any value when empty. - #[serde(default)] - pub choices: Vec, - /// What the parameter is for, shown by `--summary` and the TUI. - #[serde(default)] - pub description: Option, -} - -impl ParamDef { - /// The short form: `"name"` (required) or `"name=default"`. - fn parse(raw: &str) -> Self { - match raw.split_once('=') { - Some((name, default)) => ParamDef { - name: name.trim().to_string(), - default: Some(default.trim().to_string()), - ..Default::default() - }, - None => ParamDef { - name: raw.trim().to_string(), - ..Default::default() - }, - } - } - - /// The table form: `{ name = "env", default = "dev", choices = [...], desc = "..." }`. - fn from_toml(task: &str, value: &toml::Value) -> Result { - let invalid = |msg: &str| Error::Invalid(format!("task '{}': `params`: {}", task, msg)); - match value { - toml::Value::String(raw) => Ok(Self::parse(raw)), - toml::Value::Table(table) => { - const KEYS: [&str; 4] = ["name", "default", "choices", "desc"]; - let text = |key: &str| -> Result> { - match table.get(key) { - None => Ok(None), - Some(toml::Value::String(s)) => Ok(Some(s.clone())), - Some(_) => Err(invalid(&format!("`{}` must be a string", key))), - } - }; - for key in table.keys() { - if !KEYS.contains(&key.as_str()) { - let hint = crate::util::closest(key, KEYS) - .map(|c| format!(" (did you mean `{}`?)", c)) - .unwrap_or_else(|| { - " (expected `name`, `default`, `choices` or `desc`)".into() - }); - return Err(invalid(&format!("unknown key `{}`{}", key, hint))); - } - } - let name = text("name")? - .map(|n| n.trim().to_string()) - .filter(|n| !n.is_empty()) - .ok_or_else(|| invalid("a table needs a `name`"))?; - let choices = match table.get("choices") { - None => Vec::new(), - Some(toml::Value::Array(items)) => items - .iter() - .map(|v| match v { - toml::Value::String(s) => Ok(s.clone()), - other => Ok(other.to_string()), - }) - .collect::>>()?, - Some(_) => return Err(invalid("`choices` must be an array")), - }; - if table.contains_key("choices") && choices.is_empty() { - return Err(invalid(&format!("`choices` of '{}' is empty", name))); - } - let default = text("default")?; - if let Some(d) = &default - && !choices.is_empty() - && !choices.contains(d) - { - return Err(invalid(&format!( - "the default '{}' of '{}' is not one of its choices ({})", - d, - name, - choices.join(", ") - ))); - } - Ok(ParamDef { - name, - default, - choices, - description: text("desc")?, - }) - } - _ => Err(invalid("expected a string or a table")), - } - } - - /// All of `params` on one line. The entries are separated by commas, or - /// by semicolons when some carry choices or a description (which can - /// contain commas). - pub fn describe_all(params: &[ParamDef]) -> String { - let rich = params - .iter() - .any(|p| !p.choices.is_empty() || p.description.is_some()); - let separator = if rich { "; " } else { ", " }; - params - .iter() - .map(ParamDef::describe) - .collect::>() - .join(separator) - } - - /// One line for `--summary` and the TUI: `env=dev [dev|prod] - target`. - pub fn describe(&self) -> String { - let mut text = match &self.default { - Some(default) => format!("{}={}", self.name, default), - None => format!("{} (required)", self.name), - }; - if !self.choices.is_empty() { - text.push_str(&format!(" [{}]", self.choices.join("|"))); - } - if let Some(description) = self.description.as_deref().filter(|d| !d.is_empty()) { - text.push_str(&format!(" - {}", description)); - } - text - } -} - -/// Represents a section with its code blocks -#[derive(Debug, Clone, Serialize, Deserialize, Default)] -pub struct Section { - /// Section title - pub title: String, - /// Code blocks in this section (excludes `meta` metadata blocks) - pub codes: Vec, - /// Optional description extracted from the section content - pub description: Option, - /// Task names this section depends on (declared in a `meta` code block) - #[serde(default)] - pub depends: Vec, - /// Named parameters this task accepts (declared in a `meta` code block) - #[serde(default)] - pub params: Vec, - /// Alternate names this task can be run by (declared in a `meta` code block) - #[serde(default)] - pub aliases: Vec, - /// Hidden from `list`/`tui` output; set via `meta` or a `_`-prefixed title - #[serde(default)] - pub private: bool, - /// Default environment variables declared in a `meta` code block; - /// overridden by CLI `--env` flags of the same name - #[serde(default)] - pub env: Vec<(String, String)>, - /// Working directory declared in a `meta` code block; overridden by - /// the CLI `--dir` flag - #[serde(default)] - pub dir: Option, - /// Run this task's dependencies concurrently - #[serde(default)] - pub parallel: bool, - /// `if` condition; the task is skipped when it evaluates to false - #[serde(default)] - pub condition: Option, - /// Number of retries after a failed attempt - #[serde(default)] - pub retries: u32, - /// Delay between retries (defaults to one second) - #[serde(default)] - pub retry_delay: Option, - /// Maximum run time per attempt - #[serde(default)] - pub timeout: Option, - /// Capture stdout and hand it to dependents as `MX_OUTPUT_` - #[serde(default)] - pub capture: bool, - /// Input globs for up-to-date checks and `--watch` - #[serde(default)] - pub sources: Vec, - /// Output globs that must exist for the task to count as up to date - #[serde(default)] - pub outputs: Vec, - /// `for` loop: the task runs once per item - #[serde(default)] - pub for_each: Option, - /// Extra name for the loop item (`as = "file"` sets `MX_FILE`) - #[serde(default)] - pub loop_var: Option, - /// Tasks (optionally with arguments) run after this one finishes, - /// whatever its outcome - #[serde(default)] - pub defer: Vec, - /// A failed run (after retries) is reported but not treated as a failure - #[serde(default)] - pub ignore_error: bool, - /// Confirmation message; the task runs only on yes or `--yes` - #[serde(default)] - pub prompt: Option, - /// `.env` files, relative to the task's working directory - #[serde(default)] - pub dotenv: Vec, - /// Shell commands; the task is skipped when all exit 0 - #[serde(default)] - pub status: Vec, - /// Runs once per run even if called with different arguments (first wins) - #[serde(default)] - pub once: bool, - /// Variables that must be set and not empty - #[serde(default)] - pub requires: Vec, - /// Named values for the task, fixed or computed by a shell command - #[serde(default)] - pub vars: Vec, -} - -/// One entry of `imports`: a path (or glob) to a task file, with the options -/// of the table form. -#[derive(Debug, Clone, PartialEq)] -struct ImportSpec { - path: String, - /// Namespace to use instead of the file name. - alias: Option, - /// A missing file is not an error. - optional: bool, - /// Add the tasks under their own names, without a namespace. - flatten: bool, -} - -impl ImportSpec { - fn is_glob(&self) -> bool { - self.path.contains(['*', '?', '[']) - } - - fn from_toml(value: &toml::Value) -> Result { - let invalid = - |msg: &str| Error::Invalid(format!("document `meta` block: `imports`: {}", msg)); - let spec = match value { - toml::Value::String(path) => ImportSpec { - path: path.clone(), - alias: None, - optional: false, - flatten: false, - }, - toml::Value::Table(table) => { - const KEYS: [&str; 4] = ["path", "as", "optional", "flatten"]; - for key in table.keys() { - if !KEYS.contains(&key.as_str()) { - let hint = crate::util::closest(key, KEYS) - .map(|c| format!(" (did you mean `{}`?)", c)) - .unwrap_or_else(|| { - " (expected `path`, `as`, `optional` or `flatten`)".into() - }); - return Err(invalid(&format!("unknown key `{}`{}", key, hint))); - } - } - let text = |key: &str| match table.get(key) { - None => Ok(None), - Some(toml::Value::String(s)) if !s.trim().is_empty() => Ok(Some(s.clone())), - Some(_) => Err(invalid(&format!("`{}` must be a non-empty string", key))), - }; - let flag = |key: &str| match table.get(key) { - None => Ok(false), - Some(toml::Value::Boolean(b)) => Ok(*b), - Some(_) => Err(invalid(&format!("`{}` must be true or false", key))), - }; - ImportSpec { - path: text("path")?.ok_or_else(|| invalid("a table needs a `path`"))?, - alias: text("as")?, - optional: flag("optional")?, - flatten: flag("flatten")?, - } - } - _ => return Err(invalid("expected a path or a table")), - }; - if spec.is_glob() && spec.alias.is_some() { - return Err(invalid(&format!( - "`as` cannot be used with the glob '{}': every file is named after itself", - spec.path - ))); - } - if spec.flatten && spec.alias.is_some() { - return Err(invalid(&format!( - "`as` has no effect with `flatten` ('{}')", - spec.path - ))); - } - Ok(spec) - } - - /// The files this entry stands for under `base`, each with its - /// namespace. `current` (the importing file) is never matched by a glob. - fn resolve(&self, base: &Path, current: &Path) -> Result> { - let namespace = |path: &Path| { - path.file_stem() - .map(|s| s.to_string_lossy().into_owned()) - .ok_or_else(|| Error::Invalid(format!("invalid import path '{}'", self.path))) - }; - if self.is_glob() { - let pattern = base.join(&self.path).to_string_lossy().into_owned(); - let paths = glob::glob(&pattern).map_err(|e| { - Error::Invalid(format!("invalid import pattern '{}': {}", self.path, e.msg)) - })?; - let mut found: Vec = paths - .flatten() - .filter(|p| p.is_file() && p.canonicalize().ok().as_deref() != Some(current)) - .collect(); - found.sort(); - return found - .into_iter() - .map(|p| namespace(&p).map(|ns| (p, ns))) - .collect(); - } - let path = base.join(&self.path); - if self.optional && !path.exists() { - return Ok(Vec::new()); - } - let namespace = match &self.alias { - Some(alias) => alias.clone(), - None => namespace(&path)?, - }; - Ok(vec![(path, namespace)]) - } -} - -/// Document-wide settings found in the preamble `meta` block. -#[derive(Debug, Default)] -struct Globals { - env: Vec<(String, String)>, - dir: Option, - imports: Vec, - dotenv: Vec, - vars: Vec, - runtimes: Vec<(String, RuntimeConfig)>, - default_task: Option, -} - /// Options for [`Runner::run_task_with_options`]. #[derive(Default)] pub struct RunOptions { @@ -800,420 +220,6 @@ impl Runner { self.exec.working_dir = working_dir; } - /// Load and parse a Markdown file - pub fn load_markdown>(&self, path: P) -> Result { - fs::read_to_string(path).map_err(Error::Io) - } - - /// Extract sections from Markdown content. Does not follow `imports`; - /// use [`Runner::load_sections`] to load a file with its imports. - pub fn extract_sections(&mut self, markdown: &str) -> Result> { - let input = parse_markdown_input(markdown) - .map_err(|e| Error::Markdown(format!("Failed to parse markdown: {}", e)))?; - - let globals = match self.parse_global_meta(input.clone()) { - Ok(globals) => globals, - Err(e) => match self.collected.as_mut() { - Some(collected) => { - collected.errors.push(e); - Globals::default() - } - None => return Err(e), - }, - }; - self.exec.global_env = globals.env; - self.exec.global_dir = globals.dir; - self.exec.global_dotenv = globals.dotenv; - self.exec.global_vars = globals.vars; - self.imports = globals.imports; - self.file_default_task = globals.default_task; - for (lang, runtime) in globals.runtimes { - if !self.pinned_runtimes.contains(&lang) && self.document_runtimes.insert(lang.clone()) - { - self.exec.config.runtimes.insert(lang, runtime); - } - } - - let query = format!("{}\n | nodes | sections_with_code()", SECTIONS_QUERY); - let result = self - .engine - .eval(&query, input.into_iter()) - .map_err(|e| Error::Query(format!("Failed to execute query: {}", e)))?; - - let sections = self.parse_sections(result)?; - - Ok(sections) - } - - /// Like [`Runner::load_sections`], but carries on after a broken task or - /// import. Returns the tasks that loaded, the problems found and the - /// titles of the tasks that could not be parsed. - pub fn load_sections_lenient>( - &mut self, - path: P, - ) -> (Vec
, Vec, Vec) { - self.collected = Some(Collected::default()); - self.document_runtimes.clear(); - let mut stack = Vec::new(); - let sections = match self.load_file(path.as_ref(), &mut stack) { - Ok((sections, main)) => { - self.default_task = main.default_task; - sections - } - Err(e) => { - if let Some(collected) = self.collected.as_mut() { - collected.errors.push(e); - } - Vec::new() - } - }; - let collected = self.collected.take().unwrap_or_default(); - (sections, collected.errors, collected.failed) - } - - /// Load the tasks of a Markdown file together with those of the files - /// it `imports`. Imported tasks are named `:`. - pub fn load_sections>(&mut self, path: P) -> Result> { - let mut stack = Vec::new(); - self.document_runtimes.clear(); - let (sections, main) = self.load_file(path.as_ref(), &mut stack)?; - self.default_task = main.default_task; - // Loading imports overwrote the document defaults; the main file's - // own defaults apply at run time, so restore them. - let markdown = self.load_markdown(path.as_ref())?; - let input = parse_markdown_input(&markdown) - .map_err(|e| Error::Markdown(format!("Failed to parse markdown: {}", e)))?; - let globals = self.parse_global_meta(input)?; - self.exec.global_env = globals.env; - self.exec.global_dir = globals.dir; - self.exec.global_dotenv = globals.dotenv; - self.exec.global_vars = globals.vars; - // Spelling the path differently (`tasks.md`, `./tasks.md`) must not - // give the same file a second set of fingerprints. - let scope = path - .as_ref() - .canonicalize() - .unwrap_or_else(|_| path.as_ref().to_path_buf()); - self.exec.cache_scope = scope.to_string_lossy().into_owned(); - Ok(sections) - } - - /// Returns the file's sections (imports merged in, already namespaced) - /// and its document-wide defaults, which the caller applies. - fn load_file( - &mut self, - path: &Path, - stack: &mut Vec, - ) -> Result<(Vec
, Globals)> { - let canonical = path.canonicalize().unwrap_or_else(|_| path.to_path_buf()); - if stack.contains(&canonical) { - return Err(Error::Invalid(format!( - "circular import of {}", - path.display() - ))); - } - stack.push(canonical); - - let markdown = self.load_markdown(path)?; - let mut sections = self.extract_sections(&markdown)?; - let globals = Globals { - env: self.exec.global_env.clone(), - dir: self.exec.global_dir.clone(), - imports: std::mem::take(&mut self.imports), - dotenv: self.exec.global_dotenv.clone(), - vars: self.exec.global_vars.clone(), - runtimes: Vec::new(), - default_task: self.file_default_task.take(), - }; - - let base = path.parent().unwrap_or_else(|| Path::new(".")); - let current = path.canonicalize().unwrap_or_else(|_| path.to_path_buf()); - let mut namespaces = HashSet::new(); - for spec in &globals.imports { - let import = spec.path.as_str(); - let targets = match spec.resolve(base, ¤t) { - Ok(targets) => targets, - Err(e) => match self.collected.as_mut() { - Some(collected) => { - collected.errors.push(e); - continue; - } - None => return Err(e), - }, - }; - for (import_path, stem) in targets { - if !spec.flatten && !namespaces.insert(stem.clone()) { - let e = Error::Invalid(format!("two imports share the namespace '{}'", stem)); - match self.collected.as_mut() { - Some(collected) => { - collected.errors.push(e); - continue; - } - None => return Err(e), - } - } - - let (mut imported, child_globals) = match self.load_file(&import_path, stack) { - Ok(loaded) => loaded, - Err(e) => match self.collected.as_mut() { - Some(collected) => { - collected - .errors - .push(Error::Invalid(format!("import '{}': {}", import, e))); - continue; - } - None => return Err(e), - }, - }; - // Importing a file's own heading (its document title) as a task - // is noise; keep only sections that do something. - imported.retain(|s| !s.codes.is_empty() || !s.depends.is_empty()); - apply_globals(&mut imported, &child_globals); - if !spec.flatten { - namespace_sections(&mut imported, &stem); - } - sections.extend(imported); - } - } - - stack.pop(); - Ok((sections, globals)) - } - - /// Parse the document-wide `meta` block declared before the first - /// heading, if any, into defaults for every task. - fn parse_global_meta(&mut self, input: Vec) -> Result { - let query = format!( - "{}\n | nodes | {{\"codes\": global_code_blocks()}}", - SECTIONS_QUERY - ); - let result = self - .engine - .eval(&query, input.into_iter()) - .map_err(|e| Error::Query(format!("Failed to execute query: {}", e)))?; - - let codes = result - .into_iter() - .find_map(|value| match value { - RuntimeValue::Dict(dict) => dict.get(&Ident::from("codes")).and_then(|v| match v { - RuntimeValue::Array(arr) => self.parse_code_blocks(arr).ok(), - _ => None, - }), - _ => None, - }) - .unwrap_or_default(); - - let meta = match codes.iter().find(|c| c.lang == "meta") { - Some(block) => toml::from_str::(&block.code) - .map_err(|e| meta_error("document `meta` block", &block.code, &e, true))?, - None => GlobalMeta::default(), - }; - - Ok(Globals { - env: Self::parse_env_overrides(&meta.env)?, - dir: meta.dir.map(PathBuf::from), - imports: meta - .imports - .iter() - .map(ImportSpec::from_toml) - .collect::>()?, - dotenv: meta.dotenv, - vars: match &meta.vars { - Some(table) => vars::parse_table("document `meta` block", table)?, - None => Vec::new(), - }, - runtimes: match meta.runtimes { - Some(table) => table - .into_iter() - .map(|(lang, value)| { - value - .try_into::() - .map(|runtime| (lang.clone(), runtime)) - .map_err(|e| { - Error::Invalid(format!( - "document `meta` block: `runtimes`: '{}' must be a command or {{ command, execution_mode }}: {}", - lang, - e.message() - )) - }) - }) - .collect::>()?, - None => Vec::new(), - }, - default_task: meta.default_task, - }) - } - - fn parse_sections(&mut self, result: mq_lang::RuntimeValues) -> Result> { - let mut sections = Vec::new(); - - for value in result.into_iter() { - if let RuntimeValue::Dict(dict) = value { - match self.parse_section(&dict) { - Ok(section) => sections.push(section), - Err(e) => match self.collected.as_mut() { - Some(collected) => { - if let Some(RuntimeValue::String(title)) = - dict.get(&Ident::from("title")) - { - collected.failed.push(title.to_string()); - } - collected.errors.push(e); - } - None => return Err(e), - }, - } - } - } - - Ok(sections) - } - - fn parse_section(&self, dict: &DictMap) -> Result
{ - let title = dict - .get(&Ident::from("title")) - .and_then(|v| match v { - RuntimeValue::String(s) => Some(s.to_string()), - _ => None, - }) - .unwrap_or_default(); - - let all_codes = dict - .get(&Ident::from("codes")) - .and_then(|v| match v { - RuntimeValue::Array(arr) => Some(self.parse_code_blocks(arr)), - _ => None, - }) - .unwrap_or_else(|| Ok(Vec::new()))?; - - let meta = match all_codes.iter().find(|c| c.lang == "meta") { - Some(block) => toml::from_str::(&block.code).map_err(|e| { - meta_error( - &format!("task '{}': `meta` block", title), - &block.code, - &e, - false, - ) - })?, - None => TaskMeta::default(), - }; - - let duration = |value: &Option| -> Result> { - value - .as_deref() - .map(parse_duration) - .transpose() - .map_err(|e| Error::Invalid(format!("task '{}': {}", title, e))) - }; - let retry_delay = duration(&meta.retry_delay)?; - let timeout = duration(&meta.timeout)?; - - let params = meta - .params - .iter() - .map(|p| ParamDef::from_toml(&title, p)) - .collect::>>()?; - let private = meta.private || title.starts_with('_'); - let env = Self::parse_env_overrides(&meta.env)?; - let dir = meta.dir.map(PathBuf::from); - - let vars = match &meta.vars { - Some(table) => vars::parse_table(&format!("task '{}': `meta` block", title), table)?, - None => Vec::new(), - }; - - let for_each = meta - .for_each - .as_ref() - .map(|v| LoopSpec::from_toml(&title, v)) - .transpose()?; - if meta.loop_var.is_some() && for_each.is_none() { - return Err(Error::Invalid(format!( - "task '{}': `as` requires `for`", - title - ))); - } - - let codes: Vec = all_codes - .into_iter() - .filter(|c| c.lang != "meta") - .filter(|c| { - self.exec.config.has_runtime(&c.lang) || !DATA_LANGUAGES.contains(&c.lang.as_str()) - }) - .collect(); - - let description = dict.get(&Ident::from("description")).and_then(|v| match v { - RuntimeValue::String(s) => Some(s.to_string()), - _ => None, - }); - - Ok(Section { - title, - codes, - description, - depends: meta.depends, - params, - aliases: meta.alias, - private, - env, - dir, - parallel: meta.parallel, - condition: meta.condition, - retries: meta.retries, - retry_delay, - timeout, - capture: meta.capture, - sources: meta.sources, - outputs: meta.outputs, - for_each, - loop_var: meta.loop_var, - defer: meta.defer, - ignore_error: meta.ignore_error, - prompt: meta.prompt, - dotenv: meta.dotenv, - status: meta.status, - once: meta.once, - requires: meta.requires, - vars, - }) - } - - fn parse_code_blocks(&self, arr: &[RuntimeValue]) -> Result> { - let mut blocks = Vec::new(); - - for item in arr { - if let RuntimeValue::Dict(dict) = item { - let lang = dict - .get(&Ident::from("lang")) - .and_then(|v| match v { - RuntimeValue::String(s) => Some(s.to_string()), - _ => None, - }) - .unwrap_or_default(); - - let code = dict - .get(&Ident::from("code")) - .and_then(|v| match v { - RuntimeValue::String(s) => Some(s.to_string()), - _ => None, - }) - .unwrap_or_default(); - - let no_run = dict.get(&Ident::from("info")).is_some_and(|v| match v { - RuntimeValue::String(s) => s.split_whitespace().any(|w| w == NO_RUN), - _ => false, - }); - if no_run { - continue; - } - - blocks.push(CodeBlock { lang, code }); - } - } - - Ok(blocks) - } - pub fn find_section<'a>(&self, sections: &'a [Section], title: &str) -> Option<&'a Section> { schedule::find_section(sections, title) } @@ -1584,117 +590,6 @@ impl Runner { } } -/// Fold an imported file's document-wide defaults into its tasks so they -/// keep behaving as they would when run from that file directly. -fn apply_globals(sections: &mut [Section], globals: &Globals) { - for section in sections { - let mut env = globals.env.clone(); - env.append(&mut section.env); - section.env = env; - if section.dir.is_none() { - section.dir = globals.dir.clone(); - } - let mut dotenv = globals.dotenv.clone(); - dotenv.append(&mut section.dotenv); - section.dotenv = dotenv; - let mut vars = globals.vars.clone(); - vars.append(&mut section.vars); - section.vars = vars; - } -} - -/// Prefix titles, aliases and intra-file dependency references with -/// `:`. References to tasks outside the file are left alone. -fn namespace_sections(sections: &mut [Section], namespace: &str) { - let local: HashSet = sections - .iter() - .flat_map(|s| std::iter::once(s.title.clone()).chain(s.aliases.iter().cloned())) - .collect(); - let qualify = |name: &str| format!("{}:{}", namespace, name); - - let qualify_call = |spec: &str| -> String { - if local.contains(spec.trim()) { - return qualify(spec.trim()); - } - let words = split_words(spec); - for n in (1..=words.len()).rev() { - let name = words[..n].join(" "); - if local.contains(&name) { - let args = words[n..].iter().map(|w| quote_word(w)); - return std::iter::once(qualify(&name)) - .chain(args) - .collect::>() - .join(" "); - } - } - spec.to_string() - }; - - for section in sections { - if let Some(LoopSpec::From(source)) = &mut section.for_each - && local.contains(source.as_str()) - { - *source = qualify(source); - } - section.title = qualify(§ion.title); - section.aliases = section.aliases.iter().map(|a| qualify(a)).collect(); - for dep in section.depends.iter_mut().chain(section.defer.iter_mut()) { - *dep = qualify_call(dep); - } - } -} - -/// The key and the valid keys from a serde "unknown field" message such as -/// ``unknown field `retires`, expected one of `retries`, `timeout` ``. -fn unknown_field(message: &str) -> Option<(String, Vec)> { - let rest = message.strip_prefix("unknown field ")?; - let mut quoted = rest.split('`').skip(1).step_by(2).map(str::to_string); - let key = quoted.next()?; - Some((key, quoted.collect())) -} - -/// Turn a `meta` block parse error into one a user can act on: unknown keys -/// get a "did you mean" hint and the line they are on. -fn meta_error(context: &str, code: &str, err: &toml::de::Error, document: bool) -> Error { - let Some((key, valid)) = unknown_field(err.message()) else { - return Error::Invalid(format!("{}: {}", context, err)); - }; - let line = err - .span() - .map(|s| { - format!( - " on line {}", - code[..s.start.min(code.len())].matches('\n').count() + 1 - ) - }) - .unwrap_or_default(); - let hint = if !document && key == "imports" { - "`imports` only works in the document-wide `meta` block before the first heading" - .to_string() - } else if let Some(close) = crate::util::closest(&key, valid.iter().map(String::as_str)) { - format!("did you mean `{}`?", close) - } else { - "see the Meta keys reference for the valid keys".to_string() - }; - Error::Invalid(format!( - "{}: unknown key `{}`{} ({})", - context, key, line, hint - )) -} - -/// Quote a word for [`split_words`] when it would not survive unquoted. -fn quote_word(word: &str) -> String { - let plain = - !word.is_empty() && !word.contains(|c: char| c.is_whitespace() || c == '"' || c == '\''); - if plain { - word.to_string() - } else if word.contains('"') { - format!("'{}'", word) - } else { - format!("\"{}\"", word) - } -} - #[cfg(test)] impl Runner { fn resolve_execution_order<'a>( @@ -1710,7 +605,7 @@ impl Runner { } fn bind_params( - params: &[ParamDef], + params: &[crate::section::ParamDef], args: &[String], task_name: &str, ) -> Result> { @@ -1728,7 +623,10 @@ impl Runner { #[cfg(test)] mod tests { + use super::loader::*; use super::*; + use crate::schedule::split_words; + use crate::section::*; #[test] fn test_extract_sections() { @@ -2887,7 +1785,8 @@ true let md = "```meta\nvars = { A = \"1\", B = { sh = \"echo b\" } }\n```\n\n## t\n\n```meta\nvars = { C = 3 }\n```\n\n```bash\ntrue\n```\n"; let mut runner = Runner::with_default_config(); let sections = runner.extract_sections(md).unwrap(); - let names = |vars: &[VarDef]| vars.iter().map(|v| v.name.clone()).collect::>(); + let names = + |vars: &[crate::vars::VarDef]| vars.iter().map(|v| v.name.clone()).collect::>(); assert_eq!(names(&runner.exec.global_vars), vec!["A", "B"]); assert_eq!( sections[0].vars[0].value, diff --git a/src/runner/loader.rs b/src/runner/loader.rs new file mode 100644 index 0000000..22eb31e --- /dev/null +++ b/src/runner/loader.rs @@ -0,0 +1,583 @@ +//! Reading a task file: loading, import resolution and turning the parsed +//! Markdown into [`Section`]s. + +use std::collections::HashSet; +use std::fs; +use std::path::{Path, PathBuf}; +use std::time::Duration; + +use mq_lang::{DictMap, Ident, RuntimeValue, parse_markdown_input}; + +use super::{Collected, Runner, SECTIONS_QUERY}; +use crate::config::RuntimeConfig; +use crate::error::{Error, Result}; +use crate::schedule::split_words; +use crate::section::{ + CodeBlock, GlobalMeta, Globals, ImportSpec, LoopSpec, ParamDef, Section, TaskMeta, +}; +use crate::util::parse_duration; +use crate::vars; + +/// Info-string word that keeps a code block from running: +/// ` ```bash no-run `. +const NO_RUN: &str = "no-run"; + +/// Languages that name data or output rather than something to run. A block +/// in one of them is skipped, unless the configuration gives the language a +/// runtime, so README examples (` ```json `, ` ```text `) do not break a task. +const DATA_LANGUAGES: [&str; 22] = [ + "text", + "txt", + "plain", + "plaintext", + "console", + "output", + "log", + "json", + "jsonc", + "yaml", + "yml", + "toml", + "xml", + "html", + "css", + "csv", + "diff", + "ini", + "markdown", + "md", + "mermaid", + "dockerfile", +]; + +impl Runner { + /// Load and parse a Markdown file + pub fn load_markdown>(&self, path: P) -> Result { + fs::read_to_string(path).map_err(Error::Io) + } + + /// Extract sections from Markdown content. Does not follow `imports`; + /// use [`Runner::load_sections`] to load a file with its imports. + pub fn extract_sections(&mut self, markdown: &str) -> Result> { + let input = parse_markdown_input(markdown) + .map_err(|e| Error::Markdown(format!("Failed to parse markdown: {}", e)))?; + + let globals = match self.parse_global_meta(input.clone()) { + Ok(globals) => globals, + Err(e) => match self.collected.as_mut() { + Some(collected) => { + collected.errors.push(e); + Globals::default() + } + None => return Err(e), + }, + }; + self.exec.global_env = globals.env; + self.exec.global_dir = globals.dir; + self.exec.global_dotenv = globals.dotenv; + self.exec.global_vars = globals.vars; + self.imports = globals.imports; + self.file_default_task = globals.default_task; + for (lang, runtime) in globals.runtimes { + if !self.pinned_runtimes.contains(&lang) && self.document_runtimes.insert(lang.clone()) + { + self.exec.config.runtimes.insert(lang, runtime); + } + } + + let query = format!("{}\n | nodes | sections_with_code()", SECTIONS_QUERY); + let result = self + .engine + .eval(&query, input.into_iter()) + .map_err(|e| Error::Query(format!("Failed to execute query: {}", e)))?; + + let sections = self.parse_sections(result)?; + + Ok(sections) + } + + /// Like [`Runner::load_sections`], but carries on after a broken task or + /// import. Returns the tasks that loaded, the problems found and the + /// titles of the tasks that could not be parsed. + pub fn load_sections_lenient>( + &mut self, + path: P, + ) -> (Vec
, Vec, Vec) { + self.collected = Some(Collected::default()); + self.document_runtimes.clear(); + let mut stack = Vec::new(); + let sections = match self.load_file(path.as_ref(), &mut stack) { + Ok((sections, main)) => { + self.default_task = main.default_task; + sections + } + Err(e) => { + if let Some(collected) = self.collected.as_mut() { + collected.errors.push(e); + } + Vec::new() + } + }; + let collected = self.collected.take().unwrap_or_default(); + (sections, collected.errors, collected.failed) + } + + /// Load the tasks of a Markdown file together with those of the files + /// it `imports`. Imported tasks are named `:`. + pub fn load_sections>(&mut self, path: P) -> Result> { + let mut stack = Vec::new(); + self.document_runtimes.clear(); + let (sections, main) = self.load_file(path.as_ref(), &mut stack)?; + self.default_task = main.default_task; + // Loading imports overwrote the document defaults; the main file's + // own defaults apply at run time, so restore them. + let markdown = self.load_markdown(path.as_ref())?; + let input = parse_markdown_input(&markdown) + .map_err(|e| Error::Markdown(format!("Failed to parse markdown: {}", e)))?; + let globals = self.parse_global_meta(input)?; + self.exec.global_env = globals.env; + self.exec.global_dir = globals.dir; + self.exec.global_dotenv = globals.dotenv; + self.exec.global_vars = globals.vars; + // Spelling the path differently (`tasks.md`, `./tasks.md`) must not + // give the same file a second set of fingerprints. + let scope = path + .as_ref() + .canonicalize() + .unwrap_or_else(|_| path.as_ref().to_path_buf()); + self.exec.cache_scope = scope.to_string_lossy().into_owned(); + Ok(sections) + } + + /// Returns the file's sections (imports merged in, already namespaced) + /// and its document-wide defaults, which the caller applies. + fn load_file( + &mut self, + path: &Path, + stack: &mut Vec, + ) -> Result<(Vec
, Globals)> { + let canonical = path.canonicalize().unwrap_or_else(|_| path.to_path_buf()); + if stack.contains(&canonical) { + return Err(Error::Invalid(format!( + "circular import of {}", + path.display() + ))); + } + stack.push(canonical); + + let markdown = self.load_markdown(path)?; + let mut sections = self.extract_sections(&markdown)?; + let globals = Globals { + env: self.exec.global_env.clone(), + dir: self.exec.global_dir.clone(), + imports: std::mem::take(&mut self.imports), + dotenv: self.exec.global_dotenv.clone(), + vars: self.exec.global_vars.clone(), + runtimes: Vec::new(), + default_task: self.file_default_task.take(), + }; + + let base = path.parent().unwrap_or_else(|| Path::new(".")); + let current = path.canonicalize().unwrap_or_else(|_| path.to_path_buf()); + let mut namespaces = HashSet::new(); + for spec in &globals.imports { + let import = spec.path.as_str(); + let targets = match spec.resolve(base, ¤t) { + Ok(targets) => targets, + Err(e) => match self.collected.as_mut() { + Some(collected) => { + collected.errors.push(e); + continue; + } + None => return Err(e), + }, + }; + for (import_path, stem) in targets { + if !spec.flatten && !namespaces.insert(stem.clone()) { + let e = Error::Invalid(format!("two imports share the namespace '{}'", stem)); + match self.collected.as_mut() { + Some(collected) => { + collected.errors.push(e); + continue; + } + None => return Err(e), + } + } + + let (mut imported, child_globals) = match self.load_file(&import_path, stack) { + Ok(loaded) => loaded, + Err(e) => match self.collected.as_mut() { + Some(collected) => { + collected + .errors + .push(Error::Invalid(format!("import '{}': {}", import, e))); + continue; + } + None => return Err(e), + }, + }; + // Importing a file's own heading (its document title) as a task + // is noise; keep only sections that do something. + imported.retain(|s| !s.codes.is_empty() || !s.depends.is_empty()); + apply_globals(&mut imported, &child_globals); + if !spec.flatten { + namespace_sections(&mut imported, &stem); + } + sections.extend(imported); + } + } + + stack.pop(); + Ok((sections, globals)) + } + + /// Parse the document-wide `meta` block declared before the first + /// heading, if any, into defaults for every task. + fn parse_global_meta(&mut self, input: Vec) -> Result { + let query = format!( + "{}\n | nodes | {{\"codes\": global_code_blocks()}}", + SECTIONS_QUERY + ); + let result = self + .engine + .eval(&query, input.into_iter()) + .map_err(|e| Error::Query(format!("Failed to execute query: {}", e)))?; + + let codes = result + .into_iter() + .find_map(|value| match value { + RuntimeValue::Dict(dict) => dict.get(&Ident::from("codes")).and_then(|v| match v { + RuntimeValue::Array(arr) => self.parse_code_blocks(arr).ok(), + _ => None, + }), + _ => None, + }) + .unwrap_or_default(); + + let meta = match codes.iter().find(|c| c.lang == "meta") { + Some(block) => toml::from_str::(&block.code) + .map_err(|e| meta_error("document `meta` block", &block.code, &e, true))?, + None => GlobalMeta::default(), + }; + + Ok(Globals { + env: Self::parse_env_overrides(&meta.env)?, + dir: meta.dir.map(PathBuf::from), + imports: meta + .imports + .iter() + .map(ImportSpec::from_toml) + .collect::>()?, + dotenv: meta.dotenv, + vars: match &meta.vars { + Some(table) => vars::parse_table("document `meta` block", table)?, + None => Vec::new(), + }, + runtimes: match meta.runtimes { + Some(table) => table + .into_iter() + .map(|(lang, value)| { + value + .try_into::() + .map(|runtime| (lang.clone(), runtime)) + .map_err(|e| { + Error::Invalid(format!( + "document `meta` block: `runtimes`: '{}' must be a command or {{ command, execution_mode }}: {}", + lang, + e.message() + )) + }) + }) + .collect::>()?, + None => Vec::new(), + }, + default_task: meta.default_task, + }) + } + + fn parse_sections(&mut self, result: mq_lang::RuntimeValues) -> Result> { + let mut sections = Vec::new(); + + for value in result.into_iter() { + if let RuntimeValue::Dict(dict) = value { + match self.parse_section(&dict) { + Ok(section) => sections.push(section), + Err(e) => match self.collected.as_mut() { + Some(collected) => { + if let Some(RuntimeValue::String(title)) = + dict.get(&Ident::from("title")) + { + collected.failed.push(title.to_string()); + } + collected.errors.push(e); + } + None => return Err(e), + }, + } + } + } + + Ok(sections) + } + + fn parse_section(&self, dict: &DictMap) -> Result
{ + let title = dict + .get(&Ident::from("title")) + .and_then(|v| match v { + RuntimeValue::String(s) => Some(s.to_string()), + _ => None, + }) + .unwrap_or_default(); + + let all_codes = dict + .get(&Ident::from("codes")) + .and_then(|v| match v { + RuntimeValue::Array(arr) => Some(self.parse_code_blocks(arr)), + _ => None, + }) + .unwrap_or_else(|| Ok(Vec::new()))?; + + let meta = match all_codes.iter().find(|c| c.lang == "meta") { + Some(block) => toml::from_str::(&block.code).map_err(|e| { + meta_error( + &format!("task '{}': `meta` block", title), + &block.code, + &e, + false, + ) + })?, + None => TaskMeta::default(), + }; + + let duration = |value: &Option| -> Result> { + value + .as_deref() + .map(parse_duration) + .transpose() + .map_err(|e| Error::Invalid(format!("task '{}': {}", title, e))) + }; + let retry_delay = duration(&meta.retry_delay)?; + let timeout = duration(&meta.timeout)?; + + let params = meta + .params + .iter() + .map(|p| ParamDef::from_toml(&title, p)) + .collect::>>()?; + let private = meta.private || title.starts_with('_'); + let env = Self::parse_env_overrides(&meta.env)?; + let dir = meta.dir.map(PathBuf::from); + + let vars = match &meta.vars { + Some(table) => vars::parse_table(&format!("task '{}': `meta` block", title), table)?, + None => Vec::new(), + }; + + let for_each = meta + .for_each + .as_ref() + .map(|v| LoopSpec::from_toml(&title, v)) + .transpose()?; + if meta.loop_var.is_some() && for_each.is_none() { + return Err(Error::Invalid(format!( + "task '{}': `as` requires `for`", + title + ))); + } + + let codes: Vec = all_codes + .into_iter() + .filter(|c| c.lang != "meta") + .filter(|c| { + self.exec.config.has_runtime(&c.lang) || !DATA_LANGUAGES.contains(&c.lang.as_str()) + }) + .collect(); + + let description = dict.get(&Ident::from("description")).and_then(|v| match v { + RuntimeValue::String(s) => Some(s.to_string()), + _ => None, + }); + + Ok(Section { + title, + codes, + description, + depends: meta.depends, + params, + aliases: meta.alias, + private, + env, + dir, + parallel: meta.parallel, + condition: meta.condition, + retries: meta.retries, + retry_delay, + timeout, + capture: meta.capture, + sources: meta.sources, + outputs: meta.outputs, + for_each, + loop_var: meta.loop_var, + defer: meta.defer, + ignore_error: meta.ignore_error, + prompt: meta.prompt, + dotenv: meta.dotenv, + status: meta.status, + once: meta.once, + requires: meta.requires, + vars, + }) + } + + fn parse_code_blocks(&self, arr: &[RuntimeValue]) -> Result> { + let mut blocks = Vec::new(); + + for item in arr { + if let RuntimeValue::Dict(dict) = item { + let lang = dict + .get(&Ident::from("lang")) + .and_then(|v| match v { + RuntimeValue::String(s) => Some(s.to_string()), + _ => None, + }) + .unwrap_or_default(); + + let code = dict + .get(&Ident::from("code")) + .and_then(|v| match v { + RuntimeValue::String(s) => Some(s.to_string()), + _ => None, + }) + .unwrap_or_default(); + + let no_run = dict.get(&Ident::from("info")).is_some_and(|v| match v { + RuntimeValue::String(s) => s.split_whitespace().any(|w| w == NO_RUN), + _ => false, + }); + if no_run { + continue; + } + + blocks.push(CodeBlock { lang, code }); + } + } + + Ok(blocks) + } +} + +/// Fold an imported file's document-wide defaults into its tasks so they +/// keep behaving as they would when run from that file directly. +pub(super) fn apply_globals(sections: &mut [Section], globals: &Globals) { + for section in sections { + let mut env = globals.env.clone(); + env.append(&mut section.env); + section.env = env; + if section.dir.is_none() { + section.dir = globals.dir.clone(); + } + let mut dotenv = globals.dotenv.clone(); + dotenv.append(&mut section.dotenv); + section.dotenv = dotenv; + let mut vars = globals.vars.clone(); + vars.append(&mut section.vars); + section.vars = vars; + } +} + +/// Prefix titles, aliases and intra-file dependency references with +/// `:`. References to tasks outside the file are left alone. +pub(super) fn namespace_sections(sections: &mut [Section], namespace: &str) { + let local: HashSet = sections + .iter() + .flat_map(|s| std::iter::once(s.title.clone()).chain(s.aliases.iter().cloned())) + .collect(); + let qualify = |name: &str| format!("{}:{}", namespace, name); + + let qualify_call = |spec: &str| -> String { + if local.contains(spec.trim()) { + return qualify(spec.trim()); + } + let words = split_words(spec); + for n in (1..=words.len()).rev() { + let name = words[..n].join(" "); + if local.contains(&name) { + let args = words[n..].iter().map(|w| quote_word(w)); + return std::iter::once(qualify(&name)) + .chain(args) + .collect::>() + .join(" "); + } + } + spec.to_string() + }; + + for section in sections { + if let Some(LoopSpec::From(source)) = &mut section.for_each + && local.contains(source.as_str()) + { + *source = qualify(source); + } + section.title = qualify(§ion.title); + section.aliases = section.aliases.iter().map(|a| qualify(a)).collect(); + for dep in section.depends.iter_mut().chain(section.defer.iter_mut()) { + *dep = qualify_call(dep); + } + } +} + +/// The key and the valid keys from a serde "unknown field" message such as +/// ``unknown field `retires`, expected one of `retries`, `timeout` ``. +pub(super) fn unknown_field(message: &str) -> Option<(String, Vec)> { + let rest = message.strip_prefix("unknown field ")?; + let mut quoted = rest.split('`').skip(1).step_by(2).map(str::to_string); + let key = quoted.next()?; + Some((key, quoted.collect())) +} + +/// Turn a `meta` block parse error into one a user can act on: unknown keys +/// get a "did you mean" hint and the line they are on. +pub(super) fn meta_error( + context: &str, + code: &str, + err: &toml::de::Error, + document: bool, +) -> Error { + let Some((key, valid)) = unknown_field(err.message()) else { + return Error::Invalid(format!("{}: {}", context, err)); + }; + let line = err + .span() + .map(|s| { + format!( + " on line {}", + code[..s.start.min(code.len())].matches('\n').count() + 1 + ) + }) + .unwrap_or_default(); + let hint = if !document && key == "imports" { + "`imports` only works in the document-wide `meta` block before the first heading" + .to_string() + } else if let Some(close) = crate::util::closest(&key, valid.iter().map(String::as_str)) { + format!("did you mean `{}`?", close) + } else { + "see the Meta keys reference for the valid keys".to_string() + }; + Error::Invalid(format!( + "{}: unknown key `{}`{} ({})", + context, key, line, hint + )) +} + +/// Quote a word for [`split_words`] when it would not survive unquoted. +pub(super) fn quote_word(word: &str) -> String { + let plain = + !word.is_empty() && !word.contains(|c: char| c.is_whitespace() || c == '"' || c == '\''); + if plain { + word.to_string() + } else if word.contains('"') { + format!("'{}'", word) + } else { + format!("\"{}\"", word) + } +} diff --git a/src/schedule.rs b/src/schedule.rs index 2e43d98..21cb820 100644 --- a/src/schedule.rs +++ b/src/schedule.rs @@ -8,7 +8,7 @@ use std::sync::mpsc; use crate::error::{Error, Result}; use crate::events::{RunObserver, TaskStatus}; use crate::exec::{Executor, NodeOutcome, NodeParams, TaskIo, output_var}; -use crate::runner::{LoopSpec, Section}; +use crate::section::{LoopSpec, Section}; /// Look a task up by title or alias. pub fn find_section<'a>(sections: &'a [Section], name: &str) -> Option<&'a Section> { @@ -886,7 +886,7 @@ mod tests { } fn with_code(mut s: Section) -> Section { - s.codes = vec![crate::runner::CodeBlock { + s.codes = vec![crate::section::CodeBlock { lang: "bash".into(), code: "true".into(), }]; diff --git a/src/section.rs b/src/section.rs new file mode 100644 index 0000000..55f17e2 --- /dev/null +++ b/src/section.rs @@ -0,0 +1,558 @@ +//! The task-file document model: sections, code blocks, `meta` blocks and the +//! pieces they are parsed into. + +use std::path::{Path, PathBuf}; +use std::time::Duration; + +use serde::{Deserialize, Serialize}; + +use crate::config::RuntimeConfig; +use crate::error::{Error, Result}; +use crate::vars::VarDef; + +/// Represents a code block in a section +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct CodeBlock { + /// Language of the code block + pub lang: String, + /// Code content + pub code: String, +} + +/// What a `for` loop iterates over. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub enum LoopSpec { + /// Literal items: `for = ["a", "b"]` + List(Vec), + /// Paths matching these globs, relative to the task's directory: + /// `for = { glob = "src/*.md" }` + Glob(Vec), + /// Every combination of the axes' values: + /// `for = { matrix = { os = ["linux", "macos"], arch = ["x64"] } }` + /// Axes are kept sorted by name. + Matrix(Vec<(String, Vec)>), + /// Lines of a dependency's captured output (`capture = true`): + /// `for = { from = "list-files" }`. Items run sequentially in the task. + From(String), +} + +impl LoopSpec { + /// Short description for graph output, e.g. `a, b, c` or `glob src/*.md`. + pub fn summary(&self) -> String { + match self { + LoopSpec::List(items) => { + let shown: Vec<&str> = items.iter().take(3).map(String::as_str).collect(); + let more = if items.len() > 3 { ", ..." } else { "" }; + format!("{}{}", shown.join(", "), more) + } + LoopSpec::Glob(patterns) => format!("glob {}", patterns.join(", ")), + LoopSpec::From(task) => format!("output of {}", task), + LoopSpec::Matrix(axes) => format!( + "matrix {}", + axes.iter() + .map(|(k, _)| k.as_str()) + .collect::>() + .join(" x ") + ), + } + } + + /// Interpret the TOML value of a `for` key. + pub(crate) fn from_toml(title: &str, value: &toml::Value) -> Result { + let invalid = |msg: &str| Error::Invalid(format!("task '{}': `for`: {}", title, msg)); + let scalar = |v: &toml::Value| -> Result { + match v { + toml::Value::String(s) => Ok(s.clone()), + toml::Value::Integer(i) => Ok(i.to_string()), + toml::Value::Float(f) => Ok(f.to_string()), + toml::Value::Boolean(b) => Ok(b.to_string()), + _ => Err(invalid("items must be strings, numbers or booleans")), + } + }; + let list = |v: &toml::Value| -> Result> { + match v { + toml::Value::Array(items) => items.iter().map(scalar).collect(), + _ => Err(invalid("expected an array")), + } + }; + match value { + toml::Value::Array(_) => Ok(LoopSpec::List(list(value)?)), + toml::Value::Table(table) => { + if table.len() != 1 { + return Err(invalid( + "expected exactly one of `glob`, `matrix` or `from`", + )); + } + let (key, v) = table.iter().next().expect("one entry"); + match key.as_str() { + "glob" => match v { + toml::Value::String(s) => Ok(LoopSpec::Glob(vec![s.clone()])), + other => Ok(LoopSpec::Glob(list(other)?)), + }, + "from" => match v { + toml::Value::String(task) if !task.trim().is_empty() => { + Ok(LoopSpec::From(task.trim().to_string())) + } + _ => Err(invalid("`from` must name a task")), + }, + "matrix" => { + let toml::Value::Table(axes) = v else { + return Err(invalid("`matrix` must be a table of arrays")); + }; + let mut out = Vec::new(); + for (axis, values) in axes { + out.push((axis.clone(), list(values)?)); + } + out.sort(); + Ok(LoopSpec::Matrix(out)) + } + other => Err(invalid(&format!( + "unknown key '{}' (expected `glob`, `matrix` or `from`)", + other + ))), + } + } + _ => Err(invalid( + "expected an array, `{ glob = ... }`, `{ matrix = ... }` or `{ from = ... }`", + )), + } + } +} + +/// Document-wide defaults parsed from a `meta` code block that appears +/// before the first heading in the file. Applies to every task; a task's +/// own `meta` block overrides same-named keys, and CLI flags override both. +#[derive(Debug, Deserialize, Default)] +#[serde(deny_unknown_fields)] +pub(crate) struct GlobalMeta { + /// Default environment variables applied to every task + #[serde(default)] + pub(crate) env: Vec, + /// Default working directory applied to every task without its own `dir` + #[serde(default)] + pub(crate) dir: Option, + /// Other task files whose tasks are made available as `:`: + /// paths, globs or `{ path, as, optional, flatten }` tables + #[serde(default)] + pub(crate) imports: Vec, + /// `.env` files loaded for every task + #[serde(default)] + pub(crate) dotenv: Vec, + /// Variables available to every task + #[serde(default)] + pub(crate) vars: Option, + /// Runtimes for this file's languages, in the form of `[runtimes]` in + /// `mq-task.toml` + #[serde(default)] + pub(crate) runtimes: Option, + /// Task run when no task is named + #[serde(default)] + pub(crate) default_task: Option, +} + +/// Task metadata parsed from a `meta` code block within a section +#[derive(Debug, Deserialize, Default)] +#[serde(deny_unknown_fields)] +pub(crate) struct TaskMeta { + #[serde(default)] + pub(crate) depends: Vec, + #[serde(default)] + pub(crate) params: Vec, + #[serde(default)] + pub(crate) alias: Vec, + #[serde(default)] + pub(crate) private: bool, + /// Default environment variables, e.g. `env = ["REGION=eu", "DEBUG=1"]` + #[serde(default)] + pub(crate) env: Vec, + /// Working directory to run this task's commands in, e.g. `dir = "services/api"` + #[serde(default)] + pub(crate) dir: Option, + /// Run this task's dependencies concurrently + #[serde(default)] + pub(crate) parallel: bool, + /// Condition that must hold for the task to run + #[serde(default, rename = "if")] + pub(crate) condition: Option, + /// How many times to retry after a failure + #[serde(default)] + pub(crate) retries: u32, + /// Delay between retries, e.g. `"2s"` + #[serde(default)] + pub(crate) retry_delay: Option, + /// Kill the task if it runs longer than this, e.g. `"30s"` + #[serde(default)] + pub(crate) timeout: Option, + /// Expose this task's stdout to dependents as `MX_OUTPUT_` + #[serde(default)] + pub(crate) capture: bool, + /// Globs of input files; unchanged inputs let the task be skipped + #[serde(default)] + pub(crate) sources: Vec, + /// Globs of files the task produces; must exist for it to be skipped + #[serde(default)] + pub(crate) outputs: Vec, + /// Run the task once per item: an array, `{ glob = ... }` or `{ matrix = ... }` + #[serde(default, rename = "for")] + pub(crate) for_each: Option, + /// Extra environment variable name for the loop item, e.g. `as = "file"` + #[serde(default, rename = "as")] + pub(crate) loop_var: Option, + /// Tasks run after this one finishes, even if it failed + #[serde(default)] + pub(crate) defer: Vec, + /// Treat a failed run (after retries) as success + #[serde(default)] + pub(crate) ignore_error: bool, + /// Ask for confirmation before running, e.g. `prompt = "Deploy to production?"` + #[serde(default)] + pub(crate) prompt: Option, + /// `.env` files loaded for this task, e.g. `dotenv = [".env", ".env.local"]` + #[serde(default)] + pub(crate) dotenv: Vec, + /// Shell commands; the task is up to date when all of them exit 0 + #[serde(default)] + pub(crate) status: Vec, + /// Variables: fixed values or `{ sh = "command" }` + #[serde(default)] + pub(crate) vars: Option, + /// Run only once per run, however it is called + #[serde(default)] + pub(crate) once: bool, + /// Variables that must be set (and not empty), e.g. `["API_TOKEN"]` + #[serde(default)] + pub(crate) requires: Vec, +} + +/// A named task parameter, e.g. `params = ["env=staging", "verbose"]`. +/// A bare name is required; `name=value` supplies a default. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Default)] +pub struct ParamDef { + pub name: String, + pub default: Option, + /// The values the parameter accepts; any value when empty. + #[serde(default)] + pub choices: Vec, + /// What the parameter is for, shown by `--summary` and the TUI. + #[serde(default)] + pub description: Option, +} + +impl ParamDef { + /// The short form: `"name"` (required) or `"name=default"`. + pub(crate) fn parse(raw: &str) -> Self { + match raw.split_once('=') { + Some((name, default)) => ParamDef { + name: name.trim().to_string(), + default: Some(default.trim().to_string()), + ..Default::default() + }, + None => ParamDef { + name: raw.trim().to_string(), + ..Default::default() + }, + } + } + + /// The table form: `{ name = "env", default = "dev", choices = [...], desc = "..." }`. + pub(crate) fn from_toml(task: &str, value: &toml::Value) -> Result { + let invalid = |msg: &str| Error::Invalid(format!("task '{}': `params`: {}", task, msg)); + match value { + toml::Value::String(raw) => Ok(Self::parse(raw)), + toml::Value::Table(table) => { + const KEYS: [&str; 4] = ["name", "default", "choices", "desc"]; + let text = |key: &str| -> Result> { + match table.get(key) { + None => Ok(None), + Some(toml::Value::String(s)) => Ok(Some(s.clone())), + Some(_) => Err(invalid(&format!("`{}` must be a string", key))), + } + }; + for key in table.keys() { + if !KEYS.contains(&key.as_str()) { + let hint = crate::util::closest(key, KEYS) + .map(|c| format!(" (did you mean `{}`?)", c)) + .unwrap_or_else(|| { + " (expected `name`, `default`, `choices` or `desc`)".into() + }); + return Err(invalid(&format!("unknown key `{}`{}", key, hint))); + } + } + let name = text("name")? + .map(|n| n.trim().to_string()) + .filter(|n| !n.is_empty()) + .ok_or_else(|| invalid("a table needs a `name`"))?; + let choices = match table.get("choices") { + None => Vec::new(), + Some(toml::Value::Array(items)) => items + .iter() + .map(|v| match v { + toml::Value::String(s) => Ok(s.clone()), + other => Ok(other.to_string()), + }) + .collect::>>()?, + Some(_) => return Err(invalid("`choices` must be an array")), + }; + if table.contains_key("choices") && choices.is_empty() { + return Err(invalid(&format!("`choices` of '{}' is empty", name))); + } + let default = text("default")?; + if let Some(d) = &default + && !choices.is_empty() + && !choices.contains(d) + { + return Err(invalid(&format!( + "the default '{}' of '{}' is not one of its choices ({})", + d, + name, + choices.join(", ") + ))); + } + Ok(ParamDef { + name, + default, + choices, + description: text("desc")?, + }) + } + _ => Err(invalid("expected a string or a table")), + } + } + + /// All of `params` on one line. The entries are separated by commas, or + /// by semicolons when some carry choices or a description (which can + /// contain commas). + pub fn describe_all(params: &[ParamDef]) -> String { + let rich = params + .iter() + .any(|p| !p.choices.is_empty() || p.description.is_some()); + let separator = if rich { "; " } else { ", " }; + params + .iter() + .map(ParamDef::describe) + .collect::>() + .join(separator) + } + + /// One line for `--summary` and the TUI: `env=dev [dev|prod] - target`. + pub fn describe(&self) -> String { + let mut text = match &self.default { + Some(default) => format!("{}={}", self.name, default), + None => format!("{} (required)", self.name), + }; + if !self.choices.is_empty() { + text.push_str(&format!(" [{}]", self.choices.join("|"))); + } + if let Some(description) = self.description.as_deref().filter(|d| !d.is_empty()) { + text.push_str(&format!(" - {}", description)); + } + text + } +} + +/// Represents a section with its code blocks +#[derive(Debug, Clone, Serialize, Deserialize, Default)] +pub struct Section { + /// Section title + pub title: String, + /// Code blocks in this section (excludes `meta` metadata blocks) + pub codes: Vec, + /// Optional description extracted from the section content + pub description: Option, + /// Task names this section depends on (declared in a `meta` code block) + #[serde(default)] + pub depends: Vec, + /// Named parameters this task accepts (declared in a `meta` code block) + #[serde(default)] + pub params: Vec, + /// Alternate names this task can be run by (declared in a `meta` code block) + #[serde(default)] + pub aliases: Vec, + /// Hidden from `list`/`tui` output; set via `meta` or a `_`-prefixed title + #[serde(default)] + pub private: bool, + /// Default environment variables declared in a `meta` code block; + /// overridden by CLI `--env` flags of the same name + #[serde(default)] + pub env: Vec<(String, String)>, + /// Working directory declared in a `meta` code block; overridden by + /// the CLI `--dir` flag + #[serde(default)] + pub dir: Option, + /// Run this task's dependencies concurrently + #[serde(default)] + pub parallel: bool, + /// `if` condition; the task is skipped when it evaluates to false + #[serde(default)] + pub condition: Option, + /// Number of retries after a failed attempt + #[serde(default)] + pub retries: u32, + /// Delay between retries (defaults to one second) + #[serde(default)] + pub retry_delay: Option, + /// Maximum run time per attempt + #[serde(default)] + pub timeout: Option, + /// Capture stdout and hand it to dependents as `MX_OUTPUT_` + #[serde(default)] + pub capture: bool, + /// Input globs for up-to-date checks and `--watch` + #[serde(default)] + pub sources: Vec, + /// Output globs that must exist for the task to count as up to date + #[serde(default)] + pub outputs: Vec, + /// `for` loop: the task runs once per item + #[serde(default)] + pub for_each: Option, + /// Extra name for the loop item (`as = "file"` sets `MX_FILE`) + #[serde(default)] + pub loop_var: Option, + /// Tasks (optionally with arguments) run after this one finishes, + /// whatever its outcome + #[serde(default)] + pub defer: Vec, + /// A failed run (after retries) is reported but not treated as a failure + #[serde(default)] + pub ignore_error: bool, + /// Confirmation message; the task runs only on yes or `--yes` + #[serde(default)] + pub prompt: Option, + /// `.env` files, relative to the task's working directory + #[serde(default)] + pub dotenv: Vec, + /// Shell commands; the task is skipped when all exit 0 + #[serde(default)] + pub status: Vec, + /// Runs once per run even if called with different arguments (first wins) + #[serde(default)] + pub once: bool, + /// Variables that must be set and not empty + #[serde(default)] + pub requires: Vec, + /// Named values for the task, fixed or computed by a shell command + #[serde(default)] + pub vars: Vec, +} + +/// One entry of `imports`: a path (or glob) to a task file, with the options +/// of the table form. +#[derive(Debug, Clone, PartialEq)] +pub(crate) struct ImportSpec { + pub(crate) path: String, + /// Namespace to use instead of the file name. + pub(crate) alias: Option, + /// A missing file is not an error. + pub(crate) optional: bool, + /// Add the tasks under their own names, without a namespace. + pub(crate) flatten: bool, +} + +impl ImportSpec { + pub(crate) fn is_glob(&self) -> bool { + self.path.contains(['*', '?', '[']) + } + + pub(crate) fn from_toml(value: &toml::Value) -> Result { + let invalid = + |msg: &str| Error::Invalid(format!("document `meta` block: `imports`: {}", msg)); + let spec = match value { + toml::Value::String(path) => ImportSpec { + path: path.clone(), + alias: None, + optional: false, + flatten: false, + }, + toml::Value::Table(table) => { + const KEYS: [&str; 4] = ["path", "as", "optional", "flatten"]; + for key in table.keys() { + if !KEYS.contains(&key.as_str()) { + let hint = crate::util::closest(key, KEYS) + .map(|c| format!(" (did you mean `{}`?)", c)) + .unwrap_or_else(|| { + " (expected `path`, `as`, `optional` or `flatten`)".into() + }); + return Err(invalid(&format!("unknown key `{}`{}", key, hint))); + } + } + let text = |key: &str| match table.get(key) { + None => Ok(None), + Some(toml::Value::String(s)) if !s.trim().is_empty() => Ok(Some(s.clone())), + Some(_) => Err(invalid(&format!("`{}` must be a non-empty string", key))), + }; + let flag = |key: &str| match table.get(key) { + None => Ok(false), + Some(toml::Value::Boolean(b)) => Ok(*b), + Some(_) => Err(invalid(&format!("`{}` must be true or false", key))), + }; + ImportSpec { + path: text("path")?.ok_or_else(|| invalid("a table needs a `path`"))?, + alias: text("as")?, + optional: flag("optional")?, + flatten: flag("flatten")?, + } + } + _ => return Err(invalid("expected a path or a table")), + }; + if spec.is_glob() && spec.alias.is_some() { + return Err(invalid(&format!( + "`as` cannot be used with the glob '{}': every file is named after itself", + spec.path + ))); + } + if spec.flatten && spec.alias.is_some() { + return Err(invalid(&format!( + "`as` has no effect with `flatten` ('{}')", + spec.path + ))); + } + Ok(spec) + } + + /// The files this entry stands for under `base`, each with its + /// namespace. `current` (the importing file) is never matched by a glob. + pub(crate) fn resolve(&self, base: &Path, current: &Path) -> Result> { + let namespace = |path: &Path| { + path.file_stem() + .map(|s| s.to_string_lossy().into_owned()) + .ok_or_else(|| Error::Invalid(format!("invalid import path '{}'", self.path))) + }; + if self.is_glob() { + let pattern = base.join(&self.path).to_string_lossy().into_owned(); + let paths = glob::glob(&pattern).map_err(|e| { + Error::Invalid(format!("invalid import pattern '{}': {}", self.path, e.msg)) + })?; + let mut found: Vec = paths + .flatten() + .filter(|p| p.is_file() && p.canonicalize().ok().as_deref() != Some(current)) + .collect(); + found.sort(); + return found + .into_iter() + .map(|p| namespace(&p).map(|ns| (p, ns))) + .collect(); + } + let path = base.join(&self.path); + if self.optional && !path.exists() { + return Ok(Vec::new()); + } + let namespace = match &self.alias { + Some(alias) => alias.clone(), + None => namespace(&path)?, + }; + Ok(vec![(path, namespace)]) + } +} + +/// Document-wide settings found in the preamble `meta` block. +#[derive(Debug, Default)] +pub(crate) struct Globals { + pub(crate) env: Vec<(String, String)>, + pub(crate) dir: Option, + pub(crate) imports: Vec, + pub(crate) dotenv: Vec, + pub(crate) vars: Vec, + pub(crate) runtimes: Vec<(String, RuntimeConfig)>, + pub(crate) default_task: Option, +} diff --git a/src/summary.rs b/src/summary.rs index 04f182e..84fa7ab 100644 --- a/src/summary.rs +++ b/src/summary.rs @@ -3,7 +3,7 @@ use std::fmt::Write; use crate::graph::badge_list; -use crate::runner::{ParamDef, Section}; +use crate::section::{ParamDef, Section}; use crate::vars::VarValue; /// Describe `section`: title, description and its notable settings. @@ -123,7 +123,7 @@ pub fn list_json(sections: &[Section]) -> String { #[cfg(test)] mod tests { use super::*; - use crate::runner::CodeBlock; + use crate::section::CodeBlock; #[test] fn describes_a_task() { diff --git a/src/tui.rs b/src/tui.rs index 78b27a6..01adeed 100644 --- a/src/tui.rs +++ b/src/tui.rs @@ -28,7 +28,8 @@ use ratatui::{ use crate::error::{Error, Result}; use crate::events::{RunObserver, Stream, TaskStatus}; -use crate::runner::{RunOptions, Section}; +use crate::runner::RunOptions; +use crate::section::Section; use crate::{Config, Runner}; /// Settings for [`run_tui`]. @@ -846,7 +847,7 @@ fn section_details(s: &Section) -> Vec> { ])); } if !s.params.is_empty() { - let params = crate::runner::ParamDef::describe_all(&s.params); + let params = crate::section::ParamDef::describe_all(&s.params); lines.push(Line::from(vec![label("params"), Span::raw(params)])); } let mut flags = Vec::new(); @@ -1501,7 +1502,7 @@ fn log_widget<'a>(run: &'a RunView, area: Rect) -> Paragraph<'a> { #[cfg(test)] mod tests { use super::*; - use crate::runner::ParamDef; + use crate::section::ParamDef; use ratatui::backend::TestBackend; fn section(title: &str) -> Section { @@ -2076,7 +2077,7 @@ mod tests { s.capture = true; s.sources = vec!["src/**".into()]; s.status = vec!["true".into()]; - s.for_each = Some(crate::runner::LoopSpec::List(vec!["a".into(), "b".into()])); + s.for_each = Some(crate::section::LoopSpec::List(vec!["a".into(), "b".into()])); s.defer = vec!["cleanup".into()]; s.once = true; s.requires = vec!["TOKEN".into()]; @@ -2088,7 +2089,7 @@ mod tests { s.ignore_error = true; s.dir = Some(PathBuf::from("svc")); s.env = vec![("A".into(), "1".into()), ("B".into(), "2".into())]; - s.codes = vec![crate::runner::CodeBlock { + s.codes = vec![crate::section::CodeBlock { lang: "bash".into(), code: "echo hi".into(), }]; @@ -2140,7 +2141,7 @@ mod tests { .map(|i| format!("line{i}")) .collect::>() .join("\n"); - s.codes = vec![crate::runner::CodeBlock { + s.codes = vec![crate::section::CodeBlock { lang: "bash".into(), code, }]; diff --git a/tests/cache_state_test.rs b/tests/cache_state_test.rs index 7e2adc9..41367d2 100644 --- a/tests/cache_state_test.rs +++ b/tests/cache_state_test.rs @@ -7,7 +7,7 @@ use std::fs; use std::path::Path; use mq_task::cache::{is_fresh, record}; -use mq_task::runner::Section; +use mq_task::section::Section; fn section(title: &str, outputs: &[&str]) -> Section { Section { diff --git a/tests/cache_test.rs b/tests/cache_test.rs index 226233f..53f9558 100644 --- a/tests/cache_test.rs +++ b/tests/cache_test.rs @@ -10,7 +10,7 @@ use std::fs; use std::path::{Path, PathBuf}; use mq_task::cache::{expand, fingerprint, snapshot, snapshot_files}; -use mq_task::runner::{CodeBlock, Section}; +use mq_task::section::{CodeBlock, Section}; use proptest::prelude::*; use rstest::{fixture, rstest}; use tempfile::TempDir; diff --git a/tests/common/helpers.rs b/tests/common/helpers.rs index bd445b1..e67aa19 100644 --- a/tests/common/helpers.rs +++ b/tests/common/helpers.rs @@ -4,7 +4,7 @@ use std::sync::Mutex; use mq_task::events::{RunObserver, Stream, TaskStatus}; -use mq_task::runner::{CodeBlock, Section}; +use mq_task::section::{CodeBlock, Section}; /// A section with the given title and declared dependencies. pub fn sec(title: &str, depends: &[&str]) -> Section { diff --git a/tests/exec_test.rs b/tests/exec_test.rs index 271cea8..ce49d08 100644 --- a/tests/exec_test.rs +++ b/tests/exec_test.rs @@ -13,7 +13,7 @@ use mq_task::config::{Config, ExecutionMode, RuntimeConfig}; use mq_task::error::Error; use mq_task::events::TaskStatus; use mq_task::exec::{Executor, NodeOutcome, NodeParams, TaskIo, output_var}; -use mq_task::runner::{CodeBlock, ParamDef, Section}; +use mq_task::section::{CodeBlock, ParamDef, Section}; use proptest::prelude::*; use rstest::rstest; diff --git a/tests/graph_test.rs b/tests/graph_test.rs index a552f76..4fe51dc 100644 --- a/tests/graph_test.rs +++ b/tests/graph_test.rs @@ -8,7 +8,7 @@ use std::time::Duration; use common::sec; use mq_task::graph::{GraphFormat, render}; -use mq_task::runner::Section; +use mq_task::section::Section; use proptest::prelude::*; use rstest::rstest; diff --git a/tests/runner_test.rs b/tests/runner_test.rs index 1a9e1f0..00eb228 100644 --- a/tests/runner_test.rs +++ b/tests/runner_test.rs @@ -966,7 +966,7 @@ fn load_with( config: Config, pinned: &[&str], main: &str, -) -> (Runner, Result, Error>) { +) -> (Runner, Result, Error>) { let dir = tempfile::tempdir().unwrap(); let file = write(dir.path(), "main.md", main); let mut r = Runner::new(config); diff --git a/tests/schedule_test.rs b/tests/schedule_test.rs index cf54508..b97719c 100644 --- a/tests/schedule_test.rs +++ b/tests/schedule_test.rs @@ -11,8 +11,8 @@ use mq_task::config::Config; use mq_task::error::Error; use mq_task::events::TaskStatus; use mq_task::exec::Executor; -use mq_task::runner::Section; use mq_task::schedule::{Plan, ScheduleOptions, build_plan, find_section, run_plan}; +use mq_task::section::Section; use proptest::prelude::*; use rstest::rstest;