Weaverdance Orchestrator is a distributed task choreography engine that treats every background process, artifact, and state transition as a thread in a living tapestry. Where conventional workflow tools force you into rigid pipelines, Weaverdance lets you stitch processes together with deterministic intent — every worker lease is a knot, every artifact is a dyed fiber, and every memory snapshot is a pattern preserved for replay.
Think of it as a dance instructor for your microservices: each worker knows its steps, but the choreographer decides when the music plays, who leads, and how the final performance is recorded. This repository contains the complete control plane, worker protocol, and artifact ledger for building self-healing, auditable execution graphs that can span thousands of nodes without losing their rhythm.
Whether you're coordinating data pipelines, managing ephemeral compute clusters, or weaving together multi-stage CI/CD ceremonies, Weaverdance provides the loom — you bring the threads.
Most orchestration tools are script conductors — they tell processes when to start and stop, but they don't understand the texture of the work. Weaverdance was born from the insight that background workloads have memory, artifacts have provenance, and leases need to be more than just timestamps. This system treats every execution as a fabric swatch: you can inspect its weave, pull a single thread (a specific artifact version), and reweave the entire pattern from any point in history.
The deterministic stitching model means that if you feed the same inputs and the same orchestration graph, you get byte-identical outputs — even across retries, partial failures, or node restarts. This isn't just idempotency; it's temporal determinism through a Merkle-style ledger of every intermediate state.
The heart of Weaverdance is its pattern compiler, which converts your high-level task graph into a strict execution order where every dependency is explicitly resolved. Unlike event-driven systems that can reorder operations based on network timing, Weaverdance's stitching algorithm assigns each task a temporal slot based on its position in the logical weave. This means you can replay any historical run with perfect fidelity, making debugging and audit trails trivial.
Workers don't just claim tasks — they negotiate leases with the control plane. Each lease carries a cryptographic nonce, a time-to-live window, and a workload fingerprint. If a worker dies mid-task, the lease expires and the work is automatically re-woven into a healthy worker's queue. But crucially, the failed worker's partial artifacts are preserved as frayed threads — you can inspect exactly where the weave broke without losing the surrounding context.
Every state change in your workflow is captured as an immutable memory shard. These shards are stored in a content-addressed store, meaning identical states collapse to a single reference. When you need to roll back or branch, Weaverdance reconstructs the exact memory state at any point in the timeline — not just the latest commit, but every transaction leading up to it.
Every file, output, and intermediate product is logged with its hash, producer, and consumption history. This isn't just metadata; it's a chain of custody for your data. You can answer questions like "Which version of the model was used to generate this report?" or "Which downstream task consumed this dataset and what did it produce?" with a single query.
To experience the power of deterministic stitching, you'll need to establish your first loom workspace. The Weaverdance control plane runs as a lightweight daemon that manages your worker pool and artifact store.
Create a workspace directory where Weaverdance will maintain its ledger and artifact cache. This is your tapestry folder — all state, logs, and intermediate files will live here, organized by weave session ID.
Write a weave manifest — a declarative YAML file that describes your task graph. Each task specifies its inputs (either from external sources or from other tasks' outputs), its expected duration, and its retry policy. The manifest can also include memory anchors: checkpoints where you want to snapshot the full state for later reference.
Spin up worker processes on any machine that can reach your control plane. Workers register themselves, report their available resources, and start listening for lease offers. The control plane will handle load balancing, failover, and lease renewal automatically.
Launch the weave session. Your control plane will compile the manifest, assign leases, and begin executing. You can watch the progress in real-time via the console dashboard, which shows each task as a colored thread being pulled through the loom.
You get observability without instrumentation. Because every artifact and state change is recorded in the ledger, you don't need separate tracing tools. The weave timeline is your distributed trace — it shows you not just what happened, but the exact order and causal relationships.
Your experiments become reproducible by default. When you run a training job through Weaverdance, the entire environment — data versions, code state, hyperparameters — is captured in a single weave ID. Sharing that ID with a colleague gives them access to the exact same deterministic pipeline, even if they're running on different hardware.
Your deployments gain rollback superpowers. Because every deployment is a weave with memory shards, rolling back isn't just reverting code — it's restoring the exact runtime state, including caches, temporary files, and in-flight transactions. This eliminates the "works in staging, breaks in prod" class of problems.
Weaverdance is designed for scale without sacrificing determinism. Here's what you can expect:
- Throughput: Handles up to 50,000 lease negotiations per second on a single control plane node
- Artifact Throughput: 2.5 GB/s throughput on local storage, 850 MB/s on networked storage
- Determinism Guarantee: 100% reproducibility for weaves that don't invoke non-deterministic external services
- Recovery Time: Sub-second lease reassignment on worker failure, with no lost state
The control plane exposes a plugin system for custom stitch patterns. You can register new task types, custom artifact processors, and custom memory shard serializers. The plugin API is designed to be minimal — you implement four interfaces (Validate, Execute, Snapshot, Restore) and the system handles the rest.
- Custom serialization: Support Protobuf, Avro, or your proprietary format for memory shards
- External secret integration: Fetch credentials from your vault during task execution
- Custom lease backoff: Implement your own retry logic based on system metrics
The worker protocol is language-agnostic. Official client libraries exist for Python, Go, Java, and Rust, but any language that can speak gRPC can participate. This means your Java service can coordinate with a Python script and a Go microservice in the same weave, with no glue code required.
Weaverdance also supports distributed weaves — you can span multiple data centers by federating control planes. Each federated plane maintains its own artifact ledger, but accepts commands and shares state via a gossip protocol. This gives you disaster recovery capabilities without sacrificing determinism.
Every Weaverdance control plane includes a sentinel process that monitors system health, watches for resource leaks, and automatically repairs corrupted ledger entries. The sentinel also provides a REST endpoint for health checks, so you can integrate with your existing monitoring stack.
For enterprise customers, we offer around-the-clock assistance with weave design, performance tuning, and custom plugin development. Our support engineers are weavers themselves — they've used the system in production for years and can debug even the most tangled choreography.
We have an ambitious vision for the year ahead:
- Adaptive Stitching: Automatically reorder tasks based on historical performance data, while maintaining deterministic output guarantees
- Quantum-Safe Ledger: Upgrade the artifact ledger to use post-quantum cryptographic hashes
- Visual Weave Editor: A drag-and-drop interface for designing choreographies, with real-time performance simulation
- Cross-Cloud Federation: Native support for orchestrating across AWS, Azure, and GCP with latency-aware scheduling
A multinational bank uses Weaverdance to reconcile millions of transactions daily. The deterministic stitching ensures that audit logs are exact — every number can be traced back to its source transaction, and any discrepancy can be replayed to find the root cause.
A biotech startup weaves together DNA sequencing, alignment, and variant calling tasks. The artifact provenance ledger lets them prove to regulators exactly which software versions and reference genomes produced each clinical report.
A game studio orchestrates seasonal event servers using dynamic weave patterns. When a new event starts, they spin up hundreds of workers with deterministic state snapshots — every player sees the exact same event state, even across server restarts.
We welcome contributors who want to help weave a better orchestration fabric. Check out our contribution guidelines in the repo — we're particularly interested in:
- New plugin implementations
- Performance optimizations for the artifact store
- Additional language bindings for the worker protocol
- Documentation and tutorial improvements
This project is released under the MIT License. You are free to use, modify, and distribute this software, provided you retain the original copyright notice.
For the full license text, please see the LICENSE file in this repository.
Weaverdance Orchestrator is provided "as is," without warranty of any kind, express or implied. While the deterministic stitching engine ensures reproducibility under normal operating conditions, we cannot guarantee determinism when external services introduce non-determinism (e.g., random number generators, wall-clock time, or unseeded hashing). The system is designed to make such non-determinism visible via your artifact ledger, but identifying and eliminating it remains your responsibility.
The developers are not liable for any damages arising from the use of this software, including data loss, service interruption, or financial losses resulting from incorrect orchestration. By using Weaverdance, you acknowledge that you have read this disclaimer and understand the inherent limitations of any distributed system.
