Roadmap to v1.3.0 - #6
Merged
Merged
Conversation
zeroc0d3
commented
Jul 25, 2026
zeroc0d3
commented
Member
- OTLP-native end-to-end — OpenTelemetry future
- eBPF kernel observability — 28 metrics from syscalls, scheduler, file I/O, TCP state.
- Kubernetes super-deep — 33 sub-collectors replace 3-4 Helm charts (kube-state-metrics, cAdvisor, eventrouter, node-exporter).
- QAN (Query Analytics) — PMM-inspired. tfo-agent adds query-level analytics.
- Supervisor FSM + hot reload — Per-collector FSM, exponential backoff, SIGHUP reload.
- Property-based testing — Rare in agent codebases.
- Multi-cloud IMDS auto-detection — AWS, GCP, Azure, Alibaba, Huawei, DigitalOcean + 15 K8s provider types.
Roadmap: tfo-agent-roadmap/08-architecture-improvements.md §1 Bump version 1.2.2 → 1.3.0-dev (M1 development start). Add new internal/plugin/ package as the foundation for the M1 milestone. This coexists with the existing pkg/plugin (legacy generic registry) and internal/collector.Collector (legacy collector interface) — both are kept for backwards compatibility during the incremental migration. New package provides: - Typed plugin contracts: Collector, ServiceCollector, StreamingProcessor, SyncProcessor, Aggregator, Output, Parser, Serializer, SecretStore - Capability mixins: Initializer, PluginWithID, StatefulPlugin, ProbePlugin, ParserPlugin, SerializerPlugin - Self-registration pattern via init() + MustAddXxx helpers (mirrors Telegraf) - Universal Metric type with histogram/summary support and deep Copy() - ChannelAccumulator + DiscardAccumulator implementations - CollectorAdapter wraps legacy collector.Collector as plugin.Collector - SyncProcessorAdapter upgrades legacy SyncProcessor to StreamingProcessor - Error taxonomy: FatalError, StartupError, PartialWriteError - StartupBehavior constants (error/retry/ignore/probe)
Mirrors Telegraf's selfstat/ package. Subsystems register named counter/ gauge stats (per-agent, per-collector, per-exporter) and timing stats that report running averages cleared on read. An internal collector will emit snapshots via AllMetrics() into the normal metric pipeline. - selfstat.go: Stat/TimingStat interfaces, atomic intStat, mutex-guarded timingStatImpl, global registry with dedup by (name, labels) - agent.go: init()-registered Agent* globals (metrics written/rejected/ dropped/gathered, gather/write errors, buffer size/limit, version info) - collector.go: ForCollector helper with State gauge (stopped/running/ backoff/failed) and GatherTimeNS timing - exporter.go: ForExporter helper with WriteTimeNS timing and buffer gauges - selfstat_test.go: table-driven, property, and concurrency tests Thread-safe via sync/atomic and sync.Mutex; verified with -race.
Roadmap: tfo-agent-roadmap/08-architecture-improvements.md §10 Provides a registry for versioned config schema upgrades between agent releases. Each migration is a self-contained package that registers itself at init() time, mirroring Telegraf's 63-package migration tree (at much smaller scale today). - migration.go: Migration/Registry types, DefaultRegistry, Register / MustRegister / ApplyAll (range-based) / ApplyLatest / List / DetectVersion (reads agent.version from YAML, defaults to "1.2.0") - migration_test.go: 16 tests (dedup, ordering, range select, error propagation, version detection) - v1_3/v1_3.go: first real migration 1.2.0 → 1.3.0, tls_skip_verify_rename (extracts the inline migration currently in internal/config/loader.go for discoverability) - v1_3/v1_3_test.go: 7 tests for the rename (top-level + nested indent, no-op, multi-occurrence, substring safety) - all/all.go: blank-import aggregator for cmd/ to pull every migration ApplyAll semantics: applies migrations whose FromVersion >= fromVersion AND ToVersion <= toVersion, sorted ascending. from == to yields no-op.
Roadmap: tfo-agent-roadmap/08-architecture-improvements.md §3
Introduces a @{store:key} reference syntax for resolving secrets in any
config string, backed by pluggable SecretStore implementations. Mirrors
Telegraf's secret-store pattern with three production-grade backends.
Resolver order: ${VAR} is expanded first via os.ExpandEnv (so a Vault
token can come from env), then @{store:key} is resolved against the
instantiated stores.
Backends:
- env/env.go: os.Getenv backend with optional prefix; static resolver
- file/file.go: JSON map[string]string file on disk, RWMutex-guarded,
thread-safe for future hot reload; static resolver
- vault/vault.go: HashiCorp Vault KV-v2 over net/http (no SDK dep —
keeps go.mod lean), X-Vault-Token / X-Vault-Namespace headers,
dynamic=true resolver (Vault leases can rotate, re-fetch each call)
- all/all.go: blank-import aggregator
Resolver:
- resolver.go: StoreConfig, NewResolver, Resolve, ResolveBytes
- Regex: @\{([a-zA-Z0-9_-]+):([a-zA-Z0-9_./-]+)\}
- resolver_test.go: 23 tests (env/file/vault unit + resolver integration
incl. httptest mock for Vault)
Stdlib + zap only; no go.mod changes. Wiring into internal/config/
loader.go is a separate task.
Roadmap: tfo-agent-roadmap/06-processor-pipeline-roadmap.md Roadmap: tfo-agent-roadmap/08-architecture-improvements.md §1 Topology: inputs → pre-agg processors → aggregators → post-agg processors → outputs. Mirrors Telegraf's DAG with Go channels. The pipeline is opt-in: the existing metric_forwarder path continues to work unchanged for backwards compatibility during M1. New code registered via internal/plugin is routed through this pipeline. - pipeline.go: Pipeline struct, New, AddCollector/AddServiceCollector/ AddPreAggregatorProcessor/AddAggregator/AddPostAggregatorProcessor/ AddOutput, Run (lifecycle), Stop (graceful) - Config: QueueSize (default 10000), DropPolicy (block/drop_oldest/ drop_newest default), AggregatorPeriod (30s), FlushInterval (5s) - Channel-based backpressure via enqueue helper with central policy - Aggregator fan-out with DropOriginal semantics - Output flusher with batch_size=1000 + ticker - pipeline_test.go: 9 tests incl. race-clean end-to-end with a passthrough processor + one-shot service collector + capture output
Roadmap: tfo-agent-roadmap/06-processor-pipeline-roadmap.md
Each processor registers itself via init() in its sub-package so the
plugin registry is populated when cmd/tfo-agent blank-imports
internal/processor/all. Mirrors Telegraf's plugins/processors/<name>
layout.
Foundation processors (StreamingProcessor interface):
- filter/filter.go: rule-based keep/drop by name regex + tag
presence/value regex; first-match-wins;
configurable DefaultAction (keep/drop)
- drop/drop.go: regex-based drop shorthand
- keep/keep.go: regex-based keep shorthand (inverse of drop)
- rename/rename.go: rename measurements (Name) and tag keys
- converter/converter.go: round float values (regex-selected);
precompiled patterns
- enum/enum.go: map label values via enum tables with default
- defaults/defaults.go: apply default tag values when absent or empty
Scripting processor:
- starlark/starlark.go: Turing-complete escape hatch via embedded
Starlark (go.starlark.net). Scripts define
`apply(metric)` returning dict/None/list for
passthrough/drop/fan-out. Available keys: name,
description, type, value, timestamp (unix nanos),
unit, labels (dict[str,str]). Errors are surfaced
via accumulator.AddError and the metric is
forwarded unchanged (fail-safe).
Tests:
- filter: 7 tests incl. tag presence + value regex
- drop: 4 tests incl. invalid regex error
- keep: 3 tests incl. invalid regex error
- rename: 3 tests (measurement, tag, passthrough)
- converter: 3 tests incl. invalid regex error
- enum: 4 tests incl. default fallback
- defaults: 3 tests incl. empty-value replacement + nil labels
- starlark: 7 tests incl. fan-out, drop, syntax error, missing apply
all/all.go: blank-import aggregator for cmd/tfo-agent.
go.mod: + go.starlark.net (single new dependency).
Roadmap: tfo-agent-roadmap/08-architecture-improvements.md §2 Closes a P0 quick-win: internal/buffer/buffer.go (371 lines) was complete and tested but never instantiated in agent.go. Metrics were sent directly to OTLP without local retry — a backend outage meant data loss. Implementation: - exporter/buffer_retry_sink.go: BufferRetrySink wraps any MetricSink with disk-backed retry. On inner Export failure the batch is pushed into internal/buffer.Buffer (disk-persistent JSON) and a background goroutine retries every RetryInterval (default 5s). In-memory fallback queue (cap 100) is used when the disk buffer is nil. PartialWriteError handling is added in a later iteration (M5). - exporter/buffer_retry_sink_test.go: 5 tests covering disabled passthrough, enabled absorbs error, retry loop eventually exports, in-memory fallback, max-retries drop semantics. - agent.go: NewWithConfigFile now instantiates the disk buffer when cfg.Buffer.Enabled is true and wraps otlpBridge with BufferRetrySink before passing it to MetricForwarder. Run() launches the retry goroutine; shutdown is handled via context cancellation. No breaking change: when cfg.Buffer.Enabled is false the pipeline behaves exactly as before (passthrough). All M1 tests pass: agent + buffer + exporter + migration + persister + pipeline + plugin + processor (8 sub-packages) + secret + selfstat.
Roadmap: tfo-agent-roadmap/08-architecture-improvements.md §4, §10
Three-stage preprocessing pipeline applied to raw config bytes before
they reach viper/YAML parsing:
1. migration.ApplyLatest — schema upgrades for older configs
(e.g. tls_skip_verify_rename from 1.2.0 → 1.3.0)
2. os.ExpandEnv — ${VAR} substitution (legacy, preserved)
3. SecretResolver — @{store:key} substitution (when wired)
Failures in (1) and (3) are logged but non-fatal — the config still
parses and surfaces the actual error from viper, which is more
actionable for users.
API additions to Loader (chainable):
- WithSecretResolver(SecretResolver) — inject @{store:key} resolver
- WithLogger(*zap.Logger) — log migration/resolution warnings
- WithMigrationEnabled(bool) — toggle schema migration (default true)
SecretResolver is declared as an interface in the config package to
avoid an import cycle (config -> secret -> plugin -> collector ->
config). The concrete *secret.Resolver satisfies it; cmd/tfo-agent
wires the two together.
The existing migrateDeprecatedConfigKeys() helper remains in place —
it operates on already-parsed viper keys (post-ReadConfig) and is
orthogonal to the byte-level migrations applied in preprocess().
Roadmap: tfo-agent-roadmap/08-architecture-improvements.md §7 The persister package (internal/persister) was complete but unwired. This commit closes the loop: Config additions (internal/config/config.go): - Config.Persister field (PersisterConfig struct) - Enabled gates persister wiring (default false — opt-in) - Statefile path (default /var/lib/tfo-agent/state.json when enabled) - SaveInterval for periodic checkpoint (default 5m) Agent wiring (internal/agent/agent.go): - Load previously persisted state BEFORE collectors start so plugins can pick up their saved state during Init/Start (M3 log tail offset will be the first consumer). - StartSaveLoop in Run() for periodic checkpointing (cancellable via ctx). - Final Store() at the start of shutdown() so StatefulPlugin state is captured while plugins are still alive to serve GetState(). When no plugin implements plugin.StatefulPlugin the persister is wired but inert (zero per-tick cost — empty state map). M3 will register the log collector's tail offsets as the first stateful consumer.
Convert 7 processor test files from co-located internal package style to external '<name>_test' packages now centralized under tests/unit/domain/processor/. Qualify all source-package references (New, Config, Rule, ActionKeep/ActionDrop, TagMatch, RoundingSpec, Mapping, DefaultConfig) while leaving plugin.* imports and local helpers (captureAcc) unqualified.
Move internal-style test files into the centralized tests/ tree using external *_test packages so they exercise the public API only. Test files migrated (package -> _test): - exporter/buffer_retry_sink_test.go - migration/migration_test.go - migration/v1_3/v1_3_test.go - pipeline/pipeline_test.go - persister/persister_test.go - selfstat/selfstat_test.go - secret/resolver_test.go (package already migrated) Production accessors added for unexported members reached by tests: - migration: export CompareVersion (was compareVersion) - pipeline: add Pipeline.Config() and Pipeline.Enqueue() - persister: add Persister.RegisteredIDs() and Persister.CacheSnapshot() All 7 packages pass go vet and go test -count=1.
Roadmap: tfo-agent-roadmap/04-network-monitoring-roadmap.md
Closes a process violation: previous commits co-located *_test.go
files alongside source under internal/. Project convention (see
existing tests/unit/ tree with 227 centralized test files) is to
keep all tests under tests/unit/{domain,infrastructure,...}/ with
external *_test packages.
New collectors (M2 P0):
- collector/ping: ICMP probe via golang.org/x/net/icmp with both
privileged (raw socket) and unprivileged (UDP, Linux sysctl
net.ipv4.ping_group_range) modes. Auto-falls back to UDP when raw
sockets are denied. Emits rtt_min/avg/max/stddev_ms, packets_sent/
received, loss_percent, ttl, state per target.
- collector/dns: DNS query probe via github.com/miekg/dns. Supports
A/AAAA/TXT/MX/NS/CNAME/PTR. Emits query_time_ms, result_code,
records_returned, state per (server × query).
Test centralization (process fix):
- Moved 4 collector tests to tests/unit/domain/collector/{ping,dns,
tcp_probe,http_probe}/ with <name>_test package
- Added exports.go in ping + dns packages with exported test seams
(SetPingerExported, SetResolverExported, etc.) following the
existing postgresql/exports.go pattern
- Updated drop + filter processors with minor setter additions for
testability
go.mod additions: golang.org/x/net/icmp (promoted from indirect),
github.com/miekg/dns v1.172.
Polls SNMP v1/v2c/v3 agents for scalar OIDs (Get) and table subtrees (Walk) and emits per-agent metrics under network.snmp.*. Per-agent connection failures emit state=0 and collection continues; successful polls emit state=1. ASN.1 types map to gauge/counter with OctetStrings parsed to float when numeric and emitted as <name>_len otherwise. Includes centralized external tests (package snmp_test) covering scalar Get, table Walk, connect failure, v2c/v3 wiring, type conversion, ctx cancellation between agents, config defaults, and labels. Tests inject a fake client via SetClientFactoryExported so no real SNMP server is needed.
NetFlow v5/v9/IPFIX UDP listener with a stdlib-only v5 parser (24-byte header + 48-byte records). v9/IPFIX header-only today with TODO; unknown versions drop into a parse-error counter. Service-style collector: Start() binds the UDP socket and spawns 1 reader + N workers; Collect() snapshots+resets atomic per-window counters every cycle and emits network.netflow.* counter metrics (packets, flows, bytes, parse_errors, packets_by_version). Per-flow metric emission is deferred to avoid metric-volume explosions. Test seam (exports.go) injects a fake PacketSourceExported so tests run deterministically without binding a socket. All tests live in tests/unit/domain/collector/netflow under package netflow_test per the centralized-test convention.
Service-style collector that listens for syslog messages over UDP/TCP/Unix
sockets and parses each via github.com/leodido/go-syslog/v4 (RFC 3164,
RFC 5424, and Cisco IOS formats). Background goroutines per listener
update per-listener counters; Collect snapshots and resets them so each
cycle reports the delta. Emits network.syslog.{messages_received_total,
parse_errors_total, bytes_received_total, messages_by_severity,
messages_by_facility} labeled with listener and protocol. Per-message
logs are deferred to M3.
Roadmap: tfo-agent-roadmap/04-network-monitoring-roadmap.md Register the 7 M2 network monitoring collectors in the central CollectorConfig struct and instantiate them in agent.NewWithConfigFile so they participate in the existing metric_forwarder pipeline. Config additions to CollectorConfig (internal/config/config.go): - Ping PingCollectorConfig (mapstructure:"ping") - DNS DNSCollectorConfig (mapstructure:"dns") - TCPProbe TCPProbeCollectorConfig (mapstructure:"tcp_probe") - HTTPProbe HTTPProbeCollectorConfig (mapstructure:"http_probe") - SNMP SNMPCollectorConfig (mapstructure:"snmp") - Netflow NetflowCollectorConfig (mapstructure:"netflow") - SyslogListener SyslogListenerConfig (mapstructure:"syslog_listener") Agent wiring (internal/agent/agent.go): - Imports for 7 new collector packages (ping, dns, tcp_probe, http_probe, snmp, netflow, syslog_listener). - Per-collector enabled gate + constructor call + structured log line announcing interval/targets/listeners. Listener-style collectors (netflow, syslog_listener) handle their own goroutine lifecycle in Start/Stop. - Collectors are appended to the shared `collectors` slice consumed by MetricForwarder — zero behavioural change for users who do not set the new `collectors.<name>.enabled: true` keys. All collectors remain opt-in (Enabled=false by default); existing configs are unaffected.
StreamingProcessor that walks an ordered policy list and applies the first matching action: always-forward, drop, or deterministic FNV-1a probabilistic sampling over (name + sorted label values). Self-registers as 'tail_sampling' and is wired into the processor.all bundle. Default action is keep when no policy matches.
Implements the Telegraf-compatible 'internal' collector that snapshots selfstat.AllMetrics() into the metric pipeline on each Collect cycle. Conversion from plugin.Metric to legacy collector.Metric uses the existing plugin.ToLegacyMetric adapter. Package dir is internalstats to avoid Go's reserved 'internal' path; the user-visible Name() stays 'internal'.
… json, regex)
Streaming processors that parse metric.Description (treated as a raw log
line per the M3 convention) and emit derived metrics:
- multiline: regex-based state machine that aggregates continuation lines
into the buffered header metric. Supports Negate, per-StreamKey grouping,
and Timeout-based flush via time.AfterFunc. Header labels are preserved
across continuations.
- grok_parser: translates %{PATTERN:name} grok syntax to Go RE2 at compile
time using a self-contained pattern table (TIMESTAMP_ISO8601, LOGLEVEL,
GREEDYDATA, IP, NUMBER, ...). No external grok dependency. Telegraf-style
%{NUMBER:bytes:int} type annotations are accepted and ignored. Optional
MetricNamePrefix rename and KeepOriginal behaviour.
- json_parser: json.Unmarshal of Description with dotted-path TagKeys
(e.g. "user.id") and optional ValueKey override of metric.Value.
Non-JSON / non-object input is forwarded unchanged.
- regex_parser: thin wrapper around regexp with named captures promoted to
labels. DropWhenNoMatch toggles drop-vs-passthrough on mismatch.
All four self-register via plugin.MustAddProcessor in init() and follow
the StreamingProcessor shape from filter/converter/drop. Tests live in
tests/unit/domain/processor/<name>/ as external <name>_test packages and
pass under -race including the multiline timeout and concurrency stress
cases.
Service-style collector that receives sFlow v5 datagrams and emits aggregate counters under network.sflow.*. Mirrors the netflow collector shape: Start() opens a UDP listener with N parser workers, Collect() snapshots and resets per-window counters. Parser (stdlib only) decodes the datagram header and each sample's (format, length) envelope. Samples are bucketed by format type (flow, counter, expanded_flow, expanded_counter, unknown); detailed sample-body decoding is left as TODO. - internal/config/sflow_config.go: SflowCollectorConfig (defaults Port=6343, Workers=4, BufSize=65535, FlushInterval=30s) - internal/collector/sflow/parser_v5.go: ParseSflowV5 pure function - internal/collector/sflow/sflow.go: SflowCollector + metric builder - internal/collector/sflow/exports.go: test seams - internal/config/config.go, internal/agent/agent.go: wire Sflow field and collector registration (netflow pattern) - tests/unit/domain/collector/sflow/: external test package covering parser correctness, lifecycle, counter reset, format labels, schema
- Fix db.redis.version_info that always emitted 0 (ToFloat on the semver
string '7.2.5' returned 0). Replaced with version_major/minor/patch
gauges parsed from the semver string via ParseSemver. Valkey mirrors
this for valkey_version.
- Implement LATENCY LATEST collection gated by CollectLatency config.
Emits db.{redis,valkey}.latency_ms + latency_max_ms per event. Adds
ParseLatencyLatest helper and LatencyEvent type (reused by valkey).
- Implement CLUSTER INFO parsing when cluster_enabled=1. Emits
cluster_state (1=ok/0=fail), cluster_slots_assigned, cluster_slots_ok
gauges. Adds ParseClusterInfo helper and cluster_enabled gauge from
INFO. Both collectors reuse the same helpers.
- Fix client.go docstring to match reality (was claiming replication +
cluster stats that weren't actually collected).
- Add CollectLatency to ValkeyInstanceConfig for parity with Redis.
Update CollectLatency docstring (LATENCY LATEST, not HISTORY/RESET).
- BuildRedisMetrics / BuildValkeyMetrics now take a MetricsInput struct
so future data sources can be added without breaking the signature.
- Bug 5 (InfoInterval in DefaultConfig) was already present for both
collectors; no change needed.
- Add unit tests for ParseSemver (valid/malformed/empty/prerelease),
ParseLatencyLatest (well-formed/malformed), ParseClusterInfo, version
semver metrics, cluster_enabled flag, cluster_* metrics, latency
metrics, and end-to-end cluster+latency collection through a fake
RESP server. Existing test call sites updated to the new struct
signature; existing assertions preserved.
Comprehensive Keep a Changelog entry for the 1.3.0-dev release that
bundles M1 Foundation, M2 Network Monitoring, and the in-progress
M3 Logs & Self-Observability milestones from the tfo-agent-roadmap.
Sections:
- Added (M1 Foundation): plugin system, buffer wiring, processor
pipeline, 7 foundation processors + starlark, secret management,
persister, selfstat layer, config migration framework, error
taxonomy
- Added (M2 Network): 8 collectors — ping, dns, tcp_probe,
http_probe, snmp, netflow, syslog_listener, sflow
- Added (M3 Logs): 4 log parser processors (multiline, grok_parser,
json_parser, regex_parser) + tail_sampling + internalstats collector
- Fixed (Redis/Valkey audit): version_info always-0 bug, dead
CollectLatency field, missing cluster metrics, docstring lies,
InfoInterval default. Mirrored to both collectors.
- Fixed (Process): test centralization (18 files moved to
tests/unit/{domain,infrastructure}/)
- Added (Documentation): REDIS.md, VALKEY.md, integration tests via
testcontainers
- Added (Tests): 18 centralized test packages + 11 new redis/valkey
tests
- Changed: version 1.2.2 → 1.3.0-dev; dependency additions
(go.starlark.net, miekg/dns, gosnmp, leodido/go-syslog,
testcontainers-go)
- Compatibility: 1.2.x config works unchanged; all new features opt-in
…oard, log_to_metric) Amends the 1.3.0-dev changelog with the M3 features that landed after the initial comprehensive entry: - OTLPLogBridge wiring (LogCollector.SetLogCallback → OTLP /v1/logs) - Log collector implements StatefulPlugin — tail offsets persist across restarts via persister framework - log_to_metric processor (regex-based metric extraction from logs) - Grafana self-observability dashboard JSON (deploy/grafana/) M3 is now functionally complete (5 of 6 workstreams). Remaining M3 item: wire selfstat counters into actual MetricForwarder + buffer retry sink code paths (currently the framework exists but counters stay zero).
Three M4 collectors that were partially built by cancelled subagents are completed and wired in this commit. - collector/nginx: stub_status scraper via stdlib HTTP. Emits 7 web.nginx.* metrics (connections_active/accepted/handled, requests, reading/writing/waiting). Labels: nginx_instance, nginx_host, nginx_port. Basic auth + TLS support. - collector/haproxy: CSV stats scraper via stdlib HTTP. Emits 12 proxy.haproxy.* metrics per row (frontend/backend/server) covering sessions, bytes, HTTP response counts, errors, server weight + status. Labels include pxname/svname/type for dimensional slicing. - collector/pgbouncer: SHOW STATS + SHOW POOLS via pgx. Emits 12 db.pgbouncer.* metrics across cluster-wide totals and per-pool connection counts. Test seam via SetClientFactoryExported. All three: - Centralized tests at tests/unit/domain/collector/<name>/ with external <name>_test package - Registered in central CollectorConfig struct - Wired into agent.NewWithConfigFile lifecycle - Follow the memcache.go collector struct shape
Two-part commit: 1. tests/unit/domain/collector/pgbouncer/pgbouncer_test.go — covers lifecycle, no-instances, interface assertion, connection-failure graceful handling, CollectStatsExported happy path + error, CollectPoolsExported happy path, InstanceLabelsExported, and ApplyInstanceDefaultsExported. Closes the test gap left when the pgbouncer collector was committed without tests. 2. CHANGELOG.md — amends the 1.3.0-dev entry with M4 (10 collectors) and M5 (6 outputs + Prom remote write fix + OTLP gRPC) sections that landed after the initial comprehensive changelog commit.
Two production-ready starting points that use only TFO-native collectors (no third-party database/app integrations, no pipeline processors). - configs/examples/k8s-minimal.yaml — Kubernetes DaemonSet/Deployment: system + node_exporter + kubernetes (33 sub-collectors). Includes cluster_name/provider auto-detection, namespace scoping, sync_to_backend, and commented sections for everything that's intentionally disabled. ~250-500 metrics per node. - configs/examples/vm-minimal.yaml — Standalone Linux VM/bare metal: system + node_exporter with device/mount exclusion patterns for container runtimes and pseudo-filesystems. Includes Docker run command with proper /proc /sys /etc /var bind mounts. ~250 metrics per host. - configs/examples/README.md — quick start for K8s (kubectl + Helm), VM (systemd), and Docker. Environment variable reference table. Lists what's intentionally NOT included and how to add each collector category incrementally. Both configs: - TFO native only (system, node_exporter, kubernetes) - Opt-in buffer (50MB disk-backed retry) - OTLP HTTP export with gzip compression - JSON structured logging - All non-essential features disabled (prometheus_server, persister, agent_api, pipeline, secret_stores, outputs) with commented sections showing how to enable them
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.