Skip to content

[oncall-telemetry] Default report window to rolling 7 days ending today, Fixes AB#3683194#447

Open
shahzaibj wants to merge 1 commit into
masterfrom
shjameel-microsoft-oncall-telemetry-default-window
Open

[oncall-telemetry] Default report window to rolling 7 days ending today, Fixes AB#3683194#447
shahzaibj wants to merge 1 commit into
masterfrom
shjameel-microsoft-oncall-telemetry-default-window

Conversation

@shahzaibj

@shahzaibj shahzaibj commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Fixes AB#3683194

Feedback

From Fadi Durah's ★4 skill-effectiveness survey (Jun 2026, oncall-weekly-telemetry-report):

"The telemetry skill was certainly helpful, the biggest feedback is that it seemed to capture the wrong timeframe (didn't capture the most recent 6 days). I'm sure I could've spent some time with it to get it to target the right date, but this is probably something that should happen by default in order to save more time for the developer, and something moumita has been pushing for."

Root cause (repro'd on today's clock, Thu 2026-07-09)

bootstrap-report.ps1 was computing "reporting Sunday" as the most recent Sunday strictly before today — which is the Sunday of the currently in-progress week. Run on a Thursday it picked Sun 2026-07-05 → Sat 2026-07-11 where Saturday is 2 days in the future, so the WoW section queried a 4-day partial window and dropped the last complete week. Exactly Fadi's "missed the last 6 days" complaint. SKILL.md then compounded it by asking the model to confirm the Sunday with the user — the manual coax that the developer shouldn't have to do.

Today (UTC) Old default (broken) New default
Thu 2026-07-09 Sun 07-05 → Sat 07-11 (future!) Thu 07-02 → Thu 07-09
Mon 2026-07-13 Sun 07-12 → Sat 07-18 (future!) Mon 07-06 → Mon 07-13

Fix

Per the user's direction, deep rewrite: rolling [today−7d, today) for the primary WoW view, keeping Sun-Sat weekly bucketing only for the 60-day trend (where stable weekly denominators still matter).

  • bootstrap-report.ps1 — new -EndDate param (default: today UTC). Computes curStart = curEnd−7d, prevStart = curEnd−14d, and trendEnd = startofweek(curEnd). Stamps the resolved window into the <title>, <div class="meta"> block, and Generated banner so header dates can never drift from what was queried (this is the ADO acceptance criterion). Refuses future end-dates. Collision fingerprint stopped keying off title/meta (which bootstrap now rewrites unconditionally) — uses firstKpi + size ratio only.

  • 15 KQL templates — 12 primary/WoW queries switched to half-open [prevStart, curEnd) with wk = iff(EventInfo_Time >= curStart, curStart, prevStart); downstream JS helpers unchanged because wk still sorts lexicographically. The 3 60-day-trend queries keep startofweek() bucketing; only <END> renamed to <TREND_END> with clarified startofweek(today)-exclusive semantics. Placeholder audit is now consistent across all 15 files.

  • SKILL.md — rewrote Inputs to confirm §1: do NOT ask the user for a date by default; just run bootstrap. Override via -EndDate only when the user explicitly requests a non-default window. Filename convention <sunday><end-date> throughout. All KQL placeholder examples switched to the rolling-window form.

  • validate-report.ps1 — new check Migrate to AndroidX #11 asserts the meta-line "Last 7 days" dates are self-consistent with the filename's end-date. Catches any hand-edit that broke the auto-stamp, or a stale template stub being published fresh.

  • Docstemplate-readme.md § Date fields rewritten (bootstrap owns every date field; author never edits them). kusto-cheatsheet.md gains § 3 Rolling 7-day WoW window with the canonical let curStart/curEnd/prevStart template; the pre-existing startofweek()-alignment section is now scoped explicitly to the 60-day trend.

Breaking changes (intentional)

  • bootstrap-report.ps1: -ReportingSunday parameter removed; use -EndDate. The old parameter name encoded the exact semantic mismatch that caused the bug — keeping it would have preserved the footgun.
  • Output filename: oncall-wow-report-<end-date>.html (was <sunday>.html). Old on-disk reports are unaffected.

Verification

  • Run on 2026-07-09 → correctly resolves to [2026-07-02, 2026-07-09).
  • -EndDate 2026-07-02[2026-06-25, 2026-07-02).
  • -EndDate 2027-01-01 → refused (future-date guard).
  • Same-day re-bootstrap of unfilled stub → silently overwrites.
  • Same-day re-bootstrap after author populated KPI → HARD HALT exit 2 unless -Force.
  • Validator check Migrate to AndroidX #11 passes on fresh stub, FAILs on a hand-mutated meta line.
  • Placeholder audit across all 15 rewritten .kql files — no stale <START>/<END>/<CURR_*>/<PREV_WEEK> references remain.

…3683194

Fadi's ★4 skill-effectiveness feedback (Jun 2026): "The telemetry skill was
certainly helpful, the biggest feedback is that it seemed to capture the wrong
timeframe (didn't capture the most recent 6 days)."

Root cause (repro'd on today's clock, Thu 2026-07-09):
The old bootstrap-report.ps1 computed "reporting Sunday" as the most recent
Sunday strictly before today — i.e. the Sunday of the CURRENTLY IN-PROGRESS
week. Run on a Thursday, it picked Sun 2026-07-05 -> Sat 2026-07-11 where
Saturday is 2 days in the future, so the WoW section queried a 4-day partial
window and dropped the last complete week. Exactly Fadi's "missed the last 6
days" complaint. SKILL.md compounded this by asking the model to confirm the
Sunday with the user — a manual coax that the developer shouldn't have to do.

Fix (deep — user picked "rewrite to rolling [today-7d, today) everywhere except
60-day trend" over the smaller Sun-Sat-preserving option):

* bootstrap-report.ps1: replaced -ReportingSunday with -EndDate (default: today
  UTC). Computes curStart = curEnd-7d, prevStart = curEnd-14d, and 60d trend
  window = 8 complete Sun-Sat weeks ending startofweek(curEnd). Stamps the
  resolved window into the report's <title>, <div class="meta">, and Generated
  banner so header dates can never drift from what was queried. Refuses future
  end-dates. Collision fingerprint stopped keying off title/meta (which bootstrap
  now rewrites unconditionally) — uses firstKpi + size ratio only.

* 15 KQL templates rewritten:
  - 12 primary/WoW queries now use half-open [prevStart, curEnd) with
    wk = iff(EventInfo_Time >= curStart, curStart, prevStart) → two rows per
    key, downstream JS helpers unchanged.
  - 3 60-day-trend queries keep startofweek() bucketing (weekly denominators
    still matter for the long-horizon view); only <END> placeholder renamed to
    <TREND_END> and its semantics clarified as startofweek(today) exclusive.
  - Placeholder audit is now consistent across all 15 files
    (<CUR_END>/<CUR_START>/<PREV_START>/<TREND_START>/<TREND_END>).

* JS helpers (agg.js, bucket-trends.js, summarize-attribution.js): schema
  detection continues to work — wk sorts lexicographically so smallest = prev,
  largest = cur regardless of whether it's a Sunday label or a rolling-window
  boundary. Docstrings updated.

* report-template.html: <title>/meta stamped by bootstrap; header comment
  updated to describe stamping. template-readme.md § Date fields rewritten:
  bootstrap owns every date field; author never edits them.

* SKILL.md: rewrote "Inputs to confirm §1" — do NOT ask the user for a
  reporting date by default. Just run bootstrap; it stamps the window
  automatically. Override via -EndDate only when user explicitly requests a
  non-default window. Filename convention <sunday> → <end-date> throughout.
  All KQL placeholder examples switched to the rolling-window form.

* kusto-cheatsheet.md: added § 3 "Rolling 7-day WoW window" with canonical
  template. Kept § 7 startofweek()-alignment note but scoped it to 60-day trend.

* validate-report.ps1: new check #11 asserts meta-line "Last 7 days" dates are
  self-consistent with the filename's end-date (catches any hand-edit that
  broke the auto-stamp, or a stale template stub being published fresh).

BREAKING (called out intentionally):
  * bootstrap-report.ps1: -ReportingSunday parameter removed; use -EndDate.
    The old name encoded the exact semantic mismatch that caused the bug.
  * Filename: oncall-wow-report-<end-date>.html (was <sunday>.html).

Verification:
  Run on 2026-07-09 → correctly resolves to [2026-07-02, 2026-07-09).
  -EndDate 2026-07-02 → resolves to [2026-06-25, 2026-07-02).
  -EndDate 2027-01-01 → refused (future date).
  Same-day re-bootstrap of unfilled stub → silently overwrites.
  Same-day re-bootstrap after author populated KPI → HARD HALT exit 2 unless
  -Force. Validator check #11 passes on fresh stub, FAILs on hand-mutated
  meta line.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@shahzaibj shahzaibj requested a review from a team as a code owner July 9, 2026 23:35
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

✅ Work item link check complete. Description contains link AB#3683194 to an Azure Boards work item.

1 similar comment
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

✅ Work item link check complete. Description contains link AB#3683194 to an Azure Boards work item.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

❌ Invalid work item number: AB#3683194

##. Work item number must be a valid integer.

Click here to learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant