Skip to content

Improve metadata OOM diagnostics#18039

Open
Caideyipi wants to merge 2 commits into
masterfrom
improve-metadata-oom-diagnostics
Open

Improve metadata OOM diagnostics#18039
Caideyipi wants to merge 2 commits into
masterfrom
improve-metadata-oom-diagnostics

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

This PR improves metadata/query-analysis OOM diagnostics when result-set column expansion or schema fetch runs out of memory.

Changes:

  • Track matched, expanded, generated, and schema-fetch deserialized result columns in MPPQueryContext.
  • Enrich MemoryNotEnoughException during metadata analysis with column counts, SLIMIT/SOFFSET, ALIGN BY DEVICE guidance, memory shortage, and estimated exceeded columns.
  • Record column counts during expression expansion and schema fetch batch deserialization.
  • Add MPPQueryContextTest coverage for enriched result-column and schema-fetch OOM messages, plus the no-context fallback path.

Verification

  • mvn spotless:apply -pl iotdb-core/datanode
  • git diff --check
  • mvn "-Ddevelocity.off=true" test -pl iotdb-core/datanode -Dtest=MPPQueryContextTest currently fails before running tests during datanode main-source compilation because of pre-existing/unrelated missing symbols in AstBuilder, DataNodeMemoryConfig, TableConfigTaskVisitor, DatasetHeaderFactory, ShowCreate*Task, and DataRegionConsensusImpl.

@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 77.15736% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.70%. Comparing base (7f41f06) to head (9fa3066).
⚠️ Report is 32 commits behind head on master.

Files with missing lines Patch % Lines
...e/iotdb/db/queryengine/common/MPPQueryContext.java 78.53% 38 Missing ⚠️
...lan/analyze/schema/ClusterSchemaFetchExecutor.java 0.00% 6 Missing ⚠️
...eryengine/common/schematree/ClusterSchemaTree.java 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18039      +/-   ##
============================================
+ Coverage     41.42%   41.70%   +0.27%     
  Complexity      318      318              
============================================
  Files          5282     5296      +14     
  Lines        369246   372014    +2768     
  Branches      47786    48128     +342     
============================================
+ Hits         152978   155165    +2187     
- Misses       216268   216849     +581     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines +668 to +679
Locale.ROOT,
"Not enough memory while analyzing metadata for query result columns. "
+ "The result set has too many columns. "
+ "Before the failure, IoTDB had matched %,d source columns for result-column "
+ "expansion, expanded %,d source columns, and generated %,d result-set columns. "
+ "%s"
+ "Current series pagination is %s. "
+ "Use SLIMIT/SOFFSET to reduce returned series%s, narrow the path pattern, "
+ "or increase query memory%s. "
+ "Memory details: source-column memory for result expansion %s, "
+ "generated-result-column memory %s, requested this time %s, current free memory %s. "
+ "Original error: %s",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i18n

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in 9fa3066: moved the metadata OOM diagnostic text into DataNodeQueryMessages with both English and Chinese i18n constants.

@sonarqubecloud

sonarqubecloud Bot commented Jul 3, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants