Skip to content

Improve date/timestamp handling performance in GenericDaoBase and DateUtil - #13809

Open
sudo87 wants to merge 2 commits into
apache:4.20from
shapeblue:date-perf-optimizations
Open

sudo87 wants to merge 2 commits into
apache:4.20from
shapeblue:date-perf-optimizations

Conversation

@sudo87

@sudo87 sudo87 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

This PR contains following changes:

  • GenericDaoBase was reading and writing dates by converting them to strings via DateUtil and passing those strings to JDBC.
    This means every single DB read or write for a date column allocates a string, formats it, and then parses it back on the other side - completely unnecessary work the JDBC driver can handle natively.
    This PR replaces all of that with proper JDBC typed methods: getTimestamp() for reads, and setTimestamp()/setDate()/setTime() for writes, using a per-call GMT Calendar so timezone handling stays consistent.

  • Cleaned up DateUtil: SimpleDateFormat isn't thread-safe and was allocated fresh per
    call. Swapped in DateTimeFormatter, cached in a ConcurrentHashMap keyed by pattern + zone,
    so formatting is just a map lookup now.

  • Also, includes a minor optimization to the CIDR allow-list check in ApiServer (lookup is now skipped entirely when the check is disabled).

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 27.58621% with 42 lines in your changes missing coverage. Please review.
✅ Project coverage is 16.27%. Comparing base (549daae) to head (26b2026).

Files with missing lines Patch % Lines
...c/main/java/com/cloud/utils/db/GenericDaoBase.java 0.00% 35 Missing ⚠️
utils/src/main/java/com/cloud/utils/DateUtil.java 66.66% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##               4.20   #13809   +/-   ##
=========================================
  Coverage     16.26%   16.27%           
- Complexity    13434    13448   +14     
=========================================
  Files          5667     5667           
  Lines        500731   500740    +9     
  Branches      60803    60806    +3     
=========================================
+ Hits          81455    81486   +31     
+ Misses       410172   410146   -26     
- Partials       9104     9108    +4     
Flag Coverage Δ
uitests 4.14% <ø> (ø)
unittests 17.12% <27.58%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sudo87

sudo87 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18782

@DaanHoogland

Copy link
Copy Markdown
Contributor

looks good @sudo87.
Any tests to add to ApiServer or GenericDaoBase?
also; Can this go on 4.20?

@sudo87

sudo87 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves date/timestamp handling performance by removing string-based JDBC interactions for temporal values, modernizing DateUtil formatting/parsing internals, and avoiding unnecessary CIDR allow-list lookups when that check is disabled.

Changes:

  • Switch GenericDaoBase date/calendar JDBC reads/writes from string conversions to typed getTimestamp / setTimestamp / setDate / setTime (with a per-call GMT Calendar).
  • Replace DateUtil’s per-call SimpleDateFormat usage with cached, thread-safe DateTimeFormatter instances.
  • Optimize ApiServer CIDR allow-list enforcement by skipping account/CIDR lookup entirely when the feature flag is disabled, and add tests for the new behavior.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
utils/src/main/java/com/cloud/utils/DateUtil.java Introduces cached DateTimeFormatter usage and updates date parsing/formatting paths.
utils/src/test/java/com/cloud/utils/DateUtilTest.java Adds tests covering DateUtil parsing/formatting behavior across time zones and null handling.
framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java Migrates JDBC temporal reads/writes to typed APIs and adds GMT Calendar helper + temporal SQL type selection.
framework/db/src/test/java/com/cloud/utils/db/GenericDaoBaseTest.java Adds unit tests validating GMT calendar behavior and typed timestamp reads for Date/Calendar.
server/src/main/java/com/cloud/api/ApiServer.java Skips CIDR allow-list lookup when checks are disabled; uses whitespace deletion helper.
server/src/test/java/com/cloud/api/ApiServerTest.java Adds tests validating CIDR lookup skip/allow/deny paths and restores config defaults after each test.
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupVO.java Changes date mapping from TemporalType.DATE to TemporalType.TIMESTAMP.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread utils/src/main/java/com/cloud/utils/DateUtil.java
@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18788

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18816

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian test result (tid-16730)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 58089 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13809-t16730-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Ready

Development

Successfully merging this pull request may close these issues.

5 participants