Skip to content

feat(dsl): add MongoDB connector#816

Open
johntomcat7408-cmyk wants to merge 1 commit into
apache:masterfrom
johntomcat7408-cmyk:feat/797-mongodb-connector
Open

feat(dsl): add MongoDB connector#816
johntomcat7408-cmyk wants to merge 1 commit into
apache:masterfrom
johntomcat7408-cmyk:feat/797-mongodb-connector

Conversation

@johntomcat7408-cmyk

Copy link
Copy Markdown

What changes were proposed in this pull request?

  • Add a MongoDB source and sink connector with SPI registration.
  • Support bounded reads, explicit integer range partitions with [lowerbound, upperbound)
    boundaries, and count offsets.
  • Support ordered batch inserts and BSON type conversion.
  • Add connector configuration, unit and integration tests, GQL example, and CN/EN docs.

The source targets static collections. CDC, upsert, automatic retry, and exactly-once delivery
are not included.

Closes #797.

How was this PR tested?

  • Tests have been added for the changes

  • Production environment verified

  • MongoTableConnectorTest and MongoRowConverterTest: 12 tests passed.

  • MongoTableConnectorIT: passed with Docker in
    GitHub Actions;
    0 failures, errors, or skips.

  • Maven verify, Checkstyle, Apache RAT, and dependency tree checks passed.

@johntomcat7408-cmyk

Copy link
Copy Markdown
Author

I verified PR merge commit b42bc097a2b144e0ae9713bb44af174d1cfb5983 (head 148d4791bf2f8199b0703077b26ecba7940d2713) locally on Windows 11 with Oracle JDK 8u451, Maven 3.8.9, and protoc 3.21.7.

  • All 107 modules compiled successfully, including test sources (mvn ... -DskipTests).
  • MongoDB connector unit tests passed: 12 tests, 0 failures.

The failed JDK 8 job stopped at setup-protoc due to GitHub API rate limiting before Maven started. Could you please rerun it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a MongoDB connector

1 participant