-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[feature](paimon) support read paimon ro (read-optimized) system table #59340
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
a1fdae4 to
e3d1431
Compare
|
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for reading Paimon read-optimized (RO) system tables, which allows users to query fully compacted data without overlapping key ranges, enabling the use of native readers for better performance.
Key Changes:
- Introduces
@rosyntax for accessing Paimon read-optimized system tables - Adds validation to ensure
@rois used without parameters - Implements RO table handling in PaimonExternalTable by fetching the "ro" system table variant
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| DorisParser.g4 | Modified grammar to make parentheses optional for scan parameters, enabling @ro syntax without parentheses |
| TableScanParams.java | Added RO constant, validation logic, and isRo() method to support read-optimized table parameter |
| PaimonExternalTable.java | Implemented RO table snapshot retrieval logic following the same pattern as branch handling |
| PaimonScanNode.java | Added table name (including $ro suffix) to explain output for better query visibility |
| test_paimon_ro_read.groovy | Added comprehensive test verifying RO read behavior and error handling for invalid parameter usage |
| test_paimon_ro_read.out | Expected test output showing difference between normal and RO reads |
| run10.sql | Test data setup script creating a Paimon table with compacted and delta data |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
fe/fe-core/src/main/java/org/apache/doris/analysis/TableScanParams.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/analysis/TableScanParams.java
Outdated
Show resolved
Hide resolved
TPC-H: Total hot run time: 35565 ms |
TPC-DS: Total hot run time: 180031 ms |
ClickBench: Total hot run time: 27.61 s |
FE Regression Coverage ReportIncrement line coverage |
…rams.java Co-authored-by: Copilot <[email protected]>
…rams.java Co-authored-by: Copilot <[email protected]>
|
run buildall |
TPC-H: Total hot run time: 35864 ms |
TPC-DS: Total hot run time: 180298 ms |
ClickBench: Total hot run time: 27.2 s |
FE Regression Coverage ReportIncrement line coverage |
What problem does this PR solve?
Usage Example:
rotable, we could use the Native reader with some data latency:Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)