-
Notifications
You must be signed in to change notification settings - Fork 197
[762] Upgrade hudi version in xtable #772
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
The last test failure remaining to debug. |
|
In Hudi 1.x, all the partition paths from MDT are coming in as empty causing the failures as compared to 0.x. |
|
CI is green, still looking into the following issues. PR can be reviewed for other aspects.
|
vinishjail97
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Performed a self review on the PR as the changes were large and few tests had to be disabled for CI to be green. Looking into the disabled tests and addressing self review comments.
xtable-core/src/main/java/org/apache/hudi/stats/XTableValueMetadata.java
Show resolved
Hide resolved
xtable-core/src/main/java/org/apache/hudi/stats/XTableValueMetadata.java
Show resolved
Hide resolved
| * @param commit The current commit started by the Hudi client | ||
| * @return The information needed to create a "replace" commit for the Hudi table | ||
| */ | ||
| @SneakyThrows |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we catch/throw actual exceptions and avoid @SneakyThrows in main repo?
xtable-core/src/main/java/org/apache/xtable/hudi/BaseFileUpdatesExtractor.java
Outdated
Show resolved
Hide resolved
xtable-core/src/main/java/org/apache/xtable/hudi/BaseFileUpdatesExtractor.java
Outdated
Show resolved
Hide resolved
xtable-core/src/test/java/org/apache/xtable/ITConversionController.java
Outdated
Show resolved
Hide resolved
| "nested_record.level:SIMPLE", | ||
| "nested_record.level:VALUE", | ||
| nestedLevelFilter)), | ||
| // Different issue, didn't investigate this much at all |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the issue?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#775
Hudi 1.1 and ICEBERG partitioned filter data validation fails
| "timestamp_micros_nullable_field:DAY:yyyy/MM/dd,level:VALUE", | ||
| timestampAndLevelFilter))); | ||
| severityFilter))); | ||
| // [ENG-6555] addresses this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the issue and why is the test disabled?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#775
Hudi 1.1 and ICEBERG partitioned filter data validation fails
Important Read
What is the purpose of the pull request
Upgrades the hudi version to 1.1 which introduces many new exciting features for the lakehouse - Record Level Index, Secondary Index which can be leveraged by other table formats as well.
https://hudi.apache.org/blog/2025/11/25/apache-hudi-release-1-1-announcement/
Brief change log
(for example:)
Verify this pull request
This pull request is already covered by existing tests.