Skip to content

Improve DataFileWriter: cache FileChannel and deprecate itemsCount #19096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
akugal opened this issue May 7, 2025 · 0 comments · Fixed by #19113
Closed

Improve DataFileWriter: cache FileChannel and deprecate itemsCount #19096

akugal opened this issue May 7, 2025 · 0 comments · Fixed by #19113
Assignees
@akugal
Labels
Improvement Code changes driven by non business requirements. Java Pull requests that update Java code Performance Issues related to performance concerns. Platform Data Structures Platform Tickets pertaining to the platform
Milestone
v0.63

Comments

@akugal
Copy link
Contributor

akugal commented May 7, 2025 •
edited
Loading

Description:

  • Reuse FileChannel in DataFileWriter instead of opening a new one each time the buffer is moved
  • Deprecate itemsCount field in DataFileMetadata since it is not really used
  • Add missing unit tests for DataFileWriter about closing twice, writing data bigger than the buffer and others.
  • Use 64mb as default mapped buffer use for DataFileWriter
  • Make file reader/writer close() idempotent
@akugal akugal added this to the v0.63 milestone May 7, 2025
@akugal akugal self-assigned this May 7, 2025
@akugal akugal added Java Pull requests that update Java code Improvement Code changes driven by non business requirements. Performance Issues related to performance concerns. labels May 7, 2025
@akugal akugal added Platform Data Structures Platform Tickets pertaining to the platform labels May 7, 2025
This was referenced May 7, 2025
Merged
Closed
Closed
@akugal akugal moved this to 🛠 In Progress in Foundation Team May 8, 2025
@akugal akugal mentioned this issue May 8, 2025
Merged
@akugal akugal moved this from 🛠 In Progress to 👀 In Review in Foundation Team May 8, 2025
@artemananiev artemananiev mentioned this issue May 14, 2025
Closed
@artemananiev artemananiev moved this from 👀 In Review to ✅ Done in Foundation Team May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@akugal akugal

Labels
Improvement Code changes driven by non business requirements. Java Pull requests that update Java code Performance Issues related to performance concerns. Platform Data Structures Platform Tickets pertaining to the platform
Projects
Foundation Team
Status: ✅ Done
Milestone
v0.63
1 participant
@akugal