Skip to content

AWS, Aliyun: Fix memory leak by removing deleteOnExit() calls#13749

Merged
stevenzwu merged 1 commit into
apache:mainfrom
kamijin-fanta:fix-memory-leak-remove-deleteOnExit
Aug 6, 2025
Merged

AWS, Aliyun: Fix memory leak by removing deleteOnExit() calls#13749
stevenzwu merged 1 commit into
apache:mainfrom
kamijin-fanta:fix-memory-leak-remove-deleteOnExit

Conversation

@kamijin-fanta
Copy link
Copy Markdown
Contributor

Remove unnecessary File.deleteOnExit() calls from S3OutputStream and OSSOutputStream that were causing memory leaks in long-running servers. These calls registered files in the global DeleteOnExitHook set which never gets cleared during normal operation, leading to unbounded heap growth. The explicit cleanup logic already handles temporary file deletion properly.

Remove unnecessary File.deleteOnExit() calls from S3OutputStream and
OSSOutputStream that were causing memory leaks in long-running servers.
These calls registered files in the global DeleteOnExitHook set which
never gets cleared during normal operation, leading to unbounded heap
growth. The explicit cleanup logic already handles temporary file
deletion properly.
Copy link
Copy Markdown
Contributor

@singhpk234 singhpk234 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM too ! Thanks @kamijin-fanta

from coverage perspective is there tests for asserting this already :

The explicit cleanup logic already handles temporary file deletion properly.

@stevenzwu stevenzwu added this to the Iceberg 1.10.0 milestone Aug 6, 2025
@stevenzwu stevenzwu merged commit 05dbcae into apache:main Aug 6, 2025
42 checks passed
@stevenzwu
Copy link
Copy Markdown
Contributor

thanks @kamijin-fanta for the root cause analysis and the fix, and everybody for the review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory leak in S3OutputStream due to File.deleteOnExit() hook

5 participants