GH-3497: Fix thread-unsafe static singleton in DefaultValuesWriterFactory causing use-after-free#3498
Open
arouel wants to merge 1 commit intoapache:masterfrom
Open
GH-3497: Fix thread-unsafe static singleton in DefaultValuesWriterFactory causing use-after-free#3498arouel wants to merge 1 commit intoapache:masterfrom
DefaultValuesWriterFactory causing use-after-free#3498arouel wants to merge 1 commit intoapache:masterfrom
Conversation
…iterFactory` causing use-after-free `DefaultValuesWriterFactory` delegates to static singleton instances of `DefaultV1ValuesWriterFactory` and `DefaultV2ValuesWriterFactory`. These singletons store a mutable `parquetProperties` reference via `initialize()`, which gets overwritten by whichever `ParquetProperties` instance calls it last. When multiple Parquet writers run concurrently in the same JVM, the merger's column writers end up using the appender's `ByteBufferAllocator`. When the appender is closed and its allocator released, the merger crashes with `IllegalStateException: Already closed` on direct ByteBuffer access because its encoder slabs were allocated from a now-closed arena. Replace the static final `DEFAULT_V1_WRITER_FACTORY` and `DEFAULT_V2_WRITER_FACTORY` singletons in `DefaultValuesWriterFactory` with fresh instances created per `initialize()` call. This ensures each `ParquetProperties` instance (and its allocator) is isolated to its own factory, eliminating cross-contamination between concurrent writers sharing the same JVM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
DefaultValuesWriterFactorydelegates to static singleton instances ofDefaultV1ValuesWriterFactoryandDefaultV2ValuesWriterFactory. These singletons store a mutableparquetPropertiesreference viainitialize(), which gets overwritten by whicheverParquetPropertiesinstance calls it last. When multiple Parquet writers run concurrently in the same JVM, the merger's column writers end up using the appender'sByteBufferAllocator. When the appender is closed and its allocator released, the merger crashes withIllegalStateException: Already closedon direct ByteBuffer access because its encoder slabs were allocated from a now-closed arena.What changes are included in this PR?
Replace the static final
DEFAULT_V1_WRITER_FACTORYandDEFAULT_V2_WRITER_FACTORYsingletons inDefaultValuesWriterFactorywith fresh instances created perinitialize()call. This ensures eachParquetPropertiesinstance (and its allocator) is isolated to its own factory, eliminating cross-contamination between concurrent writers sharing the same JVM.Are these changes tested?
Yes. A new test
testFactoryIsolation_eachPropertiesUsesOwnAllocatorinDefaultValuesWriterFactoryTestverifies that two independently builtParquetPropertiesinstances with different allocators produceValuesWriterinstances that use their own respective allocators. The test fails against the original code.Are there any user-facing changes?
No API changes.
Closes #3497