progress: split PortConnectivity into bitset + sparse map#799
Draft
antiguru wants to merge 1 commit into
Draft
Conversation
Most operator edges carry the identity (default) summary. Represent those implicitly via a per-input `SmallVec<[u64; 2]>` bitset and reserve the `BTreeMap` for non-default antichains. Inline storage of the bitset covers up to 128 output ports without heap allocation. For the `event_driven` example with 100 dataflows of 1000 maps each, max RSS drops from 528 MB to 434 MB (-17.9%), and dataflow construction time drops ~15%. Profile shows zero hot-path overhead: PortConnectivity methods total 0.07% of samples; runtime loops iterate the unchanged columnar storage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
antiguru
force-pushed
the
portconnectivity-bitset
branch
from
9d4d380 to
134800a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Most operator edges carry the identity summary. Represent those implicitly via a per-input
SmallVec<[u64; 2]>bitset and reserve theBTreeMapfor non-default antichains. Inline storage covers up to 128 output ports without heap allocation.On
event_driven 100 × 1000, max RSS drops 528 → 434 MB (−17.9%) and dataflow construction time drops ~15%. Profile shows the hot path is unchanged — runtime loops iterate the same columnar tracker storage; PortConnectivity methods total 0.07% of samples.