Skip to content

skip nil values in Memberlist WatchPrefix#7429

Merged
SungJin1212 merged 2 commits intocortexproject:masterfrom
SungJin1212:fix-ha-tracker-ReplicaDesc-nil
Apr 17, 2026
Merged

skip nil values in Memberlist WatchPrefix#7429
SungJin1212 merged 2 commits intocortexproject:masterfrom
SungJin1212:fix-ha-tracker-ReplicaDesc-nil

Conversation

@SungJin1212
Copy link
Copy Markdown
Member

This PR adds a skip in WatchPrefix when a key is deleted, preventing a panic in the HA tracker caused by a failed type assertion on a nil interface value.

Related panic log

panic: interface conversion: interface {} is nil, not *ha.ReplicaDesc

goroutine 284 [running]:
github.com/cortexproject/cortex/pkg/ha.(*HATracker).loop.func3({0xc000c68ebb?, 0xc2703e459cea4e8f?}, {0x0?, 0x0?})
/__w/cortex/cortex/pkg/ha/ha_tracker.go:437 +0x629
github.com/cortexproject/cortex/pkg/ring/kv.(*prefixedKVClient).WatchPrefix.func1({0xc000c68eb0?, 0x41?}, {0x0?, 0x0?})
/__w/cortex/cortex/pkg/ring/kv/prefix.go:52 +0xb1
github.com/cortexproject/cortex/pkg/ring/kv/memberlist.(*KV).WatchPrefix(0xc000607808, {0x415fa48, 0xc000708e40}, {0xc000938060, 0xb}, {0x416ee40, 0xc00097f320}, 0xc000bf6090)
/__w/cortex/cortex/pkg/ring/kv/memberlist/memberlist_client.go:841 +0x4c7
github.com/cortexproject/cortex/pkg/ring/kv/memberlist.(*Client).WatchPrefix(0xc00097f338, {0x415fa48, 0xc000708e40}, {0xc000938060, 0xb}, 0xc000bf6090)
/__w/cortex/cortex/pkg/ring/kv/memberlist/memberlist_client.go:113 +0x6a

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
  • docs/configuration/v1-guarantees.md updated if this PR introduces experimental flags

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Apr 16, 2026
@SungJin1212 SungJin1212 merged commit 3c89123 into cortexproject:master Apr 17, 2026
62 of 65 checks passed
Comment thread pkg/ring/kv/memberlist/memberlist_client.go
friedrichg pushed a commit that referenced this pull request Apr 17, 2026
* skip nil values in Memberlist WatchPrefix

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* fix lint

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

---------

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>
friedrichg added a commit that referenced this pull request Apr 17, 2026
* Memberlist cas error code false positive (#7408)

* use errors.As in getCasErrorCode to unwrap memberlist errors

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* fix test

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

---------

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* Fix nil when ingesterQueryMaxAttempts > 1 (#7369)

* Trigger nil with test

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* Fix nil results

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* fix changelog

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

---------

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* fix: alertmanager user config disappearing when ring is unreachable  (#7372)

* Fix multitenant alertmanager user config disappearing when ring is unreachable

Signed-off-by: Kishore K G <kishorekg@google.com>

* Add change log

Signed-off-by: Kishore K G <kishorekg@google.com>

* format multitenant

Signed-off-by: Kishore K G <kishorekg@google.com>

* fix pr number

Signed-off-by: Kishore K G <kishorekg@google.com>

* use ErrNotFound for error validation in unit test

Signed-off-by: kishorekg1999 <kishorekg.github@gmail.com>

---------

Signed-off-by: Kishore K G <kishorekg@google.com>
Signed-off-by: kishorekg1999 <kishorekg@google.com>
Signed-off-by: kishorekg1999 <kishorekg.github@gmail.com>
Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* Clean Symbol Tables (#7373)

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* Fix root cause of nil return in queryWithRetry and labelsWithRetry (#7375)

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* fix regex resolver match 0 or 1 tenant bug (#7424)

* fix regex resolver match 0 or 1 tenant bug

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* fix test

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

---------

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* skip nil values in Memberlist WatchPrefix (#7429)

* skip nil values in Memberlist WatchPrefix

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* fix lint

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

---------

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* Remove duplicate CHANGELOG entry for #7373

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* Fix integration test flag name for release-1.21

The cherry-pick of #7424 brought the master flag name
-limits.query-ingesters-within, but release-1.21 still uses
-querier.query-ingesters-within (renamed in #7160, master-only).

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

---------

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>
Signed-off-by: Kishore K G <kishorekg@google.com>
Signed-off-by: kishorekg1999 <kishorekg@google.com>
Signed-off-by: kishorekg1999 <kishorekg.github@gmail.com>
Co-authored-by: SungJin1212 <tjdwls1201@gmail.com>
Co-authored-by: kishorekg1999 <kishorekg.github@gmail.com>
CharlieTLe added a commit to CharlieTLe/cortex that referenced this pull request Apr 17, 2026
CI run 24576043912: all jobs passed after merging upstream/master
(cortexproject#7424 regex resolver fix, cortexproject#7429 memberlist WatchPrefix fix).

Signed-off-by: Charlie Le <charlie.le@gmail.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Charlie Le <charlie_le@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants