test: regression guard for #101 (no prompt-template wrapping; closes #101)#106
Open
jliounis wants to merge 1 commit into
Open
test: regression guard for #101 (no prompt-template wrapping; closes #101)#106jliounis wants to merge 1 commit into
jliounis wants to merge 1 commit into
Conversation
Issue #101 reports that perplexity_reason returns irrelevant citations because the tool wraps user queries in a verbose prompt template that poisons Perplexity's search-term extraction. The reporter (SangeethsivanSivakumar) debugged the bug themselves and identified the root cause in the third-party DaInfernalCoder/researcher-mcp fork (published on npm as perplexity-mcp v0.2.3), NOT in this repository. Verified the official server: src/server.ts forwards user messages to api.perplexity.ai verbatim with zero wrapping in all four tool handlers (perplexity_ask, perplexity_research, perplexity_reason, perplexity_search). This commit adds a regression test that: 1. Calls performChatCompletion with a representative user query. 2. Asserts the outgoing request body's `messages` field equals the input verbatim. 3. Asserts none of the known poisoning keywords from the third-party fork's template ("error messages, logs, code snippets", "specific situation", "step-by-step reasoning based on the actual context") appear in the request body. If anyone ever re-introduces a wrapper template in this repo, this test will fail immediately. Closes #101 (root cause is in third-party fork, not this repo).
rbuchmayer-pplx
approved these changes
May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #101.
TL;DR
The bug reported in #101 is not in this repository. It is in the third-party fork
DaInfernalCoder/researcher-mcp, published on npm asperplexity-mcpv0.2.3. The reporter (@SangeethsivanSivakumar) confirmed this themselves while debugging:The official
@perplexity-ai/mcp-server(this repo) forwards user messages toapi.perplexity.aiverbatim. There is no template wrapping in any of the four tool handlers (perplexity_ask,perplexity_research,perplexity_reason,perplexity_search).What this PR does
Adds a regression guard so we can never accidentally introduce the third-party fork's bug:
performChatCompletionwith a representative user query.messagesfield equals the input verbatim."error messages, logs, code snippets","specific situation","step-by-step reasoning based on the actual context") appear in the outgoing JSON.If anyone ever re-introduces wrapper-template logic into this server, this test fails immediately.
Why no version bump
No code change, no behavior change — only a unit test. No 0.9.1 release is needed for this PR. The P0.1 LLM-provenance envelope PR (#105) is the change worth a minor bump; if that lands we can roll the version there.
Test plan
npm test: 79 passed / 79 (was 78). The new test "should forward user messages verbatim to the API (regression: Sonar Reasoning Pro (reason tool) returns irrelevant search results #101)" passes.Recommended issue handling
When this PR merges, close #101 with a comment pointing the reporter at
DaInfernalCoder/researcher-mcp(where the actualprompt = template + querycode lives). Credit to @SangeethsivanSivakumar for the detailed root-cause analysis — their work narrowed this down to the exact npm package and patch.