Skip to content

feat: configure cudagraph capture batch sizes#4573

Draft
CUHKSZzxy wants to merge 1 commit intoInternLM:mainfrom
CUHKSZzxy:feat/cudagraph-capture-batch-sizes
Draft

feat: configure cudagraph capture batch sizes#4573
CUHKSZzxy wants to merge 1 commit intoInternLM:mainfrom
CUHKSZzxy:feat/cudagraph-capture-batch-sizes

Conversation

@CUHKSZzxy
Copy link
Copy Markdown
Collaborator

@CUHKSZzxy CUHKSZzxy commented May 8, 2026

Summary

  • Add PyTorchEngineConfig support for explicit CUDA graph capture batch sizes.
  • Route configured sizes through CacheConfig and graph runners.
  • Fall back to eager execution when a runtime batch exceeds configured capture sizes.

@lvhan028 lvhan028 requested review from grimoire and removed request for grimoire May 8, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant