Skip to content

Issues: neuralmagic/guidellm

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

28 Open 23 Closed
28 Open 23 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Discrepancy between max-requests parameter and actual number of unique requests generated
#122 opened Apr 17, 2025 by wangchen615
1
Add Docker and Kubernetes Support for GuideLLM Benchmarking
#119 opened Apr 17, 2025 by wangchen615
1
Sweep mode with --max-seconds fails ungracefully if calculated RPS is too low priority-low
#115 opened Apr 15, 2025 by sjmonson
@sjmonson
[Bug] Possibly incorrect output dump ? (The actual data appears to be fine)
#114 opened Apr 15, 2025 by thameem-abbas
[Nice to Have] Container builds
#111 opened Apr 8, 2025 by thameem-abbas
[Bug] Single failed request fails the process
#109 opened Apr 8, 2025 by markVaykhansky
1
[FeatureReuqest] Auto-sweep with model saturation detection
#108 opened Apr 8, 2025 by markVaykhansky
[Feature Request] TTFT & ITL Timeouts
#107 opened Apr 8, 2025 by markVaykhansky
[FeatureRequest] Constraint input & output sizes no matter the dataset
#106 opened Apr 8, 2025 by markVaykhansky
[Feature Request] Max error rate parameter
#105 opened Apr 8, 2025 by markVaykhansky
[Feature Request] Testing with defined prefix lengths
#104 opened Apr 4, 2025 by thameem-abbas
Enable Batch Inferencing Benchmarking Support priorty-high
#102 opened Mar 31, 2025 by rgreenberg1
Automatic performance stabilizization - so no need to set max-requests or max-seconds analysis analysis workstream - relating to adding in features to do analysis on extrapolating results automation automation workstream priorty-high
#101 opened Mar 31, 2025 by rgreenberg1
@markurtz
Land GuideLLM UI in Upstream Repository UI Front-end workstream
#98 opened Mar 12, 2025 by rgreenberg1
@DaltheCow
INFO log level is to verbose enhancement New feature or request
#97 opened Mar 11, 2025 by sjmonson GuideLLM v0.2.0 - CI/CD Finalization, Documentation Expansion, and Backend Support
@sjmonson
Have GuideLLM kick off a vLLM server automatically to avoid having the user install vLLM and assign the target themselves enhancement New feature or request
#95 opened Mar 9, 2025 by rgreenberg1
@markurtz @rgreenberg1
1
Max RPS in sweep mode could be higher
#93 opened Mar 7, 2025 by dagrayvid
3
Integrate lm-eval into GuideLLM accuracy-eval accuracy-evaluation workstream
#90 opened Mar 6, 2025 by rgreenberg1
Enable vLLM scheduler specific Metrics metrics Metrics workstream
#86 opened Mar 4, 2025 by rgreenberg1
@markurtz @rgreenberg1
[Scoping] Enable Hardware Accelerator Monitoring metrics to be captured in benchmarks metrics Metrics workstream
#85 opened Mar 3, 2025 by rgreenberg1
Community-oriented audio dataset profile creation dataset Dataset workstream
#84 opened Feb 28, 2025 by rgreenberg1
Scope out and propose Community-oriented text dataset profiles for model validation dataset Dataset workstream
#83 opened Feb 27, 2025 by rgreenberg1 GuideLLM v0.2.0 - CI/CD Finalization, Documentation Expansion, and Backend Support
@markurtz @rgreenberg1
1
Text-dataset performance profile creation - decide what the profiles are. dataset Dataset workstream
#82 opened Feb 27, 2025 by rgreenberg1 GuideLLM v0.2.0 - CI/CD Finalization, Documentation Expansion, and Backend Support
@markurtz
Base enablement to enable a dataset configuration file to be passed in rather than standard args, allow alias's for the folder structure dataset Dataset workstream
#81 opened Feb 27, 2025 by rgreenberg1 GuideLLM v0.2.0 - CI/CD Finalization, Documentation Expansion, and Backend Support
@sjmonson
1
How to pass additional sampling parameters to the vllm server ?
#69 opened Feb 6, 2025 by pinak-p
6
ProTip! Find all open issues with in progress development work with linked:pr.