You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
when I run guidellm with the default parameters:
'''
guidellm --target "http://localhost:30020/v1/" --model "meta-llama/Llama-3.1-8B-In
struct" --data-type emulated --data "prompt_tokens=112,generated_tokens=58"
'''
I got warnings that data is not available:
| percentiles | WARNING - No data points available to calculate percentiles.
mean | WARNING - No data points available to calculate mean.
and finally, and Error:
FATAL: exception not rethrown
Aborted (core dumped)
The report is missing the metrics "Time to first token", "Inter token latency", "output token throughput"
any suggestion? I know the warnings are coming from the src/guidelln/core/ folder directory, specifically from the module distribution.py. I do not know why error is happening.
Notice there is no problem with vLLM
The text was updated successfully, but these errors were encountered:
when I run guidellm with the default parameters:
'''
guidellm --target "http://localhost:30020/v1/" --model "meta-llama/Llama-3.1-8B-In
struct" --data-type emulated --data "prompt_tokens=112,generated_tokens=58"
'''
I got warnings that data is not available:
| percentiles | WARNING - No data points available to calculate percentiles.
mean | WARNING - No data points available to calculate mean.
and finally, and Error:
FATAL: exception not rethrown
Aborted (core dumped)
The report is missing the metrics "Time to first token", "Inter token latency", "output token throughput"
any suggestion? I know the warnings are coming from the src/guidelln/core/ folder directory, specifically from the module distribution.py. I do not know why error is happening.
Notice there is no problem with vLLM
The text was updated successfully, but these errors were encountered: