Some reasons may cause this error: UNKNOWN: Deadline Exceeded?

[HoangSang1510]

Hello, I am using Milvus 2.4.5, sometime when i run this example code, i got error:

client = MilvusClient(uri="http://localhost:19530",)
search_params = {
    "metric_type": "IP",
    "params": {}
}
res = client.query(
    collection_name="db0207",
    filter="",
    output_fields=["count(*)"]
)

[2024-09-24 16:56:01,424] ERROR in decorators: grpc RpcError: [query], <_MultiThreadedRendezvous: StatusCode.DEADLINE_EXCEEDED, Deadline Exceeded>, <Time:{'RPC start': '2024-09-24 16:55:59.364782', 'gRPC error': '2024-09-24 16:56:01.424031'}>
[2024-09-24 16:56:01,424] ERROR in milvus_client: Failed to query collection: db0207

grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
        status = StatusCode.DEADLINE_EXCEEDED
        details = "Deadline Exceeded"
        debug_error_string = "UNKNOWN:Deadline Exceeded {created_time:"2024-09-24T16:56:01.365234577+07:00", grpc_status:4}"
>

Are there possible reasons for this error? Many thanks

[xiaofan-luan]

count will go through all the data. How many data is there in your collection and what is the timeout?

Could you please refer this doc to export the whole Milvus logs for investigation?
For Milvus installed with docker-compose, you can use docker-compose logs > milvus.log to export the logs.

[HoangSang1510]

i have 5 entities in my collection, my milvus.log too large but I found this warning line at the time the error occurred:

milvus-standalone  | [2024/09/24 11:54:00.288 +00:00] [WARN] [proxy/impl.go:3112] ["Query failed to WaitToFinish"] [traceID=bf9098c3e8473c3478f68e8c8af0152e] [role=proxy] [db=default] [collection=db0207] [partitions="[]"] [error="proxy TaskCondition context Done: context deadline exceeded"] [errorVerbose="proxy TaskCondition context Done: context deadline exceeded\n(1) attached stack trace\n  -- stack trace:\n  | github.com/milvus-io/milvus/internal/proxy.(*TaskCondition).WaitToFinish\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/condition.go:45\n  | github.com/milvus-io/milvus/internal/proxy.(*Proxy).query\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/impl.go:3111\n  | github.com/milvus-io/milvus/internal/proxy.(*Proxy).Query\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/impl.go:3171\n  | github.com/milvus-io/milvus/internal/distributed/proxy.(*Server).Query\n  | \t/go/src/github.com/milvus-io/milvus/internal/distributed/proxy/service.go:922\n  | github.com/milvus-io/milvus-proto/go-api/v2/milvuspb._MilvusService_Query_Handler.func1\n  | \t/go/pkg/mod/github.com/milvus-io/milvus-proto/go-api/[email protected]/milvuspb/milvus.pb.go:12730\n  | github.com/milvus-io/milvus/internal/proxy/connection.KeepActiveInterceptor\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/connection/util.go:60\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/milvus-io/milvus/internal/proxy.TraceLogInterceptor\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/trace_log_interceptor.go:36\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/milvus-io/milvus/internal/proxy/accesslog.UnaryUpdateAccessInfoInterceptor\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/accesslog/util.go:47\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/milvus-io/milvus/internal/proxy.RateLimitInterceptor.func1\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/rate_limit_interceptor.go:55\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/milvus-io/milvus/pkg/util/logutil.UnaryTraceLoggerInterceptor\n  | \t/go/src/github.com/milvus-io/milvus/pkg/util/logutil/grpc_interceptor.go:23\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/milvus-io/milvus/internal/proxy.UnaryServerInterceptor.func1\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/privilege_interceptor.go:82\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/milvus-io/milvus/internal/proxy.UnaryServerHookInterceptor.func1\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/hook_interceptor.go:118\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/milvus-io/milvus/internal/proxy.DatabaseInterceptor.func1\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/database_interceptor.go:15\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/grpc-ecosystem/go-grpc-middleware/auth.UnaryServerInterceptor.func1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/auth/auth.go:47\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryServerInterceptor.func1\n  | \t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/[email protected]/interceptor.go:342\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/milvus-io/milvus/internal/proxy/accesslog.UnaryAccessLogInterceptor\n  | \t/go/src/github.com/milvus-io/milvus/internal/proxy/accesslog/util.go:38\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n  | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1\n  | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:34\n  | github.com/milvus-io/milvus-proto/go-api/v2/milvuspb._MilvusService_Query_Handler\n  | \t/go/pkg/mod/github.com/milvus-io/milvus-proto/go-api/[email protected]/milvuspb/milvus.pb.go:12732\n  | google.golang.org/grpc.(*Server).processUnaryRPC\n  | \t/go/pkg/mod/google.golang.org/[email protected]/server.go:1360\n  | google.golang.org/grpc.(*Server).handleStream\n  | \t/go/pkg/mod/google.golang.org/[email protected]/server.go:1737\n  | google.golang.org/grpc.(*Server).serveStreams.func1.1\n  | \t/go/pkg/mod/google.golang.org/[email protected]/server.go:982\nWraps: (2) proxy TaskCondition context Done\nWraps: (3) context deadline exceeded\nError types: (1) *withstack.withStack (2) *errutil.withPrefix (3) context.deadlineExceededError"]

[xiaofan-luan]

please offer full logs as I mentioned to we can investigate on that

[HoangSang1510]

Here is my log file when error occurred: https://file.io/9mp58bFomDff

[weiliu1031]

Here is my log file when error occurred: https://file.io/9mp58bFomDff

can you upload the log again? it's deleted