Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

failed to destroy rule externalIP to clusterIP {"error": "exit status 2"} #19

Open
heartcored98 opened this issue Mar 1, 2023 · 1 comment
Open

failed to destroy rule externalIP to clusterIP {"error": "exit status 2"} #19

heartcored98 opened this issue Mar 1, 2023 · 1 comment

Comments

@heartcored98
Copy link

안녕하세요. RKE2 기반의 쿠버네티스 배포판을 온프레미스 환경에서 운영 중입니다.
yatai라고 하고 BentoML 모델을 쿠버네티스에 쉽게 배포해주는 오픈소스를 쓰는 와중에 100MB 정도되는 모델을 yatai에 올릴려고 하니까 40~50MB 정도 업로드 되었을 때 항상 connection reset by peer에러가 발생했고, 검색하다 보니 감사하게도 공유해주신 이 오픈소스를 찾게 되었습니다.

그런데

Enable Drop Invalid Packet Rule in INPUT chain 옵션은 true,
Enable External-IP to Cluster-IP DNAT Rule 옵션은 false

으로 설치하고 설정했을 때 모든 Pod들에서 Crashloopbackoff 에러가 발생하고 있습니다.
env 설정 값은 정상적으로 들어간 것 같습니다.
혹시 어디서부터 손대면 좋을지 조언을 여쭤볼 수 있을까요?

Wed, Mar 1 2023 1:27:50 pm | I0301 04:27:50.915845 1 request.go:655] Throttling request took 1.019118868s, request: GET:https://10.43.0.1:443/apis/coordination.k8s.io/v1?timeout=32s
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.096Z INFO setup starting manager
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.096Z INFO controller-runtime.manager.controller.service Starting EventSource {"reconciler group": "", "reconciler kind": "Service", "source": "kind source: /, Kind="}
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.298Z INFO controller-runtime.manager.controller.service Starting Controller {"reconciler group": "", "reconciler kind": "Service"}
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.298Z INFO controller-runtime.manager.controller.service Starting workers {"reconciler group": "", "reconciler kind": "Service", "worker count": 1}
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.298Z INFO controllers.Service.initalize initalize service contoller
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.298Z INFO controllers.Service.initalize config IPv4 pod CIDR {"IPv4 pod cIDR": "10.42.0.0/16"}
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.298Z INFO controllers.Service.initalize config IPv6 pod CIDR {"IPv6 pod cIDR": ""}
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.298Z INFO controllers.Service.initalize config for drop invalid packet in INPUT chain {"enabled": true}
Wed, Mar 1 2023 1:27:51 pm | 2023-03-01T04:27:51.298Z INFO controllers.Service.initalize config for externalIP to clusterIP {"enabled": false}
Wed, Mar 1 2023 1:27:52 pm | 2023-03-01T04:27:52.698Z ERROR controllers.Service.initalize {"error": "exit status 2"}
Wed, Mar 1 2023 1:27:52 pm | github.com/go-logr/zapr.(*zapLogger).Error
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
Wed, Mar 1 2023 1:27:52 pm | github.com/kakao/network-node-manager/pkg/rules.DestoryRulesExternalCluster
Wed, Mar 1 2023 1:27:52 pm | /workspace/pkg/rules/rule_external_cluster.go:87
Wed, Mar 1 2023 1:27:52 pm | github.com/kakao/network-node-manager/controllers.(*ServiceReconciler).Reconcile
Wed, Mar 1 2023 1:27:52 pm | /workspace/controllers/service_controller.go:133
Wed, Mar 1 2023 1:27:52 pm | sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298
Wed, Mar 1 2023 1:27:52 pm | sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
Wed, Mar 1 2023 1:27:52 pm | sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.BackoffUntil
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.JitterUntil
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.UntilWithContext
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99
Wed, Mar 1 2023 1:27:52 pm | 2023-03-01T04:27:52.699Z ERROR controllers.Service.initalize failed to destroy rule externalIP to clusterIP {"error": "exit status 2"}
Wed, Mar 1 2023 1:27:52 pm | github.com/go-logr/zapr.(*zapLogger).Error
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
Wed, Mar 1 2023 1:27:52 pm | github.com/kakao/network-node-manager/controllers.(*ServiceReconciler).Reconcile
Wed, Mar 1 2023 1:27:52 pm | /workspace/controllers/service_controller.go:134
Wed, Mar 1 2023 1:27:52 pm | sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298
Wed, Mar 1 2023 1:27:52 pm | sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
Wed, Mar 1 2023 1:27:52 pm | sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.BackoffUntil
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.JitterUntil
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
Wed, Mar 1 2023 1:27:52 pm | k8s.io/apimachinery/pkg/util/wait.UntilWithContext
Wed, Mar 1 2023 1:27:52 pm | /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99
@jc01rho
Copy link

jc01rho commented Jun 2, 2023 •
edited
Loading

동일합니다. kubeadm 온프레미스 v1.25.9 , flannel:v0.22.0 , 환경에서 아래 내용이 발생합니다. 한번 봐주실수 있을까요? @ssup2

2023-06-02T03:58:02.320Z INFO setup starting manager
2023-06-02T03:58:02.320Z INFO controller-runtime.manager.controller.service Starting EventSource {"reconciler group": "", "reconciler kind": "Service", "source": "kind source: /, Kind="}
2023-06-02T03:58:02.420Z INFO controller-runtime.manager.controller.service Starting Controller {"reconciler group": "", "reconciler kind": "Service"}
2023-06-02T03:58:02.420Z INFO controller-runtime.manager.controller.service Starting workers {"reconciler group": "", "reconciler kind": "Service", "worker count": 1}
2023-06-02T03:58:02.421Z INFO controllers.Service.initalize initalize service contoller
2023-06-02T03:58:02.421Z INFO controllers.Service.initalize config IPv4 pod CIDR {"IPv4 pod cIDR": "10.244.0.0/16"}
2023-06-02T03:58:02.421Z INFO controllers.Service.initalize config IPv6 pod CIDR {"IPv6 pod cIDR": ""}
2023-06-02T03:58:02.421Z INFO controllers.Service.initalize config for drop invalid packet in INPUT chain {"enabled": true}
2023-06-02T03:58:02.421Z INFO controllers.Service.initalize config for externalIP to clusterIP {"enabled": false}
2023-06-02T03:58:02.910Z ERROR controllers.Service.initalize  {"error": "exit status 2"}
github.com/go-logr/zapr.(*zapLogger).Error
 /go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
github.com/kakao/network-node-manager/pkg/rules.DestoryRulesExternalCluster
 /workspace/pkg/rules/rule_external_cluster.go:87
github.com/kakao/network-node-manager/controllers.(*ServiceReconciler).Reconcile
 /workspace/controllers/service_controller.go:133
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
 /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
 /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
 /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
 /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
 /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
 /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
 /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
 /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
 /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jc01rho @heartcored98