1.forbidden问题
先是安装完成后出现Exception in thread “main” io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://10.96.0.1/api/v1/namespaces/spark-operator/pods. Message: Forbidden!Configured service account doesn’t have access. Service account may have been revoked. pods “spark-pi-driver” is forbidden: error looking up service account spark-operator/spark: serviceaccount “spark” not found的错误
解决方法
这个错误是因为rbac问题
Spark驱动程序Pod需要在Pod的命名空间中具有Kubernetes服务帐户,该帐户具有创建,获取,列出和删除执行程序Pod的权限,并为该驱动程序创建Kubernetes的headless服务。在没有服务帐户的情况下驱动程序将失败并退出,除非Pod命名空间中的默认服务帐户具有所需的权限。要在命名空间中提交和运行SparkApplication,请确保在命名空间中存在具有权限的服务帐户,并将.spec.driver.serviceAccount设置为服务帐户的名称。请参考spark-rbac.yaml以获取示例RBAC设置,该示例在默认名称空间中创建名为spark的驱动程序服务帐户,并通过RBAC角色绑定为服务帐户提供所需的权限。
正如上文所示,所在的命名空间没有Kubernetes服务帐户,不能进行创建,获取,列出和删除执行程序Pod的权限。现在我们需要新建一个Kubernetes服务帐户
#
# Copyright 2018 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
apiVersion: v1
kind: ServiceAccount
metadata:
name: spark
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: spark-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["*"]
- apiGroups: [""]
resources: ["services"]
verbs: ["*"]
- apiGroups: [""]
resources: ["configMap"]
verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: spark-role-binding
namespace: default
subjects:
- kind: ServiceAccount
name: spark
namespace: default
roleRef:
kind: Role
name: spark-role
apiGroup: rbac.authorization.k8s.io
上文是在default命名空间上建立Kubernetes服务帐户,如果需要在不同的命名空间修改namespace即可
修改完成后执行kubectl create -f spark-rbac.yaml
即可