Failures when installing Kf
Istio's IP is not ready
This can happen if Istio is taking too long to get an external IP. Wait a few minutes, and re-run the Cloud Build submit step. However, it's possible that there is something else going on.
Error from server (Bad Request): Invalid character
If you see this error when setting config defaults, it is likely the value of the ${DOMAIN}
env var used in the command includes an escaped '$'. Use single quotes to define a domain with a space name or other substitution:
Error from server (BadRequest): invalid character '$' in string escape code
Errors with Kf Kubernetes objects
Follow these instructions to troubleshoot Kf Kubernetes objects.
Find the name of the Kubernetes resource type you want to troubleshoot by listing all Kf resources:
kubectl api-resources --api-group=kf.dev
Example output:
NAME SHORTNAMES APIGROUP NAMESPACED KIND apps kf.dev true App builds kf.dev true Build clusterservicebrokers kf.dev false ClusterServiceBroker routes kf.dev true Route servicebrokers kf.dev true ServiceBroker serviceinstancebindings kf.dev true ServiceInstanceBinding serviceinstances kf.dev true ServiceInstance spaces kf.dev false Space
Get the instance of the resource you want to troubleshoot using
kubectl
. If the object isn't in a namespace, omit the-n
flag:kubectl get api-resource-name.kf.dev object-name -n space-name -o yaml
For example:
kubectl get apps.kf.dev my-app -n my-space -o yaml
Select your problem from the tabs below:
Object isn't reconciling
- Check the value of
metadata.generation
. This number is incremented each time the object is updated. If the number is extremely high it's likely two Kubernetes controllers are fighting over the object. Check your cluster to see if any policies are being applied to the object which might be changing it from the desired state. - Check to see if the namespace the object belongs to is deleting. If it is, the object may not be reconciled.
- Check that the
status.observedGeneration
field exists. If it doesn't, the controller might not have executed against the object yet. Validate that the cluster and controllers are healthy usingkf doctor
. - Check that the
metadata.generation
field matches thestatus.observedGeneration
field. If it doesn't, validate that the cluster and controllers are healthy usingkf doctor
. - Check for failures in the
status.conditions
list against this list of common error reasons:
NotOwned
: There is another resource that exists in the cluster or namespace as the one this object is trying to create. Read the message to find the duplicate name and either rename the conflicting resource or the Kf object.TemplateError
: There is a misconfiguration in the Kf resource spec or Kf configuration causing the child resource to be incorrectly reconciled. Validate the settings of the Kf object and the Kf space.CacheOutdated
: The Kf controller isn't receiving updates from Kubernetes fast enough. Check the health of the Kubernetes cluster.ReconciliationError
: The Kf controller can't create the necessary child resource. Check to make sure your cluster is healthy, Kf is running, and that there are no policies being enforced that are preventing Kf from creating the object referenced in the message.
Object isn't deleting
- Check that the
metadata.deletionTimestamp
of the object was set. If it wasn't set, then the requested deletion didn't work. - Check that the
metadata.deletionTimestamp
of the object is in the past. If it's in the future, the object may not delete. - Check if a
metadata.finalizers
list exists on the object. If finalizers are present, the object must wait for them to be removed before it is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them. - Child objects may exist that are preventing the object from being deleted. Have an administrator check all objects in the namespace and cluster to see if one of them needs to be manually deleted first.
- Check the value of