Set up Project

In order to create GKE cluster and deploy Kubeflow on it, you need to set up a Google Cloud project and enable necessary APIs for the deployment.

Set up project and API scopes

Follow these steps to set up your Google Cloud project:

• Select or create a project on the Google Cloud Console. If you plan to use different Google Cloud projects for Management Cluster and Kubeflow Clusters: create one Management project for Management Cluster, and create one or more Kubeflow projects for Kubeflow Clusters.

• Make sure that you have the Owner role for the project in Cloud IAM (Identity and Access Management). The deployment process creates various service accounts with appropriate roles in order to enable seamless integration with Google Cloud services. This process requires that you have the owner role for the project in order to deploy Kubeflow.

• Make sure that billing is enabled for your project. Refer to Enable billing for a project.

• Open following pages on the Google Cloud Console and ensure that the specified APIs are enabled for all projects:

  • Compute Engine API
  • Kubernetes Engine API
  • Identity and Access Management (IAM) API
  • Service Management API
  • Cloud Resource Manager API
  • AI Platform Training & Prediction API
  • Cloud Identity-Aware Proxy API
  • Cloud Build API (It’s required if you plan to use Fairing in your Kubeflow cluster)
  • Cloud SQL Admin API
  • Config Controller (KRM API Hosting API)

  You can also enable these APIs by running the following command in Cloud Shell:

  gcloud services enable \
    compute.googleapis.com \
    container.googleapis.com \
    iam.googleapis.com \
    servicemanagement.googleapis.com \
    cloudresourcemanager.googleapis.com \
    ml.googleapis.com \
    iap.googleapis.com \
    sqladmin.googleapis.com \
    meshconfig.googleapis.com \
    krmapihosting.googleapis.com
  
  # Cloud Build API is optional, you need it if using Fairing.
  # gcloud services enable cloudbuild.googleapis.com
  

• If you are using the Google Cloud Free Program or the 12-month trial period with $300 credit, note that the free tier does not offer enough resources for default full Kubeflow installation. You need to upgrade to a paid account.

  For more information, see the following issues:

  • kubeflow/website #1065 reports the problem.
  • kubeflow/kubeflow #3936 requests a Kubeflow configuration to work with a free trial project.

  Read the Google Cloud Resource quotas to understand quotas on resource usage that Compute Engine enforces, and to learn how to check and increase your quotas.

• Initialize your project to prepare it for Anthos Service Mesh installation:

  PROJECT_ID=<YOUR_PROJECT_ID>
  

  curl --request POST \
    --header "Authorization: Bearer $(gcloud auth print-access-token)" \
    --data '' \
    https://meshconfig.googleapis.com/v1alpha1/projects/${PROJECT_ID}:initialize
  

  Refer to Anthos Service Mesh documentation for details.

  If you encounter a Workload Identity Pool does not exist error, refer to the following issue:

  • kubeflow/website #2121 describes that creating and then removing a temporary Kubernetes cluster may be needed for projects that haven’t had a cluster set up beforehand.

You do not need a running GKE cluster. The deployment process creates a cluster for you.

Next steps

• Set up an OAuth credential to use Cloud Identity-Aware Proxy (Cloud IAP). Cloud IAP is recommended for production deployments or deployments with access to sensitive data.
• Set up Management Cluster to deploy and manage Kubeflow clusters.
• Deploy Kubeflow using kubectl, kustomize and kpt.