从 Google Kubernetes Engine 集群连接到 Redis 实例

您可以从与实例位于同一区域并使用同一网络的 Google Kubernetes Engine 集群连接到 Redis 实例。

设置

如果您已安装 Cloud SDK 且已创建 Redis 实例,则可跳过这些步骤。

  1. 安装 Cloud SDK 并进行初始化:

    gcloud init
    
  2. 按照快速入门指南创建一个 Redis 实例。记下该 Redis 实例的地区、IP 地址和端口。

准备 GKE 集群

  1. 如果您尚未在您的 Redis 实例所在的区域中创建 GKE 集群,请在 gcloud 命令行工具中使用以下命令创建一个:

    1. gcloud 中为此示例应用指定项目。

      gcloud config set project [PROJECT_ID]

    2. gcloud 中设置 Compute Engine 地区配置变量。在 Redis 实例所在区域中选择任一地区。它不必与您的 Redis 实例位于同一地区,但可以如此。

      gcloud config set compute/zone [ZONE]

    3. 创建一个名为 visitcount-cluster 的 GKE 集群。

      gcloud container clusters create visitcount-cluster --num-nodes=3 --enable-ip-alias

  2. 如果未通过 gcloud 创建集群,请使用此命令检索集群凭据:

    gcloud container clusters get-credentials [CLUSTER_NAME] --zone [CLUSTER_ZONE] --project [PROJECT_ID]
    
    1. CLUSTER_NAME 是 GKE 集群的名称。
    2. CLUSTER_ZONE 是集群所在的地区。
    3. PROJECT_ID 是集群和 Redis 实例所在的项目。
  3. 如果您的集群是版本 1.8 或更高版本启用了 IP 别名,请跳过此步骤。如果您的集群是版本 1.7 或更低版本,或者集群为版本 1.8 或更高版本但未启用 IP 别名,请在尝试连接到实例之前,按照以下解决方法步骤操作:

    1. 运行以下命令,将 RESERVED_IP_RANGE 替换为实例的预留 IP 范围:

      git clone https://github.com/bowei/k8s-custom-iptables.git
      cd k8s-custom-iptables/
      TARGETS="RESERVED_IP_RANGE" ./install.sh
      cd ..
      
    2. 如果您不清楚实例的预留 IP 范围,可通过 Console(高级选项)或输入以下命令找到:

      gcloud redis instances describe INSTANCE_ID --region=REGION
      

    如需详细了解 IP 别名(包括如何创建启用了此设置的集群),请参阅 IP 别名文档

示例应用

此示例 HTTP 服务器应用通过 Google Kubernetes Engine 集群建立与 Redis 实例的连接。

克隆您所需编程语言的代码库,并转到包含示例代码的文件夹:

Go

git clone https://github.com/GoogleCloudPlatform/golang-samples
cd golang-samples/memorystore/redis

Java

git clone https://github.com/GoogleCloudPlatform/java-docs-samples
cd java-docs-samples/memorystore/redis

Node.js

git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples
cd nodejs-docs-samples/memorystore/redis

Python

git clone https://github.com/GoogleCloudPlatform/python-docs-samples
cd python-docs-samples/memorystore/redis

每次访问 / 端点时,此示例应用都会增加一个 Redis 计数器。

Go

此应用使用 github.com/gomodule/redigo/redis 客户端。通过运行以下命令进行安装:

go get github.com/gomodule/redigo/redis

// Command redis is a basic app that connects to a managed Redis instance.
package main

import (
	"fmt"
	"log"
	"net/http"
	"os"

	"github.com/gomodule/redigo/redis"
)

var redisPool *redis.Pool

func incrementHandler(w http.ResponseWriter, r *http.Request) {
	conn := redisPool.Get()
	defer conn.Close()

	counter, err := redis.Int(conn.Do("INCR", "visits"))
	if err != nil {
		http.Error(w, "Error incrementing visitor counter", http.StatusInternalServerError)
		return
	}
	fmt.Fprintf(w, "Visitor number: %d", counter)
}

func main() {
	redisHost := os.Getenv("REDISHOST")
	redisPort := os.Getenv("REDISPORT")
	redisAddr := fmt.Sprintf("%s:%s", redisHost, redisPort)

	const maxConnections = 10
	redisPool = redis.NewPool(func() (redis.Conn, error) {
		return redis.Dial("tcp", redisAddr)
	}, maxConnections)

	http.HandleFunc("/", incrementHandler)

	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
	}
	log.Printf("Listening on port %s", port)
	if err := http.ListenAndServe(":"+port, nil); err != nil {
		log.Fatal(err)
	}
}

Java

此应用基于 Jetty 3.1 servlet。

它使用 Jedis 库:

<dependency>
  <groupId>redis.clients</groupId>
  <artifactId>jedis</artifactId>
  <version>3.3.0</version>
</dependency>

AppServletContextListener 类用于创建长效 Redis 连接池:


package com.example.redis;

import java.io.IOException;
import java.util.Properties;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

@WebListener
public class AppServletContextListener implements ServletContextListener {

  private Properties config = new Properties();

  private JedisPool createJedisPool() throws IOException {
    String host;
    Integer port;
    config.load(
        Thread.currentThread()
            .getContextClassLoader()
            .getResourceAsStream("application.properties"));
    host = config.getProperty("redis.host");
    port = Integer.valueOf(config.getProperty("redis.port", "6379"));

    JedisPoolConfig poolConfig = new JedisPoolConfig();
    // Default : 8, consider how many concurrent connections into Redis you will need under load
    poolConfig.setMaxTotal(128);

    return new JedisPool(poolConfig, host, port);
  }

  @Override
  public void contextDestroyed(ServletContextEvent event) {
    JedisPool jedisPool = (JedisPool) event.getServletContext().getAttribute("jedisPool");
    if (jedisPool != null) {
      jedisPool.destroy();
      event.getServletContext().setAttribute("jedisPool", null);
    }
  }

  // Run this before web application is started
  @Override
  public void contextInitialized(ServletContextEvent event) {
    JedisPool jedisPool = (JedisPool) event.getServletContext().getAttribute("jedisPool");
    if (jedisPool == null) {
      try {
        jedisPool = createJedisPool();
        event.getServletContext().setAttribute("jedisPool", jedisPool);
      } catch (IOException e) {
        // handle exception
      }
    }
  }
}

VisitCounterServlet 类是一个网络 Servlet,用于增加 Redis 计数器。


package com.example.redis;

import java.io.IOException;
import java.net.SocketException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;

@WebServlet(name = "Track visits", value = "")
public class VisitCounterServlet extends HttpServlet {

  @Override
  public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
    try {
      JedisPool jedisPool = (JedisPool) req.getServletContext().getAttribute("jedisPool");

      if (jedisPool == null) {
        throw new SocketException("Error connecting to Jedis pool");
      }
      Long visits;

      try (Jedis jedis = jedisPool.getResource()) {
        visits = jedis.incr("visits");
      }

      resp.setStatus(HttpServletResponse.SC_OK);
      resp.getWriter().println("Visitor counter: " + String.valueOf(visits));
    } catch (Exception e) {
      resp.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, e.getMessage());
    }
  }
}

Node.js

此应用使用 redis 模块。

{
  "name": "memorystore-redis",
  "description": "An example of using Memorystore(Redis) with Node.js",
  "version": "0.0.1",
  "private": true,
  "license": "Apache Version 2.0",
  "author": "Google Inc.",
  "engines": {
    "node": ">=8.0.0"
  },
  "dependencies": {
    "redis": "^3.0.0"
  }
}

'use strict';
const http = require('http');
const redis = require('redis');

const REDISHOST = process.env.REDISHOST || 'localhost';
const REDISPORT = process.env.REDISPORT || 6379;

const client = redis.createClient(REDISPORT, REDISHOST);
client.on('error', (err) => console.error('ERR:REDIS:', err));

// create a server
http
  .createServer((req, res) => {
    // increment the visit counter
    client.incr('visits', (err, reply) => {
      if (err) {
        console.log(err);
        res.status(500).send(err.message);
        return;
      }
      res.writeHead(200, {'Content-Type': 'text/plain'});
      res.end(`Visitor number: ${reply}\n`);
    });
  })
  .listen(8080);

Python

此应用使用 Flask 进行网络服务,并使用 redis-py 软件包与 Redis 实例进行通信。

Flask==1.1.2
gunicorn==20.0.4
redis==3.5.3
import logging
import os

from flask import Flask
import redis

app = Flask(__name__)

redis_host = os.environ.get('REDISHOST', 'localhost')
redis_port = int(os.environ.get('REDISPORT', 6379))
redis_client = redis.StrictRedis(host=redis_host, port=redis_port)

@app.route('/')
def index():
    value = redis_client.incr('counter', 1)
    return 'Visitor number: {}'.format(value)

@app.errorhandler(500)
def server_error(e):
    logging.exception('An error occurred during a request.')
    return """
    An internal error occurred: <pre>{}</pre>
    See logs for full stacktrace.
    """.format(e), 500

if __name__ == '__main__':
    # This is used when running locally. Gunicorn is used to run the
    # application on Google App Engine and Cloud Run.
    # See entrypoint in app.yaml or Dockerfile.
    app.run(host='127.0.0.1', port=8080, debug=True)

构建容器映像

构建容器映像并将其推送到 Container Registry:

cp gke_deployment/Dockerfile .
export PROJECT_ID="$(gcloud config get-value project -q)"
docker build -t gcr.io/${PROJECT_ID}/visit-counter:v1 .
gcloud docker -- push gcr.io/${PROJECT_ID}/visit-counter:v1

将应用部署到 Google Kubernetes Engine

更新 gke_deployment/visit-counter.yaml,将 <PROJECT_ID> 替换为您的 Google Cloud 项目 ID。此文件包含部署和服务的配置。

Go

为避免对 Redis 实例 IP 进行硬编码,您可以创建 redishost ConfigMap

    export REDISHOST_IP=XXX.XXX.XXX.XXX
    kubectl create configmap redishost --from-literal=REDISHOST=${REDISHOST_IP}

使用以下命令验证配置:

    kubectl get configmaps redishost -o yaml
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: visit-counter
  labels:
    app: visit-counter
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: visit-counter
    spec:
      containers:
      - name: visit-counter
        image: "gcr.io/<PROJECT_ID>/visit-counter:v1"
        env:
        - name: REDISHOST
          valueFrom:
            configMapKeyRef:
              name: redishost
              key: REDISHOST
        ports:
        - name: http
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: visit-counter
spec:
  type: LoadBalancer
  selector:
    app: visit-counter
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP

Java

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: visit-counter
  labels:
    app: visit-counter
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: visit-counter
    spec:
      containers:
      - name: visit-counter
        image: "gcr.io/<PROJECT_ID>/visit-counter:v1"
        ports:
        - name: http
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: visit-counter
spec:
  type: LoadBalancer
  selector:
    app: visit-counter
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP

Node.js

为避免对 Redis 实例 IP 进行硬编码,您可以创建 redishost ConfigMap

    export REDISHOST_IP=XXX.XXX.XXX.XXX
    kubectl create configmap redishost --from-literal=REDISHOST=${REDISHOST_IP}

使用以下命令验证配置:

    kubectl get configmaps redishost -o yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: visit-counter
  labels:
    app: visit-counter
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: visit-counter
    spec:
      containers:
      - name: visit-counter
        image: "gcr.io/<PROJECT_ID>/visit-counter:v1"
        env:
        - name: REDISHOST
          valueFrom:
            configMapKeyRef:
              name: redishost
              key: REDISHOST
        ports:
        - name: http
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: visit-counter
spec:
  type: LoadBalancer
  selector:
    app: visit-counter
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP

Python

为避免对 Redis 实例 IP 进行硬编码,您可以创建 redishost ConfigMap

    export REDISHOST_IP=XXX.XXX.XXX.XXX
    kubectl create configmap redishost --from-literal=REDISHOST=${REDISHOST_IP}

使用以下命令验证配置:

    kubectl get configmaps redishost -o yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: visit-counter
  labels:
    app: visit-counter
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: visit-counter
    spec:
      containers:
      - name: visit-counter
        image: "gcr.io/<PROJECT-ID>/visit-counter:v1"
        env:
        - name: REDISHOST
          valueFrom:
            configMapKeyRef:
              name: redishost
              key: REDISHOST
        ports:
        - name: http
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: visit-counter
spec:
  type: LoadBalancer
  selector:
    app: visit-counter
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP

将配置应用到您的集群:

    kubectl apply -f gke_deployment/visit-counter.yaml

通过运行以下命令,确定此示例应用的 [EXTERNAL-IP] 地址:

    kubectl get service visit-counter

通过浏览器查看 http://[EXTERNAL-IP] 处托管的应用,或者通过 cURL 或浏览器发送 GET 请求:

    curl http://[EXTERNAL-IP]

删除 Redis 实例的 IP 表条目

如果您按照本演示中准备 GKE 集群部分的步骤 3 进行了操作,则您已将 Redis 实例的预留 IP 范围安装到了 GKE 实例的 IP 表中。如果要从 GKE 实例的 IP 表中移除此 Redis IP 范围条目,请从 k8s-custom-iptables/ 目录运行以下命令:

    ./uninstall.sh