클라이언트 왕복 지연 시간이 길지만 GFE 지연 시간과 Spanner API 요청 지연 시간은 짧은 경우 애플리케이션 코드에 문제가 있을 수 있습니다. 또한 클라이언트와 리전 GFE 간의 네트워킹 문제를 나타낼 수도 있습니다. 애플리케이션에 성능 문제가 발생하여 일부 코드 경로가 느려지면 각 API 요청의 클라이언트 왕복 지연 시간이 늘어날 수 있습니다. 또한 이전 단계에서 감지되지 않은 클라이언트 컴퓨팅 인프라에 문제가 있을 수 있습니다.
이러한 측정기준별로 그룹화하여 문제가 특정 데이터베이스, 상태 또는 메서드로 제한되는지 확인합니다. 이중 리전 또는 멀티 리전 워크로드의 경우 문제가 특정 리전으로 제한되는지 확인합니다.
GFE 지연 시간은 길지만 Spanner API 요청 지연 시간이 짧은 경우 다음 중 하나가 원인일 수 있습니다.
다른 리전에서 데이터베이스에 액세스하는 경우. 이렇게 하면 GFE 지연 시간이 길어지고 Spanner API 요청 지연 시간은 짧아질 수 있습니다. 예를 들어 us-central1 리전에 인스턴스가 있는 us-east1 리전의 클라이언트에서 들어오는 트래픽은 GFE 지연 시간이 길지만 Spanner API 요청 지연 시간은 더 짧을 수 있습니다.
GFE 레이어에 문제가 있는 경우. Google Cloud 상태 대시보드에서 리전에 진행 중인 네트워킹 문제가 있는지 확인합니다. 문제가 없는 경우 지원 엔지니어가 GFE 문제 해결을 지원할 수 있도록 지원 케이스를 열고 이 정보를 포함하세요.
[[["이해하기 쉬움","easyToUnderstand","thumb-up"],["문제가 해결됨","solvedMyProblem","thumb-up"],["기타","otherUp","thumb-up"]],[["이해하기 어려움","hardToUnderstand","thumb-down"],["잘못된 정보 또는 샘플 코드","incorrectInformationOrSampleCode","thumb-down"],["필요한 정보/샘플이 없음","missingTheInformationSamplesINeed","thumb-down"],["번역 문제","translationIssue","thumb-down"],["기타","otherDown","thumb-down"]],["최종 업데이트: 2025-09-05(UTC)"],[],[],null,["# Identify where latency occurs\n\n| **Note:** [OpenCensus is deprecated](https://opentelemetry.io/blog/2023/sunsetting-opencensus/). We recommend using OpenTelemetry to capture and visualize Spanner observability metrics. For more information, see [Capture custom client-side metrics using OpenTelemetry](/spanner/docs/capture-custom-metrics-opentelemetry).\n\nThis page describes how to troubleshoot Spanner components to find the\nsource of the latency. To learn more about possible latency points in a\nSpanner request, see\n[Latency points in a Spanner request](/spanner/docs/latency-points).\n\n1. In your client application that affects your service, confirm there's a\n latency increase from client round-trip latency. Check the following dimensions\n from your client-side metrics.\n\n - Client Application Name\n - Client locality (for example, Compute Engine VM zones) and Host (that is, VM names)\n - Spanner API method\n - Spanner API status\n\n Group by these dimensions to see if the issue is limited to a specific\n client, status, or method. For dual-region or multi-regional workloads, see\n if the issue is limited to a specific client or Spanner region.\n2. Check your client application health, especially the computing\n infrastructure on the client side (for example, VM, CPU, or memory\n utilization, connections, file descriptors, and so on).\n\n3. Check latency in Spanner components:\n\n a. Check client round-trip latency [with OpenTelemetry](/spanner/docs/capture-custom-metrics-opentelemetry#capture-client-round-trip-latency)\n or [with OpenCensus](/spanner/docs/capture-visualize-latency-opencensus#capture_and_visualize_client_round-trip_latency).\n\n b. Check Google Front End (GFE) latency [with OpenTelemetry](/spanner/docs/capture-custom-metrics-opentelemetry#capture-gfe-latency)\n or [with OpenCensus](/spanner/docs/capture-visualize-latency-opencensus#capture_and_visualize_gfe_latency).\n\n c. Check Spanner API request latency [with OpenTelemetry](/spanner/docs/capture-custom-metrics-opentelemetry#capture-spanner-api-request-latency)\n or [with OpenCensus](/spanner/docs/capture-visualize-latency-opencensus#capture_and_visualize_api_request_latency).\n\n If you have high client round-trip latency, but low GFE latency, and a low\n Spanner API request latency, the application code might\n have an issue. It could also indicate a networking issue between the client\n and regional GFE. If your application has a performance issue that causes\n some code paths to be slow, then the client round-trip latency for each API\n request might increase. There might also be an issue in the client computing\n infrastructure that was not detected in the previous step.\n4. Check the following dimensions for\n [Spanner metrics](/spanner/docs/latency-metrics):\n\n - Spanner Database Name\n - Spanner API method\n - Spanner API status\n\n Group by these dimensions to see if the issue is limited to a specific\n database, status, or method. For dual-region or multi-regional workloads,\n check to see if the issue is limited to a specific region.\n\n If you have a high GFE latency, but a low Spanner API request\n latency, it might have one of the following causes:\n - Accessing a database from another region. This action can lead to high GFE\n latency and low Spanner API request latency. For example,\n traffic from a client in the `us-east1` region that has an instance in the\n `us-central1` region might have a high GFE latency but a lower\n Spanner API request latency.\n\n - There's an issue at the GFE layer. Check the [Google Cloud Status Dashboard](https://status.cloud.google.com/)\n to see if there are any ongoing networking issues in your region. If there\n aren't any issues, then open a support case and include this information so\n that support engineers can help with troubleshooting the GFE.\n\n5. [Check the CPU utilization of the instance](/spanner/docs/cpu-utilization).\n If the CPU utilization of the instance is above the recommended level, you\n should manually add more nodes, or set up auto scaling. For more information,\n see [Autoscaling overview](/spanner/docs/autoscaling-overview).\n\n6. Observe and troubleshoot potential hotspots or unbalanced access patterns\n using [Key Visualizer](/spanner/docs/key-visualizer)\n and try to roll back any application code changes that strongly correlate\n with the issue timeframe.\n\n | **Note:** We recommend you follow [Schema design best practices](/spanner/docs/schema-design) to ensure your access is balanced across Spanner computing resources.\n7. Check any traffic pattern changes.\n\n8. Check [Query insights](/spanner/docs/using-query-insights) and\n [Transaction insights](/spanner/docs/use-lock-and-transaction-insights) to\n see if there might be any query or transaction performance bottlenecks.\n\n9. Use procedures in [Oldest active queries](/spanner/docs/introspection/oldest-active-queries)\n to see any expense queries that might cause a performance bottleneck and\n cancel the queries as needed.\n\n10. Use procedures in the troubleshooting sections in the following topics to\n troubleshoot the issue further using Spanner introspection\n tools:\n\n - [Query statistics](/spanner/docs/introspection/query-statistics)\n - [Read statistics](/spanner/docs/introspection/read-statistics)\n - [Transaction statistics](/spanner/docs/introspection/transaction-statistics)\n - [Lock statistics](/spanner/docs/introspection/lock-statistics)\n\nWhat's next\n-----------\n\n- Now that you've identified the component that contains the latency, explore the problem further using OpenCensus. For more information, see [Capture custom client-side metrics using OpenTelemetry](/spanner/docs/capture-custom-metrics-opentelemetry) or [with OpenCensus](/spanner/docs/capture-visualize-latency-opencensus).\n- Learn how to use [metrics](/spanner/docs/latency-metrics) to diagnose latency.\n- Learn how to [troubleshoot Spanner deadline exceeded errors](/spanner/docs/deadline-exceeded)."]]
