Datastream은 PostgreSQL WAL(미리 쓰기 로그) 트랜잭션 로그를 사용하여 PostgreSQL 스트림을 읽습니다. 이 로그는 데이터베이스 서버의 WAL 파일에 저장됩니다. WAL 로그의 각 레코드는 데이터베이스의 테이블 중 하나에 있는 실제 데이터의 단일 변경사항을 나타냅니다.
PostgreSQL WAL 파일의 구성 매개변수 설정
PostgreSQL 데이터베이스에 다음 구성 설정을 적용하는 것이 좋습니다.
max_slot_wal_keep_size: 복제 슬롯에서 사용하는 스토리지의 양을 제한하도록 이 매개변수를 설정합니다(PostgreSQL 13 이상에서만 사용 가능). 이는 WAL 파일 크기가 전체 스토리지를 차지하고 데이터베이스가 비정상 종료될 수 있는 장기 실행 트랜잭션에서 특히 중요합니다.
statement_timeout: 장기 실행 트랜잭션으로 인한 지연 시간을 줄이려면 이 매개변수를 선택한 값으로 설정합니다. max_slot_wal_keep_size을 지원하지 않는 데이터베이스에 대한 대안으로 statement_timeout를 사용할 수도 있습니다.
wal_sender_timeout: 이 매개변수를 0(제한 시간 중지) 또는 10분 이상의 값으로 설정합니다.
스트림을 10개 넘게 만들려는 경우 또는 계획된 스트림 수 외에 다른 리소스에서 사용하는 논리 복제 슬롯 수가 10개를 초과하는 경우 다음 매개변수를 수정해야 합니다.
max_replication_slots: 데이터베이스에 설정된 복제 슬롯 수에 따라 이 매개변수의 값을 늘립니다(스트림당 복제 슬롯 1개 필요). 서버 시작 시 max_replication_slots만 설정할 수 있습니다.
max_wal_senders: max_replication_slots 매개변수 값보다 크도록 이 매개변수의 값을 늘립니다. 서버를 시작할 때만 max_wal_senders를 설정할 수 있습니다.
[[["이해하기 쉬움","easyToUnderstand","thumb-up"],["문제가 해결됨","solvedMyProblem","thumb-up"],["기타","otherUp","thumb-up"]],[["이해하기 어려움","hardToUnderstand","thumb-down"],["잘못된 정보 또는 샘플 코드","incorrectInformationOrSampleCode","thumb-down"],["필요한 정보/샘플이 없음","missingTheInformationSamplesINeed","thumb-down"],["번역 문제","translationIssue","thumb-down"],["기타","otherDown","thumb-down"]],["최종 업데이트: 2025-09-04(UTC)"],[[["\u003cp\u003eDatastream utilizes the PostgreSQL WAL transaction log, stored in WAL files, to capture changes made to the database tables.\u003c/p\u003e\n"],["\u003cp\u003eSetting the \u003ccode\u003emax_slot_wal_keep_size\u003c/code\u003e parameter is recommended to prevent the WAL file from consuming excessive storage, especially during long-running transactions, though it is not supported by certain databases like Cloud SQL and AlloyDB.\u003c/p\u003e\n"],["\u003cp\u003eThe \u003ccode\u003estatement_timeout\u003c/code\u003e parameter can be configured to mitigate latency from prolonged transactions, serving as an alternative control for databases not supporting \u003ccode\u003emax_slot_wal_keep_size\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eIf you need more than 10 streams, you must adjust the \u003ccode\u003emax_replication_slots\u003c/code\u003e and \u003ccode\u003emax_wal_senders\u003c/code\u003e parameters based on the number of streams or replication slots you are using in your database.\u003c/p\u003e\n"],["\u003cp\u003eSetting \u003ccode\u003ewal_sender_timeout\u003c/code\u003e to \u003ccode\u003e0\u003c/code\u003e or a value of 10 minutes or greater is advised for better performance.\u003c/p\u003e\n"]]],[],null,["# Work with PostgreSQL database WAL log files\n\nDatastream uses the PostgreSQL WAL (Write Ahead Log) transaction log to\nread PostgreSQL streams. The log is stored in WAL files on the database server.\nEach record in the WAL log represents a single change to the actual data in one\nof the tables in the database.\n\nSet configuration parameters for PostgreSQL WAL files\n-----------------------------------------------------\n\nIt is recommended that you apply the following configuration settings to your\nPostgreSQL database:\n\n- `max_slot_wal_keep_size`: set this parameter (available only for PostgreSQL\n 13 and above) to limit the amount of storage used by the replication slot.\n This is particularly important for long-running transactions, which, in\n extreme cases, can lead to the WAL file size taking up the entire storage\n and crashing the database.\n\n | **Note:** Some managed databases, such as Cloud SQL and AlloyDB for PostgreSQL, don't support `max_slot_wal_keep_size`.\n- `statement_timeout`: set this parameter to a selected value to reduce\n latency caused by long-running transactions. You can also use\n `statement_timeout` as an alternative precaution measure for databases that\n don't support `max_slot_wal_keep_size`.\n\n- `wal_sender_timeout`: set this parameter to `0` (to disable the\n timeout) or to a value greater than or equal to 10 minutes.\n\nIf you plan to create more than 10 streams, or the number of logical replication\nslots that is used by other resources in addition to the number of planned\nstreams exceeds 10, make sure to modify the following parameters:\n\n- `max_replication_slots`: increase the value of this parameter, depending on\n the number of replication slots set for your database (you need 1\n replication slot per stream). You can only set `max_replication_slots`\n at server start.\n\n- `max_wal_senders`: increase the value of this parameter, so that it's\n greater than the value of the `max_replication_slots` parameter.\n You can only set `max_wal_senders` when you start the server.\n\nOptimize WAL log files\n----------------------\n\nTo avoid high latency of your streams and rapid growth in the size of WAL log\nfiles when replicating data from a PostgreSQL source, consider applying the\nfollowing precautions:\n\n- Avoid large long-running operations because they can significantly increase the size of your WAL file.\n- Use `UNLOGGED` or `TEMPORARY` tables during batch operations.\n- Check your WAL configuration and consider reducing the checkpoint frequency. For more information, see [WAL configuration](https://www.postgresql.org/docs/current/wal-configuration.html)\n- Check for large `DELETE` operations and consider replacing them with `TRUNCATE` operations. Doing this can significantly reduce WAL file data, however you need to be cautious, because Datastream doesn't replicate `TRUNCATE` operations.\n\nWhat's next\n-----------\n\n- Learn more about [PostgreSQL as a\n source](/datastream/docs/sources-postgresql).\n- Learn more about [configuring a source PostgreSQL\n database](/datastream/docs/configure-your-source-postgresql-database)."]]
