目录标题VastBase 高可用集群完整操作报告目录架构概览组件层次结构命令映射关系当前集群状态增加备库详细流程一、K8s CRD 层面1.1 查看当前配置1.2 指定新备库调度节点可选1.3 增加 masterReplicas二、K8s StatefulSet 层面2.1 StatefulSet 创建2.2 Pod 创建过程三、HAS/Patroni 层面3.1 HAS 引导日志3.2 HAS 配置四、数据库控制层 (gs_ctl)4.1 备库构建命令自动执行4.2 流复制过程4.3 构建完成五、数据库 SQL 层5.1 验证复制状态主库5.2 验证复制槽主库5.3 验证备库状态备库六、验证集群状态释放备库详细流程一、K8s CRD 层面1.1 查看当前配置1.2 减少 masterReplicas二、K8s StatefulSet 层面2.1 StatefulSet 缩容2.2 Pod 删除过程三、HAS/Patroni 层面3.1 主备切换日志3.2 配置重载四、数据库控制层 (gs_ctl)4.1 原主库降级4.2 检查构建状态五、PVC 处理5.1 查看 PVC 状态5.2 删除 PVC可选六、最终验证命令验证结果patronictl (hasctl) 命令patronictl listpatronictl show-configpatronictl switchoverpatronictl failoverpatronictl reinitpatronictl historygs_ctl 命令gs_ctl statusgs_ctl buildgs_ctl querybuildgs_ctl startgs_ctl stopgs_ctl reloadgs_ctl switchovergs_ctl failovervsql (psql) 命令查看复制状态查看复制槽查看备库状态计算复制延迟日志路径说明容器内日志路径K8s 日志查看进入容器查看日志关键日志事件故障排查1. 备库无法启动2. 复制连接失败3. 复制延迟过高4. 主备切换失败5. 复制槽堆积附录A. 环境变量B. 配置文件位置C. 端口说明D. CRD 字段说明VastBase 高可用集群完整操作报告环境信息: Kubernetes 1.20 | VastBase 2.2.15 | HAS (定制 Patroni)集群名称: vastbase-222373ec命名空间: qfusion-admin报告日期: 2026-02-08目录架构概览增加备库详细流程释放备库详细流程命令验证结果日志路径说明故障排查架构概览组件层次结构┌────────────────────────────────────────────────────────────────────────┐ │ K8s PostgresCluster CRD │ │ postgres.woqutech.com/v1 │ │ ┌─────────────────────────────────────────────────────────────────────┐│ │ spec.pg.masterReplicas: 3 ← 控制副本数量 ││ │ └─────────────────────────────────────────────────────────────────────┘│ └─────────────────────────────────────────────────────────────────────────┘ │ ┌───────────────┴───────────────┐ ▼ ▼ ┌──────────────────────────────────┐ ┌──────────────────────────────────┐ │ HAS (High Availability Server) │ │ VastBase Database │ │ /home/vastbase/local/vastbase/bin/has │ │ /home/vastbase/local/vastbase/ │ │ │ │ bin/vastbase │ │ patroni → has (软链接) │ │ gs_ctl → vb_ctl (软链接) │ │ patronictl → hasctl (软链接) │ │ │ └──────────────────────────────────┘ └──────────────────────────────────┘命令映射关系标准命令VastBase 实现位置patronihas/home/vastbase/local/vastbase/bin/haspatronictlhasctl/home/vastbase/local/vastbase/bin/hasctlgs_ctlvb_ctl/home/vastbase/local/vastbase/bin/vb_ctlpsqlvsql/home/vastbase/local/vastbase/bin/vsql当前集群状态节点IP角色状态同步模式vastbase-222373ec-2-0245.0.3.45Leaderrunning-vastbase-222373ec-1-0245.0.2.252Sync StandbyrunningSyncvastbase-222373ec-0-0245.0.4.101ReplicarunningAsync增加备库详细流程一、K8s CRD 层面1.1 查看当前配置kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin get postgrescluster vastbase-222373ec -o yaml关键字段:spec:pg:masterReplicas:2# 当前副本数status:readyReplicas:2replicas:2customStatus:available1.2 指定新备库调度节点可选kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin annotate postgrescluster vastbase-222373ecvastbase-222373ec-0/nodeSelectorqfusion2输出:postgrescluster.postgres.woqutech.com/vastbase-222373ec annotated1.3 增加 masterReplicaskubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin patch postgrescluster vastbase-222373ec --typemerge-p{spec:{pg:{masterReplicas:3}}}输出:postgrescluster.postgres.woqutech.com/vastbase-222373ec patchedCRD 状态变化:spec:pg:masterReplicas:3# ← 2 → 3status:customStatus:availablefsmStatus:AWAWAUACreadyReplicas:3# ← 持续更新中replicas:3二、K8s StatefulSet 层面2.1 StatefulSet 创建自动创建的 StatefulSet:NAME READY AGE vastbase-222373ec-0 1/1 2m ← 新创建 vastbase-222373ec-1 1/1 31m vastbase-222373ec-2 1/1 26m2.2 Pod 创建过程kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin get pod -lAppNamevastbase-222373ec -w时间线:21:46:32 - Pod vastbase-222373ec-2-0 Pending (调度中) 21:46:33 - Pod vastbase-222373ec-2-0 ContainerCreating 21:46:34 - Init Container init 启动 21:46:34 - Main Container postgres 启动 21:46:41 - Pod vastbase-222373ec-2-0 Ready (3/3)三、HAS/Patroni 层面3.1 HAS 引导日志[2026-02-08 21:46:32] INFO: Initializing cluster bootstrap with command: /opt/crunchy/bin/tini -- /usr/local/bin/patroni /tmp/postgres-ha-bootstrap.yaml [2026-02-08 21:46:33] INFO: trying to bootstrap from leader vastbase-222373ec-0-03.2 HAS 配置# patronictl show-config 输出failsafe_mode:truepostgresql:basebackup:checkpoint:fastuse_pg_rewind:trueuse_slots:truesynchronous_mode:trueprimary_start_timeout:0四、数据库控制层 (gs_ctl)4.1 备库构建命令自动执行实际执行的命令:gs_ctl build -D /pgdata/data/vastbase-222373ec -b full -M standby详细日志:[2026-02-08 21:46:34.340][vb_ctl]: gs_ctl full build ,datadir is /pgdata/data/vastbase-222373ec [2026-02-08 21:46:34.341][vb_ctl]: fopen build pid file success [2026-02-08 21:46:34.342][vb_ctl]: set gaussdb state file when full build: db state(BUILDING_STATE), server mode(STANDBY_MODE), build mode(FULL_BUILD) [2026-02-08 21:46:34.383][gs_ctl]: build try host(245.0.2.252) port(5433) failed [2026-02-08 21:46:34.401][gs_ctl]: build try host(245.0.4.101) port(5433) success [2026-02-08 21:46:34.401][gs_ctl]: connected to server success, build started [2026-02-08 21:46:34.402][gs_ctl]: get system identifier success [2026-02-08 21:46:34.403][gs_ctl]: receiving and unpacking files... [2026-02-08 21:46:34.403][gs_ctl]: create backup label success4.2 流复制过程INFO: The starting position of the xlog copy of the full build is: 0/500A9B8 [2026-02-08 21:46:35.022][gs_ctl]: xlog start point: 0/500A9B8 [2026-02-08 21:46:35.022][gs_ctl]: begin get xlog by xlogstream [2026-02-08 21:46:35.022][gs_ctl]: starting background WAL receiver [2026-02-08 21:46:35.022][gs_ctl]: starting walreceiver [2026-02-08 21:46:35.064][gs_ctl]: check identify system success [2026-02-08 21:46:35.067][gs_ctl]: send START_REPLICATION 0/5000000 success4.3 构建完成[2026-02-08 21:46:36.452][gs_ctl]: finish receive tar files [2026-02-08 21:46:36.452][gs_ctl]: xlog end point: 0/6000058 [2026-02-08 21:46:40.118][gs_ctl]: starting fsync all files come from source [2026-02-08 21:46:41.218][gs_ctl]: finish fsync all files [2026-02-08 21:46:41.230][gs_ctl]: full build build completed(/pgdata/data/vastbase-222373ec)五、数据库 SQL 层5.1 验证复制状态主库SELECT*FROMpg_stat_replication;输出:pid | usesysid | usename | application_name | client_addr | state | sender_sent_location | sync_state ---------------------------------------------------------------------------------------------------------------------------------------------- 140542871533312 | 10 | vastbase | WalSender to Standby[vastbase-222373ec-1-0] | 245.0.2.252 | Streaming | 0/80016B8 | Sync 140542728992512 | 10 | vastbase | WalSender to Standby[vastbase-222373ec-0-0] | 245.0.4.101 | Streaming | 0/80016B8 | Async5.2 验证复制槽主库SELECT*FROMpg_replication_slots;输出:slot_name | plugin | slot_type | active | restart_lsn --------------------------------------------------------------- vastbase-222373ec-1-0 | | physical | t | 0/80016B8 vastbase-222373ec-0-0 | | physical | t | 0/80016B85.3 验证备库状态备库SELECTpg_is_in_recovery(),pg_last_xlog_receive_location(),pg_last_xlog_replay_location(),pg_last_xact_replay_timestamp();输出:pg_is_in_recovery | pg_last_xlog_receive_location | pg_last_xlog_replay_location | pg_last_xact_replay_timestamp --------------------------------------------------------------------------------------------------------------- t | 0/80016B8 | (1,0/80016B8) | 2026-02-08 21:55:20.82570408六、验证集群状态kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-adminexecvastbase-222373ec-2-0 -c postgres -- patronictl list vastbase-222373ec输出: Cluster: vastbase-222373ec ----------------------------------------------- | Member | Host | Role | State | TL | Lag in MB | -------------------------------------------------------------------------- | vastbase-222373ec-0-0 | 245.0.4.101 | Replica | running | 1 | 0 | | vastbase-222373ec-1-0 | 245.0.2.252 | Sync Standby | running | 1 | 0 | | vastbase-222373ec-2-0 | 245.0.3.45 | Leader | running | 1 | | --------------------------------------------------------------------------释放备库详细流程一、K8s CRD 层面1.1 查看当前配置kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin get postgrescluster vastbase-222373ec -o yaml|grep-E(masterReplicas|readyReplicas)输出:masterReplicas: 3 readyReplicas: 31.2 减少 masterReplicaskubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin patch postgrescluster vastbase-222373ec --typemerge-p{spec:{pg:{masterReplicas:2}}}输出:postgrescluster.postgres.woqutech.com/vastbase-222373ec patchedCRD 状态变化:spec:pg:masterReplicas:2# ← 3 → 2status:customStatus:warningfsmStatus:WAWAUACreadyReplicas:3# ← 等待 Pod 停止replicas:3statusDetails:|1.[vastbase-222373ec,cluster,wait database to ready. ]二、K8s StatefulSet 层面2.1 StatefulSet 缩容自动缩容:NAME READY AGE vastbase-222373ec-0 0/0 17m ← 缩容到 0 vastbase-222373ec-1 1/1 17m vastbase-222373ec-2 1/1 12m2.2 Pod 删除过程kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin get pod -lAppNamevastbase-222373ec -w时间线:21:55:19 - vastbase-222373ec-2-0 promoted to leader 21:55:27 - vastbase-222373ec-0-0 starts following new leader 21:56:00 - StatefulSet vastbase-222373ec-0 scaled to 0 21:56:01 - Pod vastbase-222373ec-0-0 Terminating三、HAS/Patroni 层面3.1 主备切换日志[2026-02-08 21:55:19] INFO: Cleaning up failover key after acquiring leader lock... [2026-02-08 21:55:19] INFO: promoted self to leader by acquiring session lock [2026-02-08 21:55:19] INFO: pre_promote script exited with 0 [2026-02-08 21:55:19.595][vb_ctl]: vb_ctl failover ,datadir is /pgdata/data/vastbase-222373ec [2026-02-08 21:55:19.618][vb_ctl]: Getting state from gaussdb.state! [2026-02-08 21:55:19.618][vb_ctl]: server starting failover3.2 配置重载[2026-02-08 21:55:20] INFO: Assigning synchronous standby status to [vastbase-222373ec-1-0] [2026-02-08 21:55:20.719][vb_ctl]: vb_ctl reload ,datadir is /pgdata/data/vastbase-222373ec [2026-02-08 21:56:40] INFO: Changed application_name from HAS to vastbase-222373ec-2-0 [2026-02-08 21:56:40.701][vb_ctl]: vb_ctl reload ,datadir is /pgdata/data/vastbase-222373ec四、数据库控制层 (gs_ctl)4.1 原主库降级原主库 vastbase-222373ec-0-0 日志:[2026-02-08 21:55:27] INFO: on_role_change callback called (actionon_role_change rolereplica) [2026-02-08 21:55:30] INFO: Lock owner: vastbase-222373ec-2-0; I am vastbase-222373ec-0-0 [2026-02-08 21:55:30] INFO: no action. I am (vastbase-222373ec-0-0), a secondary, and following a leader (vastbase-222373ec-2-0)4.2 检查构建状态gs_ctl querybuild -D /pgdata/data/vastbase-222373ec输出:[2026-02-08 21:52:57] gs_ctl build query ,datadir is /pgdata/data/vastbase-222373ec HA state: local_role : Standby static_connections : 2 db_state : Build completed sync_mode : Async五、PVC 处理5.1 查看 PVC 状态kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin get pvc|grepvastbase-222373ec-0输出:NAME STATUS VOLUME CAPACITY>5.2 删除 PVC可选kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin delete pvc>六、最终验证kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-adminexecvastbase-222373ec-2-0 -c postgres -- patronictl list vastbase-222373ec输出: Cluster: vastbase-222373ec ----------------------------------------------- | Member | Host | Role | State | TL | Lag in MB | -------------------------------------------------------------------------- | vastbase-222373ec-1-0 | 245.0.2.252 | Sync Standby | running | 1 | 0 | | vastbase-222373ec-2-0 | 245.0.3.45 | Leader | running | 1 | | --------------------------------------------------------------------------命令验证结果patronictl (hasctl) 命令patronictl listpatronictl list vastbase-222373ec验证结果: Cluster: vastbase-222373ec ----------------------------------------------- | Member | Host | Role | State | TL | Lag in MB | -------------------------------------------------------------------------- | vastbase-222373ec-0-0 | 245.0.4.101 | Replica | running | 1 | 0 | | vastbase-222373ec-1-0 | 245.0.2.252 | Sync Standby | running | 1 | 0 | | vastbase-222373ec-2-0 | 245.0.3.45 | Leader | running | 1 | | --------------------------------------------------------------------------patronictl show-configpatronictl show-config vastbase-222373ec验证结果:failsafe_mode:truepostgresql:basebackup:checkpoint:fastcustom_conf:/pgconf/postgres.base.confparameters:archive_mode:onmax_connections:200wal_level:hot_standbyhot_standby:truesynchronous_commit:trueuse_pg_rewind:trueuse_slots:truesynchronous_mode:trueprimary_start_timeout:0patronictl switchoverpatronictl switchover vastbase-222373ec\--leader vastbase-222373ec-2-0\--candidate vastbase-222373ec-1-0patronictl failoverpatronictl failover vastbase-222373ec\--leader vastbase-222373ec-2-0\--candidate vastbase-222373ec-1-0patronictl reinitpatronictl reinit vastbase-222373ec vastbase-222373ec-0-0patronictl historypatronictlhistoryvastbase-222373ec验证结果:---------------------------------------- | TL | LSN | Reason | Timestamp | New Leader | ---------------------------------------- (历史记录为空 - 因为是切换后重新初始化)gs_ctl 命令gs_ctl statusgs_ctl status -D /pgdata/data/vastbase-222373ec验证结果:[2026-02-08 22:12:26] gs_ctl status,datadir is /pgdata/data/vastbase-222373ec gs_ctl: server is running (PID: 314) /home/vastbase/local/vastbase/bin/vastbase -D /pgdata/data/vastbase-222373ec \ --config-file/pgdata/data/vastbase-222373ec/postgresql.conf \ --listen_addresses* --port5432 --wal_levelhot_standby \ --hot_standbyTrue --max_connections200 --max_wal_senders10 \ --max_replication_slots10 -M primarygs_ctl build# 全量构建备库手动执行非自动场景gs_ctl build -D /pgdata/data/vastbase-222373ec -b full -M standbygs_ctl querybuildgs_ctl querybuild -D /pgdata/data/vastbase-222373ec验证结果:[2026-02-08 21:52:57] gs_ctl build query ,datadir is /pgdata/data/vastbase-222373ec HA state: local_role : Standby static_connections : 2 db_state : Build completed sync_mode : Asyncgs_ctl startgs_ctl start -D /pgdata/data/vastbase-222373ec -M standbygs_ctl stopgs_ctl stop -D /pgdata/data/vastbase-222373ec -m fastgs_ctl reloadgs_ctl reload -D /pgdata/data/vastbase-222373ecgs_ctl switchovergs_ctl switchover -D /pgdata/data/vastbase-222373ec -m fastgs_ctl failovergs_ctl failover -D /pgdata/data/vastbase-222373ec -T1vsql (psql) 命令查看复制状态SELECT*FROMpg_stat_replication;验证结果:pid | usesysid | usename | application_name | client_addr | state | sender_sent_location | sync_state ---------------------------------------------------------------------------------------------------------------------------------------------- 140542871533312 | 10 | vastbase | WalSender to Standby[vastbase-222373ec-1-0] | 245.0.2.252 | Streaming | 0/80016B8 | Sync 140542728992512 | 10 | vastbase | WalSender to Standby[vastbase-222373ec-0-0] | 245.0.4.101 | Streaming | 0/80016B8 | Async查看复制槽SELECT*FROMpg_replication_slots;验证结果:slot_name | plugin | slot_type | active | restart_lsn --------------------------------------------------------------- vastbase-222373ec-1-0 | | physical | t | 0/80016B8 vastbase-222373ec-0-0 | | physical | t | 0/80016B8查看备库状态SELECTpg_is_in_recovery(),pg_last_xlog_receive_location(),pg_last_xlog_replay_location(),pg_last_xact_replay_timestamp();验证结果:pg_is_in_recovery | pg_last_xlog_receive_location | pg_last_xlog_replay_location | pg_last_xact_replay_timestamp --------------------------------------------------------------------------------------------------------------- t | 0/80016B8 | (1,0/80016B8) | 2026-02-08 21:55:20.82570408计算复制延迟SELECTclient_addr,application_name,sync_state,pg_size_pretty(pg_xlog_location_diff(sender_sent_location,receiver_replay_location))ASlag;日志路径说明容器内日志路径日志类型路径说明HAS/Patroni 日志标准输出kubectl logs pod -c postgresVastBase 数据库日志/pgdata/data/pglog/postgresql-*.csvCSV 格式按小时轮转构建日志同上包含 gs_ctl build 输出K8s 日志查看# 查看容器日志kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin logs vastbase-222373ec-2-0 -c postgres --tail100# 查看特定时间范围日志kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin logs vastbase-222373ec-2-0 -c postgres --since-time2026-02-08T21:46:00# 实时跟踪日志kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-admin logs vastbase-222373ec-2-0 -c postgres -f进入容器查看日志# 进入容器kubectl --kubeconfig/etc/kubernetes/admin.conf -n qfusion-adminexec-it vastbase-222373ec-2-0 -c postgres --bash# 查看数据库日志tail-f /pgdata/data/pglog/postgresql-*.csv# 过滤关键事件grep-igs_ctl build\|bootstrap\|switchover/pgdata/data/pglog/postgresql-*.csv关键日志事件事件关键字说明备库构建开始gs_ctl full build标识备库开始初始化流复制开始START_REPLICATIONWAL 流复制开始构建完成full build build completed备库数据同步完成角色变更on_role_change主备角色切换领导者选举promoted self to leader新主库产生配置重载vb_ctl reload配置文件重新加载故障排查1. 备库无法启动症状: Pod 状态非 Running# 查看 Pod 状态kubectl get pod vastbase-222373ec-0-0 -o wide# 查看 Pod 事件kubectl describe pod vastbase-222373ec-0-0# 查看 Pod 日志kubectl logs vastbase-222373ec-0-0 -c postgres常见原因:PVC 绑定失败节点资源不足镜像拉取失败2. 复制连接失败症状:pg_stat_replication无记录# 查看数据库日志kubectl logs vastbase-222373ec-0-0 -c postgres|grep-iconnection\|replication# 检查网络连通性kubectlexecvastbase-222373ec-0-0 --ping245.0.4.101# 检查复制配置kubectlexecvastbase-222373ec-0-0 --cat/pgdata/data/vastbase-222373ec/postgresql.conf|grepreplication3. 复制延迟过高症状:Lag in MB数值较大# 查看复制延迟patronictl list vastbase-222373ec# 查询详细延迟vsql -cSELECT client_addr, pg_xlog_location_diff(sender_sent_location, receiver_replay_location) AS lag_bytes FROM pg_stat_replication;# 检查网络带宽# 检查磁盘 I/O4. 主备切换失败症状: 切换后无主库# 查看切换历史patronictlhistoryvastbase-222373ec# 查看集群配置patronictl show-config vastbase-222373ec# 手动触发切换patronictl switchover vastbase-222373ec --force5. 复制槽堆积症状: WAL 目录增长过快# 查看复制槽状态vsql -cSELECT slot_name, active, restart_lsn, pg_xlog_location_diff(pg_current_xlog_location(), restart_lsn) AS lag_bytes FROM pg_replication_slots;# 删除不活跃复制槽vsql -cSELECT pg_drop_replication_slot(slot_name);附录A. 环境变量# HAS/Patroni 相关PATRONI_SCOPEvastbase-222373ecPATRONI_POSTGRESQL_DATA_DIR/pgdata/data/vastbase-222373ecPATRONI_KUBERNETES_NAMESPACEqfusion-adminPATRONI_KUBERNETES_SCOPE_LABELqfusion-pgha-scope# VastBase 相关PGDATA/pgdata/data/vastbase-222373ecPGPORT5432PGHOST/tmpB. 配置文件位置配置路径Patroni 配置/tmp/postgres-ha-bootstrap.yaml数据库配置/pgdata/data/vastbase-222373ec/postgresql.conf复制配置/pgconf/vbreplconn/证书配置/pgconf/tls/C. 端口说明端口服务5432VastBase 数据库服务5433VastBase 复制服务8009Patroni REST APID. CRD 字段说明spec:pg:masterReplicas:3# 主备库总数1主N备slaveReplicas:0# 独立备库数当前未使用masterDatabaseResources:ActiveSetName:vastbase-222373ec-2# 当前活动主库名称报告生成时间: 2026-02-08 22:15:00环境: Kubernetes VastBase 2.2.15 HAS