redis cluster 구성

# cluster 구성

cluster-enabled yes

cluster-config-file nodes.conf

cluster-node-timeout 3000

 

redis-cli --cluster create 100.51.6.77:3001 100.51.5.35:3001 100.51.0.162:3001

slave추가
동일하게 cluster enabled yes설정한 노드 추가 기동 후 slave로 추가
redis-cli -p 3001 --cluster add-node 100.51.5.55:3001 100.51.6.77:3001 --cluster-slave

[ec2-user@ip-100-51-5-55 ~]$ redis-cli -p 3001 --cluster add-node 100.51.5.55:3001 100.51.6.77:3001 --cluster-slave
>>> Adding node 100.51.5.55:3001 to cluster 100.51.6.77:3001
>>> Performing Cluster Check (using node 100.51.6.77:3001)
M: a7a520b14f11d37429a818785baf98156bf15bc8 100.51.6.77:3001
   slots:[0-5460] (5461 slots) master
M: 6a40345006a5ad115da770281e6e5aa508dfda44 100.51.5.35:3001
   slots:[5461-10922] (5462 slots) master
M: 512f523684b321acb466baf75b209410f0bb78a5 100.51.0.162:3001
   slots:[10923-16383] (5461 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Automatically selected master 100.51.6.77:3001
>>> Send CLUSTER MEET to node 100.51.5.55:3001 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 100.51.6.77:3001.
[OK] New node added correctly.

slave기동시 master서버 로그

13290:M 21 Apr 2022 09:57:01.456 * Starting BGSAVE for SYNC with target: disk
13290:M 21 Apr 2022 09:57:01.456 * Background saving started by pid 13662
13662:C 21 Apr 2022 09:57:01.460 * DB saved on disk
13662:C 21 Apr 2022 09:57:01.461 * RDB: 0 MB of memory used by copy-on-write
13290:M 21 Apr 2022 09:57:01.549 * Background saving terminated with success
13290:M 21 Apr 2022 09:57:01.549 * Synchronization with replica 100.51.5.55:3001 succeeded

cluster 노드 확인

127.0.0.1:3001> cluster nodes
a7a520b14f11d37429a818785baf98156bf15bc8 100.51.6.77:3001@13001 master - 0 1650503361599 1 connected 0-5460
694a8b9e36c493cf8980538a41df37a29d09de42 100.51.5.55:3001@13001 myself,slave a7a520b14f11d37429a818785baf98156bf15bc8 0 1650503362000 1 connected
512f523684b321acb466baf75b209410f0bb78a5 100.51.0.162:3001@13001 master - 0 1650503362102 3 connected 10923-16383
6a40345006a5ad115da770281e6e5aa508dfda44 100.51.5.35:3001@13001 master - 0 1650503361599 2 connected 5461-10922

[ec2-user@ip-100-51-5-55 ~]$ redis-cli --cluster info 127.0.0.1:3001
100.51.6.77:3001 (a7a520b1...) -> 23 keys | 5461 slots | 1 slaves.
100.51.0.162:3001 (512f5236...) -> 15 keys | 5461 slots | 0 slaves.
100.51.5.35:3001 (6a403450...) -> 16 keys | 5462 slots | 0 slaves.
[OK] 54 keys in 3 masters.
0.00 keys per slot on average.

cluster 환경에서 cli 접속 : redlic-cli -c 옵션으로 하면, 다른 노드에서 가져와서 보여줌

100.51.5.55:3001> hgetall adFrequency:161134
-> Redirected to slot [14739] located at 100.51.0.162:3001
1) "_class"
2) "uplus.redistest.domain.FrequencyAds"
3) "adsNumber"
4) "145498"
5) "adsViewTime"
6) "2022-04-19T09:03:52.438986"
7) "id"
8) "161134"

 

redis-cli --cluster 명령

[ec2-user@ip-100-51-5-55 cluster]$ redis-cli --cluster help
Cluster Manager Commands:
  create         host1:port1 ... hostN:portN
                 --cluster-replicas <arg>
  check          host:port
                 --cluster-search-multiple-owners
  info           host:port
  fix            host:port
                 --cluster-search-multiple-owners
                 --cluster-fix-with-unreachable-masters
  reshard        host:port
                 --cluster-from <arg>
                 --cluster-to <arg>
                 --cluster-slots <arg>
                 --cluster-yes
                 --cluster-timeout <arg>
                 --cluster-pipeline <arg>
                 --cluster-replace
  rebalance      host:port
                 --cluster-weight <node1=w1...nodeN=wN>
                 --cluster-use-empty-masters
                 --cluster-timeout <arg>
                 --cluster-simulate
                 --cluster-pipeline <arg>
                 --cluster-threshold <arg>
                 --cluster-replace
  add-node       new_host:new_port existing_host:existing_port
                 --cluster-slave
                 --cluster-master-id <arg>
  del-node       host:port node_id
  call           host:port command arg arg .. arg
                 --cluster-only-masters
                 --cluster-only-replicas
  set-timeout    host:port milliseconds
  import         host:port
                 --cluster-from <arg>
                 --cluster-from-user <arg>
                 --cluster-from-pass <arg>
                 --cluster-from-askpass
                 --cluster-copy
                 --cluster-replace
  backup         host:port backup_directory
  help

 

slave 노드 제거

[ec2-user@ip-100-51-5-55 cluster]$ redis-cli --cluster del-node 100.51.5.55:3001 694a8b9e36c493cf8980538a41df37a29d09de42
>>> Removing node 694a8b9e36c493cf8980538a41df37a29d09de42 from cluster 100.51.5.55:3001
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.

 

lettuce를 활용한 cluster 테스트
기본 설정은 cluster에서 master가 다운되면 다시 master로 된 노드로 접속을 못한다. 설정을 바꿔준다.

spring.redis.cluster.nodes=100.51.6.77:3001,100.51.5.35:3001,100.51.0.162:3001,100.51.5.55:3001,100.51.3.120:3001,100.51.1.172:3001
spring.redis.timeout=60s
spring.redis.lettuce.cluster.refresh.period=60s
spring.redis.lettuce.cluster.refresh.adaptive=true