Differences

This shows you the differences between two versions of the page.

--- gpfs:gpfs_ces [2021/01/01 21:24]
127.0.0.1 external edit
+++ gpfs:gpfs_ces [2025/04/28 12:07] (current)
manu [enable SMB / NFS]
@@ Line 147: / Line 147: @@
 </cli>
+=== CES address failover and distribution policies ===
+Add IP address for cluster NFS and CIFS
+<cli prompt='#'>
+# mmces address policy [even-coverage | balanced-load | node-affinity | none]
+</cli>
+  * **even-coverage**
+  Distributes the addresses among the available nodes. The even-coverage policy is the default address distribution policy.
+  * **balanced-load**
+  Distributes the addresses to approach an optimized load distribution. The load (network and CPU) on all the nodes are monitored. Addresses are moved based on given policies for optimized load throughout the cluster.
+  * **node-affinity**
+  Attempts to keep an address on the node to which the user manually assigned it. If the mmces address add command is used with the --node option, the address is marked as being associated with that node. Similarly, if an address is moved with the mmces address move command, the address is marked as being associated with the destination node. Any automatic movement, such as reassigning a down node's addresses, does not change this association. Addresses that are enabled with no node specification do not have a node association.
+  Addresses that are associated with a node but assigned to a different node are moved back to the associated node if possible.
+Force the balance
+<cli prompt='#'>
+# mmces address move --rebalance
+</cli>
+Or
+<cli prompt='#'>
+# mmces address move --ces-ip {IP[,IP...]} --ces-node Node
+</cli>
+<cli prompt='#'>
+# mmces address list --full-list
+cesAddress    cesNode                  attributes                                   cesGroup     cesPrefix   preferredNode            unhostableNodes
+------------- ------------------------ -------------------------------------------- ------------ ----------- ------------------------ -----------------
+.128.1.171   gpfsa01.mydom.lu   object_database_node,object_singleton_node   nfsgroup01   none        gpfsa01.mydom.lu   none
+.128.1.172   gpfsb01.mydom.lu   none                                         nfsgroup01   none        gpfsb01.mydom.lu   none
+</cli>
+https://www.ibm.com/docs/en/storage-scale/5.1.8?topic=reference-mmces-command
 ==== configure authentification ====
+Here only local authentification, so users creation must be done on all cluster nodes. Support also LDAP, AD,...
 <cli prompt='#'>
 [root@gpfs01 ~]# mmuserauth service list
@@ Line 439: / Line 476: @@
 samba    /gpfs01/samba   yes          no         auto
 </cli>
+==== Authentification local ====
+Create a local user on all GPFS nodes and SMB user
+<cli prompt='#'>
+[root@prscale-a-01 ces]# /usr/lpp/mmfs/bin/smbpasswd -a gpfsveeam01
+New SMB password:
+Retype new SMB password:
+Added user gpfsveeam01.
+</cli>
+<cli prompt='#'>
+[root@prscale-b-01 ~]# groupadd -g 10000001 gpfsveeam01
+[root@prscale-b-01 ~]# useradd -c "user connect veeam" -M  -u 10000001 -g 10000001    -s /sbin/nologin gpfsveeam01
+</cli>
+So you can assign UID to shared SMB folder
+===== CTDB state =====
+<cli prompt='#'>
+[root@prscale-a-01 log]# ctdb --debug=3 -v status
+Number of nodes:2
+pnn:0 10.255.7.11      OK
+pnn:1 10.255.7.10      OK (THIS NODE)
+Generation:1948871161
+Size:2
+hash:0 lmaster:0
+hash:1 lmaster:1
+Recovery mode:NORMAL (0)
+Recovery master:0
+[root@prscale-a-01 log]# ctdb --debug=3 -v getdbmap
+Number of databases:16
+dbid:0x3ef19640 name:passdb.tdb path:/var/lib/ctdb/persistent/passdb.tdb.1 PERSISTENT
+dbid:0x2ca251cf name:account_policy.tdb path:/var/lib/ctdb/persistent/account_policy.tdb.1 PERSISTENT
+dbid:0xa1413774 name:group_mapping.tdb path:/var/lib/ctdb/persistent/group_mapping.tdb.1 PERSISTENT
+dbid:0xc3078fba name:share_info.tdb path:/var/lib/ctdb/persistent/share_info.tdb.1 PERSISTENT
+dbid:0x06916e77 name:leases.tdb path:/var/lib/ctdb/volatile/leases.tdb.1
+dbid:0x83b22c33 name:share_entries.tdb path:/var/lib/ctdb/volatile/share_entries.tdb.1
+dbid:0x7a19d84d name:locking.tdb path:/var/lib/ctdb/volatile/locking.tdb.1
+dbid:0x4e66c2b2 name:brlock.tdb path:/var/lib/ctdb/volatile/brlock.tdb.1
+dbid:0x68c12c2c name:smbXsrv_tcon_global.tdb path:/var/lib/ctdb/volatile/smbXsrv_tcon_global.tdb.1
+dbid:0x6b06a26d name:smbXsrv_session_global.tdb path:/var/lib/ctdb/volatile/smbXsrv_session_global.tdb.1
+dbid:0x477d2e20 name:smbXsrv_client_global.tdb path:/var/lib/ctdb/volatile/smbXsrv_client_global.tdb.1
+dbid:0x521b7544 name:smbXsrv_version_global.tdb path:/var/lib/ctdb/volatile/smbXsrv_version_global.tdb.1
+dbid:0x7132c184 name:secrets.tdb path:/var/lib/ctdb/persistent/secrets.tdb.1 PERSISTENT
+dbid:0x4d2a432b name:g_lock.tdb path:/var/lib/ctdb/volatile/g_lock.tdb.1
+dbid:0x6cf2837d name:registry.tdb path:/var/lib/ctdb/persistent/registry.tdb.1 PERSISTENT
+dbid:0x6645c6c4 name:ctdb.tdb path:/var/lib/ctdb/persistent/ctdb.tdb.1 PERSISTENT
+</cli>
+In case of failure check the IPs
+<cli prompt='#'>
+[root@prscale-b-01 ~]# ctdb --debug=3 -v status
+connect() failed, errno=111
+Failed to connect to CTDB daemon (/var/run/ctdb/ctdbd.socket)
+Failed to detect PNN of the current node.
+Is this node part of CTDB cluster?
+[root@prscale-b-01 ~]# cat "/usr/lpp/mmfs/lib/ctdb/nodes"
+.10.10.11
+.10.10.12
+</cli>
+===== Debug protocols =====
+<cli prompt='#'>
+[root@prscale-a-01 ~]# gpfs.snap --protocol authentication
+gpfs.snap: started at Wed Dec  1 15:18:06 CET 2021.
+Gathering common data...
+Gathering Linux specific data...
+Gathering extended network data...
+Gathering local callhome data...
+Gathering local perfmon data...
+Gathering local msgqueue data...
+Gathering local auth data...
+Gathering local sysmon data...
+Gathering local cnss data...
+Gathering local gui data...
+Gathering trace reports and internal dumps...
+Gathering Transparent Cloud Tiering data at level BASIC...
+The Transparent Cloud Tiering snap data collection completed for node prscale-a-01
+Gathering QoS data at level FULL...
+gpfs.snap:  No QoS configuration was found for this cluster.
+gpfs.snap:  QoS configuration collection complete.
+Gathering cluster wide gui data...
+Gathering cluster wide sysmon data...
+Gathering cluster wide cnss data...
+Gathering cluster wide callhome data...
+Gathering cluster wide perfmon data...
+Gathering cluster wide msgqueue data...
+gpfs.snap:  Spawning remote gpfs.snap calls. Master is prscale-a-01.
+...
+gpfs.snap completed at Wed Dec  1 15:21:33 CET 2021
+###############################################################################
+Send file /tmp/gpfs.snapOut/3243658/all.20211201151806.3243658.tar to IBM Service
+Examine previous messages to determine additional required data.
+###############################################################################
+</cli>
+==== NFS tuning ====
+https://www.ibm.com/docs/en/spectrum-scale/5.1.5?topic=dr-tuning-nfs-server-homesecondary-cluster-nfs-server

wiki

User Tools

Site Tools

Differences

Page Tools