Saucepan

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Grid Master Candidates ======

Marking a Grid Member as a GMC will not cause a reboot but it will cause services restart on that specific Grid Member alone because it needs to receive a complete binary database from the Grid Master and apply it rather than the partial XML database it currently has. The database switch-over involves an automated restart.

Promoting a Grid Master Candidate to Grid Master will restart service on every device in the Grid (but the devices will not reboot). There is an option to do this sequentially but you cannot controll the order. The GMC being promoted will restart first, then the old GM, then every other Grid Member in FQDN order.

===== Test Promote Grid Master Candidate =====
In the GM GUI, you can run GMC Promote Test where you can select all non GM/GMC nodes and test their communication to a GMC of your choice.

Connectivity to Port 1194 (Grid Communication) and Port 2114 (Key Exchange) between the non GM/GMC members and the GMC you selected will then be tested and the results made available.

Note, the test command doesn't test all communication channels.

Note: During the GMC promotion test, communication through ports (2114/1194) will be checked. The test cannot be performed when MGMT port is used for communication (this is known behavior). As a workaround we can use the "show network_connectivity" command to confirm whether the Member is reachable on ports 2114 and 1194 which is required for the Grid Master and the Grid Member to communicate with each other. The result of the command shows that the Host is up and both the ports 2114 and 1194 are open between the Grid Master Candidate and the Grid Member. For more details on "show network_connectivity" see [[infoblox_nios:troubleshooting#show_port_is_open|this page]].


The service name newheights was given for the port 2114 by the nmap in the output. In general, the port 2114 is a registered port and given the title newheights. But in our environment port 2114 is used for communication which is OpenVPN.

The issue with the GMC promotion test ( for members using MGMT port) was identified approximately around October 2021 and it is tentatively targeted to be fixed in the NIOS 8.6.4.
===== Promote Grid Master Candidate =====
<code>set promote_master</code>
You can do one of the following to promote a Master Candidate:
  * Immediately notify all Grid members about the promotion.
  * Set a sequential notification to provide wait time for Grid members to join the new Grid Master. Staggering the restarts of Grid members can minimize DNS outages. The sequential order for Grid members to join the new Grid Master begins with the old Grid Master and then the Grid members in FQDN order. The default delay time is 120 seconds. You can configure the delay time from a minimum of 30 seconds up to 600 seconds (10 minutes).

In my lab, the TE-815 NIOS DNS servers took 3.5 minutes (ish) to reboot and rejoin the new Grid Master. For this reason, I would normally put 4 or 5 minutes as the delay time.

During a promotion, the members will connect to the new GM, re-sync and perform a services restart (not a product restart and no reboot)

When you promote a HA GMC to GM
 
  *Upon running "set promote_master" from CLI on the Grid Master Candidate, the request goes to the active node of the Grid Master Candidate.
  *The active node of the Grid Master Candidate restarts in the process of transitioning from Grid Master Candidate to Grid Master.
  *When the passive node of the Grid Master Candidate misses multiple consecutive heartbeats from the active node, the passive node assumes the active node role of the Grid Master Candidate.
  *When the previously active node of the Grid Master Candidate finishes the restart process, the previously active node of the Grid Master Candidate comes back up as the new Grid Master.
  *The current active node of the Grid Master Candidate logs "restart: new master found" and restarts to come back up as the passive node of the new Grid Master.
  *All Members, including the previous Grid Master, should log the same message "restart: new master found" and restart.
  *If the previous Grid Master and other Grid Members are HA pairs, the HA pairs will restart almost simultaneously because the active nodes will see the new master, log the message "restart: new master found" and restart. Then, when the passive nodes transition to active nodes, these newly active nodes will also see the new Grid Master and immediately restart.


For HA members, both nodes (active and passive) restart almost at the same time. This is because as soon as the active node restarts and the passive becomes active, it also detects the new grid manager and restarts immediately.

Find the below KB article for the order of restart:
  * https://support.infoblox.com/s/article/4307
  * https://support.infoblox.com/s/article/98


===== GMC Error =====
If you have a VM Grid and the GM says that the GMC is not suitable for GMC role, it could be that the GMC hard drive is too small. Years ago I had this when I deployed a NIOS 7.3.11 VM that had a 55Gb hard drive. This image is really for Multi-GM only. GM and GMC should use the 250Gb image. As of 2025, the minimum image you can deploy is 150GB (resizable image). 150GB should not be used in production but is okay for simple labbing.