Differences

This shows you the differences between two versions of the page.

--- infoblox_nios:troubleshooting [2025/03/31 14:57] – [CLI Modes] bstafford
+++ infoblox_nios:troubleshooting [2026/02/16 02:58] (current) – [TCPDUMP] bstafford
@@ Line 38: / Line 38: @@
 RAID_DISK4: ONLINE, IB-Type14
 RAID_BATTERY: RAID battery OK</code>
+<code>hostname > show hardware-type
+Member hardware type: IB-2225</code>
+<code>hostname  > show version
+Version : 8.5.4-419474
+SN   : 2205202223333123
+Hotfix : N/A</code>
 ===== TCPDUMP =====
@@ Line 53: / Line 62: @@
 To list interfaces <code>show interface</code>
 To list all interfaces quickly <code>show interface_mtu</code>
+To capture traffic on a server (192.168.11.153) where the client (1921.68.99.74) is accessing TCP-443 on the server.
+<code>tcpdump -i eth1 -n '(src 192.168.99.74 and dst 192.168.11.153 and dst port 443) or (src 192.168.11.153 and dst 192.168.99.74 and src port 443)'</code>
+===== Automated Traffic Capture =====
+Traffic Capture can be automated on events. [[https://docs.infoblox.com/space/nios90/280760742/Enabling+Automated+Traffic+Capture|docs]]
 ===== Hardware=====
 (Don't try this without support)
@@ Line 69: / Line 83: @@
 ===== Joining Grid =====
 Remember, when you try and join a grid, if the Grid name is wrong, the GM will log an error. If the Grid name is correct but the shared secret is wrong, there will be no log. The new member will just silently fail to join.
+Also, if you try and join a Grid and tell the joining member to use the MGMT port, if the Grid configuration hasn't got Grid > Grid Manager > Members > [Member] Network > Advanced > "Enable VPN on MGMT Port" ticked, then the member won't be able to join. If you want to use the MGMT port to connect, you have to tick "Enable VPN on MGMT Port" first in the GM configuration for that member.
 ===== Show Port is Open =====
 <code>set maintenacemode
 Maintenance Mode > show network_connectivity proto udp x.x.x.x 1194</code>
+<code>set maintenacemode
+Maintenance Mode > show network_connectivity type 4 proto tcp x.x.x.x 22
+Starting Nmap 7.80 ( https://nmap.org ) at 2025-09-10 09:27 UTC
+Nmap scan report for x.x.x.x
+Host is up (0.00091s latency).
+PORT   STATE SERVICE
+/tcp open  ssh
+Nmap done: 1 IP address (1 host up) scanned in 0.03 seconds
+Maintenance Mode ></code>
+In addition, the following is a bit of a hacky way of showing connectivity. You can you misuse the dig command in expertmode to test the OpenVPN Ports -
+Grid Member:
+<code>dig -v <local interface ip>#1194 @<gridmaster vip> -p 1194 dummy.domain
+dig -v <local interface ip>#2114 @<gridmaster vip> -p 2114 dummy.domain</code>
+This will send a UDP packet from the specified interface IP address and local port to the specified port on the Gridmaster VIP.
+Grid Master:
+To verify the incoming packet, you will need to start a traffic capture or tcpdump on the CLI.
 ===== Traceroute =====
 <code>traceroute -U -s GMCip memberIP-p 2114 -n -w 0.75 -f 30-q 1
@@ Line 85: / Line 124: @@
 ===== Disk Issues =====
+[[misc#disk|Info on disk size]]
 <code>show disk_usage_sorted config</code>
 <code>Expert Mode > set maintenancemode
 Maintenance Mode > show cores
-Maintenance Mode > show file
+Maintenance Mode > show logfiles
 Maintenance Mode > show backup  [ grid | dtc ]
@@ Line 169: / Line 209: @@
 ===== DB Queue Dump Data =====
-Below are the steps to get DB Queue Dump Data on each appliance:
+Below are the steps to get DB Queue Dump Data on each appliance. Do not run this unless told to do so by support.
   - Access CLI
-  - Execute "set maintenancemode"
+  - Execute "set maintenancemode on"
+  - Execute "set txn_trace on"
+  - Wait for 10 minutes
   - Execute "set debug_tools db_queue_dump"
+  - Wait until the command is complete, it may take couple of minutes till you see the cursor again.
+  - Execute "set txn_trace off"
   - Wait until the command is complete, it may take couple of minutes (or longer) till you see the cursor again.
   - Execute "set maintenancemode off"
@@ Line 181: / Line 225: @@
 Please follow the instructions below to collect the requested data. Enabling the CLI command will only generate additional logs and is not expected to impact your environment. If you have any follow-up queries, feel free to reach out.
+===== EA Bug in NIOS <9.0.7 ======
+To verify if any object data is missing, you may do an XML database dump on the GM and GMC, download the bundles, and compare the files for mismatched object values. To perform this test, take the following steps:
+  - Login to the CLI of the GM and GMC (active node if in an HA pair)
+  - Run "set maintenancemode" and then run "set debug_tools db_xml_dump" on both GM and GMC
+  - Once complete on both the GM and GMC exit the CLI and download a Support Bundle for each
+  - Uncompress the Support Bundle file and locate the onedb.xml file inside the /storage/debug_db_xml directory
+  - Compare the entries in the two DB files and note the objects containing ".com.infoblox.one.extensible_attributes_value" (see OS-specific examples below) or ".com.infoblox.one.hier_rule"
 ===== Core Dump Files =====
 <code>Expert Mode > set maintenancemode
@@ Line 203: / Line 254: @@
 Maintenance Mode></code>
+===== Syslog Log Severity =====
+Standard syslog error levels explained (from [[https://signoz.io/guides/syslog-levels/|here]])
+^ Level ^ Severity ^ Keyword ^ Description ^
+| 0	| Emergency | emerg | System is unusable |
+| 1	| Alert | alert | Action must be taken immediately |
+| 2	| Critical | crit | Critical conditions |
+| 3	| Error | err| Error conditions |
+| 4	| Warning | warning| Warning conditions |
+| 5	| Notice | notice| Normal but significant condition |
+| 6	| Informational | info| Informational messages |
+| 7	| Debug	| debug | Debug-level messages |
+===== Syslog Types =====
+^ Filter Name ^ Server Name ^ Facility ^
+| CDISCOVERY |  |  |
+| Cisco ISE |  |  |
+| Cloud API |  |  |
+| Cloud DNS |  |  |
+| DHCP | dhcpd | daemon |
+| Discovery |  |  |
+| DNS | named | daemon |
+| DNS Traffic Control | idns_healthd | kern |
+| File Distribution | httpd/in.tftpd | daemon |
+| FTP |  |  |
+| HTTP | httpd | daemon |
+| MS Server | msconnectd | daemon |
+| NTP | ntpd/ntpdate | daemon |
+| Outbound API |  |  |
+| Subscriber Services |  |  |
+| TFTP | in.tftpd | daemon |
+| Threat Insight |  |  |
+| Threat Protection | threat-protect-log |  |
+facility/server
+  * kern/idns_healthd (message contains "monitor")
+  * user/gunicorn (message contains "net_autodiscovery")
+  * user/monitor (message contains "State:")
+  * daemon/pidof (message contains "can't read from")
+  * daemon/systemd (message contains "dpkg" and "Rotate")
+  * user/debug_umount (message contains "umount")
+  * daemon/dbus-daemon (message container "dbus")
+  * daemon/dpkg-db-backup (message has dbpkg-db-backup
+  * kern/kernel (message contains "mounted filesystem")
+  * user/controld (message contains "Distribution Complete"/"Distribution Started")
+  * authpriv/su (message has "rabbitmq")
+  * auth/su (message has "rabbitmq")
+  * daemon/ntpd (message contains "NTP service")
+  * daemon/ntpdate (message contains "ntpdate")
+  * daemon/openvpn-member  (message contains "Peer Connection")
+  * auth/sshd (message contains "Server listenting")
+  * authpriv/chpasswd (message contains "pam_unix")
+  * daemon/in.tftpd (message contains "connection refused")
 ===== Show Logs in CLI =====
@@ Line 223: / Line 327: @@
 -01-30T10:04:57+00:00 daemon dnsmember.fqdn.example named[11807]: notice running on Linux x86_64 4.9.58 #1 SMP Mon Jan 31 20:10:08 PST 2022
 -01-30T10:06:17+00:00 daemon dnsmember.fqdn.example named[11807]: notice running</code>
+    * Note: look for the message "all zones loaded" to see when BIND has fully restarted.
+===== Show BIND and ISC Version =====
+Show DNS and DHCP software version. Restart the service and look in syslog
+    * Note: look for the message "all zones loaded" to see when BIND has fully restarted.
+NIOS 9.0.6
+  * Facility  daemon
+  * Level = NOTICE
+  * Server = named[xxx]
+  * Message = ''starting BIND 9.16.23-S1 (Supported Preview Version) <id:70b08b2>''
+NIOS 9.0.6
+  * Facility  daemon
+  * Level = INFO
+  * Server = dhcpd[xxx]
+  * Message = ''Internet Systems Consortium DHCP Server 4.3.3-P1''
 ===== Show Hotfix Action =====