Differences

This shows you the differences between two versions of the page.

--- infoblox_nios:troubleshooting [2025/06/27 10:38] – [Joining Grid] bstafford
+++ infoblox_nios:troubleshooting [2026/02/16 02:58] (current) – [TCPDUMP] bstafford
@@ Line 62: / Line 62: @@
 To list interfaces <code>show interface</code>
 To list all interfaces quickly <code>show interface_mtu</code>
+To capture traffic on a server (192.168.11.153) where the client (1921.68.99.74) is accessing TCP-443 on the server.
+<code>tcpdump -i eth1 -n '(src 192.168.99.74 and dst 192.168.11.153 and dst port 443) or (src 192.168.11.153 and dst 192.168.99.74 and src port 443)'</code>
+===== Automated Traffic Capture =====
+Traffic Capture can be automated on events. [[https://docs.infoblox.com/space/nios90/280760742/Enabling+Automated+Traffic+Capture|docs]]
 ===== Hardware=====
 (Don't try this without support)
@@ Line 83: / Line 88: @@
 <code>set maintenacemode
 Maintenance Mode > show network_connectivity proto udp x.x.x.x 1194</code>
+<code>set maintenacemode
+Maintenance Mode > show network_connectivity type 4 proto tcp x.x.x.x 22
+Starting Nmap 7.80 ( https://nmap.org ) at 2025-09-10 09:27 UTC
+Nmap scan report for x.x.x.x
+Host is up (0.00091s latency).
+PORT   STATE SERVICE
+/tcp open  ssh
+Nmap done: 1 IP address (1 host up) scanned in 0.03 seconds
+Maintenance Mode ></code>
+In addition, the following is a bit of a hacky way of showing connectivity. You can you misuse the dig command in expertmode to test the OpenVPN Ports -
+Grid Member:
+<code>dig -v <local interface ip>#1194 @<gridmaster vip> -p 1194 dummy.domain
+dig -v <local interface ip>#2114 @<gridmaster vip> -p 2114 dummy.domain</code>
+This will send a UDP packet from the specified interface IP address and local port to the specified port on the Gridmaster VIP.
+Grid Master:
+To verify the incoming packet, you will need to start a traffic capture or tcpdump on the CLI.
 ===== Traceroute =====
 <code>traceroute -U -s GMCip memberIP-p 2114 -n -w 0.75 -f 30-q 1
@@ Line 96: / Line 124: @@
 ===== Disk Issues =====
+[[misc#disk|Info on disk size]]
 <code>show disk_usage_sorted config</code>
 <code>Expert Mode > set maintenancemode
 Maintenance Mode > show cores
-Maintenance Mode > show file
+Maintenance Mode > show logfiles
 Maintenance Mode > show backup  [ grid | dtc ]
@@ Line 180: / Line 209: @@
 ===== DB Queue Dump Data =====
-Below are the steps to get DB Queue Dump Data on each appliance:
+Below are the steps to get DB Queue Dump Data on each appliance. Do not run this unless told to do so by support.
   - Access CLI
-  - Execute "set maintenancemode"
+  - Execute "set maintenancemode on"
+  - Execute "set txn_trace on"
+  - Wait for 10 minutes
   - Execute "set debug_tools db_queue_dump"
+  - Wait until the command is complete, it may take couple of minutes till you see the cursor again.
+  - Execute "set txn_trace off"
   - Wait until the command is complete, it may take couple of minutes (or longer) till you see the cursor again.
   - Execute "set maintenancemode off"
@@ Line 192: / Line 225: @@
 Please follow the instructions below to collect the requested data. Enabling the CLI command will only generate additional logs and is not expected to impact your environment. If you have any follow-up queries, feel free to reach out.
+===== EA Bug in NIOS <9.0.7 ======
+To verify if any object data is missing, you may do an XML database dump on the GM and GMC, download the bundles, and compare the files for mismatched object values. To perform this test, take the following steps:
+  - Login to the CLI of the GM and GMC (active node if in an HA pair)
+  - Run "set maintenancemode" and then run "set debug_tools db_xml_dump" on both GM and GMC
+  - Once complete on both the GM and GMC exit the CLI and download a Support Bundle for each
+  - Uncompress the Support Bundle file and locate the onedb.xml file inside the /storage/debug_db_xml directory
+  - Compare the entries in the two DB files and note the objects containing ".com.infoblox.one.extensible_attributes_value" (see OS-specific examples below) or ".com.infoblox.one.hier_rule"
 ===== Core Dump Files =====
 <code>Expert Mode > set maintenancemode
@@ Line 214: / Line 254: @@
 Maintenance Mode></code>
+===== Syslog Log Severity =====
+Standard syslog error levels explained (from [[https://signoz.io/guides/syslog-levels/|here]])
+^ Level ^ Severity ^ Keyword ^ Description ^
+| 0	| Emergency | emerg | System is unusable |
+| 1	| Alert | alert | Action must be taken immediately |
+| 2	| Critical | crit | Critical conditions |
+| 3	| Error | err| Error conditions |
+| 4	| Warning | warning| Warning conditions |
+| 5	| Notice | notice| Normal but significant condition |
+| 6	| Informational | info| Informational messages |
+| 7	| Debug	| debug | Debug-level messages |
+===== Syslog Types =====
+^ Filter Name ^ Server Name ^ Facility ^
+| CDISCOVERY |  |  |
+| Cisco ISE |  |  |
+| Cloud API |  |  |
+| Cloud DNS |  |  |
+| DHCP | dhcpd | daemon |
+| Discovery |  |  |
+| DNS | named | daemon |
+| DNS Traffic Control | idns_healthd | kern |
+| File Distribution | httpd/in.tftpd | daemon |
+| FTP |  |  |
+| HTTP | httpd | daemon |
+| MS Server | msconnectd | daemon |
+| NTP | ntpd/ntpdate | daemon |
+| Outbound API |  |  |
+| Subscriber Services |  |  |
+| TFTP | in.tftpd | daemon |
+| Threat Insight |  |  |
+| Threat Protection | threat-protect-log |  |
+facility/server
+  * kern/idns_healthd (message contains "monitor")
+  * user/gunicorn (message contains "net_autodiscovery")
+  * user/monitor (message contains "State:")
+  * daemon/pidof (message contains "can't read from")
+  * daemon/systemd (message contains "dpkg" and "Rotate")
+  * user/debug_umount (message contains "umount")
+  * daemon/dbus-daemon (message container "dbus")
+  * daemon/dpkg-db-backup (message has dbpkg-db-backup
+  * kern/kernel (message contains "mounted filesystem")
+  * user/controld (message contains "Distribution Complete"/"Distribution Started")
+  * authpriv/su (message has "rabbitmq")
+  * auth/su (message has "rabbitmq")
+  * daemon/ntpd (message contains "NTP service")
+  * daemon/ntpdate (message contains "ntpdate")
+  * daemon/openvpn-member  (message contains "Peer Connection")
+  * auth/sshd (message contains "Server listenting")
+  * authpriv/chpasswd (message contains "pam_unix")
+  * daemon/in.tftpd (message contains "connection refused")
 ===== Show Logs in CLI =====
@@ Line 236: / Line 329: @@
+    * Note: look for the message "all zones loaded" to see when BIND has fully restarted.
 ===== Show BIND and ISC Version =====
 Show DNS and DHCP software version. Restart the service and look in syslog
+    * Note: look for the message "all zones loaded" to see when BIND has fully restarted.
 NIOS 9.0.6
   * Facility  daemon