User Tools

Site Tools


paloaltonetworks:troubleshooting:firewall_resources

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
paloaltonetworks:troubleshooting:firewall_resources [2020/05/28 11:16] – [Show Resources] bstaffordpaloaltonetworks:troubleshooting:firewall_resources [2022/11/23 12:49] (current) – external edit 127.0.0.1
Line 1: Line 1:
 ====== Troubleshooting Firewall Performance ====== ====== Troubleshooting Firewall Performance ======
 +===== CPU Spikes =====
 +Could be caused by [[https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u000000HAcfCAG|genindex.sh]].
  
 +Could be caused by [[https://knowledgebase.paloaltonetworks.com/kcSArticleDetail?id=kA10g000000CmQv|IPsec tunnels]].
 +
 +===== Performance Issues =====
 +Possible but in PAN-OS 10.0.6 but we have seen a case where disabling "Forward segments exceeding TCP content inspection queue" cause massive throughput hit.
 +===== Show Session Limits =====
 +<code>show session info</code>
 +<code>show session info | match active</code>
 ===== Show Resources ===== ===== Show Resources =====
-<code>show running resource-monitor second<code> +Show record of each second for the last 60 seconds. 
-<code>show running resource-monitor minute<code> +<code>show running resource-monitor second</code> 
-<code>show running resource-monitor hour<code> +Show record of each minutefor the last 60 minutes. 
-<code>show running resource-monitor day<code> +<code>show running resource-monitor minute</code> 
-<code>show running resource-monitor week<code> +Show record of each hourfor the last 24 hours. 
-<code>show running resource-monitor<code>+<code>show running resource-monitor hour</code> 
 +Show record of each dayfor the last 7 days. 
 +<code>show running resource-monitor day</code> 
 +Show record of each weekfor the last 13 weeks. 
 +<code>show running resource-monitor week</code> 
 +Show all records - seconds, minutes, hours, days, weeks. 
 +<code>show running resource-monitor</code> 
 This is the most important command in getting dataplane CPU usages over different time intervals. Usually, if the CPU stays high (>90), traffic would feel sluggish, latency would also rise. The best strategy is to determine a regular 24-hour usage ("baseline") and then compare it to the times when spikes are experienced. This is the most important command in getting dataplane CPU usages over different time intervals. Usually, if the CPU stays high (>90), traffic would feel sluggish, latency would also rise. The best strategy is to determine a regular 24-hour usage ("baseline") and then compare it to the times when spikes are experienced.
 Resource utilization gives the % usage of sessions and buffers. If the output of packet descriptors or buffers stays > 80%, it indicates the device is over loaded and may lead to packet loss and abnormal behavior of the device. Resource utilization gives the % usage of sessions and buffers. If the output of packet descriptors or buffers stays > 80%, it indicates the device is over loaded and may lead to packet loss and abnormal behavior of the device.
 The same output can be obtained from the dp-monitor log. The same output can be obtained from the dp-monitor log.
 <code>less mp-log dp-monitor.log</code> <code>less mp-log dp-monitor.log</code>
-  * **VM-100****VM-300**, **PA-220** - Have Core 0 - 3 listed but only have data for Core 1 - 2.+  * **VM-100** - Has Core 0 - 3 listed but only have data for Core 1. 
 +  * **VM-300**, **PA-220** - Have Core 0 - 3 listed but only have data for Core 1 - 2.
   * **PA-850** - Has Core 0 - 7 listed but only has data for Core 1 - 5.   * **PA-850** - Has Core 0 - 7 listed but only has data for Core 1 - 5.
   * **PA-5220** - Has Core 0 - 7 listed but only has data for Core 1 - 7.   * **PA-5220** - Has Core 0 - 7 listed but only has data for Core 1 - 7.
 ===== Buffer Usage ===== ===== Buffer Usage =====
 <code>debug dataplane pool statistics</code> <code>debug dataplane pool statistics</code>
 +<code>debug dataplane pool statistics | match Packet</code>
 This command shows the packet buffers, resource pools and memory cache usages by different processes. If the pools deplete, traffic performance will be affected corresponding to that particular resource pool. Regarding pools, the number of the left shows the remaining while the number on the right shows the total capacity. The total capacity can vary based on platforms, models and OS versions. Likewise, if a certain process uses too much memory, that can also cause issues related to that process. This command shows the packet buffers, resource pools and memory cache usages by different processes. If the pools deplete, traffic performance will be affected corresponding to that particular resource pool. Regarding pools, the number of the left shows the remaining while the number on the right shows the total capacity. The total capacity can vary based on platforms, models and OS versions. Likewise, if a certain process uses too much memory, that can also cause issues related to that process.
 +
 +For Packet Buffer Usage, run the following filter in the System Logs view
 +<code>( description contains 'Packet buffer congestion is' )</code>
 +
 +To show the top five session using more than 2% of packet buffers: (this only works on physical firewalls)
 +<code>show running resource-monitor ingress-backlocs </code>
 +shows top five session using more than 2% of packet bufer.
 +Look for ''unknown'' or ''undecided'' App-ID and kill off with 
 +<code>request session-discard id <session_id></code>
  
 ===== Show Counters ===== ===== Show Counters =====
Line 40: Line 67:
 This command is used to monitor the ssl decryption memory usage; the first sz malloc size is the value to track. This value should increment/decrement, it is a concern if this value only increments. The max value is around 16 mb. This command is used to monitor the ssl decryption memory usage; the first sz malloc size is the value to track. This value should increment/decrement, it is a concern if this value only increments. The max value is around 16 mb.
  
 +The following command shows all decrypted sessions currently in the session table
 +<code>show session all filter ssl-decrypt yes</code>
 +<code>show session all filter ssl-decrypt yes count yes</code>
 ===== Show GlobalProtect Data ==== ===== Show GlobalProtect Data ====
 <code>show resource limit ssl-vpn</code> <code>show resource limit ssl-vpn</code>
-This command shows the number of current SSL VPN sessions (GlobalProtect).+This command shows the number of current number of connected GlobalProtect sessions (both the SSL sessions and the IPsec VPN sessions). 
 + 
 +You can show the GlobalProtect cookie cache on the firewall. 
 +<code>show system setting ssl-decrypt gp-cookie-cache</code>
paloaltonetworks/troubleshooting/firewall_resources.1590664596.txt.gz · Last modified: (external edit)