====== Packet Captures ====== ===== Packet Capture Types ===== * RX - Pre-decryption, pre-NAT * FW - Post-decryption, pre-NAT * TX - Post-decryption, post-NAT * DR - Dropped packets Putting RX and TX into the same file will, if NAT is involved, result in the packet capture putting both the pre-NAT packet and the post-NAT packet in the PCAP. Including the FW stream will result in duplicate errors as it will clash with RX. I've seen drops recorded when a packet was tranmisted and then a routing loop pushed the packet straight back at the firewall. So the packet was shown in TX and DR. ===== Packet Capture IP Style ===== NAT affects what IP you should put in the packet capture filter. I've noticed that you don't really need to put the reverse stream into the capture. Just source and destination will end up capturing both c2s and s2c streams of the traffic flow. An exception may be sesssions that can be bi-directional like IKE on udp-500. In the following example there are two devices 10.2.2.22 10.3.3.33 Client 10.2.2.22 will try to access server 10.3.3.33. However, client 10.2.2.22 will source NAT behind 192.168.2.22 and will try and access the sever on IP 192.168.3.33 which is then destination NAT'd to 10.3.3.33. In the example below, we have * actual client (pre-NAT client) * actual server (post-NAT server) * fake client (post-NAT server) * fake server (pre-NAT server) If we capture with no IP filter, we get * rx * c2s actual client to fake server * s2c actual server to fake client * fw * c2s actual client to fake server * s2c actual server to fake client * tx * c2s fake client to actual server * s2c fake server to actual client If we filter by IP * 10.2.2.22 > 10.3.3.33 doesn't work * 192.168.2.22 > 192.168.3.33 doesn't work However * 10.2.2.22 > 192.168.3.33 works * rx * c2s actual client to fake server * s2c actual server to fake client * fw * c2s actual client to fake server * s2c none (nothing captured) * tx * c2s none (nothing captured) * s2c fake server to actual client Also * 192.168.2.22 > 10.3.3.33 works * rx * c2s actual client to fake server * s2c actual server to fake client * fw * c2s none (nothing captured) * s2c actual server to fake client * tx * c2s fake client to actual server * s2c none (nothing captured) If you set the capture to have two filters * 10.2.2.22 > 192.168.3.33 * 192.168.2.22 > 10.3.3.33 * rx * c2s actual client to fake server * s2c actual server to fake client * fw * c2s actual client to fake server * s2c actual server to fake client * tx * c2s fake client to actual server * s2c fake server to actual client Conclusions for double NAT'd traffic: * Capturing "recieved" traffic is easy. * Capturing "firewalled" and "transmitted" traffic that included both directions of traffic flow requires you to include two filters, the pre-NAT IP address and the post-NAT IP addresses If you have destination NAT only where 10.2.2.22 connects to 192.158.3.33 which is D-NAT'd to 10.3.3.33, then * 10.2.2.22 > 192.168.3.33 works * rx * c2s actual client to fake server * s2c actual server to actual client * fw * c2s actual client to fake server * s2c none (nothing captured) * tx * c2s none (nothing captured) * s2c fake server to actual client Also * 10.2.2.22 > 10.3.3.33 works * rx * c2s actual client to fake server * s2c actual server to actual client * fw * c2s none (nothing captured) * s2c actual server to actual client * tx * c2s actual client to actual server * s2c none (nothing captured) If you set the capture to have two filters * 10.2.2.22 > 192.168.3.33 * 10.2.2.22 > > 10.3.3.33 * rx * c2s actual client to fake server * s2c actual server to actual client * fw * c2s actual client to fake server * s2c actual server to actual client * tx * c2s actual client to actual server * s2c fake server to actual client If you have Source NAT only where 10.2.2.22 connects to 10.3.3.33 and S-NAT's behind 192.168.2.22, then * 10.2.2.22 > 10.3.3.33 works * rx * c2s actual client to actual server * s2c actual client to actual server * fw * c2s actual client to actual server * s2c actual client to actual server * tx * c2s actual client to actual server * s2c actual client to actual server Also * 192.168.2.22 > 10.3.3.33 doesn't work ===== Data Plane Packet Capture ===== Base on [[https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClTJCA0|this article]] and [[https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000CleECAS|this article]]. * ''delta yes'' indicates I want to view counters that have incremented since the last time I executed this command. * ''packet-filter yes'' indicates I want to see only global counters that match my filters. show counter global filter packet-filter yes delta yes Next you're going to configure the stages—there are 4: * **drop** stage is where packets get discarded. The reasons may vary and, for this part, the global counters may help identify if the drop was due to a policy deny, a detected threat, or something else. * **receive** stage captures the packets as they ingress the firewall before they go into the firewall engine. When NAT is configured, these packets will be pre-NAT. * **transmit** stage captures packets how they egress out of the firewall engine. If NAT is configured, these will be post-NAT. * **firewall** stage captures packets in the firewall stage. Start packet captures debug dataplane packet-diag set capture on Stop packet captures debug dataplane packet-diag set capture off ===== Managment Interface Packet Capture ===== To Troubleshoot connectivity issues with the management plane, the built-in tcpdump command can be used to capture useful information: tcpdump filter "port 53" tcpdump filter "host 10.16.0.106 and not port 22" view-pcap mgmt-pcap mgmt.pcap ===== Flow Basic ===== First we're going to verify that nothing's been configured yet that could interfere with our new settings: debug dataplane packet-diag show setting If anything's still configured, we can clear out all filters and previous flow basic logs using these commands: debug dataplane packet-diag clear all debug dataplane packet-diag clear log log We can now go ahead and create and enable the filters, making sure pre-parse is disabled. A second filter from the server to the NAT IP on the external interface of the firewall will help capture returning packets before they are NAT'ed in the 'ingress stage.' More about that below: debug dataplane packet-diag set filter match source 192.168.0.34 destination 198.51.100.97 destination-port 80 protocol 6 non-ip exclude debug dataplane packet-diag set filter match source 198.51.100.97 destination 198.51.100.230 source-port 80 protocol 6 non-ip exclude debug dataplane packet-diag set filter on debug dataplane packet-diag show setting debug dataplane packet-diag set log feature flow basic When you're ready to initiate traffic make sure any existing sessions have been terminated, then disable session offloading to ensure all packets are captured even if the session would normally be offloaded into hardware and finally go ahead and enable the logging feature. show session all filter source 192.168.0.34 destination 198.51.100.97 If there are still active sessions you can clear them by using the clear session command: clear session all filter source 192.168.0.34 destination 198.51.100.97 set session offload no debug dataplane packet-diag set log on You can now go ahead and start the session you want to capture, wait for it to gracefully end, then disable logging: show session all filter source 192.168.0.34 destination 198.51.100.97 debug dataplane packet-diag set log off set session offload yes A nifty little tool is provided to aggregate these files into a single file: debug dataplane packet-diag aggregate-logs The final output file is then stored on the management plane as ''pan_packet_diag.log'': less mp-log pan_packet_diag.log =Packet Filter= debug dataplane packet-diag set filter match source 192.0.2.1 non-ip exclude debug dataplane packet-diag set filter on Then show your counters as a delta with just that filter: show counter global filter delta yes packet-filter yes