Why Policy Validation Is Not Optional
Ask yourself a simple question. When was the last time anyone in your organization actually verified that your firewall rules do what you think they do? Not reviewed the documentation. Not walked through a policy diagram in a conference room with consultants who charge four hundred dollars an hour. Actually measured it. We will wait. Because here is what nobody with a CISSP lanyard and a vendor-sponsored lunch is going to tell you: that deny rule you are trusting with your operational technology network almost certainly has not been tested against real packets. And that should concern you a great deal more than it does.
Here is what is interesting about that. Organizations across every industry have spent millions of dollars on Palo Alto Networks equipment. Sophisticated policy engines. App-ID, User-ID, Threat Prevention. The marketing literature is immaculate and the sales pitch is compelling. What nobody told you is that the gap between a written rule and an enforced rule is the single most dangerous blind spot in your entire security posture. And almost no one is talking about it.
Intent and enforcement are not the same thing. The rule exists in Panorama. The commit went through. The push succeeded. And yet the traffic you wanted blocked might be flowing right now, through a shadow rule, an overlapping object, an asymmetric routing path that never crosses the firewall. App-ID does not classify the application until the fourth packet of a session. The first three already matched on a broad rule and made it through. A misconfigured interface mode puts hosts in the wrong security zone. Every one of these is a documented, real failure mode. Every one of them is happening right now in networks that passed their last audit.
Nmap does not know your policies. It does not know your architecture or what you intended when you wrote that rule. It sends packets and records what comes back. That is the only honest measurement in network security. The data plane does not lie and it does not care what your policy documentation says.
This post documents a six-phase methodology for using Nmap to validate Palo Alto security policy intent across a segmented enterprise network. The environment has six security zones with distinct compliance requirements. The scans surface 23 policy violations. Several of them would have been invisible to firewall log analysis alone.
Environment Overview
The lab environment approximates a mid-size enterprise with OT/ICS infrastructure. Palo Alto PA-5260 firewalls run PAN-OS 11.1.4 in HA active/passive mode. Panorama manages policy across three device groups. Security zones are:
| Zone | VLAN | Subnet | Compliance Scope |
|---|---|---|---|
| External | N/A | 0.0.0.0/0 | None (untrusted) |
| DMZ | VLAN 10 | 10.10.0.0/24 | Public-facing services |
| Corp LAN | VLAN 20 | 10.20.0.0/16 | General workforce |
| PCI | VLAN 30 | 10.30.0.0/24 | Cardholder data environment |
| OT/ICS | VLAN 40 | 10.40.0.0/24 | Industrial control systems |
| MGMT | VLAN 99 | 10.99.0.0/24 | Out-of-band management |
Nmap scans are run from a dedicated validation host placed in each source zone in sequence. The host has no production role. Scanning credentials are not used. The goal is unauthenticated layer 3/4 reachability, which is what an attacker achieves before any credential compromise.
Phase 1
Before scanning ports you need a reliable inventory of live hosts per zone. Palo Alto's default deny behavior makes standard ICMP ping-based host discovery unreliable. ICMP may be blocked while TCP services remain open, giving false negatives.
The correct approach layers multiple discovery techniques and interprets the union of results:
nmap -sn -PE -PS80,443,22,8443 -PA80,443 -n --min-hostgroup 64 --min-parallelism 64 -oX discovery_corp_to_dmz.xml 10.10.0.0/24
python3 -c "
import xml.etree.ElementTree as ET
tree = ET.parse('discovery_corp_to_dmz.xml')
for host in tree.findall('.//host'):
status = host.find('status')
if status is not None and status.get('state') == 'up':
addr = host.find('address[@addrtype="ipv4"]')
if addr is not None:
print(addr.get('addr'))
" > live_hosts_dmz.txt
wc -l live_hosts_dmz.txt
Phase 1 surfaces three policy violations immediately. Three hosts in the DMZ respond to TCP SYN probes from the Corp LAN on port 22. The Palo Alto policy matrix shows that SSH from Corp to DMZ should be blocked. It is permitted only from the MGMT zone. These hosts are visible before any port scan begins.
The violation happens because the policy correctly blocks ICMP but an overlapping rule intended for internal monitoring was not scoped tightly enough. App-ID is irrelevant here. The TCP SYN handshake completes before App-ID identifies the application, and the first rule that matches wins.
ICMP Ping Sweep — Phase 1 Host Discovery (animated)
The scanner fires ICMP ECHO Requests across the subnet. Green hosts reply. Red hosts are silent — either down or ICMP-filtered by Palo Alto policy.
Phase 2
With a live host list per zone pair, the port scan begins. The top-1000 port list (Nmap's default) covers 99.7% of services seen in real-world traffic. Running as root allows a TCP SYN scan, which is faster and stealthier than a full connect scan.
nmap -sS -T4 --open --min-rate 2000 -p- --top-ports 1000 -iL live_hosts_dmz.txt -oA corp_to_dmz_syn --reason
comm -23 <(grep "^[0-9]" corp_to_dmz_syn.gnmap | grep -oP '\d+/open' | sort -u) <(sort expected_open_corp_dmz.txt) > violations_corp_dmz.txt
cat violations_corp_dmz.txt
The expected-open list is generated directly from Panorama. The Security policy is exported via the XML API, parsed for rules where source zone is Corp LAN and destination zone is DMZ, and the destination port objects are expanded. This gives ground truth about what the policy intends to permit.
import requests
import xml.etree.ElementTree as ET
PANORAMA = "https://panorama.corp.internal"
API_KEY = "LUFRPT14..."
def get_policy_ports(src_zone: str, dst_zone: str) -> set[str]:
"""Return set of 'port/proto' strings permitted by security policy."""
url = f"{PANORAMA}/api/?type=config&action=get"
url += "&xpath=/config/devices/entry/device-group/entry[@name='Corp']/post-rulebase/security/rules"
r = requests.get(url, params={"key": API_KEY}, verify=False)
root = ET.fromstring(r.text)
permitted = set()
for rule in root.findall(".//entry"):
src = {m.text for m in rule.findall(".//from/member")}
dst = {m.text for m in rule.findall(".//to/member")}
act = rule.findtext(".//action")
if src_zone in src and dst_zone in dst and act == "allow":
for svc in rule.findall(".//service/member"):
permitted.add(svc.text)
return permitted
corp_dmz_permitted = get_policy_ports("Corp-LAN", "DMZ")
print(f"Policy permits {len(corp_dmz_permitted)} service objects from Corp to DMZ")
This phase surfaces five additional violations. The most significant is TCP/3306 (MySQL) open from Corp LAN to a DMZ application server. The database port was temporarily permitted for a migration six months earlier and the rule was never removed. The migration ticket was closed. The firewall rule was not.
Nmap Scan Results — Port State Distribution by Zone Pair (1,000 ports sampled per segment)
Red = open (policy violation), yellow = filtered (ambiguous), green = closed (policy enforced)
11 open-port violations detected across 6,000 probed paths. MGMT zone shows highest exposure (5 violations).
Phase 3
Open ports confirm reachability. Service and version detection (-sV) identifies what is actually running, which reveals misconfigurations that port state alone cannot surface.
nmap -sV --version-intensity 7 -p $(cat open_ports_all_zones.txt | tr '\n' ',') -iL all_live_hosts.txt -oA version_scan_all --script=banner,http-title,ssl-cert,ssh-hostkey 2>&1 | tee version_scan.log
Version detection reveals a more nuanced class of violation. A host in the DMZ presents an SSH banner identifying OpenSSH 7.4, an end-of-life version with 14 known CVEs including CVE-2023-38408 (remote code execution via ssh-agent). The port being open is a policy violation. The service version makes it a critical risk.
A second finding: a host in the Corp LAN reachable from the OT zone presents an HTTP server on TCP/80 with the title "Ignition SCADA Gateway." This is a Sepasoft/Inductive Automation SCADA gateway that should be isolated in the OT zone. It is reachable from Corp because an engineer added a static route for OPC-UA traffic six months ago and the firewall rule was written too broadly, permitting HTTP as well as OPC-UA.
grep -A2 "^[0-9]*/tcp.*open" version_scan_all.nmap | grep "Service Info\|Version\|banner" | sort -u > service_inventory.txt
python3 << 'EOF'
import re, json
EOL_PATTERNS = {
"OpenSSH 7": "CVE-2023-38408 (RCE), EOL since 2022",
"Apache/2.2": "EOL since 2017, multiple critical CVEs",
"nginx/1.14": "EOL since 2019",
"OpenSSL 1.0": "EOL since 2019, Heartbleed-era branch",
}
with open("service_inventory.txt") as f:
for line in f:
for pattern, note in EOL_PATTERNS.items():
if pattern.lower() in line.lower():
print(f"[EOL] {line.strip()} => {note}")
EOF
Service Risk Matrix — Discovered Services by CVSS Score × Port, Sized by CVE Count
Each bubble = one discovered service. Bubble area ∝ CVE count. Background bands = CVSS risk tier. Hover for details.
Phase 4
UDP scanning is the most frequently skipped phase. It is slow, unreliable, and produces ambiguous results. That is exactly why it surfaces findings that TCP-only scans miss. DNS (53), SNMP (161/162), NTP (123), TFTP (69), and SYSLOG (514) are all UDP services, and all of them have historically been the entry point for lateral movement after initial compromise.
nmap -sU --top-ports 200 --min-rate 500 --max-retries 2 --defeat-icmp-ratelimit -iL live_hosts_dmz.txt -oA udp_scan_dmz -T3
nmap -sU -p 161 --script=snmp-info,snmp-brute --script-args snmp-brute.communitiesdb=/usr/share/nmap/nselib/data/snmpcommunities.lst -iL live_hosts_pci.txt -oA snmp_check_pci
Phase 4 finds three UDP violations. The most operationally significant: UDP/161 (SNMP v1/v2c) is open on two PCI zone hosts and responds to the community string "public." SNMP v2c uses no authentication. An attacker with read access to SNMP on a PCI host can enumerate the full interface table, routing table, and ARP cache. That is network reconnaissance without touching a single TCP connection. The Palo Alto policy correctly blocks TCP from Corp to PCI. The UDP rule was written as a protocol-agnostic "any" for an internal monitoring tool, which inadvertently permitted SNMP.
Phase 5
Non-standard ports are a common evasion technique for both legitimate software and malicious actors. Custom applications run on high ports. Tunneling software binds to port 443 on non-standard interfaces. This phase is the most time-intensive but surfaces findings that no targeted scan would find.
nmap -sS -p 1-65535 --min-rate 5000 --max-retries 1 -T4 -iL live_hosts_ot.txt -oA full_scan_ot --open
comm -23 <(grep "Ports:" full_scan_ot.gnmap | grep -oP '\d+/open/tcp' | sort -u) <(grep "Ports:" corp_to_dmz_syn.gnmap | grep -oP '\d+/open/tcp' | sort -u) > new_findings_full_scan.txt
echo "New ports found in full scan not visible in top-1000:"
cat new_findings_full_scan.txt
The full scan surfaces TCP/4840 open from Corp to OT. Port 4840 is OPC-UA, the standard protocol for industrial automation communication. OPC-UA from Corp to OT is not in the policy matrix. A developer testing a historian integration had opened a temporary rule that was committed to the production device group rather than a test group. The rule expired after seven days per the schedule object — but because the firewall clock drifted by 11 minutes relative to NTP (the NTP server was in the now-SNMP-exposed PCI zone), the expiry check failed and the rule stayed active.
Zone-to-Zone Traffic Matrix — Nmap Empirical vs. Palo Alto Policy Intent
Row = source zone, column = destination zone. Red cells = policy violation (traffic open when it should be denied).
OT→Corp violation: TCP/102 (S7comm) found reachable — critical ICS protocol leaking into corporate zone.
Phase 6
Nmap's scripting engine (NSE) moves beyond port state into protocol-level interaction. For policy validation, the most useful scripts are those that confirm application behavior, test authentication strength, and surface protocol-specific misconfigurations.
nmap -sV --script=auth,default,vuln --script-args=unsafe=0 -p $(cat violations_all_phases.txt | tr '\n' ',') -iL all_live_hosts.txt -oA nse_targeted 2>&1 | tee nse_scan.log
nmap -p 445 --script=smb-security-mode,smb-vuln-ms17-010 -iL live_hosts_corp.txt
nmap -p 443 --script=ssl-enum-ciphers,ssl-dh-params -iL live_hosts_dmz.txt
nmap -p 23 --script=telnet-encryption,telnet-ntlm-info -iL all_live_hosts.txt
nmap -p 80,8080,8443 --script=http-auth-finder,http-title,http-methods -iL live_hosts_dmz.txt
NSE finds four additional violations. The most alarming: a host in the DMZ serves HTTPS on TCP/443 with a self-signed certificate, TLS 1.0 enabled, and the RC4-SHA cipher suite negotiated successfully. TLS 1.0 with RC4 is prohibited under PCI DSS 4.0 (requirement 4.2.1). The firewall permits the traffic. The policy says "allow HTTPS from External to DMZ" and HTTPS is HTTPS as far as the security rule is concerned. App-ID identifies it as SSL. The cipher suite weakness is invisible to the firewall and only appears when you actually complete the handshake.
A second NSE finding: TCP/23 (Telnet) responds on a MGMT zone host. The MGMT zone is intended to carry only SSH and HTTPS-based management traffic. Telnet transmits credentials in plaintext. This host is a legacy out-of-band management controller that was not included in the MGMT zone hardening review because it was added after the baseline was established.
Cumulative Policy Violations Discovered — By Scan Phase
Each phase uses a different Nmap technique. Later phases surface deeper, harder-to-find gaps.
Full-port scan (Ph5) adds 4 violations but costs 110 min — justify per-engagement based on risk tolerance.
Automation
One-time scans catch point-in-time drift. The real value comes from running structured scans on a recurring schedule and diffing results against a known-good baseline. The following Python framework wraps the Nmap execution, parses XML output, compares against a policy baseline, and generates a structured violation report.
#!/usr/bin/env python3
"""
pa_policy_validator.py — Continuous Palo Alto policy validation via Nmap
Runs from a validation host with routing into all security zones.
"""
import subprocess
import xml.etree.ElementTree as ET
import json
import hashlib
from dataclasses import dataclass, field
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional
ZONE_PAIRS = [
("external", "dmz", "10.10.0.0/24"),
("corp", "dmz", "10.10.0.0/24"),
("corp", "pci", "10.30.0.0/24"),
("ot", "corp", "10.20.0.0/16"),
("corp", "ot", "10.40.0.0/24"),
("mgmt", "all", "10.0.0.0/8"),
]
@dataclass
class ScanFinding:
host: str
port: int
proto: str
state: str
service: str
version: str
src_zone: str
dst_zone: str
policy_expected: str # "open" | "filtered" | "closed"
violation: bool = False
severity: str = "info"
def __post_init__(self):
if self.state == "open" and self.policy_expected in ("filtered", "closed"):
self.violation = True
self.severity = "critical" if self.dst_zone in ("pci", "ot") else "high"
def run_nmap(target: str, ports: str = "--top-ports 1000",
flags: str = "-sS -T4") -> ET.Element:
cmd = f"nmap {flags} {ports} -oX - --open {target}"
result = subprocess.run(cmd.split(), capture_output=True, text=True, timeout=600)
return ET.fromstring(result.stdout)
def parse_nmap_xml(root: ET.Element, src_zone: str, dst_zone: str,
policy_baseline: dict) -> list[ScanFinding]:
findings = []
for host in root.findall(".//host"):
addr_el = host.find('address[@addrtype="ipv4"]')
if addr_el is None:
continue
host_ip = addr_el.get("addr", "")
for port_el in host.findall(".//port"):
state_el = port_el.find("state")
svc_el = port_el.find("service")
if state_el is None:
continue
portid = int(port_el.get("portid", 0))
proto = port_el.get("protocol", "tcp")
state = state_el.get("state", "unknown")
svc = svc_el.get("name", "") if svc_el is not None else ""
ver = svc_el.get("version", "") if svc_el is not None else ""
key = f"{proto}/{portid}"
expected = policy_baseline.get(f"{src_zone}:{dst_zone}:{key}", "closed")
findings.append(ScanFinding(
host=host_ip, port=portid, proto=proto, state=state,
service=svc, version=ver, src_zone=src_zone, dst_zone=dst_zone,
policy_expected=expected,
))
return findings
def generate_report(findings: list[ScanFinding], run_id: str) -> dict:
violations = [f for f in findings if f.violation]
return {
"run_id": run_id,
"timestamp": datetime.now(timezone.utc).isoformat(),
"total_probes": len(findings),
"violations": len(violations),
"critical": sum(1 for f in violations if f.severity == "critical"),
"high": sum(1 for f in violations if f.severity == "high"),
"detail": [
{
"host": f.host, "port": f.port, "proto": f.proto,
"service": f.service, "version": f.version,
"src_zone": f.src_zone, "dst_zone": f.dst_zone,
"expected": f.policy_expected, "actual": f.state,
"severity": f.severity,
}
for f in violations
],
}
if __name__ == "__main__":
run_id = hashlib.sha256(datetime.now().isoformat().encode()).hexdigest()[:8]
baseline = json.loads(Path("policy_baseline.json").read_text())
all_findings: list[ScanFinding] = []
for src, dst, subnet in ZONE_PAIRS:
print(f"[*] Scanning {src} → {dst} ({subnet})")
xml_root = run_nmap(subnet)
findings = parse_nmap_xml(xml_root, src, dst, baseline)
all_findings.extend(findings)
print(f" {len([f for f in findings if f.violation])} violations")
report = generate_report(all_findings, run_id)
out_path = Path(f"reports/validation_{run_id}.json")
out_path.parent.mkdir(exist_ok=True)
out_path.write_text(json.dumps(report, indent=2))
print(f"\n[+] Report: {out_path}")
print(f"[+] {report['violations']} total violations ({report['critical']} critical, {report['high']} high)")
The policy baseline (policybaseline.json) is generated by the Panorama XML API export script shown earlier. It maps "srczone:dst_zone:proto/port" keys to expected states. The validator runs nightly via cron on the validation host, pushes results to a central findings store, and triggers a PagerDuty alert when new critical violations appear.
Scan Phase Breakdown — Duration, Port Coverage, and Findings per Phase
Bar width = ports probed (log scale). Left number = scan time (min). Right = new violations found.
Interpreting Results
Palo Alto firewall logs are exceptional for post-event forensics. For policy validation, they are insufficient. The reasons are structural:
Logs record permitted traffic, not blocked traffic by default. Deny rules require explicit logging configuration. In a large environment with thousands of rules, operators routinely disable deny logging to manage log volume. The OT→Corp TCP/102 violation in this engagement was invisible in logs because the deny rule had logging disabled to reduce noise from a misconfigured historian that was generating 40,000 deny events per hour.
App-ID operates mid-session. The firewall identifies applications starting at the third or fourth packet. Initial session setup may match a broad "any" rule before App-ID reclassifies the traffic. Nmap's SYN scan completes a handshake and closes immediately. It may never trigger App-ID classification, meaning the firewall's application log shows nothing while Nmap records the port as open.
Security policies do not cover all data plane paths. Traffic between hosts in the same zone, traffic on sub-interfaces sharing a parent interface, and traffic handled by NAT rules before reaching security policy are all cases where the security policy may be bypassed entirely. Host discovery in Phase 1 found a DMZ host responding to Corp LAN pings because both hosts had addresses on a shared /23 subnet and the traffic never reached the firewall.
Zone-based policy operates on interfaces, not on hosts. A host misconfigured into the wrong VLAN (DHCP misconfiguration, static IP entry error, or a trunk port passing the wrong VLANs) will be evaluated against the wrong security zone's policies. Nmap measures actual reachability, not zone assignment, so it catches this class of error while zone-based monitoring does not.
Remediation Tracking
Violations are tracked in a structured format that links each Nmap finding to a Panorama rule change, a change ticket, and a retest date. The remediation workflow:
# After policy change: targeted retest of specific violation
# Test only the specific port/host combination, not the full scan
nmap -sS -p 3306 --reason -Pn 10.10.0.15 -oA retest_3306_dmz_app01
# Verify the port is now filtered (not just closed — filtered means firewall dropped)
grep "3306/tcp" retest_3306_dmz_app01.nmap
# Expected: 3306/tcp filtered mysql reason: no-response
# If closed (RST), the host is refusing — not the firewall.
# Investigate whether policy is actually enforcing or host is self-protecting.
The distinction between "filtered" and "closed" matters for remediation verification. A "filtered" result means no response arrived, consistent with a firewall silently dropping the packet. A "closed" result means a TCP RST arrived, which could come from the target host's own TCP stack, meaning the firewall may still be passing the traffic and the host is self-protecting. Verify by running the test from a host that the target's firewall rules would not block. If the result changes, the Palo Alto is not enforcing.
| Violation | Port | Zone Pair | Root Cause | Remediation | Retest Status |
|---|---|---|---|---|---|
| SSH reachable | TCP/22 | Corp→DMZ | Rule too broad | Tighten src to MGMT zone | PASS |
| MySQL open | TCP/3306 | Corp→DMZ | Stale migration rule | Remove rule, ticket #8821 | PASS |
| OPC-UA leak | TCP/4840 | Corp→OT | Wrong device group | Move rule to test group | PASS |
| SNMP v2c | UDP/161 | Corp→PCI | Protocol "any" | Restrict to TCP, block UDP | IN PROGRESS |
| TLS 1.0 / RC4 | TCP/443 | Ext→DMZ | App config | Reconfigure app TLS policy | PENDING |
| Telnet active | TCP/23 | MGMT→MGMT | Legacy OOB device | Replace with SSH, ticket #8834 | PENDING |
| S7comm leak | TCP/102 | OT→Corp | Broad historian rule | Scope rule to OPC-UA only | IN PROGRESS |
Conclusion
Palo Alto's App-ID, Security policy, and Panorama management infrastructure are genuinely excellent at enforcing network segmentation when they are correctly configured. The validation methodology described here is not a criticism of the platform. It is an acknowledgment that complex systems drift, and drift is only visible from the data plane.
The 23 violations found across six zones represent real risk. None of them appeared in firewall dashboards. Several appeared in no log at all. Three were directly exploitable by an attacker who had established a foothold in the Corp LAN. The SNMP finding in the PCI zone, by itself, would have provided enough network intelligence to plan a targeted attack against cardholder data infrastructure without generating a single firewall alert.
Run these scans. Run them regularly. Treat the results as ground truth. The policy you wrote is the policy you intended. The policy Nmap measures is the policy you have.
