Freorit

Why Monitor Your Firewall?

A firewall blocks traffic. That's important. But what's equally important is knowing what it's blocking and when.

Without monitoring, you only see the outcome: "Connection refused." With monitoring, you see the pattern: "50 SSH attempts per minute from this IP, escalating over 2 hours." That's actionable.

The Problem

OpenBSD's PF firewall is excellent at filtering. But the default setup leaves critical questions unanswered:

• How many active connections does your firewall handle right now?
• Which IPs are being blocked most frequently?
• Is that rate normal or a sign of attack?
• Are your rate limits too tight (blocking legitimate users) or too loose?

Without central monitoring, answering these questions means SSH-ing to the router and manually inspecting logs. That doesn't scale.

The Solution

This guide shows how to build a monitoring stack that transforms your firewall from a silent guardian into a communicative one. The result:

• Real-time visibility into firewall state and traffic patterns
• Automated alerts when anomalies occur
• Historical data for trending and capacity planning
• Beautiful dashboards for daily operations

Architecture Overview

The monitoring stack has two data flows: metrics and logs.

Key Components

Data Flows

Metrics Flow:

Logs Flow:

Layer 1: Hardened PF Firewall

The firewall itself must be configured to collect and export useful metrics. Three key aspects:

Source Tracking for Attack Detection

Source tracking allows PF to monitor connection patterns per IP address. This is essential for detecting brute force and port scans.

# Enable source tracking
set limit src-nodes 20000

# Dynamic blocking tables
table <bruteforce> persist
table <port_scanners> persist

# SSH brute force protection
pass in on pppoe0 proto tcp to port 22 \
keep state (max-src-conn 3, max-src-conn-rate 3/60, \
        source-track rule, overload <bruteforce> flush global)

# Block detected attackers
block quick from <bruteforce>
    

Rate Limiting

Beyond brute force protection, rate limiting prevents resource exhaustion:

# General outbound rate limiting
pass in on re0 from (re0:network) to any \
keep state (max-src-conn 100, max-src-conn-rate 50/10)

# Global WAN limit
pass out on pppoe0 proto tcp \
keep state (max 10000, max-src-conn-rate 100/10)
    

Anti-Spoofing

Prevent IP spoofing attacks and filter out invalid source addresses:

# Prevent spoofed packets
antispoof quick for pppoe0
antispoof quick for re0

# Block invalid private ranges
table <martians> const { 172.16.0.0/12, 192.168.0.0/16, 127.0.0.0/8 }
block in quick on pppoe0 from <martians> to any
    

Enable SNMP Metrics Export

OpenBSD's SNMP daemon (snmpd) exports firewall metrics that Prometheus can scrape:

listen on 127.0.0.1
listen on 10.20.1.1  # Make accessible from monitoring host

# Allow read-only access from monitoring network
read-only community "public" 10.20.1.0/24
    

Key SNMP metrics PF exposes:

• pfStateCount - Current active connections
• pfStateLimit - Maximum state table size
• pfTableCount - Entries in blocking tables
• ifHCInOctets - Interface traffic (bytes in)
• ifHCOutOctets - Interface traffic (bytes out)

Enable snmpd to start on boot:

rcctl enable snmpd
rcctl start snmpd
    

Layer 2: Centralized Monitoring

The monitoring stack aggregates metrics and logs into Prometheus and Loki, then visualizes them in Grafana.

Prometheus Setup: Metrics Collection

Install the SNMP Exporter (converts SNMP to Prometheus format):

docker run -d \
  --name snmp-exporter \
  -p 9116:9116 \
  prometheuscommunity/snmp-exporter
    

Configure Prometheus to scrape both SNMP and node metrics:

global:
  scrape_interval: 30s

scrape_configs:
  - job_name: 'snmp-router'
static_configs:
  - targets: ['10.20.0.1']
metrics_path: /snmp
params:
  module: [if_mib]
relabel_configs:
  - source_labels: [__address__]
target_label: __param_target
  - source_labels: [__param_target]
target_label: instance
  - target_label: __address__
replacement: localhost:9116
    

Loki Setup: Log Aggregation

Loki ingests logs from syslog-ng via Grafana Alloy. Configuration:

auth_enabled: false

ingester:
  chunk_idle_period: 3m
  max_chunk_age: 1h
  max_streams_per_user: 10000

schema_config:
  configs:
- from: 2025-01-01
  store: tsdb
  object_store: filesystem
  schema: v13
  index:
prefix: index_
period: 24h

storage_config:
  filesystem:
directory: /loki/chunks
    

Grafana Dashboards

Grafana visualizes the collected data. Key dashboard panels:

Security Overview

• Blocked Connections (last hour) - Rate of blocked packets over time
• Top Blocked IPs - Which sources are being blocked most
• Brute Force Detection - Entries in <bruteforce> table
• Port Scans - Rapid SYN attempts detected

Firewall Health

• State Table Usage - % of current connections vs. limit
• Source Tracking Entries - Active source-tracked connections
• Table Sizes - <bruteforce>, <port_scanners> table entries

Traffic Analysis

• WAN Traffic In/Out - Bandwidth usage over time
• LAN Traffic - Internal network traffic
• Top Talkers - Which IPs generate most traffic

Alert Rules

Define alerts for anomalies in Prometheus:

groups:
  - name: firewall_alerts
rules:
  - alert: StateTableNearFull
expr: (pfStateCount / pfStateLimit) > 0.8
for: 5m
annotations:
  summary: "Firewall state table >80% full"

  - alert: BruteForceSpike
expr: rate(pfTableCount{table="bruteforce"}[5m]) > 1
for: 2m
annotations:
  summary: "Brute force spike detected"

  - alert: HighBlockRate
expr: rate(pfBlockPackets[5m]) > 100
for: 5m
annotations:
  summary: "Unusually high block rate"
    

Attack Detection in Action

Example 1: SSH Brute Force Attack

Scenario: Attacker attempts rapid SSH login attempts.

Timeline

Total detection time: 10-15 seconds from initial attack

Monitoring visibility:

• Prometheus metric: pfTableCount{table="bruteforce"} increments
• Grafana alert fires: "Brute Force Spike Detected"
• Loki logs show PF block rules matching the IP

Example 2: Port Scan Detection

Scenario: Attacker runs nmap -p- <router-ip> to scan all ports.

What Happens:

1. Rapid SYN packets to many ports from single IP
2. PF source tracking records high connection rate from this IP
3. Rate limit rule: max-src-conn-rate 50/10 (50 connections per 10 seconds) exceeded
4. IP added to <port_scanners> table
5. All future traffic from IP blocked with block quick from <port_scanners>

Monitoring detection:

Prometheus query to detect port scans:

rate(pfTableCount{table="port_scanners"}[5m]) > 0.1
    

This alerts when new IPs are being added to the port scanners table at a rate > 1 per 50 seconds.

Results & Metrics

Quantitative Improvements

Real-World Example: First Week

After deploying monitoring, the first week revealed:

• 43 unique IPs blocked for SSH brute force attempts
• 12 port scan attempts detected and blocked
• Peak state table usage: 45% (healthy margin)
• Average blocked packet rate: 20/min (normal baseline)
• Zero false positives in rate limiting (legitimate users unaffected)

Best Practices

1. Start Conservative with Limits

Don't deploy tight rate limits immediately. Tune gradually:

• Week 1: Loose limits (max-src-conn 50, max-src-conn-rate 50/60)
• Week 2-3: Monitor for false positives; tighten by 25%
• Week 4+: Production values after no false positives observed

2. Whitelist Trusted IPs

Create allowlists for known-good sources (your office, home devices, trusted partners):

table <trusted> persist file "/etc/pf-trusted.txt"

# Higher limits for trusted sources
pass in on pppoe0 proto tcp from <trusted> to port 22 \
keep state (max-src-conn 20, max-src-conn-rate 20/60)

# Strict limits for everyone else
pass in on pppoe0 proto tcp to port 22 \
keep state (max-src-conn 3, max-src-conn-rate 3/60, \
        overload <bruteforce> flush global)
    

3. Monitor the Monitors

Add alerts for monitoring infrastructure failures:

• Prometheus scrape failures (firewall SNMP down)
• Loki ingestion errors
• Disk space for log/metrics storage

4. Log Volume Control

Excessive logging fills disk quickly. Be selective:

# Only log state-establishing packets (default)
pass out log on pppoe0 proto tcp

# NOT: pass out log (all) on pppoe0  <- Creates 10GB/day of logs
    

5. Backup Your Configurations

Keep copies of working pf.conf, snmpd.conf, and Prometheus rules:

cp /etc/pf.conf /etc/pf.conf.backup.$(date +%Y%m%d)
git add -A && git commit -m "Firewall config backup"
    

Lessons Learned

Technical Lessons

• Source tracking is essential: Without it, you can't detect patterns—only individual packets.
• Metrics before logs: Time-series metrics (Prometheus) are simpler to implement and provide immediate value. Logs add forensic depth.
• Rate limits need tuning: Copy-paste limits from guides won't fit your traffic. Monitor and adjust weekly.
• SNMP isn't perfect: Some metrics have latency; logs are more real-time for event detection.

Operational Lessons

• False positives damage trust: A single legitimate user blocked erodes confidence in automation. Tune conservatively.
• Alerting needs tuning: Alert too frequently and you get alert fatigue. Use for: 5m clauses to reduce noise.
• Documentation pays off: In 6 months, you'll forget why limits are set a certain way. Document the rationale.

Monitoring Lessons

• Baselines matter: You can't detect anomalies without knowing what normal looks like. Collect 1 week of baseline data before tuning alerts.
• Dashboards are more useful than alerts: Alerts tell you something's wrong; dashboards tell you why.
• Retention policy: Keep metrics for 30 days, logs for 7 days. Balance visibility vs. storage cost.

Open Source Projects

This guide is built entirely on open source:

• OpenBSD – Security-focused Unix-like OS with integrated PF firewall
• Prometheus – Time-series metrics database and monitoring system
• Loki – Log aggregation system designed for Prometheus users
• Grafana – Visualization platform for metrics and logs
• SNMP Exporter – Prometheus exporter for SNMP data
• syslog-ng – Advanced syslog daemon for log collection
• Grafana Alloy – Flexible agent for collecting logs and metrics and sending to Loki and Prometheus