Today, without no reason the website (oneill - FreeBSD 6.2) was down, but somehow, it responded to pings.

Apache and everything else is running inside a jail and I said that maybe the server rebooted (power failures are quite frequent) and the jail didn't start because of some filesystem problem.

After logging in to the host I see that the jail is running... hmm...

[oneill:root]-~# jexec 2 /usr/local/bin/bash [www:root]-/# [www:root]-/# sockstat -4l USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS www httpd 20293 18 tcp4 192.168.1.10:80 *:* www httpd 20292 18 tcp4 192.168.1.10:80 *:* www httpd 20291 18 tcp4 192.168.1.10:80 *:* www httpd 20290 18 tcp4 192.168.1.10:80 *:* www httpd 20289 18 tcp4 192.168.1.10:80 *:* www httpd 20288 18 tcp4 192.168.1.10:80 *:* www httpd 20287 18 tcp4 192.168.1.10:80 *:* www httpd 20286 18 tcp4 192.168.1.10:80 *:* www httpd 20285 18 tcp4 192.168.1.10:80 *:* www httpd 20284 18 tcp4 192.168.1.10:80 *:* www httpd 20283 18 tcp4 192.168.1.10:80 *:* www httpd 20282 18 tcp4 192.168.1.10:80 *:* www httpd 20281 18 tcp4 192.168.1.10:80 *:* www httpd 20280 18 tcp4 192.168.1.10:80 *:* www httpd 20279 18 tcp4 192.168.1.10:80 *:* www httpd 20278 18 tcp4 192.168.1.10:80 *:* www httpd 20277 18 tcp4 192.168.1.10:80 *:* www httpd 20276 18 tcp4 192.168.1.10:80 *:* www httpd 20275 18 tcp4 192.168.1.10:80 *:* www httpd 20274 18 tcp4 192.168.1.10:80 *:* root httpd 19394 18 tcp4 192.168.1.10:80 *:* root sendmail 9430 3 tcp4 192.168.1.10:25 *:* mysql mysqld 1311 13 tcp4 192.168.1.10:3306 *:* root sshd 1178 3 tcp4 192.168.1.10:22 *:* nobody proftpd 1168 0 tcp4 192.168.1.10:21 *:* [www:root]-/#

Well... I see from here that the jail is up, apache is running so I start tcpdump (port forwarding) to pflog:

[oneill:root]-~# tcpdump -nettti pflog0 port 80 tcpdump: WARNING: pflog0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 96 bytes

And I start making requests to port 80 (browser and telnet to port 80)... nothing... Then I think of checking PF and rules, maybe there is no port forwarding.

I checked pf and pflog if the modules are loaded into kernel with kldstat and I wanted to restart the rules (pfctl -F all && pfctl -f /etc/pf.conf) and I get:

[oneill:root]-~# pfctl -F all && pfctl -f /etc/pf.conf pfctl: SIOCGIFMTU: Device not configured

Hmm.... Beeing the first time i get this error from pf i thought that maybe there was some brakein and someone altered pf binaries and I start finding files that are newer than yesterday. Nothing relevant here.

One reboot came in the hope that it will be ok after it. Nothing new :( same error: pfctl: SIOCGIFMTU: Device not configured.

After asking opinions I was thinking of a new buildworld.

Google didn't provide a lot of help, only one result in which someone had written that this error is related to an interface. I quickly check ifconfig -a and see that rl0 (the interface with the jail) wasn't there... upsy... I check dmesg | grep rl0 and see some erros related to this realtek card:

[oneill:root]-~# dmesg|grep rl0 rl0: port 0xb800-0xb8ff mem 0xdfeefe00-0xdfeefeff irq 19 at device 3.0 on pci3 rl0: unknown device ID: 0 device_attach: rl0 attach returned 6 rl0: port 0xb800-0xb8ff mem 0xdfeefe00-0xdfeefeff irq 19 at device 3.0 on pci3 rl0: unknown device ID: 0 device_attach: rl0 attach returned 6 rl0: port 0xb800-0xb8ff mem 0xdfeefe00-0xdfeefeff irq 19 at device 3.0 on pci3 rl0: unknown device ID: 0 device_attach: rl0 attach returned 6

Luckily I had on spare network card dc0: installed on the server. From this point I only had to replace rl0 with dc0 in every config file (for FreeBSD and in my case /etc/rc.conf and /etc/pf.conf). So this helped instantly:

[oneill:root]-~# for i in /etc/rc.conf /etc/pf.conf; do perl -i -p -e "s/rl0/dc0/" $i; done

The command perl -i -p -e "s/rl0/dc0/" /path/to/file substitutes rl0 with dc0 and the loop above replaced it in both /etc/rc.conf and /etc/pf.conf

Ater this all went well, the port forwarding was working and www.ivorde.ro is up and running

Bookmarks:

Echo "Packet Filter broken on FreeBSD 6.2 ? Unusual pf error" around:

-------------------advertising-----------------