SmallWall Version 1.8.x, May 2015
Copyright © 2015 SmallWall Documentation Project
All rights reserved.
Redistribution and use in any form, with or without modification, are permitted provided that the following conditions are met:
Redistributions must retain the above copyright notice, this list of conditions and the following disclaimer.
Neither the name of the SmallWall Documentation Project nor the names of its contributors may be used to endorse or promote products derived from this documentation without specific prior written permission.
THIS DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENTATION OR THE ASSOCIATED SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
May 2015
Table of Contents
- 1. The complete guide to building a SmallWall image from scratch
- 2. SmallWall Hacker's Guide
- 3. Other Documentation
- 4. Development Frequently Asked Questions
- 4.1. I don't have a FreeBSD box to develop new features for SmallWall, is there an easier way?
- 4.2. What patches have been applied to the FreeBSD source that is used in SmallWall, and are they available?
- 4.3. Where can I find the SmallWall source code?
- 4.4. Is there any information available on the unsupported extensions support?
Table of Contents
This guide explains, in detail, all the steps that are required in order to build a complete SmallWall image from scratch. The primary intention is not that people will use the guide to actually build their own images from scratch, as that is rarely necessary, but to document and preserve all the details so that people who would like to make changes to an existing image can see how SmallWall and all of its parts are built.
This guide assumes that FreeBSD 8.4 i386 is used. It is preferred that a separate, dedicated (possibly virtual) machine is going to be used for this task, since a clean environment is best, and many steps need to be done as root. (among other things to get the file ownership and permissions right)
Note that this is not "SmallWall image building for dummies", and readers are expected to know what they're doing. Nor is it an introduction to FreeBSD and Unix like operating systems. As such, not all necessary commands and full file paths are listed (the obvious ones have been left out). It is expected that the user have some Unix like experience before using this guide.
Finally, many of these tasks are automated (especially the image building part). How this is done is left to the reader, and if the scripts do not run it is often a problem with the environment.
Install FreeBSD 8.4 i386 as usual, but use one file system only (i.e. don't create a separate partition for /usr)! See the note below for an explanation why this is necessary. Make sure that you choose the "Developer" distribution set. Installing the ports collection is not required if space is tight. The scripts will install the few ports needed.
Note
FreeBSD has all the essential system binaries (mostly those in /bin and /sbin) linked statically so that the system can boot even if /usr (which holds important libraries like libc) is not available (which is often the case when it's on a separate file system). This takes up huge amounts of space though, so we obviously want all the binaries to be linked dynamically for our m0n0wall image.
SmallWall will always lag behind FreeBSD -RELEASE versions. However, in order to get the latest security patches, the freebsd-update system is used. Additionally, some packages are needed and are installed by the pkg system, which will need initializing and updating.
Simply run pkg as root for the first time and follow the prompts to set it up.
pkg
In rare cases, this will generate an error saying the pkg system needs configuration. If so, simple copy the sample configuration file.
cp /usr/local/etc/pkg.conf.sample /usr/local/etc/pkg.conf
Update the entire system with freebsd-update. This will take some time, and list all of the update files, which will take many pages. First the updates must be fetched, and then installed.
freebsd-update fetch
freebsd-update install
The SmallWall build scripts require bash, which is not installed by default. I have also find nano to be a much more user friendly editor then vi or edit. So I install those tools. First, update the package list, and then install the packages. FreeBSD will install the latest packages, so there is no need to run freebsd-update again.
pkg update
pkg install bash nano
Almost everything you will need to do to build an image will require root. Also, any mounting of the file system by another OS will need to mount as root to even see many of the needed files. So for this reason, enabling ssh root logins can be handy. Of course, this is a huge security risk and should not be done on any system that is facing the Internet, or left on all the time, or that you even care about at all. (This is why we recommend this is done in a VM.) This is also optional, and totally at your own risk.
nano /etc/ssh/sshd_config
Use ctl-w to search for "root" and remove the has from PermitRootLogin and change no to yes. Then reboot the OS to restart sshd and reload all of your updated programs.
Now that the latest FreeBSD patches have been installed, it's time to download the svn and create the build environment.
You can either check out the svn directly, or your can get the go.sh script that will check out the svn and create the build environment.
/usr/local/bin/svn checkout http://svn.smallwall.org/SmallWall/freebsd8
svn.smallwall.org/SmallWall/freebsd8/build/scripts/go.sh
If you directly checkout the svn, you will need to move go.sh to the directory containing the "freebsd8" directory.
Update svn if this is an existing svn checkout that may not be current and execute the go.sh file as root from the directory containing the freebsd8 directory. This will set many environment variables, and then run a bash shell. You will be in this "build shell" until you type "exit." You must complete the build process before exiting.
Note
This script and all of the ones following will take a while.
At this point you can run the doall.sh script which will complete each of the following steps as one. But the documentation will cover executing them individually to assist with troubleshooting and to give a better understanding of what is happening.
This script creates the directory structure at /usr/SmallWall for the build environment and completed images. It also sets up paths and other environment variables.
This script creates all of the binaries for SmallWall. They come from packages, ports, and in some cases are compiled with SmallWall specific patches. This does mean a lot of downloading, and compiling, however.
This is a separate section for tools that require more complex patching and compiling. It is separated to make troubleshooting easier, as it has considerably less packages to compile.
The default kernel is quite different from what is used in SmallWall. To begin with, everything not needed is removed. This saves both space, and makes for a more efficient system. And since network speeds have surpassed buss speeds, every little efficiency helps. But it is not just removal. Several default have been changed or increased to better handle network forwarding.
As this is a tiem consuming and critical step, it is also logically separated to make troubleshooting easier.
And finally! This is where the image is actually built. The mfsroot is the disk image that is loaded into memory on boot. By default, it has 4 meg free for logs and other modification. In some rare cases this can fill up, and this is where you can change it.
the "img" is what is written to disk. It contains the current config, and the mfsroot image. By default it has 2 meg free, and unless you plan on having a very large config file (Complex captive portal, or many users) you probably do not need to ever change this.
There are a few other interesting tools in the scripts directory that should be mentions.
README - A very basic walk through in image building.
TODO - Things to work on, but not a key priority.
experimental-syslinux.sh - An untested script to build a bootable USB FAT32 based image.
Table of Contents
This chapter has a bit of history. It originated from Chris Buechler's original m0n0wall documentation that took a lot from Rudi van Drunen's m0n0wall Hackers Guide. However, sections 1 and 2 are all new.
SmallWall is open-source software: If it does not quite do what you want, you can change it yourself, or have someone else of your choice change it for you. There are many ways to do this, and the official way is documented in the image building section. But for those who just want a minor change quickly, there are other options.
Note that the instructions in this guide are meant as guidelines, your mileage may vary. Also, hacking low level things may seriously mess up your development system, target system, or other systems, please take care.
There are three different methods to "hack" SmallWall that we will cover here.
Modifying a running firewall with exec.php
Modifying an image with workon.sh
Working with a running file system with pxe boot.
Some times you do not need a full image, but just a quick test of some change. You can use SmallWall's built in features to test newly developed features. With this method you are not able to compile new binaries on the box, but are able to add php pages into the environment, as well as pre-compiled binaries.
Develop the page
Open a browser and point it to http://SmallWall-ip/exec.php or use https if you have enabled it. It is generally easiest if you simply log into your firewall, and then append exec.php to the end of the url.
Upload the page and any binaries needed for it to function properly via exec.php.
-
Execute the following to place php pages into the www root.
mv /tmp/*.php /usr/local/www
If you need any binaries for the page to work, copy them into the appropriate places.
-
Use chmod to make your pages readable and executable by the web server. For example, to make all your web pages read and execute;
chmod 755 /usr/local/www/*
Type in the address of your new page to try out your experiment.
Using this method, the image will revert after a reboot in case
you messed something up. You may wish to write a script to move everything
into its proper place and upload that along with your pages / binaries if
the feature requires many files. Then simply execute the script in
exec.php. After you have successfully tested the feature, and are happy with
the result, submit your work to the forums at SmallWall.freeforums.net or
email <lee@smallwall.org> for it to be added to the project.
This is a way to permanently modify a SmallWall image. It allows changes to be persistent across reboots. An example might be a custom configuration to detect the installed network cards, or additional drivers, or perhaps just turning the entire webGUI red. However, it has more requirements as well. The key one being that you must use FreeBSD to modify the image.
First download the workon.sh script from www.SmallWall.org/downloads/workon.sh.tar.gz
Now you will need to install FreeBSD 8 on an older system or in a VM. www.freebsd.org/
You will need a copy of the SmallWall image you want to modify, and both it and workon.sh will need to be in the same directory.
As root your workon.sh generic-pc-1.8.2.img (or whatever is appropriate) to mount the compressed images. It will exit into a bash shell and you will see mnt1 and mnt2 directories. The mnt1 directory is the image that is written to the hard drive. (Or CF card, or USB stick, or DOM) It contains the msfroot and the working config.xml file. The mnt2 directory is the uncompressed msfroot, and is the filesystem that is loaded in memory when SmallWall is running.
-
Some special locations in the file system are here.
mnt2/conf.default/config.xml -> Default Config mnt1/conf/config.xml -> Current Config mnt2/etc/version.buildtime -> Image Build time mnt2/etc/version -> Image Version (Change 1.8.2 to 1.8.2-special) mnt2/usr/local/www -> Web Files mnt2/usr/local/www/fbegin.inc -> Header mnt2/usr/local/www/fend.inc -> Footer mnt2/usr/local/www/gui.css -> Color defaults
Once you have edited all of the files you want (And added a notation to the version so you can identify it) type "exit" to leave the bash shell, and allow workon.sh to decompress all of the now modified images. Needless to say, a reboot, crash, or killing of the shell without a clean exit will result in a broken image. Also, your modified image will no longer be signed.
Your new image can now be uploaded to an existing SmallWall installation, or written to a hard drive.
This is the oldest section of the hacking guide. It is also the only way to fully access the filesystem of a running firewall. This is because the filesystem is on another computer that the firewall pxe boots from. Setting up a firewall for pxe boot, and setting up the tftp server is outside the scope of this document, and let to the reader.
Note
This section is woefully out of date, and need a lot of correction. It was compiled before the buildscripts automated much of the SmallWall image building. It will continue to refer to "m0n0wall" until it is updated.In order to get m0n0wall to build we have to compile the kernel for m0n0wall using the kernel config file as found on http://m0n0.ch/wall/downloads. Place this config file (M0N0WALL_NET45XX) in /usr/src/sys/i386/conf. Now build the kernel:
cd /usr/src/sys/i386/conf; config M0N0WALL_NET45XX cd /usr/src/sys/compile/M0N0WALL_NET45XX; make depend all strip kernel strip --remove-section=.note --remove-section=.comment kernel gzip -9v kernel
Copy the kernel to /tftpboot:
cp kernel.gz /tftpboot
cd /usr/src/sys/compile/M0N0WALL_NET45XX; make modules
Then, move the needed modules to the modules directory in the m0n0wall root filesystem. In the pb8 version of m0n0wall the following modules are needed:
dummynet.ko ipfw.ko
These newly-built modules can be found in /usr/src/sys/compile/M0N0WALL_NET45XX/modules/usr/src/sys/modules. modules directory).
Fetch the root filesystem tar file from the m0n0wall web site to a directory, uncompress and untar. The contents of this directory will be in the root of the target system later on. In this just created directory you will be making the changes you like. As we will not not have mounted compact flash card on-line (under /cf), (you could, just put it in, but make sure it boots from the net instead of the flash) we will have to relocate the (default) config file in the root directory:
mkdir cf/conf ; cp conf.default/config.xml cf/conf
Now make a tarfile again to be put onto the to imagefile:
tar cfz ./rootfs.tgz <path to your rootfs-dir>
Now, you can create an imagefile (mfsroot) from this rootfilesystem. This imagefile has to be put into /tftpboot to be downloaded during boot.
dd if=/dev/zero of=./mfsroot.bin bs=1k count=10240 vnconfig -s labels -c vn0 ./mfsroot.bin disklabel -rw vn0 auto newfs -b 8192 -f 1024 /dev/vn0c
Now mount this file as device and copy the m0n0wall root filesystem in:
mount /dev/vn0c /mnt cd /mnt tar xfzP rootfs.tgz cd / umount /mnt vnconfig -u vn0
Now your file mfsroot.bin file is the rootfilesystem image. When this image is put into /tftpboot it will be loaded and unpacked in memory once the kernel boots.
mv mfsroot.bin /tftpboot
Another way to get the kernel.gz file without compiling is extracting it from the net45xx-pbxrxxx.bin.gz image. To do just that, uncompress the image file and mount it as device under /mnt.
The net45xx-pbxrxxx.img files have also to be uncompressed first (check with file < filename >) . Just append a .gz at the filename and gzip -d the resulting file.
gzip -d net45xx-pbxrxxx.bin.gz vnconfig -s labels -c vn0 ./net45xx-pbxrxxx.bin mount /dev/vn0a /mnt cp /mnt/kernel.gz /tftpboot umount /mnt vnconfig -u vn0
The root file system is also in the abovementioned image as the file mfsroot.gz. You can use this file to reconstruct the root file system by uncompressing and mounting it as device /dev/vx0c under /mnt.
gzip -d mvfsroot.gz vnconfig -s labels -c vn0 ./mfsroot mount /dev/vn0c /mnt cd /mnt tar cvf /tmp/mfs.tgz . umount /mnt vnconfig -u vn0 cd tar xvfzP /tmp/mfs.tgz
The bootloader has to be available in the /tftpboot directory and has to be configured to load kernel.gz and the mfsroot.bin file. To do that make the following changes to the loader and configure pxeboot: create the following files: loader.conf:
rootfs_load="YES" rootfs_name="mfsroot.bin" rootfs_type="mfs_root" autoboot_delay=1
loader.rc:
include /boot/loader.4th start
and populate the /tftpboot directory:
mkdir -m 0755 -p /tftpboot/boot/defaults cp -p /boot/loader /tftpboot/boot/ cp -p /boot/*.4th /tftpboot/boot/ cp -p /boot/defaults/loader.conf /tftpboot/boot/defaults/ cp -p loader.conf loader.rc /tftpboot/boot/ chown -R root:wheel /tftpboot
Now boot the stuff....
Remember to turn on dhcp (if needed):
/usr/local/sbin/dhcpd
Now you can test you m0n0wall system. If you edit / cange something in the root filesystem, or build a new kernel, do not forget to update your mfsroot.bin or kernel.gz file in the /tftpboot directory. Also remember that you have a virtual read-only memory filesystem, (nothing will be written back to the mfsroot.bin file on the host) and no flash, so changes in configuration will not be stored.
The next thing is, if you are really confident with your system to create a new image that you can use to upgrade tha m0n0wall flash. The best way to do that is build an image like Manual has on his web site, so you can update the m0n0wall using the GUI tool. First reconstruct the root filesystem to its initial state with respect to the link and the location of the config file: (not really needed, the CF card will be mounted "over" the /cf directory even when not empty) but to keep everything as clean as possible you might do that
rm -rf cf/conf
Creating a flash image works about the same way as creating a rootfs file, but we will need a disklabel that suits the flash card. After creating a file device, we will be putting the kernel.gz file, the rootfilesystem file, the conf directory and the default configuration on the card (file). Also needed is the /boot directory, containing the boot loader files A suitable disklabel (put it in the label.proto file) might be:
# /dev/vn0c: type: unknown disk: amnesiac label: flags: bytes/sector: 512 sectors/track: 32 tracks/cylinder: 64 sectors/cylinder: 2048 cylinders: 5 sectors/unit: 10240 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # milliseconds track-to-track seek: 0 # milliseconds drivedata: 0 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 10240 0 4.2BSD 1024 8192 26 # (Cyl. 0 - 4) c: 10240 0 unused 0 0 # (Cyl. 0 - 4)
First you might compress the rootfilesystem to save space:
gzip -9 mfsroot.bin; mv mfsroot.bin mfsroot.gz
Create the boot directory for inclusion on the flash image, and populate it with the appropriate files:
mkdir -m 0755 -p boot; cd boot cp /boot/boot? . cp /boot/loader . cp /boot/loader.help . cp /boot/loader.4th . cp /boot/mbr . cp /boot/support.4th . mkdir -m 0755 -p defaults cp /boot/defaults/loader.conf defaults
Now create the custom files for the loader:
loader.conf:
kernfs_load="NO" # Kernel filesystem
loader.rc:
load /kernel load -t mfs_root /mfsroot autoboot 0
Now you might start building the actual memory filesystem image ....
dd if=/dev/zero of=image.bin bs=1k count=5120 vnconfig -s labels -c vn0 image.bin disklabel -BR vn0 label.proto newfs -b 8192 -f 1024 /dev/vn0a mount /dev/vn0a /mnt cp -Rp boot /mnt cp -p mfsroot.bin kernel.gz /mnt mkdir /mnt/conf cp -p /conf.default/config.xml /mnt/conf umount /mnt vnconfig -u vn0 gzip -9 image.bin
Now your new-and-improved-with-your-most-wanted-feature m0n0wall image is ready to be loaded !!
To submit any changes you've made to m0n0wall, post them to the forum or email them to Lee directly.
It's best to post them to the forum, as that way if they do not get included in the base m0n0wall for some reason, they are available for others to use who might find them helpful. It also can allow others to show enough interest that it ends up being included. (And prevents things from being lost in e-mail)
Over the years, many talanted people have been involved with m0n0wall development, and several have written additional documentation for m0n0wall development which are beyond the scope of this manual, or which is out of date and no longer applies. However, a lot of this documentation can give a greater understanding of how small FreeBSD based appliances (Such as SmallWall, t1t1wall, pfSense, OPNsense, nas4free and freeenas) work. This chapter provides a reference to some of those sources to help you when you find yourself in a situation not covered in detail in this manual.
An Introduction to m0n0wall Development - Michael Iedema
Custom m0n0wall images howto - Jean-Francois Theroux (Link is broken. If you have a copy, let me know!)
m0n0wall Documentation Project - Chris Buechler
If you have written something on SmallWall, or other FreeBSD appliance based development, please email Lee Sharp to have it listed here.
Table of Contents
- 4.1. I don't have a FreeBSD box to develop new features for SmallWall, is there an easier way?
- 4.2. What patches have been applied to the FreeBSD source that is used in SmallWall, and are they available?
- 4.3. Where can I find the SmallWall source code?
- 4.4. Is there any information available on the unsupported extensions support?
This chapter contains development-related FAQ's. For non-development related FAQ's, see the main SmallWall FAQ.
You can use SmallWall's built in features to test newly developed features. With this method you are not able to compile new binaries on the box, but are able to add php pages into the environment, as well as pre-compiled binaries.
Develop the page
Open a browser and point it to http://SmallWall-ip/exec.php or use https if you have enabled it. It is generally easiest if you simply log into your firewall, and then append exec.php to the end of the url.
Upload the page and any binaries needed for it to function properly via exec.php.
-
Execute the following to place php pages into the www root.
mv /tmp/*.php /usr/local/www
If you need any binaries for the page to work, copy them into the appropriate places.
-
Use chmod to make your pages readable and executable by the web server. For example, to make all your web pages read and execute;
chmod 755 /usr/local/www/*
Type in the address of your new page to try out your experiment.
Using this method, the image will revert after a reboot in case
you messed something up. You may wish to write a script to move everything
into its proper place and upload that along with your pages / binaries if
the feature requires many files. Then simply execute the script in
exec.php. After you have successfully tested the feature, and are happy with
the result, submit your work to the forums at smallwall.freeforums.net or
email <lee@smallwall.org> for it to be added to the project.
The best way to know what is current is to look in the svn at the patches directory svn.smallwall.org/SmallWall/freebsd8/build/patches/
to see what is currently patched.In the past, the following patches have been applied to the FreeBSD kernel source that was used to build early m0n0wall releases:
clock.c: fix writing the day of week back to the RTC (Soekris BIOS resets the date to 1-1-1970 if it's invalid - 0 is not a valid weekday as per the AT specification)
if_ethersubr.c: disable multicast warning in bridge code
if_xl.c: disable hardware TX checksumming
ip_input.c: fix problem with packets from dummynet pipes getting NATed again
ip_nat.c: fix ipfilter bug (only required for 3.4.31)
ip_output.c: reverse ipfilter/ipfw processing order
ng_pptpgre.c: reduce ACK timeout to 1 second, disable PPTP windowing
subr_diskslice.c: disable warning about partition size
They are available to download from http://m0n0.ch/wall/downloads/kernel-patches.tgz.
The SmallWall source code can be found in the svn repository at svn.SmallWall.org. Full build scripts are also available.
Support for extensions was added in m0n0wall 1.1b1, and while it is still there, I can not recall it ever being used. Here is how it was supposed to work.
During the final stage of the boot phase, the boot scripts check for the existence of the directory /etc/inc/ext. If it is found, the boot scripts check all subdirectories of that directory for a file or files starting with "rc" and execute each one.
-
The webGUI checks for the existence of the directory /usr/local/www/ext - if it is populated, a new "Extensions" section is created in the navigation bar, and any files named "menu.inc" that are found in subdirectories of /usr/local/www/ext are included in the navigation bar just below the "Extensions" heading (by means of a PHP include() call). Note: webGUI PHP files (except .inc files) must be marked executable, saved in UNIX format (LF line breaks) and have the following string as their first line:
#!/usr/local/bin/php
-
See these posts on the old m0n0wall list as well.
This, or course, a totally unsupported feature of SmallWall which was last developed as of m0n0wall version 1.1.