FreeBSD tech

Create a rescue USB on FreeBSD

In this post, I document how to create a bootable rescue USB stick on FreeBSD. Having such a stick is handy in case you want to quickly inspect a broken or suspect system.

There are many ways to Create a rescue USB on FreeBSD. One can also use the FREEBSD official installer to choose to get into a live shell.

I use mfsBSD as the contents of this rescue disk. Normally, one can directly use mfsBSD as a rescue risk but below I specifically create an MBR based USB image in order to boot on those older hardware. Many a time, older hardware is unable to boot using GPT.


Preparing the device

  1. Download the mfsBSD ISO of interest. In this case, it is mfsbsd-13.0-RELEASE-amd64.iso
  2. Have the USB stick ready. In this example we use an archaic 256MB stick
  3. Ensure that the USB stick does not have any data. This example assumes that it doesn’t have any partitions as well.
  4. Caution: Use the right device id. Identify the device id – you can find this after plugging in the USB stick and checking dmesg. In this example, we use da1
  5. Setup the device partitions on da1. We keep a second partition which can be used as a writable scratch area which would be useful during recovery.
# gpart create -s MBR da1
da1 created

# gpart add -t freebsd da1
da1s1 added

# gpart create -s BSD da1s1
da1s1 created

# gpart add -t freebsd-ufs -s 150M da1s1
da1s1a added

# gpart add -t freebsd-ufs da1s1
da1s1b added

# gpart bootcode -b /boot/boot0 da1
gpart bootcode -b /boot/boot0 da1

# newfs /dev/da1s1a
/dev/da1s1a: 150.0MB (307200 sectors) block size 32768, fragment size 4096
       using 4 cylinder groups of 37.53MB, 1201 blks, 4864 inodes.
super-block backups (for fsck_ffs -b #) at:
 192, 77056, 153920, 230784

# newfs /dev/da1s1b
/dev/da1s1b: 95.0MB (194520 sectors) block size 32768, fragment size 4096
        using 4 cylinder groups of 23.75MB, 760 blks, 3072 inodes.
super-block backups (for fsck_ffs -b #) at:
 192, 48832, 97472, 146112

Setting up the rescue stick

# mdconfig -a -f mfsbsd-13.0-RELEASE-amd64.iso
# mount -t cd9660 /dev/md0 /mnt

# mkdir /mnt2

# mount /dev/da1s1a /mnt2

# cd /mnt

# find . -print -depth| cpio -dump /mnt2
177244 blocks

Cleaning up

So, we have now copied all the files from the mfsBSD ISO file to the USB stick root partition. Now it is time to cleanup and then we are ready to use our freshly created rescue stick.

# cd

# umount /mnt2

# umount /mnt

# mdconfig -d -u md0

# rmdir /mnt2

And that’s it. You now know how to create a rescue USB on FreeBSD.

tech time

Serving Time on the Internet

One of the top things taken for granted is Time on the internet. Think about it. We have millions, billions, gazillions of devices – servers in data centers running the Googles and the Facebooks, laptops etc. needing to have the right time.

Why? So that financial transactions have accurate timestamps, or that you can see the right time when you want to. The needed accuracy depends on the nature of the application – be in a few sec, few ms or sub-ms. How does all of this work seamlessly in the internet?

Almost all of the internet synchronizes using the Network Time Protocol (NTP). NTP is a protocol or a method for computer clocks to synchronize over a network – including the internet. NTP is one of the oldest protocols active on the internet. Implementations of NTP started coming into use well before 1985 when RFC 958 was published. NTPv4 was published in 2010 with RFC 7822 bringing in NTPv4 field extensions as recently as 2016.

About NTP

In the NTP protocol, there is a client and a server. The client is the one seeking time, the server serves it.

How does it work? In simple terms, the client sends a request to the server asking for the time. The client believes that the server has a more accurate time. The request contains the client side timestamp. The server receives the request, and responds back with the timestamp on the server side. The client factors in the delay for packet transit and finally computes the offset (difference between the servers’ time and its own). This allows the client to optionally adjust its own clock. The client polls the server periodically – but not too frequently so that the server can cater to a large number of clients if it would like to do so.

OK, I admit the above is an oversimplification; but the intent of this write-up is not to explain the protocol.

When we talk about time synchronization on the internet, we have many servers providing time using NTP to hundreds or thousands of clients on a regular basis. These are run on a voluntary basis by government bodies, corporations or organizations and individuals. is the home of the Network Time Protocol project. The NTP Pool project is one which enables and provides the framework of NTP pools.

What is a NTP pool? The NTP pool project creates a pools of servers and groups them in geographical zones (with country as the lowest granularity). As an example, if a client from Japan requests for time from the NTP Japan pool, the client is assigned one of the servers who are participating in the Japan pool. This allows clients to be connected with servers that are in their proximity thereby allowing for faster synchronization.

State of affairs

Number of NTP servers across zones
Number of servers across zones

As can be seen in the image above, certain areas like Europe have good participation in the pool whereas Asia, South America have extremely poor participation.

If you have a server on the internet with a static IP address, the best way to help is to join the pool. You can join the pool by following the instructions here.

NTP servers

Running an NTP server on the internet does not require a larger amount of maintenance. Some things to sort out before you embark as a timekeeper on the internet.

  • Selecting the server software – ntpd, ntpsec, chrony etc. There are slight differences between each of these though the overall concept and configuration is the same.
  • Ensuring your server has a static address – one that is fixed for the long term (think years!)
  • Ensuring you have the right connectivity (bandwidth & a stable connection)
  • In terms of hardware resources – both CPU and memory, the ask of an NTP server is minimal. However if you are handling a tens of thousands of clients, CPU starts to become a consideration
  • Which hardware – there are different opinions about virtual machines vs dedicated hardware – virtual machines can very much hold their own in the NTP Pool being extremely stable. You just need to choose a suitable provider

Mental note: At some point, I will put together a brief comparision of ntpd, ntpsec and chrony in their default configuration.

How many sources is good enough?

The NTP Pool project suggests to setup 4-7 servers to synchronize time with. I love this quote.

A man with a watch knows what time it is. A man with two watches is never sure

Unknown, taken from here

Once you configure the 4-7 servers to synchronize with, your NTP server will start polling them to maintain its own time.

After joining the pool, the NTP Pool project starts monitoring your servers time via NTP. Your server is assigned a score based on the offset and anything more than 100ms is not acceptable. An unacceptable score starts degrading your score and eventually you are kicked out of the pool available for clients. If your server recovers, your score gradually improves and once over 10, you are again added into the pool available for clients. You can monitor your server as in the below image.

Offset monitoring in
Offset monitoring in

The burden of serving time

Don’t underestimate the network traffic requirements of running a NTP server. If you are not careful, your bandwidth can easily run into many TB per month. Depending on your bandwidth usage plan for the server, this might result in huge additional costs that you did not factor in. One good set of articles available to check this in detail are here, here and here.

The NTP Pool project allows you to manage your servers and tune the bandwidth you want for your server. This is not an exact setting since the NTP Pool project only connects clients to you (via DNS) but clients can continue to be attached to you from a few hours to days to even years.

Individual clients do not pose a bigger challenge as long as they are using standard NTP client software; but additional challenges come with misbehaving clients who will poll too frequently or pool continuously – even many many times per second. There are ways to address these – using rate limitation rules in your NTP server software or implementing a firewall.

Finally, once you have your server configured and setup exactly how you like, a picture like this will bring a big smile on your face.

Monitoring NTP packets per second on the server
Monitoring NTP packets per second on the server


Running a NTP server is a great project to contribute to the Internet infrastructure as well as to learn about running a production system on the internet. There are a number of considerations before embarking on such a project, but it could be lead towards a very satisfying experience.