tech time

Serving Time on the Internet

One of the top things taken for granted is Time on the internet. Think about it. We have millions, billions, gazillions of devices – servers in data centers running the Googles and the Facebooks, laptops etc. needing to have the right time.

Why? So that financial transactions have accurate timestamps, or that you can see the right time when you want to. The needed accuracy depends on the nature of the application – be in a few sec, few ms or sub-ms. How does all of this work seamlessly in the internet?

Almost all of the internet synchronizes using the Network Time Protocol (NTP). NTP is a protocol or a method for computer clocks to synchronize over a network – including the internet. NTP is one of the oldest protocols active on the internet. Implementations of NTP started coming into use well before 1985 when RFC 958 was published. NTPv4 was published in 2010 with RFC 7822 bringing in NTPv4 field extensions as recently as 2016.

About NTP

In the NTP protocol, there is a client and a server. The client is the one seeking time, the server serves it.

How does it work? In simple terms, the client sends a request to the server asking for the time. The client believes that the server has a more accurate time. The request contains the client side timestamp. The server receives the request, and responds back with the timestamp on the server side. The client factors in the delay for packet transit and finally computes the offset (difference between the servers’ time and its own). This allows the client to optionally adjust its own clock. The client polls the server periodically – but not too frequently so that the server can cater to a large number of clients if it would like to do so.

OK, I admit the above is an oversimplification; but the intent of this write-up is not to explain the protocol.

When we talk about time synchronization on the internet, we have many servers providing time using NTP to hundreds or thousands of clients on a regular basis. These are run on a voluntary basis by government bodies, corporations or organizations and individuals. is the home of the Network Time Protocol project. The NTP Pool project is one which enables and provides the framework of NTP pools.

What is a NTP pool? The NTP pool project creates a pools of servers and groups them in geographical zones (with country as the lowest granularity). As an example, if a client from Japan requests for time from the NTP Japan pool, the client is assigned one of the servers who are participating in the Japan pool. This allows clients to be connected with servers that are in their proximity thereby allowing for faster synchronization.

State of affairs

Number of NTP servers across zones
Number of servers across zones

As can be seen in the image above, certain areas like Europe have good participation in the pool whereas Asia, South America have extremely poor participation.

If you have a server on the internet with a static IP address, the best way to help is to join the pool. You can join the pool by following the instructions here.

NTP servers

Running an NTP server on the internet does not require a larger amount of maintenance. Some things to sort out before you embark as a timekeeper on the internet.

  • Selecting the server software – ntpd, ntpsec, chrony etc. There are slight differences between each of these though the overall concept and configuration is the same.
  • Ensuring your server has a static address – one that is fixed for the long term (think years!)
  • Ensuring you have the right connectivity (bandwidth & a stable connection)
  • In terms of hardware resources – both CPU and memory, the ask of an NTP server is minimal. However if you are handling a tens of thousands of clients, CPU starts to become a consideration
  • Which hardware – there are different opinions about virtual machines vs dedicated hardware – virtual machines can very much hold their own in the NTP Pool being extremely stable. You just need to choose a suitable provider

Mental note: At some point, I will put together a brief comparision of ntpd, ntpsec and chrony in their default configuration.

How many sources is good enough?

The NTP Pool project suggests to setup 4-7 servers to synchronize time with. I love this quote.

A man with a watch knows what time it is. A man with two watches is never sure

Unknown, taken from here

Once you configure the 4-7 servers to synchronize with, your NTP server will start polling them to maintain its own time.

After joining the pool, the NTP Pool project starts monitoring your servers time via NTP. Your server is assigned a score based on the offset and anything more than 100ms is not acceptable. An unacceptable score starts degrading your score and eventually you are kicked out of the pool available for clients. If your server recovers, your score gradually improves and once over 10, you are again added into the pool available for clients. You can monitor your server as in the below image.

Offset monitoring in
Offset monitoring in

The burden of serving time

Don’t underestimate the network traffic requirements of running a NTP server. If you are not careful, your bandwidth can easily run into many TB per month. Depending on your bandwidth usage plan for the server, this might result in huge additional costs that you did not factor in. One good set of articles available to check this in detail are here, here and here.

The NTP Pool project allows you to manage your servers and tune the bandwidth you want for your server. This is not an exact setting since the NTP Pool project only connects clients to you (via DNS) but clients can continue to be attached to you from a few hours to days to even years.

Individual clients do not pose a bigger challenge as long as they are using standard NTP client software; but additional challenges come with misbehaving clients who will poll too frequently or pool continuously – even many many times per second. There are ways to address these – using rate limitation rules in your NTP server software or implementing a firewall.

Finally, once you have your server configured and setup exactly how you like, a picture like this will bring a big smile on your face.

Monitoring NTP packets per second on the server
Monitoring NTP packets per second on the server


Running a NTP server is a great project to contribute to the Internet infrastructure as well as to learn about running a production system on the internet. There are a number of considerations before embarking on such a project, but it could be lead towards a very satisfying experience.



April 2022, the world has been in lockdown for two years.

A rumble of thunder awoke Varun from his sleep. He checked the clock by the bed; it was 6 AM. Sitting up on the bedside, he stretched his arms upwards. The pain in his elbows had subsided – it was like this in the mornings and got worse during the day. He stepped out to the balcony – he loved the view he had from the 15th floor. This morning though, it was raining, and he couldn’t see much.

After brushing his teeth, he looked at the softboard in the kitchen where he had his housework schedule mapped out. Today was a Friday – the following items were listed

Change curtains (last Fri)


Friday party!

Today though was different – today the lockdown was to be lifted. It was difficult to imagine that more than two years had passed since it had started. Varun had taken an off from work today – he thought he needed the day to be able to digest such an event. He had been prepared that the lockdown would be extended the previous evening or by mid-night. After all, it had happened before – twelve times in all.  He spent almost fifteen minutes checking his phone for news on the topic. Nothing indicated otherwise. It was two weeks in a row when the number of new cases had reduced to single digits; and finally, the lockdown was to be lifted: today.

After years of thinking of what to do after the lockdown – his mind was now blank. He had had so many ideas – go out for a run, go for a movie, eat at his favorite Chinese restaurant, hit the pub – but he was now unsure. It was now almost four months since he had stepped out of his apartment. Finally, he made up his mind – he would go to the neighborhood park. He had a cup of coffee and put on his shoes.

He found an umbrella and picked up the keys to the apartment. He hesitated a moment – donned his mask and stepped out.


Randomness: The subconscious impacts of COVID-19

I wonder about the subconscious impacts of the COVID-19 outbreak. There are a few things that make me cringe.

Almost 80% of the people around my locality wear masks when outside. They pull the mask down when they feel like it. And again, when they want to talk to someone. The mask is there for a reason. To protect yourself – and to protect others. Use it.

During my walks in the evening, I notice people who walk right in the center of the walkway. Some of them will wear masks. Why do you want to throw social distancing in the gutter? Walk on the side: people can walk in both directions. Lanes are good – not just for cars.

This has also affected me while watching TV. If there is a scene of a gathering at home or a party, a shot of a beach – it makes me cringe. It takes me a moment to realize, recall the virus and remind myself that I’m watching TV.

We hear phrases like “the new normal”, “live with the virus”. We’ve started versioning the lockdown – currently on 4.0. Will it reached the number 12 that I wrote about in a fictional piece many weeks ago?


Lines #1 – Rocking shore

Boat rocking, stormy sea, water splashing everywhere

Holes appear, water leaking here and there

There is no way out

Far ahead, shore’s seen, hang on, hang on

There is no way out

Above lines are also published in The Writing Gym

My first bake

I tried my hand at the no-knead bread recipe. The highly recommended one is the NYT recipe.

As a personal touch, I added some herbs and cheese to the dough while folding it.

It turned out great. I don’t have pictures of the cut unfortunately! That is for the next time.

No knead bread - the dough, the intermediate and the final look


To write or not to write?

I have attempted to write before. Mostly in private; short-lived private blogs, journals, tech notes. But never took it seriously enough.

Duirng the COVID-19 lockdown, I took up a writing course. I never wanted to be a writer – but I took the course to explore a new dimension, something to keep the mind busy: learning.

What will I write about? My experiments with writing, food, tech and more.

No limits.