Click to learn more about author Thomas LaRock.
Change is good — that’s what “progress” is. Usually, changes make our apps better and improve our tools. But no one likes it when a bad change hobbles those apps and causes us to waste hours sifting through code to figure out where it all went wrong. And change is even worse when it’s imposed on us by hackers.
Change monitoring is a viable first line of defense — especially now, with many offices still closed or running at reduced capacity because of COVID-19.
When HQ is a kitchen countertop, it’s logical to address matters on an ad-hoc basis. Nobody knows how this script is going to play out — and while we’re all trying to do the right thing, we’re also improvising. Change monitoring can help us become reasonable gatekeepers.
Change monitoring can detect and compare configuration changes to your servers, databases, and applications. It can correlate changes to performance issues (i.e., last night, there was an update, and today everything is running slower) — it’s the first line of defense.
If suddenly, you’re seeing a lot of files hitting a shared server somewhere, it could be remote worker activity. But it could also mean a segment of the network is having more action because your VPN is misconfigured.
Change monitoring logs activity — which then helps you perform the necessary forensics. It can also track service-affecting software and operating system updates. And importantly, it helps you identify breaches.
Suppose we have a complex custom app, and we’re using Kubernetes for continuous deployment. Updates are made continually, but there are occasional hiccups in the service. Your site has angry customers, and you’re losing sales. What’s going wrong? Is there a failure somewhere, or did someone throw a wrench in the works?
You can use change management to perform forensics. Let’s say New Guy Brad made a change he shouldn’t have. Change monitoring can help you discover who made the error and when the error was made. Or maybe Brad pings you while you’re on vacation. No one has made changes, he says, but the site is slow. Go into your change monitoring tool, and you can discover if there was a wayward patch or a system update Brad didn’t know about.
The same principle applies to breaches — that command inserted last night? Keep that from happening by disallowing a change.
Change monitoring can help you identify breaches — or make you aware of installed malware or ransomware. (And remember, malware isn’t always about injecting or installing an executable file. Sometimes it modifies permissions and leaves things alone for a while, so it can come back later.)
As a tool, change monitoring is flexible. You can also use it as an application-centric tool: You can monitor for file attributes. You can use it for version control and for looking at past code revisions, too.
But there’s one catch: Since it can monitor every change to files and systems in your environment, how do you wrangle the information overload?
Cut Through the Noise
Too much information is noise. If you write a document and save it to your server, it becomes a file that is logged. If you modify it, it’s also registered. But if the file is an incidental one — perhaps it’s a document reminding employees to wash their hands — maybe you don’t need up-to-the-minute data on it.
You also need to filter out the extraneous information, the noise. I’ve seen too many cases of alert fatigue — the tech pros were over-alerted, and therefore no one could tell what information was important.
The need to filter out the noise has given rise to tools designed to log every change, but also filter out the extraneous information. We use a server configuration monitor, or SCM, to capture those events.
SCM gives users visibility into configuration changes on servers and applications, while also correlating performance issues to authorized or unauthorized changes to files, binaries, configuration files, the Windows Registry, and hardware configurations.
You can narrow things down when it comes to alerts. You can cut out the noise (the entirely expected humdrum changes) and find the one change that has made life miserable for everyone.
You can set which parts of your environments are no-go zones and which ones are allowable for New Guy Brad to do his work. You can easily see when a software patch was introduced.
Change monitoring bridges the gap between too much information and no information at all. It’s lightweight since you can simply point it at a server and then monitor the registry for changes.
Just Get Started
Get started — but start simple. Point it at your vital database servers. Set up alerts — but don’t over-alert — and create areas you want to protect. If you require something a little more involved, there are tools available.
And remember, if you aren’t practicing change monitoring, then you can’t have an effective change management process. To my way of thinking, companies without a change management process in place are going to fail at some point, perhaps spectacularly.
We’re living in an unsettled time. It is good to have a set of tools to help keep us steady. So, why not give change monitoring a try?