28 February 2023

Better Logging for the Best Sleep

I enjoy my sleep. An uninterrupted night after completing a coding project and having it deployed into a production environment is some of the best sleep I can have.

Being woken in the middle of the night because my code is failing in production and the support team not knowing the cause is something I have learned over the year to try to address long before my head hits the pillow. My logging should help them understand if the problem is really bad code or really something else such as a network problem or bad data. Well constructed logging can point the spotlight on the right place to look.

I was going through logs of some automation software at work trying to find a flow to the process. I will admit I did not know the package that well, so I was trying to gain an understanding reading through the logs. I've found using an application's logs to be helpful to understanding how it works without taking a class on the software. This got me thinking about how I log my own code. I've been writing log statements for years, but I know I could always improve my own logging skills. The following aspects are a few things I've learned over the years.

Logging is communication

Relationships are always better with good communication. The clearer the communication, the better the relationship. Assumptions can be made when it comes to believing what the code is doing, and we all know what is said about assumptions. In fact, the logical flow one expects might actually be part of the problem which logging could reveal. Being explicit when logging to remove any guesswork will save so much time when troubleshooting. Log clearly and concisely. Get to the point.

Understand the levels

Logging generally has about five levels: Debug, Info, Warning, Error, and Critical. And most loggers allow for customizing these levels or even adding levels among these defaults. Knowing what goes into a general flow of knowledge when reading logs and what belongs at the debug level should be obvious, but it might take a little time to make those judgment calls. At the Info level, I normally want to give more of an overall stepping through the code at points which mark functionality so that a first pass of the logs can help me at least see where in the code it is failing. I will throw in more variables and values into my Debug level statements when I need to see beyond the obvious logic. Warnings are for giving notice of things to not do or things which could be done better. Errors should be obvious as are Critical responses.

Logging is documentation

To be clear, logging does not replace documentation. It is a form of documentation. It goes hand-in-hand with documentation. Instead of explaining how the code functions, it shows how the process flows. It's a real-time walkthrough of the logical execution of the code. It can tell the story of what the program is doing while you watch it play out before you. If you consider it a part of the of the documentation, how you construct the statements might vary a little from what might seem obvious when in the middle of coding.

Another idea is to combine the documentation with the logging. Sometimes it is better to put longer explanations and suggestions for solving in the actual documentation and use custom codes in the logging to direct the support team to text that will give them a better understanding of the causes and outline the solutions to take.

Write for a larger audience

Sawing logs is an expression defining heavy sleeping. But when someone in a support capacity who does not have an intimate knowledge of the code, what they have in the logs can be enough for them to know what is wrong or leave them confused even a little bit which will prompt a next level call to the development team and possibly the developer.

My approach to logging has always been to write it so that no one needs to call me in the middle of the night because they have reached a point where it cannot be solved by them. In most cases, other pieces in the system could be the cause and easily determined with clear logging. When certain common problems have been experiences, adjusting the logs to better pinpoint the place to check and even suggest a course of action can help keep notifications at bay.

Keep it safe

Logging is generally put on a protected file system (or it should be) and/or ingested by a secure logging service, but logging credentials is generally unnecessary. Login ids should be enough to know which credentials are being used. Sometimes connection strings logged as a lump of information. Measures should be taken to mask passwords in those blocks of strings. I've seen functions built to parse that output before it's put out to mask the sensitive information. Knowing again the audience who will view the logs should also be considered for this.

I'm sure more could be said about logging. Clear and thorough logging can keep you sleeping when your code is being reviewed by a support team. Support teams can do their jobs more efficiently and effectively with skilled logging. Developers who will manage projects you leave behind will herald you wise. It should be every developer's goal to become a master at logging.