Logging is a big part of software development for many years now. One can argue that a logging mechanism is a must-have part of any application or library. I would agree with that statement.
Logging has a crucial part to play in a scenario where you can’t use interactive debugging (that is, attaching a debugger like Visual Studio). It allows us to investigate errors after the problem already happened. In some cases, like Production Debugging, logs might be the only information you have.
Even if you can debug your own process, logs can give you priceless information on other components like 3rd party libraries, the .NET framework itself, and the CLR.
Alongside logging, a new term is becoming popular: Monitoring. Monitoring means there’s an automatic tool that reports information on your application. That information is usually errors (Error Monitoring, Crash Monitoring), but also information on requests and on performance (Performance Monitoring). This is very different from logging where your code actively writes messages and exceptions to log. We’re going to talk just about logging today.
Logging can also be used to gather data and statistics on your users. This data can be used to research usage patterns, demographics, and behavior. Needless to say, this kind of data is priceless in some products.
Logging Target Types
When we say logging, we traditionally mean saving the message to a file. That’s indeed logging, but far from the only type of logging. Here are some common logging targets to consider:
A database. Logging to a database has many advantages
- You can retrieve the logs from anywhere, without access to the production machine.
- It’s easy to aggregate logs from multiple machines.
- There’s no chance the logs will be deleted from a local machine.
- You can easily search and extract statistics from the logs. This is especially useful if you’re using Structured Logging. We’ll talk about that later on.
There’s a myriad of choices for a database to store your logs. We can categorize them as follows:
- Relational Databases are always an option. They’re easy to set up, can be queried with SQL and most engineers are already familiar with them.
- NoSQL Databases like CouchDB. These are perfect for structured logs that are stored in JSON format.
- Time-series Databases like InfluxDB are optimized to store time-based events. This means your logging performance will be better and your logs will take less storage space. This is a good choice for intense high-load logging.
Searchable Solutions like Logstash + Elastic Search + Kibana (The “Elastic Stack”) provide a full service for your logs. They will store, index, add search capabilities and even visualize your logs` data. They work best with structured logging.
Error Monitoring tools can also act as logging targets. This is very beneficial since they can display errors alongside log messages that were in the same context. The context might be the same Http request for example. A couple of choices are Elmah and Azure Application Insights.
Logging to File is still a good logging target. It doesn’t have to be exclusive, you can log both to file and a database for example. For desktop applications, logging to file is very effective. Once a problem has happened, the customer can easily find and send their log files to investigate.
Logging to Standard Output (Console) and Debug Output (Trace) – Logging to Console, also known as Standard Output, is very convenient, especially during development. Windows also supports a similar logging target called Debug Output, which you can log to with
System.Diagnostics.Trace("My message"). What’s nice about both Console and Trace logging is that you can log messages from native code as well, easily achieving a shared logging system. You can view the Debug Output live with a program like DebugView. You can also use the ConsoleTraceListener class to direct this information anywhere, like to a database.Logging to Event Viewer – If your application is on Windows, you can use Windows Event Log to log messages. It’s pretty easy to do and you can view the message with the Event Viewer program. As a bonus, all crashes are automatically added as event logs. So after any .NET process crash, you can enter the Event Viewer and see the Exception and its Call Stack. This is pretty costly in terms of performance, so it’s best to use just for critical notifications, like fatal errors.
Log to ETW – Windows has an extremely fast built-in logging system called Event Tracing for Windows (ETW). You can use it to see logs from the .NET framework, the operating system, and even the kernel. But you can also use ETW for logging yourself with
System.Diagnostics.Tracing.EventSource. This is the fastest logging option available in .NET. If you’ve got a hot path that’s executing 100,000 a second, then ETW might be for you. .NET Core 3.0 Preview 5+ now has an ETW alternative called dotnet-trace which is cross-platform.
Structured Logging Revolution
In traditional logging, we log a simple string message. To that message, we’re usually adding a Timestamp, a log level, and possibly additional context like an Exception.
try{ _log.Debug("About to do something"); // ...}catch (Exception ex){ _log.Error("Doing something failed with", ex);}In Structured Logging we’re also adding structured fields to the message. That is, we’re marking some data as fields and giving it a name. Later, we will be able to search in those fields, filter according to them, and gather data. According to your logging library, a structured log message might look something like this:
var requestInfo = new { Url = "https://myurl.com/data", Payload = 12 };_log.Information("Request info is {@RequestInfo}", requestInfo);When sent to the server, this is saved as JSON rather than a regular string. The implications are huge. Now, we can find all log messages with a certain Payload value. Or filter log messages according to the request URL. We can save consumer data and try and find a correlation with their usage. Perhaps we’ll find that women between 30-35 are more likely to buy shoes during Summer. This means we can suggest more shoes, get more sales and have big Christmas bonuses. All with the power of structured logging.
All popular logging frameworks support custom logging, though I believe Serilog was the first to implement structured logging as a first-class citizen.
Logging Frameworks
There are 4 logging frameworks that pretty much dominate the .NET space. Those are log4net, NLog, Serilog, and Microsoft.Extensions.Logging (only for .NET Core and ASP.NET Core). All of them are great, free, and offer similar functionality.
Let’s talk first of the three community logging frameworks: log4net, NLog, and Serilog.
log4net
Apache log4net is the oldest of the three frameworks. It was originally ported from Java’s log4j project. You’ll find it in most older .NET projects.
To set up, you’ll need to build an XML configuration (log4net.config) file that looks something like this:
<log4net> <appender name="RollingFile" type="log4net.Appender.RollingFileAppender"> <file value="my_log.log" /> <appendToFile value="true" /> <maximumFileSize value="50KB" /> <maxSizeRollBackups value="2" /> <layout type="log4net.Layout.PatternLayout"> <conversionPattern value="%date %level %message%newline" /> </layout> </appender> <root> <level value="ALL" /> <appender-ref ref="RollingFile" /> </root></log4net>Then, add the following to AssemblyInfo.cs:
[assembly: log4net.Config.XmlConfigurator(ConfigFile = "log4net.config")]Then, we can start logging:
class MyClass{ private static readonly log4net.ILog _log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType); public void Foo() { _log.Debug("Foo started"); }}log4net supports structured logging, but not as innately as the others. It supports a bunch of logging targets called appenders. Although it supports fewer targets than others out of the box, log4net is so huge that whatever isn’t supported, you can find a custom implementation on the internet, like this log4net-to-Elasticsearch appender.
log4net’s biggest problem is probably the rather difficult configuration. Let’s see how its competitors handled this.
NLog
NLog appeared second after log4net and gained a lot of popularity.
Like other libraries, NLog starts with a NuGet package. Then, you can configure either in XML like log4net or in code. I’ll show how to do it in code (from documentation):
var config = new NLog.Config.LoggingConfiguration();// Targets where to log to: File and Consolevar logfile = new NLog.Targets.FileTarget("logfile") { FileName = "file.txt" };var logconsole = new NLog.Targets.ConsoleTarget("logconsole"); // Rules for mapping loggers to targets config.AddRule(LogLevel.Info, LogLevel.Fatal, logconsole);config.AddRule(LogLevel.Debug, LogLevel.Fatal, logfile); // Apply config NLog.LogManager.Configuration = config;Now, start logging:
class MyClass{ private static readonly NLog.Logger _log_ = NLog.LogManager.GetCurrentClassLogger(); public void Foo() { _log.Debug("Foo started"); // structured logging: _log.Info("Hello {Name}", "Michael"); }}NLog has an easy setup and API. By several accounts, it’s also faster than log4net. It supports structured logging, and 84 targets out of the box including all the popular databases. Like with any of the libraries, you can extend NLog to write logs wherever you want.
Serilog
Serilog was last to join the party but added a much-needed feature: Structure logging as a first-class citizen. We’ll get back to that.
To use Serilog, first install their NuGet. Then, add the setup in code (see official documentation):
class Program{ static void Main() { Log.Logger = new LoggerConfiguration() .MinimumLevel.Debug() .WriteTo.Console() .WriteTo.File("logs\\my_log.log", rollingInterval: RollingInterval.Day) .CreateLogger();And use:
class MyClass{ public void Foo() { Log.Debug("Foo started"); // structured logging: Log.Debug("Requesting from URL {A} with {B}", myUrl, myPayload); }}So if you thought NLog is as simple as it gets, Serilog was able to make the setup even easier. Serilog also supports a big group of targets (Sinks) out of the box.
As far as performance, Serilog seems to be about twice faster than NLog according to this Benchmark. As far as performance, NLog seems to be faster than Serilog according to these benchmarks.
Thanks to Rolf Kristensen for correcting the incorrect benchmark that showed Serilog as faster than NLog.
Which to choose?
First of all, all 3 frameworks are good and provide a rather similar value. Before answering this question, let’s do some popularity research. Looking at NuGet package statistics of the last 6 weeks (up to 11th Aug 2019) we can see the following:
Serilog is first at 3rd place, NLog is second at 20th and log4net is last at 28th place. A pretty significant margin in favor of Serilog.
Google Trends paints a very different picture:
According to that, NLog and log4net are competing for the first place, and Serilog is 3rd with a significant margin.
Another interesting statistic is the number of questions in StackOverflow. According to the tags, log4net has about 3700 questions, NLog is with about 2000 questions and Serilog with 832.
Support is also a big consideration. Looking at the NuGet packages, we can see that NLog and Serilog packages are released a few times a month. Whereas log4net is released once or twice a year.
Verdict: Due to the more difficult setup, worse support for structured logging, less maintenance and worse performance, I don’t recommend using log4net for new projects. There are exceptions to this rule. For example, you might have a custom appender made that you don’t want to rewrite for another framework.
As for choosing Serilog or NLog, I think both choices are good. Serilog seems to have better support for structured logging, whereas NLog seems to have better performance. Both are popular and well maintained. So I can’t name a clear winner.
Microsoft.Extensions.Logging (aka ASP.NET Core Logging)
.NET Core and ASP.NET Core come with its own built-in logging framework. When you create a new ASP.NET Core project from the default template, Microsoft.Extensions.Logging will already be included in the project. This is partly why this framework is much more popular than the others as seen in nuget Trends.
ASP.NET Core Logging framework is both an abstraction and implementation. It primarily makes sure you can get the ILogger<T> interface in your ASP.NET Core dependency injection system. You’ll be able to do the following in your controllers:
public class MyController : Controller{ private readonly ILogger _logger; public MyController(ILogger<MyController> logger){ logger.LogInformation("Hello world"); _logger = logger; }}Where does the logger write to? This depends on its providers, which you can specify on initialization in Program.cs. Providers are just another name for Logging Targets. Like Sinks in Serilog and Appenders in log4net.
public static IHostBuilder CreateHostBuilder(string[] args) => Host.CreateDefaultBuilder(args) .ConfigureLogging(logging => { logging.ClearProviders(); logging.AddConsole();//Adding Console provider }) .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); });In this case, we added the Console provider, so all log messages will write to Console. Some of the other logging providers by Microsoft are: File, Debug, EventSource, TraceSource, and ApplicationInsights. But you can add any providers yourself.
There are logging providers for all of the big community logging frameworks. So you can use Microsoft.Extensions.Logging to log messages with Serilog, NLog, or log4net. For Serilog, for example, it’s as simple as adding the Serilog.AspNetCore nuget package and adding the following code. First, define Serilog’s logger in Program.cs:
public static void Main(string[] args){ Log.Logger = new LoggerConfiguration() .Enrich.FromLogContext() .WriteTo.Console() .CreateLogger();Now, add it to the Host Builder:
public static IHostBuilder CreateHostBuilder(string[] args) => Host.CreateDefaultBuilder(args) .UseSerilog(); .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); }) Should I use Microsoft.Extensions.Logging in all ASP.NET Core apps?
I found that the newer frameworks, Serilog and NLog, are better than ASP.NET Core’s logging framework. They have more features and better usability. This includes better-structured logging support and better contextual data support. When you integrate Serilog or NLog as a provider in Microsoft.Extensions.Logging, you lose some of those abilities if working with the ILogger interface (although you can work directly with the 3rd-party logger).
Having said that, I’m a big believer in doing what’s popular. Developers will be more familiar with popular technologies, you’ll have more documentation, a bigger community, and more questions on StackOverflow. So I would still choose to go with Microsoft.Extensions.Logging for new projects. It provides most of the functionality you’ll need and the abstraction allows to easily change it to a different framework later on.
Logging Best Practices
Whatever framework and logging targets you choose, there are some common best practices to follow.
1. Use Log Levels appropriately
Every logging frameworks associate by default a logging level to each message. The levels are usually Debug, Info, Warn, Error, Fatal or similar. Those are important to convey the type of information logged. Debug and Info show debug information and contextual information that’s helpful to understand the current flow. Warn indicates something is probably wrong. Error indicates an error has occurred. Usually, we’ll want to log Error messages when we catch exceptions. Fatal usually means a major error has occurred that requires to terminate the application.
2. Enable only high-severity logs in Production
As a rule of thumb, we don’t want to enable Debug and Info levels in production. We want to log just Warn messages and above or just Error and above. The reason is better performance, avoid using more storage and possibly avoid sending sensitive information.
That’s why you should make sure to change your logging configuration when deploying to production. It can be as easy as #IF DEBUG clause in code or a special step to change the configuration file in your CI/CD pipeline.
3. Log Exceptions
For production debugging, logging Exceptions is crucial. It’s usually the most important piece of information we need to solve the bug. That means both handled exceptions and unhandled exceptions. For handled exceptions, make sure to log.Error() in the catch clause. For unhandled exceptions, you can register to a special event that fires when an exception is thrown. Or a middleware in ASP.NET Core, like here.
4. Log Context
To understand production problems, we need context. While the Exception is the most important thing, the Context is the 2nd most important. We need to know the Http Request, current thread, current machine, state, user information, etc.
You can log context with every message, when transitioning, or just on exceptions. All logging libraries support reporting of context. It’s best to use structured logging for context, so you’ll be able to search and filter by it later.
5. Use Structured Logging
Structured Logging became extremely popular for a reason. As applications grow in size, their logs are growing with it. We need more sophisticated tools to find logs, aggregate them according to context and analyze them. Structured logging gives us the means to do just that. Even if you don’t use the benefits of structured logging immediately, consider this to be a long term investment.
6. Redact Sensitive Information
You should be aware that production logs shouldn’t contain sensitive or private information. That might include passwords, credit card numbers, social security numbers, etc. But it also might include private information like preferences and usage patterns. Although, according to your needs, perhaps this is exactly the kind of information you do want to log.
By setting minimum log level to “Warn” or “Error” in production, you are minimizing this problem, but even error logs can contain sensitive information. You can then scrub the data, or avoid sending it altogether.
You can find a very good list of additional best practices in this article by Jason Skowronski.
Summary
In this article, I tried to show a good bird’s eye view on modern logging practices in .NET. My goal was for the reader to make sense of the abundance of available technologies and when to use which. Hopefully, this helped you to make sense of things and now you can research further on a more specific technology.
NOTE: There’s a myriad of logging-related technologies out there. Tech to store logs, query logs and to visualize logs. The technologies described here are just to convey the types of solutions you can find.
