Evolution of .NET Configuration

Every software developer imagined or at least might want to imagine they are a pilot in the situation when it comes to a huge project with a large panel of sensors, metrics, and switches using which one can easily configure everything. That is, following our example, the pilot does not have to manually lift the chassis. Both metrics and graphs are a good thing, but in this article, we would like to focus on those switches and buttons that help you set the parameters for your ‘aircraft’.

Everyone knows that configuration is a very important part of software development. Different specialists use different approaches to configure their applications. While it might seem that there is nothing complicated about it, the question is whether it really that simple. In our article, we will use the before-and-after approach to understand the configuration process and find out how various items work, which new features we’ve got available, and how to use them. Those who are not familiar with.NET Core configuration will get the basics, while those who are already knowledgeable in it, will get food for thought and will be able to use new approaches in their daily work.

Before .NET Core configuration

.NET Framework was introduced in 2002, and since it was the time of the XML hype, Microsoft developers wanted it everywhere. As a result, we have got XML configurations that are still there. The main thing here is the ConfigurationManager static class, using which you can get string representations for settings. The configuration itself looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="Title" value=".NET Configuration evo" />
    <add key="MaxPage" value="10" />
  </appSettings>
</configuration>

This way, the task got solved, as the developers got an option for settings that was still better than INI files; it did have its own particular features, though. For instance, to support different setting values of various application environment types, one has to use XSLT transformations of the configuration file. You can still define your own XML schemas for elements and attributes if you want more complex data grouping. The only type for the key-value pairs is a string, so in case you wanted it number or date, you’ve got to invent a workaround:

string title = ConfigurationManager.AppSettings["Title"];
int maxPage = int.Parse(ConfigurationManager.AppSettings["MaxPage"]);

In 2005, Microsoft added configuration sections that enabled grouping parameters, building your own schemes, and avoiding naming conflicts. Besides, we got .settings files and a dedicated designer for them.

As a result, the developer got an option to have a generated strongly typed class representing configuration data. The designer enables conveniently editing the values with an option to sort by editor columns. The data is retrieved through the Default property of the generated class, which provides the Singleton configuration object.

DateTime date = Properties.Settings.Default.CustomDate;
int displayItems = Properties.Settings.Default.MaxDisplayItems;
string name = Properties.Settings.Default.ApplicationName;

Finally, .NET also received areas for configuration parameter values. The User field, for instance, is intended for user data, which can be changed by the user and saved while the app is running. Data is saved in a separate file at %AppData%\*Application name*. Meanwhile, the Application field enables retrieving parameter values without the user’s being able to change them.

Despite the good intentions, all these updates only complicated matters:

• In fact, it is all about the same XML files whose size increased faster, which means they became inconvenient to read.
• The configuration is read from the XML file only once, so you need to reboot the application in order to change the configuration data.
• Classes generated from .settings files are marked as Sealed, which is why this class cannot be inherited. In addition, this file can be changed, but in case anything gets generated again, all previous data get lost.
• You are limited to working with data only on a key-value basis. To get a structured approach to working with configuration, you need to implement additional features yourself.
• The data source can only be a file, external providers are not supported.
• There is also a human factor: private parameters can get into the version control system and become disclosed.

All of these issues are here to stay in.NET Framework up to now.

Configuration in .NET Core

In .NET Core, configuration got redesigned and everything was made from scratch. With the ConfigurationManager static class removed, many other issues were resolved, too. As before, there were stages of forming the configuration data and consuming them, but with a more flexible and extended life cycle.

Customization and adding configuration data

In .NET Core, one can use multiple sources for data generation, not only files. To customize configuration, one uses IConfgurationBuilder, the basis to which one can add data sources. There are NuGet packages available for various types of sources:

Each source is added as a new layer and redefines the parameters with matching keys. Here is an example of the Program.cs file that comes by default with the ASP.NET Core Application Template (version 3.1):

public static IHostBuilder CreateHostBuilder(string[] args) => 
    Host.CreateDefaultBuilder(args).ConfigureWebHostDefaults(webBuilder => 
        { webBuilder.UseStartup<Startup>(); });

The main thing here is the CreateDefaultBuilder method, which shows the initial configuration of sources:

public static IWebHostBuilder CreateDefaultBuilder(string[] args)
{
    var builder = new WebHostBuilder();
    ...
    builder.ConfigureAppConfiguration((hostingContext, config) =>
    {
        IHostingEnvironment env = hostingContext.HostingEnvironment;
        config.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
              .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: true);
        if (env.IsDevelopment())
        {
            Assembly appAssembly = Assembly.Load(new AssemblyName(env.ApplicationName));
            if (appAssembly != null)
            {
                config.AddUserSecrets(appAssembly, optional: true);
            }
        }
        config.AddEnvironmentVariables();
        if (args != null)
        {
            config.AddCommandLine(args);
        }
    })
            
    ...
    return builder;
}

Thus, the appsettings.json file serves as a base for the overall configuration. If there is a file for a specific environment, it will have a higher priority, and thereby redefine the matching values of the base file. The same is true for each subsequent source. The final value depends on what is added first and what next, like on this chart:

If you want to change the order, you can simply clear the existing one and define your own:

Host.CreateDefaultBuilder(args)
    .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); })
    .ConfigureAppConfiguration((context,
                                builder) =>
     {
         builder.Sources.Clear();
         
         //Custom source order
     });

Each configuration source consists of two parts:

• Implementing IConfigurationSource, which provides a source for configuration values.
• Implementing IConfigurationProvider that converts the original data to the resulting key-value pair.

By implementing these components, you can get your own data source for configuration. You can learn how to get parameters from a database through the Entity Framework here.

How to use and retrieve data

Now that we explained how to customize and add data, let’s see how you can use these data and retrieve them in a more convenient manner. The new approach to configuring projects is very much centered around the popular JSON format. This makes sense, as it can be used to build any data structures, group data, and have a readable file at the same time. Let’s use the following configuration file as an example:

{
  "Features" : {
    "Dashboard" : {
      "Title" : "Default dashboard",
      "EnableCurrencyRates" : true
    },
    "Monitoring" : {
      "EnableRPSLog" : false,
      "EnableStorageStatistic" : true,
      "StartTime": "09:00"
    }
  }
}

All data form a flat key-value dictionary, while the configuration key is formed from the entire file key hierarchy for each value. Such a structure will have the following data set:

To get the value, you can use the IСonfiguration object; for instance, here is how you can get parameters:

string title = Configuration["Features:Dashboard:Title"];
string title1 = Configuration.GetValue<string>("Features:Dashboard:Title");
bool currencyRates = Configuration.GetValue<bool>("Features:Dashboard:EnableCurrencyRates");
bool enableRPSLog = Configuration.GetValue<bool>("Features:Monitoring:EnableRPSLog");
bool enableStorageStatistic = Configuration.GetValue<bool>("Features:Monitoring:EnableStorageStatistic");
TimeSpan startTime = Configuration.GetValue<TimeSpan>("Features:Monitoring:StartTime");

This is already fine, as you get a convenient way to retrieve data of a required type; at the same time, it could still be much better. If you retrieve data as in the above example, you will end up with repetitive code and errors in the key names. Instead of individual values, however, you can have an entire configuration object through binding data using the Bind method. Here is an example of a class and how to retrieve data:

public class MonitoringConfig
{
    public bool EnableRPSLog { get; set; }
    public bool EnableStorageStatistic { get; set; }
    public TimeSpan StartTime { get; set; }
}
var monitorConfiguration = new MonitoringConfig();
Configuration.Bind("Features:Monitoring", monitorConfiguration);
var monitorConfiguration1 = new MonitoringConfig();
IConfigurationSection configurationSection = Configuration.GetSection("Features:Monitoring");
configurationSection.Bind(monitorConfiguration1);

In the first case, the section is bound by name, while in the second one, you retrieve a section first and then bind the data from it. The section allows you to work with the configuration partial view, which means you can control the data set you are working with. Sections are also used in standard extension methods; for instance, the ConnectionStrings section is used to get a connection string:

string connectionString = Configuration.GetConnectionString("Default");
public static string GetConnectionString(this IConfiguration configuration, string name)
{
    return configuration?.GetSection("ConnectionStrings")?[name];
}

Options: typed configuration view

Creating a configuration object manually and binding it to the data is not feasible; instead, you should use Options to get a typed view of a configuration. The view class must be public with a constructor without parameters and public properties for assigning a value, while the object gets filled through reflection. You can view the source code to learn more details.

To start using Options, you need to register the configuration type through the Configure extension method for IserviceCollection, specifying the section that will be projected onto the class:

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();
    services.Configure<MonitoringConfig>(Configuration.GetSection("Features:Monitoring"));
}

After that, you can get configuration by injecting a dependency to the IOptions, IOptionsMonitor, and IOptionsSnapshot interfaces. To get the MonitoringConfig object from the IOptions interface, use the Value property:

public class ExampleService
{
    private IOptions<MonitoringConfig> _configuration;
    public ExampleService(IOptions<MonitoringConfig> configuration)
    {
        _configuration = configuration;
    }
    public void Run()
    {
        TimeSpan timeSpan = _configuration.Value.StartTime; // 09:00
    }
}

The IOptions interface has a special feature: in the dependency injection container, the configuration is registered as an object with the Singleton life cycle. When a value is first requested by the Value property, an object is initialized with data that exists as long as this object does. IOptions does not support updating data; for that option, you can use the IOptionsSnapshot and IOptionsMonitor interfaces.

IOptionsSnapshot is registered with the Scoped life cycle in the dependency injection container, which enables getting a new configuration object when responding to a request with a new container field. For example, a certain web request will return the same object, but a new one will return a new object with updated data.

Meanwhile, IOptionsMonitor is registered as Singleton, with the only difference that each configuration is retrieved with the actual data at the time of the request. In addition, IOptionsMonitor allows you to register a configuration change event handler if you need a response to the data change event.

public class ExampleService
{
    private IOptionsMonitor<MonitoringConfig> _configuration;
    public ExampleService(IOptionsMonitor<MonitoringConfig> configuration)
    {
        _configuration = configuration;
        configuration.OnChange(config =>
        {
            Console.WriteLine("Configuration has changed");
        });
    }
    
    public void Run()
    {
        TimeSpan timeSpan = _configuration.CurrentValue.StartTime; // 09:00
    }
}

You can also get IOptionsSnapshot and IOptionsMonitor by name, which is handy in case you have multiple configuration sections belonging to a single class, and you want to retrieve a specific one. For example, let’s assume we have the following data:

{
  "Cache": {
    "Main": {
      "Type": "global",
      "Interval": "10:00"
    },
    "Partial": {
      "Type": "personal",
      "Interval": "01:30"
    }
  }
}

Here is the type to use for projection:

public class CachePolicy
{
    public string Type { get; set; }
    public TimeSpan Interval { get; set; }
}

The configuration gets registered with a specific name:

services.Configure<CachePolicy>("Main", Configuration.GetSection("Cache:Main"));
services.Configure<CachePolicy>("Partial", Configuration.GetSection("Cache:Partial"));

The values are retrieved as follows:

public class ExampleService
{
    public ExampleService(IOptionsSnapshot<CachePolicy> configuration)
    {
        CachePolicy main = configuration.Get("Main");
        TimeSpan mainInterval = main.Interval; // 10:00
            
        CachePolicy partial = configuration.Get("Partial");
        TimeSpan partialInterval = partial.Interval; // 01:30
    }
}

The source of the extension method being used to register the configuration type will show that the default name is Options.Default, which is an empty string. This means that, implicitly, we always have a name for any configuration:

public static IServiceCollection Configure<TOptions>(this IServiceCollection services, IConfiguration config) where TOptions : class
            => services.Configure<TOptions>(Options.Options.DefaultName, config);

Since a configuration can be represented by a class, you can also add parameter value validation by marking up properties through the validation attributes belonging to the System.ComponentModel.DataAnnotations namespace. For example, if you specify that the value for the Type property is required, you also need to indicate that validation is mandatory when registering a configuration. For this purpose, you can use the ValidateDataAnnotations extension method:

public class CachePolicy
{
    [Required]
    public string Type { get; set; }
    public TimeSpan Interval { get; set; }
}
services.AddOptions<CachePolicy>()
        .Bind(Configuration.GetSection("Cache:Main"))
        .ValidateDataAnnotations();

What is special about such validation is that it will happen only at the moment of getting the configuration object. This makes it difficult to understand that the configuration is not valid when the application starts. There is a task on GitHub to solve this issue. One of the solutions includes using the approach described here: Adding validation to strongly typed configuration objects in ASP.NET Core.

Disadvantages of options and how to bypass them

Working with configuration through Options also has its disadvantages. Thus, you need to add a dependency to use it, and also need to access the Value or CurrentValue property each time to get a value object. You can get a better code by retrieving a configuration object without Options. The easiest solution is an additional registration in the container of a clear configuration type dependency:

services.Configure<MonitoringConfig>(Configuration.GetSection("Features:Monitoring"));
services.AddScoped<MonitoringConfig>(provider => provider.GetRequiredService<IOptionsSnapshot<MonitoringConfig>>().Value);

The is a straightforward solution: the final code may not necessarily ‘know’ IOptions, but you lose flexibility for additional configuration actions if you need such. To solve this issue, you can use the Bridge pattern that will allow you to get an additional layer to perform additional actions before receiving an object.

This task requires refactoring the existing example code. Since the configuration class has a restriction, namely a constructor without parameters, you cannot transfer the IOptions, IOptionsSnapshot, or IOptionsMontitor object to the constructor; otherwise, the configuration reading process will be separated from the final view.

For instance, let’s assume you want to specify the StartTime property of the MonitoringConfig class with a string representation of minutes with a value of 09, which does not fit the standard format:

public class MonitoringConfigReader
{
    public bool EnableRPSLog { get; set; }
    public bool EnableStorageStatistic { get; set; }
    public string StartTime { get; set; }
}
public interface IMonitoringConfig
{
    bool EnableRPSLog { get; }
    bool EnableStorageStatistic { get; }
    TimeSpan StartTime { get; }
}
public class MonitoringConfig : IMonitoringConfig
{
    public MonitoringConfig(IOptionsMonitor<MonitoringConfigReader> option)
    {
        MonitoringConfigReader reader = option.Value;
        
        EnableRPSLog = reader.EnableRPSLog;
        EnableStorageStatistic = reader.EnableStorageStatistic;
        StartTime = GetTimeSpanValue(reader.StartTime);
    }
    
    public bool EnableRPSLog { get; }
    public bool EnableStorageStatistic { get; }
    public TimeSpan StartTime { get; }
    
    private static TimeSpan GetTimeSpanValue(string value) => TimeSpan.ParseExact(value, "mm", CultureInfo.InvariantCulture);
}

To get clear configuration, you need to register it in the dependency injection container:

services.Configure<MonitoringConfigReader>(Configuration.GetSection("Features:Monitoring"));
services.AddTransient<IMonitoringConfig, MonitoringConfig>();

This approach allows you to create a completely separate life cycle for the configuration object. You can also add your own data validation, or additionally implement a data decryption stage if you get them in an encrypted form.

Ensuring data security

Data security is a very important task when it comes to configuration. File configuration is insecure as the data are stored as easy-to-read text; in addition, the files are often in the same directory as the application. Furthermore, one may commit the values to the version control system by mistake, which can disclose the data; this could be a critical issue when it comes to public code. The situation is so common that there is a ready-made tool called Gitleaks that can help find such data leaks. You can also refer to this article to find statistics and the variety of disclosed data.

There are cases when a project must have specific parameters for various environments, such as Release, Debug, etc. A possible solution would be substituting final values through the CI/CD tools; however, this will not protect the data at the design stage.

To protect developers, you can use the User Secrets tool, which is included into the .NET Core SDK package (3.0.100 and higher). Let us see how it works. First, you need to initialize your project with the init command:

dotnet user-secrets init

The command adds a UserSecretsId element to the .csproj project file. With this parameter, you get a private storage with a regular JSON file. The difference is that it is not located in your project directory, so it will only be available on the local machine at the following path: %APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json for Windows, and ~/.microsoft/usersecrets/<user_secrets_id>/secrets.json for Linux and MacOS.

Now let us add the value from the example above using the set command:

dotnet user-secrets set "Features:Monitoring:StartTime" "09:00"

You can find the complete list of the available commands in the relevant documentation.

When it comes to production, data security is best ensured with specialized storage, such as AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, Consul, or ZooKeeper. To implement some storages, you will have ready-made NuGet packages, while in other cases, it is easy to implement storages on your own, since you will have access to the REST API.

Conclusion

If you have any issues that have not been around for long, you will need some up-to-date solutions to tackle them. As we are moving away from monoliths to dynamic infrastructures, configuration approaches have changed a lot, too. You can no longer depend on the configuration data source location or type and need to promptly respond to data changes. With .NET Core, you get a good tool to implement all kinds of application configuration scenarios.