Use AWS S3 as a secure storage for secure .Net Core app data

When it comes to encryption, key storage becomes a major problem for developers. Most of the developers hard cording keys into the source codes or add them in the config file. Both these methods are in-secure in the context of application security. 

One of the best ways to secure your secrets are using a HSM ( Hardware Security Module). These devices are comparatively expensive and need  regular maintenance. Is there any cheaper method to do this ? especially when it comes to cloud hosting. 

The answer is Yes. If you are going for an Azure deployment , there is Azure Vault. Azure vault manages all your keys securely and most of the security features can be encapsulated from the application. 

But today, I am going to explain how we can use the AWS S3 bucket as a secure storage for your keys using the AWS Key Management service in .Net core 2.0 web application. 

To begin with, add following nuget packages which facilitates the AWS SDK functionality. 

AWS SDK Core : Install-Package AWSSDK.Core

AWS SDK Extensions for .Net Core : Install-Package AWSSDK.Extensions.NETCore.Setup -Version 3.3.4

AWS SDK Key management service :  Install-Package AWSSDK.KeyManagementService -Version 3.3.5

AWS SDK S3 : Install-Package AWSSDK.S3 -Version 3.3.17

Next step is to create key pair for a AWS user using the AWS IAM console. Think link provides a comprehensive guide. Make sure you save the generated AWS access keys at the end of the process and keep them securely. You need to grant admin privileges for S3 bucket for this newly created user. 

Create a S3 bucket using the AWS console. 

The next thing you need to do is, add the config entry to the appsettings.json to configure the AWS settings. Following is the entry you need to add

"AWS": {
    "Profile": "default",
    "Region": "ap-south-1",
    "ProfilesLocation": "/config/awsprofile"
  }

You need to persist your user's keys in the awsprofile file which is mentioned in the ProfilesLocation attribute. Make sure that you are keep this file in a secured place in your drive and not the app folder itself. ( This file persist your AWS access keys in plain text format which is not secured. AWS provides best practices how to keep your AWS key file securely in following link) 

ProfilesLcoation file should be like this 

[aunex-aws-profile]
aws_access_key_id=<your access key ID>
aws_secret_access_key=<Your secret access key>

As the next step, you need to load AWSOptions into your application. In windows you do not need to do anything than call the following code segment.

Configuration.GetAWSOptions();

Since this method not working in Linux, I had to manually load the AWS params into application's Environment variables as follows. 

I added the keys in two lines in my key file.

private void LoadAwsConfig(string fileLocation, string region)
        {
            try
            {
                string line;
                int counter = 0;
                System.IO.StreamReader file =
                    new System.IO.StreamReader(fileLocation);
                while ((line = file.ReadLine()) != null)
                {
                    if (counter == 0)
                        Environment.SetEnvironmentVariable("AWS_ACCESS_KEY_ID", line);
                    if (counter == 1)
                        Environment.SetEnvironmentVariable("AWS_SECRET_ACCESS_KEY", line);
                    counter++;
                }
                Environment.SetEnvironmentVariable("AWS_REGION", region);
                file.Close();
            }
            catch{}
        }

Then Add following interface and class which implements date securing and retrieval  functionality. You need to inject this into .net core default IoC container. 

The Interface

public interface ISecureEnclave
    {
        Task<string> ReadValue(string key);

        Task<bool> WriteValue(string key, string value);
    }

The Implementation ( Write secured data into AWS S3) 

public async Task<bool> WriteValue(string key, string value)
        {
            using (var kmsClient = new AmazonKeyManagementServiceClient())
            {
                var response = await kmsClient.CreateKeyAsync(new CreateKeyRequest());
                string kmsKeyID = response.KeyMetadata.KeyId;

                var keyMetadata = response.KeyMetadata; 
                var bucketName = "<your s3 bucket name>";
                var objectKey = key;

                var kmsEncryptionMaterials = new EncryptionMaterials(kmsKeyID);
                var config = new AmazonS3CryptoConfiguration()
                {
                    StorageMode = CryptoStorageMode.ObjectMetadata
                };

                using (var s3Client = new AmazonS3EncryptionClient(config, kmsEncryptionMaterials))
                {
                    var putRequest = new PutObjectRequest
                    {
                        BucketName = bucketName,
                        Key = objectKey,
                        ContentBody = value
                    };
                    var obj = await s3Client.PutObjectAsync(putRequest);
                    if (obj.HttpStatusCode == HttpStatusCode.OK)
                        return true;

                }
            }
            return false;

The Implementation ( Read secured data from AWS S3 based on given key)

public async Task<string> ReadValue(string key)
        {
            using (var kmsClient = new AmazonKeyManagementServiceClient())
            {
                string kmsKeyID = null;
                var response = await kmsClient.CreateKeyAsync(new CreateKeyRequest());
                 kmsKeyID = response.KeyMetadata.KeyId;

                var keyMetadata = response.KeyMetadata; 
                var bucketName = "<your s3 bucket name>";
                var objectKey = key;

                var kmsEncryptionMaterials = new EncryptionMaterials(key);
                var config = new AmazonS3CryptoConfiguration()
                {
                    StorageMode = CryptoStorageMode.ObjectMetadata
                };

                using (var s3Client = new AmazonS3EncryptionClient(config, kmsEncryptionMaterials))
                {
                    var getRequest = new GetObjectRequest
                    {
                        BucketName = bucketName,
                        Key = objectKey
                    };

                    using (var getResponse = await s3Client.GetObjectAsync(getRequest))
                    using (var stream = getResponse.ResponseStream)
                    using (var reader = new StreamReader(stream))
                    {
                        return reader.ReadToEnd();
                    }
                }
            }
        }

Now you can use this implemented functionality in your web application using the dependency that injected in the startup class.

private ISecureEnclave _secureEnclave;

        public HomeController( ISecureEnclave secureEnclave)
        {
            _secureEnclave = secureEnclave;
        }
        
        public IActionResult Index()
        {
            _secureEnclave.WriteValue("MyKey123", "This is the data to secure");
            return View();
        }

You can find the git repo in this location. 

Stay tuned for another secure programming post :)