Authenticate AWS workloads to HCP Vault Dedicated

14min
|
HCP
Vault

In the previous tutorial, you learned how to configure Vault to authenticate users using OIDC with the OIDC auth method. Workloads such as virtual machines and containers also need to authenticate to Vault. Vault supports authentication for many different platforms such as Amazon Web Services, Azure, and Google Cloud Platform.

This tutorial will walk you through the steps to authenticate AWS workloads with Vault. You will enable and configure the Amazon Web Services (AWS) auth method, and verify workloads can access Vault.

You will learn how to set up the necessary resources in your AWS account and Vault Dedicated cluster.

Create a AWS IAM policy to allow Vault Dedicated to validate the identity of AWS services.
Create a AWS IAM user to allow Vault Dedicated to access your AWS resources.
Create AWS IAM role which will be assigned to AWS services and trusted by Vault Dedicated.
Enable the AWS auth method for Vault Dedicated.
Assign the AWS IAM user credentials to the AWS auth method configuration.
Configure a Vault Dedicated role to authenticate AWS services with a trusted AWS IAM role.
Validate the configuration using a EC2 instance and the Vault binary.

hcp-vault-auth-aws-overview

Note

Deploying and configuring the items in this tutorial may lead to additional charges in your AWS or HCP account.

Personas

The end-to-end scenario described in this tutorial involves two personas:

admin with privileged permissions to configure an auth method
app is the consumer of secrets stored in Vault

Prerequisites

To complete this tutorial you should have familiarity with, and access to the HashiCorp Cloud Platform (HCP), Vault Dedicated, and AWS.

Visit the Getting Started with HCP Vault Dedicated tutorials for a refresher on how to deploy and configure Vault Dedicated.

Note

This tutorial will use the naming conventions from the Getting Started with HCP Vault Dedicated tutorials when referencing HCP resources.

Please verify all prerequisites have been met for Vault Dedicated, AWS, and your local workstation.

HCP Vault Dedicated

The following items be must be deployed in the HCP Portal to complete this tutorial.

HashiCorp Virtual Network (HVN) created.
A Vault cluster with public address. (Refer to the Create a Vault Cluster on HCP tutorial.)

Warning

Making the HCP Vault Dedicated cluster publicly accessible is not recommended, however for ease of completing this tutorial we suggest you have it enabled.

Note

You can learn how to connect to a private HCP Vault Dedicated cluster in the Connect an Amazon Transit Gateway to your HashiCorp Virtual Network or Peering an AWS VPC with HashiCorp Cloud Platform (HCP) documentation.

AWS

During this tutorial, you will create several Identity and Access Management (IAM) resources, create an Amazon Linux2 instance and connect via SSH to demonstrate authentication with Vault Dedicated.

Access to an AWS account with a Virtual Private Cloud (VPC), attached Internet Gateway (IGW), routing tables, and public subnet.
User credentials that have permissions to create Identity and Access Management (IAM) users, policies, and roles.
User credentials that have permission to create Elastic Compute (EC2) instances, security groups, attach IAM roles, and access to or ability to create key pairs to connect to the EC2 instance.

Workstation

This tutorial was developed and tested using OSX, however you can complete this tutorial by making the necessary changes to support your preferred Operating System (OS).

HashiCorp Vault binary installed. Please see the Install Vault tutorial for more information.
Access to a terminal/command line interface and ability to create SSH sessions.

Lab setup

Note

If you do not have access to an HCP Vault Dedicated cluster, visit the Create a Vault Cluster on HCP tutorial.

Launch the HCP Portal and login.
Click Vault in the left navigation pane.
In the Vault clusters pane, click vault-cluster.
Under Cluster URLs, click Public Cluster URL.
In a terminal, set the VAULT_ADDR environment variable to the copied address.
```
$ export VAULT_ADDR=<Public_Cluster_URL>
```
Return to the Overview page and click Generate token.
Within a few moments, a new token will be generated.
Copy the Admin Token.
Return to the terminal and set the VAULT_TOKEN environment variable.
```
$ export VAULT_TOKEN=<token>
```
Set the VAULT_NAMESPACE environment variable to admin.
```
$ export VAULT_NAMESPACE=admin
```

Type vault status to verify your connectivity to the Vault cluster.

$ vault status

Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    1
Threshold                1
Version                  1.9.2+ent
Storage Type             raft
...snipped...

With the environment variables set, you can now explore your Vault cluster. Type vault auth list to view the list of available auth methods.

$ vault auth list

Path      Type        Accessor                  Description
----      ----        --------                  -----------
token/    ns_token    auth_ns_token_d6310fa6    token based credentials

All new Vault clusters will have the token auth method enabled.

Create AWS resources

(Persona: admin)

Before you configure the Vault Dedicated AWS auth method, you must create the necessary resources in AWS. The AWS auth method will require an IAM policy that permits the appropriate access for the auth method, an IAM user with programmatic access, and one or more roles that you will assign to other AWS services that require authentication to Vault.

Create AWS IAM policy for HCP Vault Dedicated auth method

Launch the AWS Console and log in with a user that has permission to create IAM users, policies, and roles such as the AWS accounts root user.
Open the IAM dashboard and click Policies in the left navigation pane.
Click the blue Create Policy button.
Click the JSON tab.
Clear all existing content in the IAM policy editor.

Copy and paste the sample IAM policy into the IAM policy editor.

{
"Version": "2012-10-17",
"Statement": [
 {
   "Effect": "Allow",
   "Action": [
     "ec2:DescribeInstances",
     "iam:GetInstanceProfile",
     "iam:GetUser",
     "iam:ListRoles",
     "iam:GetRole"
   ],
   "Resource": "*"
 }
 ]
}

Click the blue Next: Tags button.
Click the blue Next: Review button.
In the Name* field enter aws-iampolicy-for-vault-authmethod.
Click the blue Create policy button.

Create AWS IAM user for HCP Vault Dedicated auth method

Open the IAM dashboard and click Users in the left navigation pane.
Click the blue Add users button.
In the User name* field enter aws-iamuser-for-vault-authmethod.
Click the Access key - Programmatic access checkbox.
Click the blue Next: Permissions button.
Click the Attach existing policies directly button.
Search for, then click the checkbox for aws-iampolicy-for-vault-authmethod.
Click the blue Next: Tags button.
Click the blue Next: Review button.
Click the blue Create user button.
On the Add user Success page, copy the Access key ID.
Return to the terminal and create an environment variable named TMP_VAULT_ACCESS_KEY.
```
$ export TMP_VAULT_ACCESS_KEY=<ActualAccessKeyID>
```
Replace <ActualAccessKeyID> with the Access key ID provided by the AWS Add user wizard.
Return to AWS Console Add user Success page.
Click the Show link and copy the Secret access key.
Return to the terminal and create an environment variable named TMP_VAULT_SECRET_KEY.
```
$ export TMP_VAULT_SECRET_KEY=<ActualSecretAccessKey>
```
Replace <ActualSecretAccessKey> with the Secret access key provided by the AWS Add user wizard.
Return to the AWS Console Add user Success page and click the Close button.

Create AWS role for HCP Vault Dedicated auth method

Open the IAM dashboard and click Roles in the left navigation pane.
Click the blue Create role button.
Click the AWS service radio button.
Click the EC2 radio button.
Click the blue Next button.
Do not attach a policy.
When you configure the AWS auth method, you specify an AWS IAM role that Vault will trust during the authentication process. There are no special permissions required by Vault beyond knowing which AWS IAM roles to trust.
Click the blue Next button.
In the Role name* field enter aws-ec2role-for-vault-authmethod.
Click the blue Create role button.
Repeat the steps in the Create AWS Role for Vault Dedicated Auth Method section. Select Lambda as the service and enter aws-lambdarole-for-vault-authmethod for the Role name.

Configure Vault auth method

(Persona: admin)

Now that you have created the necessary resources in AWS, you can configure Vault Dedicated with the KV secret engine, a policy to allow read access to secrets, and the AWS auth method.

Return to the terminal and enable the KV v2 secret engine at the default path.

$ vault secrets enable -version=2 kv

Success! Enabled the kv secrets engine at: kv/

Store api-key with a value of ABCDEFG9876 at the path kv/test/ec2.

$ vault kv put kv/test/ec2 api-key="ABCDEFG9876"

Key                Value
---                -----
created_time       2022-02-10T23:11:37.035373572Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

Create a Vault policy named vault-policy-for-aws-ec2role that allows read at the path kv/test/ec2.

$ vault policy write vault-policy-for-aws-ec2role - << EOF
# Grant 'read' permission to paths prefixed by 'kv/data/test/ec2'
path "kv/data/test/ec2" {
  capabilities = [ "read" ]
}
EOF

Successful output example:

Success! Uploaded policy: vault-policy-for-aws-ec2role

View the list of policies.
```
$ vault policy list

default
hcp-root
vault-policy-for-aws-ec2role
```
The default and hcp-root policies are created with all new HCP Vault clusters.

Enable the AWS auth method at the default path.

$ vault auth enable aws

Success! Enabled aws auth method at: aws/

Type vault auth list to view the list of available auth methods.

$ vault auth list

Path      Type        Accessor                  Description
----      ----        --------                  -----------
aws/      aws         auth_aws_ebfaa1b9         n/a
token/    ns_token    auth_ns_token_d6310fa6    token based credentials

In addition to the token auth method that is enabled with all new Vault Clusters, you now have the aws auth method enabled.

Configure the aws auth method with access to your AWS account using the Access key ID and Secret Access Key previously created and stored at TMP_VAULT_ACCESS_KEY and TMP_VAULT_SECRET_KEY.
```
$ vault write auth/aws/config/client secret_key=$TMP_VAULT_SECRET_KEY access_key=$TMP_VAULT_ACCESS_KEY

Success! Data written to: auth/aws/config/client
```
Configure the aws auth method to trust the AWS IAM role previously created and attach the vault-policy-for-aws-ec2role to the token provided by the aws auth method. Replace YourAWSAccountID with the actual account ID for your AWS account.
```
$ vault write auth/aws/role/vault-role-for-aws-ec2role \
     auth_type=iam \
     bound_iam_principal_arn=arn:aws:iam::<YourAWSAccountID>:role/aws-ec2role-for-vault-authmethod \
     policies=vault-policy-for-aws-ec2role
```
Successful output example:
```
Success! Data written to: auth/aws/role/vault-role-for-aws-ec2role
```
Thus far you have created several resources in AWS, and configured several settings to support the Vault AWS auth method. Take moment to review the command to better understand each part, and where they were configured.
If there are multiple AWS IAM roles that Vault should trust, you can create additional Vault roles by replacing the Vault role name. In the Create AWS Role for Vault Dedicated Auth Method section you created an additional AWS IAM role called called aws-lambdarole-for-vault-authmethod. To configure Vault to trust this role, in addition to the aws-ec2role-for-vault-authmethod run:
```
$ vault write auth/aws/role/vault-role-for-aws-lambdarole \
     auth_type=iam \
     bound_iam_principal_arn=arn:aws:iam::<YourAWSAccountID>:role/aws-lambdarole-for-vault-authmethod \
     policies=vault-policy-for-aws-ec2role
```
Successful output example:
```
Success! Data written to: auth/aws/role/vault-role-for-aws-lambdarole
```

View the roles created for the aws auth method.

$ vault list /auth/aws/role

Keys
----
vault-role-for-aws-ec2role
vault-role-for-aws-lambdarole

View the configuration of a specific role.

$ vault read auth/aws/role/vault-role-for-aws-ec2role

Key                               Value
---                               -----
allow_instance_migration          false
auth_type                         iam
bound_account_id                  []
bound_ami_id                      []
bound_ec2_instance_id             <nil>
bound_iam_instance_profile_arn    []
bound_iam_principal_arn           [arn:aws:iam::186150483639:role/aws-ec2role-for-vault-authmethod]
bound_iam_principal_id            [AROASWV3O623UKSNMYSRT]
bound_iam_role_arn                []
bound_region                      []
bound_subnet_id                   []
bound_vpc_id                      []
disallow_reauthentication         false
inferred_aws_region               n/a
inferred_entity_type              n/a
policies                          [vault-policy-for-aws-ec2role]
resolve_aws_unique_ids            true
role_id                           1ff0b395-603c-71b6-3b5b-cf795e8a4b15
role_tag                          n/a
token_bound_cidrs                 []
token_explicit_max_ttl            0s
token_max_ttl                     0s
token_no_default_policy           false
token_num_uses                    0
token_period                      0s
token_policies                    [vault-policy-for-aws-ec2role]
token_ttl                         0s
token_type                        default

You have now configured the necessary resources in AWS and Vault Dedicated. In the next section, you will deploy an Amazon EC2 instance and test authentication with Vault.

Authenticate with HCP Vault Dedicated

(Persona: app)

This section demonstrates how to authenticate with Vault from an Amazon EC2 instance. To simulate an application running on your EC2 instance, you will install the Vault binary and authenticate to your Vault Dedicated instance.

Return to AWS Console and log in with a user that can create Amazon EC2 instances.
Open the EC2 dashboard and click Instances in the left navigation pane.
Click the orange Launch Instances button.
Launch an Amazon Linux 2 AMI with a family/size of t2.micro, in a public subnet or subnet you can SSH to, a security group that will allow SSH access, and a new or existing key pair you have access to. Do not attach a role at this time.
Note
If you used any sensitive information instead of the sample data provided in this tutorial you should encrypt the EBS volume.
SSH to the new EC2 instance. Replace <mykey.pem> and <public_ip_address> with the values for your Amazon EC2 instance.
```
$ ssh -i <mykey.pem> ec2-user@<public_ip_address>
```
Install yum-utils.
```
$ sudo yum install -y yum-utils
```

Add the HashiCorp Linux repository

$ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo

Install Vault.
```
$ sudo yum -y install vault
```

Verify Vault was installed properly by executing vault.

$ vault

Usage: vault <command> [args]

Common commands:
 read        Read data and retrieves secrets
 write       Write data, configuration, and secrets
 delete      Delete secrets and configuration
 ...snipped...

Return to the HCP Portal and click the Public link in the Quick actions page to copy the public URL for your Vault cluster.
Return to the SSH session for your Amazon EC2 instance.
In the terminal, set the VAULT_ADDR environment variable to the copied address.
```
$ export VAULT_ADDR=<Public_Cluster_URL>
```
Set the VAULT_NAMESPACE environment variable to admin.
```
$ export VAULT_NAMESPACE="admin"
```
Try to authenticate to Vault using the aws auth method.
```
$ vault login -method=aws role=vault-role-for-aws-ec2role

Error authenticating: failed to retrieve credentials from credential chain:
NoCredentialProviders: no valid providers in chain.
```
Recall that when you created the new Amazon EC2 instance you did not specify an IAM role. With no IAM role attached, Vault does not trust the EC2 instance.
Return to the AWS Console and access the EC2 >> Instances dashboard.
Click the check box for the EC2 instance.
Click the Actions pull down menu and navigate to Security >> Modify IAM role.
In the Choose IAM role pull down menu select aws-ec2role-for-vault-authmethod.
Click the orange Save button.
Return to the SSH session for your EC2 instance.

Try to authenticate to Vault using the aws auth method again.

$ vault login -method=aws role=vault-role-for-aws-ec2role

token                    s.aBcD3FgH1jkkLMnoPCavo9xd.SpK2b
token_accessor           CgA5asdfas2YeGf8ptn84KS.SpK2b
token_duration           1h
token_renewable          true
token_policies           ["default" "vault-policy-for-aws-ec2role"]
identity_policies        []
policies                 ["default" "vault-policy-for-aws-ec2role"]
token_meta_account_id    186150483639
token_meta_auth_type     iam
token_meta_role_id       1ff0b395-603c-71b6-3b5b-cf795e8a4b15

You were able to successfully authenticate to Vault Dedicated by attaching the trusted AWS IAM role to your EC2 instance.

The output displays details about your token such as the token, token_accessor, whether the token is renewable, and the token_policies which you created and associated to the Vault role.

Cleanup

To avoid any unnecessary charges to your AWS or HCP account, you should destroyed any resources that are no longer needed.

The following is a list of resources created in AWS for this tutorial

EC2 instance for testing Vault authentication.
AWS IAM roles aws-ec2role-for-vault-authmethod and aws-lambdarole-for-vault-authmethod.
AWS IAM policy aws-iampolicy-for-vault-authmethod.
AWS IAM user aws-iamuser-for-vault-authmethod IAM user.
Any EC2 key pairs created specifically for this tutorial.

Next steps

You learned how to set up and configure the AWS auth method to allow an EC2 instance to authenticate with Vault Dedicated so that it can read secrets. Refer to the Vault documentation for the code example that uses the AWS auth method to authenticate with Vault.

In practice, you can reduce the amount of custom coding by delegating the authentication to Vault Agent. To learn more about Vault Agent, see the Vault Agent with AWS tutorial.