Securing S3

To access an S3 bucket, users need AWS access key ID and AWS secret. Both of these are generated by going to AWS website. When you use Ozone’s S3 protocol, you need the same AWS access key and secret.

Under Ozone, the clients can download the access key directly from Ozone. The user needs to kinit first and once they have authenticated via kerberos they can download the S3 access key ID and AWS secret. Just like AWS S3, both of these are secrets that needs to be protected by the client since it gives full access to the S3 buckets.

Obtain Secrets

S3 clients can get the secret access id and user secret from OzoneManager.

Using the command line

For a regular user to get their own secret:

ozone s3 getsecret

An Ozone administrator can get a secret for a specific user by using the -u flag:

ozone s3 getsecret -u <username>

Using the REST API

A user can get their own secret by making a PUT request to the /secret endpoint:

curl -X PUT --negotiate -u : https://localhost:9879/secret

An Ozone administrator can get a secret for a specific user by appending the username to the path:

curl -X PUT --negotiate -u : https://localhost:9879/secret/<username>

This command will talk to ozone, validate the user via Kerberos and generate the AWS credentials. The values will be printed out on the screen. You can set these values up in your .aws file for automatic access while working against Ozone S3 buckets.

• Now you can proceed to setup these secrets in aws configs:

aws configure set default.s3.signature_version s3v4
aws configure set aws_access_key_id ${accessId}
aws configure set aws_secret_access_key ${secret}
aws configure set region us-west-1

Please refer to AWS S3 documentation on how to use S3 via command line or via S3 API.

Revoking Secrets via REST API

To invalidate/revoke the secret, use ozone s3 revokesecret command. Alternatively, you can use the REST API endpoint to revoke the secret. Ozone now provides a REST API endpoint that allows administrators to revoke S3 access secrets. This operation invalidates a secret, ensuring it can no longer be used for authentication.

Endpoint Details

• URL: http://localhost:9879/secret
• HTTP Method: DELETE

Authentication

The API leverages SPNEGO (Kerberos) authentication. The following curl options are used:

• --negotiate enables SPNEGO.
• -u : uses the current Kerberos ticket (an empty username is provided).

Example 1: Revoke Secret for the Current User

This command revokes the secret for the currently authenticated user:

curl -X DELETE --negotiate -u : -v http://localhost:9879/secret

Example 2: Revoke Secret by Username

This command revokes the secret for a specific user by appending the username as a query parameter. Replace testuser with the desired username:

curl -X DELETE --negotiate -u : -v "http://localhost:9879/secret?username=testuser"

Response

• Success: Returns HTTP 200 OK along with a confirmation message in JSON format.
• Failure: Returns an appropriate HTTP error status and message if there are issues (e.g., authentication failures).

Testing and Verification

For a working example of these operations, refer to the Secret Revoke Robot Test. This test demonstrates both the default secret revocation and the revocation by username.

Note: Ensure your Kerberos authentication is correctly configured, as secret revocation is a privileged operation.

External S3 Secret Storage with HashiCorp Vault

By default, S3 secrets are stored in the Ozone Manager’s RocksDB. For enhanced security, Ozone can be configured to use HashiCorp Vault as an external secret storage backend.

Configuration

To enable Vault integration, you need to configure the following properties in ozone-site.xml:

Example

Here is an example of how to configure Ozone to use Vault for S3 secret storage with token authentication:

<property>
  <name>ozone.secret.s3.store.provider</name>
  <value>org.apache.hadoop.ozone.s3.remote.vault.VaultS3SecretStorageProvider</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.address</name>
  <value>http://localhost:8200</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.enginever</name>
  <value>2</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.secretpath</name>
  <value>secret</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.auth</name>
  <value>TOKEN</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.auth.token</name>
  <value>your-vault-token</value>
</property>

Example with SSL

Here is an example of how to configure Ozone to use Vault for S3 secret storage with SSL:

<property>
  <name>ozone.secret.s3.store.provider</name>
  <value>org.apache.hadoop.ozone.s3.remote.vault.VaultS3SecretStorageProvider</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.address</name>
  <value>https://localhost:8200</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.enginever</name>
  <value>2</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.secretpath</name>
  <value>secret</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.auth</name>
  <value>TOKEN</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.auth.token</name>
  <value>your-vault-token</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.trust.store.path</name>
  <value>/path/to/truststore.jks</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.trust.store.password</name>
  <value>truststore-password</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.key.store.path</name>
  <value>/path/to/keystore.jks</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.key.store.password</name>
  <value>keystore-password</value>
</property>

References

• HashiCorp Vault Documentation