Mounting an Amazon S3 Bucket as a Filesystem on Ubuntu Linux

```

Mounting an Amazon S3 Bucket as a Filesystem on Ubuntu Linux

This guide provides a comprehensive, step-by-step approach to mounting an Amazon S3 bucket as a local filesystem on a Linux Ubuntu server. This allows you to interact with your S3 bucket using standard file system operations, making it easier to manage and integrate with applications. We will primarily use s3fs, a FUSE (Filesystem in Userspace) based tool, for this process, and also briefly touch on Mountpoint for Amazon S3 for specific use cases.

Prerequisites

Before you begin, ensure you have the following:

An active AWS account.
An existing S3 bucket. Note the bucket name and region.
An IAM user with programmatic access and appropriate S3 permissions (e.g., s3:ListBucket, s3:GetObject, s3:PutObject).
The IAM user's Access Key ID and Secret Access Key.
An Ubuntu server (e.g., Ubuntu 20.04, 22.04, or 23.04) with root or sudo access.
Basic command-line knowledge.
A stable internet connection.

Step 1: Update System Packages

Start by updating your Ubuntu system's package list and upgrading installed packages to their latest versions. This ensures you have the most recent software and security patches.

sudo apt-get update -y
sudo apt-get upgrade -y

Step 2: Install s3fs and Dependencies

Install the s3fs package along with any necessary dependencies. This package allows you to mount S3 buckets as local file systems.

sudo apt-get install -y s3fs fuse

Step 3: Configure AWS Credentials

s3fs needs your AWS credentials to access your S3 bucket. There are two primary methods for providing these credentials:

Method 1: Using a Credentials File (Traditional)

Create a file to store your AWS credentials:
```
touch ~/.passwd-s3fs
```
Edit the file and add your AWS Access Key ID and Secret Access Key in the following format:
```
Your_AWS_Access_Key:Your_AWS_Secret_Key
```
For example:
```
AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
```
Set the file permissions to ensure only the owner can read it:
```
chmod 600 ~/.passwd-s3fs
```

Method 2: Using AWS Credentials File (Recommended)

Create the .aws directory if it doesn't exist:
```
mkdir -p ~/.aws
```
Create a credentials file:
```
nano ~/.aws/credentials
```
Add your AWS credentials in the following format:
```
[default]
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_ACCESS_KEY
```
Replace YOUR_ACCESS_KEY_ID and YOUR_SECRET_ACCESS_KEY with your actual AWS credentials.
Set the file permissions to ensure only the owner can read it:
```
chmod 600 ~/.aws/credentials
```

Using the AWS credentials file is generally recommended as it aligns with AWS best practices and can be used by other AWS tools.

Step 4: Create a Mount Point

Choose a directory on your server where you want to mount the S3 bucket. Create this directory if it doesn't exist.

sudo mkdir -p /mnt/s3bucket

You can replace /mnt/s3bucket with any directory you prefer.

Step 5: Mount the S3 Bucket

Use the s3fs command to mount your S3 bucket to the created mount point. The basic command format is:

s3fs your-s3-bucket-name /path/to/local/mountpoint -o passwd_file=~/.passwd-s3fs -o use_path_request_style

Replace your-s3-bucket-name with the name of your S3 bucket and /path/to/local/mountpoint with the path to the directory you created in the previous step. The -o passwd_file=~/.passwd-s3fs option specifies the path to your credentials file. The -o use_path_request_style option is necessary for buckets in non-default regions.

For example:

sudo s3fs my-s3-bucket /mnt/s3bucket -o passwd_file=~/.passwd-s3fs -o use_path_request_style

If you are using the AWS credentials file, you can omit the -o passwd_file option. s3fs will automatically look for credentials in ~/.aws/credentials.

To use the AWS credentials file, the command would look like this:

sudo s3fs my-s3-bucket /mnt/s3bucket -o use_path_request_style

Step 6: Verify the Mount

After mounting, verify that the S3 bucket is accessible by listing the contents of the mount point:

ls /mnt/s3bucket

You should see the files and directories from your S3 bucket.

You can also use the mount command to check if the bucket is mounted:

mount | grep s3fs

And the df -h command to see the mounted file systems:

df -h | grep s3fs

Step 7: Automate Mounting on Boot

To ensure the S3 bucket is mounted automatically when your server starts, you need
December 17, 2024