I have multiple VPS servers hosted with different cloud providers, and regularly backing up server data has become a tricky problem. Normally, I would use NFS mounting, using s3fs-fuse to mount the S3 bucket to the local partition, and then write the files that need to be backed up to that partition.
However, the stability of S3 access has been very poor recently, even in AWS's home country, the United States. The partition mounted using s3fs-fuse is not stable for reading and writing, especially when dealing with a large number of files and continuous reading and writing.
Recently, I discovered that using AWS Storage Gateway allows the virtual machine to directly connect to the iscsi target and operate on the corresponding LUN and volume. The iscsi protocol, which uses a lower TCP/IP protocol than HTTP/S, is more stable compared to the s3fs method.
In addition, incoming traffic to AWS Storage Gateway is free, and only storage fees for S3 and EBS are charged.
Server version: Ubuntu 18.04 LTS
Device target: /dev/sdc
- Installing and configuring the iscsi initiator program
- Install open-iscsi and open-iscsi-utils
- Discover the iscsi target
- Login to the target
- Mounting the hard disk
- Discover and partition/format the disk
- Partition and format the LUN
- Mount to the specified location
- Unmount the hard disk
- Automatically connect to the LUN and mount the hard disk at startup
- Automatically login to the target. Modify /etc/iscsi/iscsid.conf to enable automatic login to the Gateway.
- Modify /etc/fstab to achieve automatic mounting at startup
A few things to note:
- If an unknown interruption occurs during the connection with AWS Storage Gateway, you can try forcing a device refresh.
- If a connection timeout occurs, it is usually related to network quality. AWS recommends modifying iscsid.conf.
Reference: Customize Linux iSCSI Settings