A New Old Thing:
(Bucket cannot have acls set with object ownership’s bucket owner enforced setting)
Amazon has announced support for S3 buckets without access control lists (ACLs), using the new “Bucket Owner Enforced“ setting for S3 Object Ownership. This setting removes the complexity of object ownership and ACLs on S3 buckets, dramatically simplifying permissions management for S3 buckets. But, although this is new for Amazon S3, this is not a new concept for Cloud Storage vendors. Google Cloud Storage has had their Uniform bucket-level access feature since late 2019. Azure Data Lake Storage Gen2 also supports role-based access control (RBAC).
What does this new S3 setting mean for users?
Simplified Permissions Management
With the new setting for Amazon S3 buckets (and, indeed, for all the other storage vendors that support similar options), the goal is simplification.  When the “Bucket Owner Enforced” option is set for an S3 bucket, all objects put into that bucket immediately become owned by the bucket owner. As we had mentioned in a previous post, there are some peculiarities around S3 buckets that had been cause for concern. For example, uploading any object to an S3 bucket using an account different than the bucket owners would put that file into the destination bucket, but it will still be owned by the uploading party. The bucket owner could not even access the data in that bucket, unless the uploading party remembered to send along a header (metadata) that informed the receiving bucket to grant the bucket owner full control of the uploaded file. Thus, it was up to the sending party to make the conscious effort to mark an uploaded file as owned by the bucket owner, otherwise the file would be inaccessible to the recipient. (Luckily, Diplomat MFT supports this option with a simple checkbox.)
With “Bucket Owner Enforced” turned on for a bucket, there is no need to bother with ACLs again. No more granular policy writing, or updating permissions with each new user account, access request, change in data locations, and so on. The entire bucket has (as Google Cloud so aptly puts it) a uniform access across all objects in the bucket, both now and in the future. All access is controlled through simple role based policies, making ACL management headaches a thing of the past.
Ensuring Security
Amazon S3 also provides a great policy validation tool on S3 buckets. A bucket owner can use the S3 tooling ecosystem to run over 100 actionable policy checks on a bucket. This tool provides a simple and intuitive way to double check that your S3 buckets are set up to be as secure for your business requirements.
This is important, too, when you are sending data to someone else’s bucket. Even if your own company’s S3 buckets are policy checked and super safe, when you transfer data to another account’s bucket, you need to be sure that you are not accidentally sending to an untrusted party. Since S3 buckets are based upon domain names, you are just one typo away from sending to the wrong bucket. Or perhaps you are sending to a bucket name that has since been renamed by your trading partner, and now you are using the old bucket name and sending your sensitive data to the wrong location. We see this in the wild quite a bit, which is why Diplomat MFT supports the “bucket owner condition“ feature for S3 transfers. When defining a job in Diplomat MFT to move files to an S3 bucket, you can easily specify the account number to which you are intending to send files. If the destination bucket is owned by any other account, the transaction is aborted. With Diplomat MFT’s notification system using e-mail, Slack, or MS Teams, you can easily learn when this error occurs.
Buckets, Buckets Everywhere
Cloud storage provides storage that is cheap and reliable, and now it is getting easier to use across all storage vendors. We have seen a rise in the use of Cloud Storage vendors in the market, and expect that trend to continue to grow. If you are using any cloud storage buckets for your business processes, Diplomat MFT is the easiest and best value way to automate and secure transfers to and from those buckets — as well as automating transfers within your organization and across your clients, suppliers, and trading partners.
FAQ
Q: How do I transfer files to and from an Amazon AWS S3 Bucket?
A: S3 buckets support file transfers over the HTTPS protocol, using a standard API (application programming interface). You can use the AWS CLI to transfer files from a command line, or write your own applications in Python, Java, C#, and so on using SDK provided by Amazon. To automate and provide operational security for S3 file transfers, consider using Diplomat MFT.
Q: How do I ensure that the files in AWS S3 are transferred and stored securely?
A: AWS S3 file transfers take place over HTTPS, which protects the privacy of the data as it flows between any computer and the S3 bucket. In addition, S3 supports both client-side and server-side encryption of the data to ensure that data at rest is secure. If you are transferring files to a third-party S3 bucket (that is, one that you do not own), be sure to validate bucket ownership so that you confirm files are only uploaded to the intended recipient.
Q: Why use AWS S3 Buckets to store and receive files?
A: AWS S3 buckets provide a secure, scalable, and low-latency file storage subsystem with extremely high reliability — all at a very reasonable cost. Amazon S3 is designed for 99.999999999% (11 9’s) of durability, and stores data for millions of customers all around the world. Companies choose S3 buckets to store data to reduce their operational costs and risks, while maintaining high levels of reliability and security.
Q: Can I use S3 buckets to exchange files with my customers, vendors, and other external parties?
A: Yes, as long as those external parties have a tool that supports transferring files to and from AWS S3 — such as Diplomat MFT — then you can give those external parties access to your buckets so that they can upload and download files. With a rich and versatile permissions model, your S3 bucket can be configured so that only the intended 3rd parties can upload data, or download only the data to which they have the permission to see. Coviant Software strongly recommends using the “Bucket Owner Enforced” permission model when using S3 buckets to allow external parties to transfer files to and from S3 buckets.
Q: Can I automate file transfers to and from an AWS S3 bucket?
A: There are numerous tools to automate file transfers to and from S3 buckets. Amazon AWS provides a CLI (Command Line Interface), an API (Application Programming Interface), and SDKs (Software Development Kits) which allow customers to write their own automation tools in their favorite scripting or development language. Alternatively, a tool such as Diplomat MFT provides an intuitive and flexible mechanism to schedule file transfers to and from AWS S3 (and many other protocols), and includes auditing, reporting, data archiving, and job alerting.
Use the link below to pick a time that is convenient for a demonstration. In just a few minutes of your time you will see how you can use Diplomat MFT to effortlessly automate and secure your file transfers from anywhere, to anywhere.
# # #
