This piece discusses the use of the aws s3 rm cli in manipulating and operating the s3 bucket objects.
It touches on the rm command for deleting a single object, and the –recursive parameter used to delete many objects inside a bucket at once.
It shows how prefixes such as ‘—include’ can be combined with the ‘—recursive’ parameter for multiple object deletion.
The rm wildcard characters for matching patterns are also discussed as well as troubleshooting skills in case the rm command is not working
AWS s3 rm cli
The aws s3 rm cli (Command Line Interface) is used to manage Amazon S3 buckets and objects by typing and executing short high-level commands rather than using the aws s3 graphical user interface (GUI).
Using the rm command in the aws s3 cli enables you to manage the deletion of contents of Amazon S3 within itself and in the local directory.
To use rm cli commands, you need to have the AWS CLI installed. If it is already installed, then it is always good to have the updated latest version of the AWS CLI.
For better command results, properly configure your CLI profile and enable permissions for the AWS operations. You must have a good understanding of the Bucket, Prefix, and Object operations.
AWS s3 rm
aws s3 rm command is used to delete S3 objects contained in your local directory or a bucket.
Used without any flag, the aws s3 command deletes only a single S3 object.
When deleting an object, you must pass the name of the object to the rm command.
If you have an object named myobject.txt, contained in in a bucket named mybucket, then to delete the object using the aws s3 rm command, you would type and execute the following command.
aws s3 rm s3://mybucket/myobject.txt
The output would be
delete: s3://mybucket/myobject.txt
AWS s3 rm recursive
The aws s3 rm command supplied with the –recursive parameter and specified bucket is used to delete objects contained in a bucket all at once.
This is a faster way to recursively delete the contents of a folder or bucket as well as the folder. Take, for instance, that your folder structure in aws S3 is as follows:
mybucket1/myobject2.txt
mybucket2/myobject1.txt
mybucket2/myobject2.txt
The bucket mybucket1 contains two objects myobject1.txt and myobject2.txt If you want to recursively delete all the files under the bucket mybucket1, then the command will look as
aws s3 rm s3://mybucket1 –recursive
The output on cli will appear as shown
delete: s3://mybucket1/ myobject1.txt
delete: s3://mybucket1/ myobject2.txt
AWS S3 rm multiple files
The –recursive parameter can be executed using a combination of a prefix and some arguments to delete multiple files or objects in a bucket while excluding others.
For instance, you can recursively delete all objects under mybucket2, that is myobject1 and exclude myobject2 with the use of –exclude parameter.
The command is as shown below.
aws s3 rm s3://mybucket2/ –recursive –exclude “* myobject2.txt”
The output would be
delete: s3://mybucket2/myobject1.txt
The exclude parameter can also be used together with the recursive parameter to recursively delete all objects under a specified bucket and a prefix of your choice while excluding all objects under a prefix that you specify.
From our example, let mybucket1 contain another/myobject1.txt. Then to exclude another/myobject1.txt from the recursive deletion of mybucket1, use
aws s3 rm s3://mybucket1/ –recursive –exclude “another/*”
To produce
delete: s3://mybucket1/myobject1.txt
delete: s3://mybucket1/myobject2.txt
It is important to note that when you are using multiple filters, then the order of the filtering parameters is paramount. The rule followed is that the filters that are put later in the command take precedence over the ones that appear earlier in the command. Hence
aws s3 rm s3:// mybucket1/ –recursive –exclude “*” –include “myobject2.txt” –include “another/*”
In this case above, all the files will be excluded from the command except for myobject2.txt and another/*.
Note that this command will also delete any files in sub-folders matching the –include patterns. In case you are not sure, always confirm with the parameter –dryrun first to inspect which of the files will be deleted.
AWS S3 rm wildcard
An asterisk is a special character that is used as a wildcard in the s3 rm command to match any specified keyword.
This enables one to match all objects for a certain pattern.
For example, when you run aws s3 rm *, it will delete all files. In case you indicate a prefix of ‘*’ or ‘/*’ it only applies to objects that begins with the asterisk and not all other objects.
Although now AWS CLI doesn’t offer support for UNIX wildcards in the command’s path argument, it is very simple to repeat this functionality utilizing the –exclude and –include parameters with the s3 rm commands.
The wildcards that can be used with the aws s3 rm command are: “*” which matches everything, “?” to match any single character, “[]” for matching any single character between the brackets, and “[!]” which matches any single character not between the brackets.
AWS s3 rm not working
Have you been wondering why your aws s3 rm command to delete your s3 bucket is not working even if you have full root permissions on the cli?
Well, it is impossible to delete any s3 bucket using the rm cli command when the AWS CLI versioning is enabled.
Suspend the versioning, then set up a lifecycle rule to enable deletion of objects in the buckets.
If you receive errors when you execute the aws s3 rm command, it may be possible that you are using an outdated AWS cli version. You need to update the latest version to eliminate this pain in the neck.
Final Thoughts
This guide has shown you how to use the aws s3 rm cli command to delete a single object in a bucket, how to recursively delete files inside a folder, and deleting multiple files using specified prefixed and wildcard characters for matching and some troubleshooting tips when the aws s3 rm command fails to work.