1

I have a task to go through all the s3 buckets in my account and find the ones that are not encrypted by the default kms encryption. I have drafted the following two commands with a loop to go through each bucket name and check its encryption level

output="$(aws s3api list-buckets --query 'Buckets[*].Name')"

for i in $output; do aws s3api get-bucket-encryption --bucket $i; done

The script gives me the following error:

Invalid bucket name ""cdktoolkit-stagingbucket-30v8nlr122c0",": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:s3:[a-z\-0-9]+:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-]{1,63}$"

it works for the first part now, the output variable doesn't contain the question mark anymore, thanks for the help.

But as i go further, when i ran the second cli command "for i in $output; do aws s3api get-bucket-encryption --bucket $i; done", it returns another JSON format output, how could I grep the bucket name without AES256 encryption enabled.

Jasons-Air:~ jason$ for i in $output; do aws s3api get-bucket-encryption --bucket $i; done
{
    "ServerSideEncryptionConfiguration": {
        "Rules": [
            {
                "ApplyServerSideEncryptionByDefault": {
                    "SSEAlgorithm": "aws:kms"
                }
            }
        ]
    }
}

An error occurred (ServerSideEncryptionConfigurationNotFoundError) when calling the GetBucketEncryption operation: The server side encryption configuration was not found


An error occurred (ServerSideEncryptionConfigurationNotFoundError) when calling the GetBucketEncryption operation: The server side encryption configuration was not found


An error occurred (ServerSideEncryptionConfigurationNotFoundError) when calling the GetBucketEncryption operation: The server side encryption configuration was not found
{
    "ServerSideEncryptionConfiguration": {
        "Rules": [
            {
                "ApplyServerSideEncryptionByDefault": {
                    "SSEAlgorithm": "AES256"
                }
            }
        ]
    }
}

I tried to append "--query 'ServerSideEncryptionConfiguration[].Rules[].ApplyServerSideEncryptionByDefault[*].SSEAlgorithm'" to my command, but it shows result as "null" instead of "AES256".

Jasons-Air:~ jason$ for i in $output; do aws s3api get-bucket-encryption --bucket $i --query 'ServerSideEncryptionConfiguration[*].Rules[*].ApplyServerSideEncryptionByDefault[*].SSEAlgorithm'; done
null

An error occurred (ServerSideEncryptionConfigurationNotFoundError) when calling the GetBucketEncryption operation: The server side encryption configuration was not found


An error occurred (ServerSideEncryptionConfigurationNotFoundError) when calling the GetBucketEncryption operation: The server side encryption configuration was not found


An error occurred (ServerSideEncryptionConfigurationNotFoundError) when calling the GetBucketEncryption operation: The server side encryption configuration was not found
null
null
null
condescendent
  • 11
  • 1
    What does aws s3api list-buckets --query 'Buckets[*].Name' return? (Please add to your question, and if the information is sensitive replace it with made-up data that exactly matches the style of the real output.) – Chris Davies Jun 15 '20 at 17:43
  • [ "cdktoolkit-stagingbucket-30v8nlr122c0", "elasticbeanstalk-us-east-1-194639014949", "elasticbeanstalk-us-west-1-194639014949", "elasticbeanstalk-us-west-2-194639014949", "intangible-dev-terraform-state", "intangible-ssp-global-tf-states", "testbucketwithencryptionenabled", "thedevopguy" ] – condescendent Jun 15 '20 at 17:47
  • It looks like it returns all the bucket names with a double quotation mark. Is there a way I can remove the quotation mark ? – condescendent Jun 15 '20 at 17:48
  • It returns a JSON document. You may want to pass the result of the aws command trough jq -r '.[]'. – Kusalananda Jun 15 '20 at 17:55
  • it works by using the jq filter. but the result of the loop still shows a json output. If i want to just grep the bucket name not having aws kms enabled ("SSEAlgorithm"= "aws:kms"), how could I filter the result. Sorry I am new to aws and just started my new job. the result is like this. 
    Jasons-Air:~ jason$ for i in $output; do aws s3api get-bucket-encryption --bucket $i; done
{
    "ServerSideEncryptionConfiguration": {
        "Rules": [
            {
                "ApplyServerSideEncryptionByDefault": {
                    "SSEAlgorithm": "aws:kms"
                }
    – condescendent Jun 15 '20 at 18:19
  • That's completely unreadable. As I asked before, please put the update in your question – Chris Davies Jun 15 '20 at 18:23
  • I just updated my question. I appreciate your help, I will learn the "jq" command after this question. – condescendent Jun 15 '20 at 18:37
  • Same as my first comment: what does aws s3api list-buckets --query 'Buckets[*].Name' return? – Chris Davies Jun 15 '20 at 21:27
  • I resolved the issue by switching to boto3 library, it works now. Thank you all for the help. – condescendent Jun 17 '20 at 14:53

0 Answers0