aws_s3_file IAM role support

[defunct73]

Would it be possible to add IAM role support to the aws_s3_file provider? It currently requires aws_access_key_id and aws_secret_access_key be set.

Thanks for consideration.

[defunct73]

I tried to do the following, which does get me the credentials:

include_recipe "aws"

::Chef::Recipe.send(:include, Opscode::Aws::Ec2)

creds = query_role_credentials

aws_s3_file "/var/chef/cache/myfile.tar.gz" do
  bucket "mybucket"
  remote_path "/path/to/myfile.tar.gz"
  aws_access_key_id creds['AccessKeyId']
  aws_secret_access_key creds['SecretAccessKey']
  action :create
end

However, I get a timeout error. What am I doing wrong? (p.s. I'm a chef/ruby noob)

[2014-06-19T15:02:32-04:00] ERROR: aws_s3_file[/var/chef/cache/myfile.tar.gz](test-cookbook::default line 19) had an error: Errno::ETIMEDOUT: remote_file[/var/chef/cache/myfile.tar.gz](/tmp/vagrant-chef-1/chef-solo-1/cookbooks/aws/providers/s3_file.rb line 36) had an error: Errno::ETIMEDOUT: Error connecting to https://mybucket.s3.amazonaws.com/path/to/myfile.tar.gz?Signature=XXXXXXXXXXXXXXX&Expires=1403290889&AWSAccessKeyId=XXXXXXXXXXXX - Connection timed out - connect(2)
I, [2014-06-19T15:01:29.168216 #11453] INFO -- : New RightAws::S3Interface using shared connections mode

[fred]

@defunct73 I was able to achieve it using your code. thanks.
It works great for me. Does your machine have proper internet connectivity?

it should be trivial to implement this on the S3 provider, just like it's done on EC2:

if new_resource.aws_access_key and new_resource.aws_secret_access_key
  aws_interface.new(new_resource.aws_access_key, new_resource.aws_secret_access_key, {:logger => Chef::Log, :region => region})
else
  creds = query_role_credentials
  aws_interface.new(creds['AccessKeyId'], creds['SecretAccessKey'], {:logger => Chef::Log, :region => region, :token => creds['Token']})
end

[defunct73]

@fred, The instance does have proper connectivity. The s3 bucket I'm referencing is private, though, and I wonder if the token retrieved from the IAM role is also required. It's currently not being passed along.

[ubiquitousthey]

Pull request #83 adds this feature

[skinney6]

The readme says we can use IAM roles instead of passing keys but i'm getting:

RightAws::AwsError
    ------------------
    AWS access keys are required to operate on S3

When running:

aws_s3_file "/tmp/local_file" do
     bucket "my_bucket"
     remote_path "filename"
    #  aws_access_key_id aws['aws_access_key_id']
    #  aws_secret_access_key aws['aws_secret_access_key']
end

I can see in my ec2 console that this instance is running a role with full s3 rights.

[ubiquitousthey]

I created a PR to add this for the S3 provider, but after adding it, I realized that my test had a problem. I withdrew the PR. The unerlying right_aws library for the s3 provider does not support it, and unfortunately, this project does not appear to be accepting PRs. I think this cookbook would have to move away from using the right_aws library to implement this fully.

[cwebberOps]

I think in an ideal world, a move to the official aws would be the right thing to do. It is just a matter of a PR or finding time to make it happen.

[ubiquitousthey]

I can probably submit a PR with the s3_file provider converted to use the aws sdk.

[zrisher]

+1

[ubiquitousthey]

PR #96 converts everything to use the aws-sdk. It is using the pre release version 2.0.9.pre. You might want to keep this in a branch until it is in general release.

[kgf]

Hoping this will be merged soon, as we use IAM roles for all our EC2 instances.

Thanks

[ubiquitousthey]

@kgf It has been merged. It does have an issue on non-EC2 instances. PR #115 fixes this.

[tas50]

Closing this out since we have support for roles and the fix for non-ec2 instances has been merged