Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

PERL to AWS

by gdmichaels (Initiate)
on May 06, 2020 at 18:44 UTC ( #11116518=perlquestion: print w/replies, xml ) Need Help??

gdmichaels has asked for the wisdom of the Perl Monks concerning the following question:

I have been struggling all day with trying to get perl to talk to AWS/S3. It has been a rough day. First off, I am using Strawberry Perl, and perhaps this is the root of all my issues. I failed to install Net::Amazon::S3, and I had separate issues with Amazon::S3. I made the most progress with AWS::S3 from LEEJO. Assuming for a moment Strawberry Perl is not the issue, the current roadblock I am hitting is trying to use the $bucket->add_file function to an existing bucket. The code keeps generating this message:

"Use of uninitialized value $etag in substitution (s///) at C:/Strawberry/perl/site/lib/AWS/S3/File.pm line 153."

use AWS::S3; my $aws_access_key_id = "XXX"; my $aws_secret_access_key = "XXX"; my $s3 = AWS::S3->new( access_key_id => $aws_access_key_id, secret_access_key => $aws_secret_access_key, honor_leading_slashes => 0, # set to allow leading slashes in bucket + names, defaults to 0 ); my @bucket_list = $s3->buckets; my $bucket = @bucket_list[0]; my $name = $bucket->name; my $acl = $bucket->acl; print "Name: " . $name . ", ACL: " . acl . "\n"; # Add a file: my $new_file = $bucket->add_file( key => 'foo/bar.txt', contents => \'This is the contents of the file', );

Replies are listed 'Best First'.
Re: PERL to AWS
by 1nickt (Abbot) on May 06, 2020 at 19:58 UTC

    Hi gdmichaels and welcome to the Monastery!

    I have struggled and failed with various modules to connect to S3 over the years including the one you mentioned. In my experience you need to either use Paws, or if you don't need the Full Monty, Net::Amazon::S3.

    I use the latter most of the time. I was working on an upload script just this morning as a matter of fact:

    my $s3 = Net::Amazon::S3->new({ aws_access_key_id => $conf->{access_key}, aws_secret_access_key => $conf->{secret_key}, }); my $bucket = $s3->bucket($conf->{ bucket }); $log->info("Uploading $key to S3"); $bucket->add_key($key, join("\n", @rows, '')) or $log->fatal($s3->err . ': ' . $s3->errstr);

    If you explain your installation issues, someone can likely assist.

    Hope this helps!


    The way forward always starts with a minimal test.
Re: PERL to AWS (updated)
by haukex (Bishop) on May 07, 2020 at 07:55 UTC

    TBH, I've fallen back to using the aws command-line tool. I use my module IPC::Run3::Shell:

    use warnings; use strict; use IPC::Run3::Shell [ aws => {fail_on_stderr=>1}, "/path/to/aws" ]; use Cpanel::JSON::XS qw/decode_json/; use Data::Dumper; my $PROFILE = 'MyProfile'; # aws configure --profile=MyProfile my $buckets = decode_json( aws(qw/ s3api --output json --profile /, $PROFILE, 'list-buckets') ); print Dumper($buckets);

    Update: See also.

Re: PERL to AWS
by jo37 (Friar) on May 07, 2020 at 15:53 UTC

    I've been using AWS::CLIWrapper for some time. It has the advantage that you can do everything the CLI offers.

    Greetings,
    -jo

    $gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$
      I've been using AWS::CLIWrapper for some time.

      Thank you for pointing this out, it inspired me to release IPC::Run3::Shell::CLIWrapper, which is a more generic tool, but allows me to write my previous example as the following. The set-up may look a little more complicated, but the actual calls are much simplified. And since I wrote IPC::Run3::Shell, I can be certain of its behavior :-)

      use warnings; use strict; use IPC::Run3::Shell::CLIWrapper; use JSON::PP qw/decode_json/; use Data::Dumper; my $s3api = IPC::Run3::Shell::CLIWrapper->new( { fail_on_stderr => 1, stdout_filter => sub { $_=decode_json($_) } }, qw/ aws --profile MyProfile --output json s3api /); my $buckets = $s3api->list_buckets; print Dumper($buckets);

        Taking a real life example, I don't see an advantage in using IPC::Run3::Shell::CLIWrapper over AWS::CLIWrapper:

        #!/usr/bin/perl use strict; use warnings; use AWS::CLIWrapper; use Data::Dumper; my $aws = AWS::CLIWrapper->new(profile => 'my_profile') or die $AWS::CLIWrapper::Error->{Message}; sub gethosts { my $name = shift; my $hosts = $aws->ec2('describe-instances' => { filters => [{Name => 'tag:Name', Values => [$name]}], query => 'Reservations[*].Instances[*].{Id:InstanceId,Addr:Pri +vateIpAddress,Zone:Placement.AvailabilityZone,Name:(Tags[?Key==`Name` +])[0].Value}'}) or die "$AWS::CLIWrapper::Error->{Message}"; return [map {@$_} @$hosts]; } my $hosts = gethosts('*something*'); print Dumper($hosts);

        Doesn't this feel very "perlish"? :-)
        Maybe I'm missing the point.

        Greetings,
        -jo

        $gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11116518]
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (5)
As of 2021-04-12 22:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?