Amazon S3 Command Line Copy for Windows

A couple of months ago I started to use Amazon S3 (Amazon Simple Storage Service) as off site backup for one of my servers. It’s dirt cheap and so far it works very well (I’m paying less than $1 per month for my daily uploads and storage!)

The server is running Windows Server 2008 and my backup routine is simply a scheduled bat/cmd file to copy everything to a temp folder, then zipping it to a single file that I move off site (I used to ftp it to another host but you cannot use ftp with S3)

So, I needed to be able to upload my zip file to Amazon S3 using the command line. But unfortunately the only S3 copy or backup tools I found for Windows was either graphical and user driven or required Java, Perl or such and I didn’t really feel like installing any of that stuff on my server just for this simple task.

The solution was to write my own copy routine in C# (.NET Framework 2.0) which I did using one of the many S3 libraries available, in my case I selected the Affirma Threesharp C# Library. The result is a very simple copy routine, you can only copy one file at a time, no wild cards etc. I will probably extend it in the future, for example a delete routine so I can also automate deleting old backups (I’m currently using the s3fox plugin for Firefox to check on my backups and do deletes about once a month)

Syntax: s3copy Localpath Bucket Filename AccessKeyId SecretAccessKey

Example: s3copy.exe c:\backup\ mybackupbucket backup.rar MyAccessKeyId MySecretAccessKey

It’s a single exe file. No need to install anything, the dll is contained in the exe. Feel free to use it if you like!

Click button below download S3Copy.exe from Download.com
Get it from CNET Download.com!

[ Edit/10 May 2009: Thanks to Brad! Corrected post + exe so that Access Key Id and Secret Access Key can be entered (you find your keys at Amazon’s AWS: Your Account + Access Identifiers) ]
[Edit/15 June 2009: Moved file download to download.com]

31 comments

  1. Great idea. I’ve been looking for something just like it. How do you authenticate to access a private bucket though? Seems that the only way to use the program is to give write access to everyone.

  2. Hey Brad – do I feel stupid…
    Uploaded a corrected version where you can add your info to authenticate. Please download and try again, use same download link as before (in post above) Thanks/Max

  3. Hi, thanks for posting this. I am trying to add S3 support to an old ASP site and this will be a big help.

    I seem to be having trouble directing the file to a folder in my bucket. Is there support for copying to a folder? Next release 🙂

    thx

  4. Hi, thanks for sharing! Just what I was looking for. One question (I don’t have S3 yet so I can’t test): does the program give a meaningful exit code?

    Thanks,
    Kevin

  5. How does this handle ACL permissions? I want all copied files to inherit the permissions of the bucket.

    Primarily I need Public Read for all copied files.

  6. Hopefully I caught you before moderation. I made the silly mistake of not escaping my second “. Thanks so much for a great script. It works like a charm.

  7. Hi,

    great work but I got an error:

    status Code: Moved Permanantly

    The bucket you are attempting to access must be addressed using a specific endpoint. How do I submit this endpoint?

    Best Regards
    Simon

  8. Program works great, except it seems to be limited on how much bandwidth it can consume. I have an open 10Mbps connection and it only uses about 20% of that, while other tools will use all available bandwidth. Any chance that could be corrected.

  9. Well this is weird — I’m also called Max and have written a single-EXE S3 client for Windows! Maybe we are related. It does wildcards and a few other very simple things

  10. Hi,

    I am getting a funny message saying application failed to initialize properly 0x0000135. Any thoughts? I’m running windows server 2008 core.

    Thanks,

    Tom.

  11. Nice. I was looking for a Linux command line or shell script and found this…works great for initial testing from my PC before I could move the solution to my Linux hosting servers 🙂

    Thanks a lot.

  12. Hi, I have a problem.
    Credentials checked, pathes checked.
    When I run from Windows Server 2008 (Amazon EC2 Instance) I get Access Denied message:
    Copying file to s3… An exception occurred.
    Message: Access Denied
    HTTP Status Code: Forbidden
    Error Code: AccessDenied
    XML: AccessDeniedAccess Denied
    EA5FBF7D2889AC70TaqwfXXjs07al4ELyS147gjCz6qj/FMosLkxSJcZ6sxQ
    0KnoqLw8UnmlVIefOpTV

  13. Hi, again
    I’ve found where was my fault. For bucket name I used word reserved for S3.
    So use unique name.

  14. How can I download a file from S3 to local PC or an EC2 instance using s3copy?

    Thanks in advance

  15. Hello Phil,

    Yes you can do this using my very own tool Bucket Explorer, You can use its commander version and set the bucket default permissions and meta-data as desired.
    On uploading,copying the stuff in your S3 bucket it got automatically set for all the content being copied.
    If you want to set Bucket Default structure and interested in 3rd party java based tool else please ignore

    Thanks and Regards
    Kirti

  16. I’m wondered about some antivirus detecting malicious software in this file. I followed the link, clicked ‘Download’, got a copy of cnet2_s3copy.exe and uploaded it to Virustotal. 3 of 42 anti-viruses detected something malicious. Am I doing something wrong?

  17. I can’t get files to a bucket and they and they company I’m trying to get it to even created me my own bucket and I still can’t. I get

    Message: The request signature we calculated does not match the signature you pr
    ovided. Check your key and signing method.
    HTTP Status Code: Forbidden

    I don’t know what I’m doing wrong.

    The bucket just starts with s3.//ThierCompany.name.import.prod.julie

Leave a Reply to Tom Cancel reply

Your email address will not be published. Required fields are marked *