A couple of months ago I started to use Amazon S3 (Amazon Simple Storage Service) as off site backup for one of my servers. It’s dirt cheap and so far it works very well (I’m paying less than $1 per month for my daily uploads and storage!)
The server is running Windows Server 2008 and my backup routine is simply a scheduled bat/cmd file to copy everything to a temp folder, then zipping it to a single file that I move off site (I used to ftp it to another host but you cannot use ftp with S3)
So, I needed to be able to upload my zip file to Amazon S3 using the command line. But unfortunately the only S3 copy or backup tools I found for Windows was either graphical and user driven or required Java, Perl or such and I didn’t really feel like installing any of that stuff on my server just for this simple task.
The solution was to write my own copy routine in C# (.NET Framework 2.0) which I did using one of the many S3 libraries available, in my case I selected the Affirma Threesharp C# Library. The result is a very simple copy routine, you can only copy one file at a time, no wild cards etc. I will probably extend it in the future, for example a delete routine so I can also automate deleting old backups (I’m currently using the s3fox plugin for Firefox to check on my backups and do deletes about once a month)
Syntax: s3copy Localpath Bucket Filename AccessKeyId SecretAccessKey
Example: s3copy.exe c:\backup\ mybackupbucket backup.rar MyAccessKeyId MySecretAccessKey
It’s a single exe file. No need to install anything, the dll is contained in the exe. Feel free to use it if you like!
Click button below download S3Copy.exe from Download.com
[ Edit/10 May 2009: Thanks to Brad! Corrected post + exe so that Access Key Id and Secret Access Key can be entered (you find your keys at Amazon’s AWS: Your Account + Access Identifiers) ]
[Edit/15 June 2009: Moved file download to download.com]
Great idea. I’ve been looking for something just like it. How do you authenticate to access a private bucket though? Seems that the only way to use the program is to give write access to everyone.
Hey Brad – do I feel stupid…
Uploaded a corrected version where you can add your info to authenticate. Please download and try again, use same download link as before (in post above) Thanks/Max
Hi, thanks for posting this. I am trying to add S3 support to an old ASP site and this will be a big help.
I seem to be having trouble directing the file to a folder in my bucket. Is there support for copying to a folder? Next release 🙂
thx
Hi, thanks for sharing! Just what I was looking for. One question (I don’t have S3 yet so I can’t test): does the program give a meaningful exit code?
Thanks,
Kevin
You can do subfolders. Just put on the bucket such as
bucket/subfolder
Works like a charm
It needs nothing to install? And what about .Net? Do I need to install it or not?
How does this handle ACL permissions? I want all copied files to inherit the permissions of the bucket.
Primarily I need Public Read for all copied files.
I’m getting an ‘invalid number of arguments’ error.
What am I doing wrong?
http://pastie.org/570207
Hopefully I caught you before moderation. I made the silly mistake of not escaping my second “. Thanks so much for a great script. It works like a charm.
Hi,
great work but I got an error:
status Code: Moved Permanantly
The bucket you are attempting to access must be addressed using a specific endpoint. How do I submit this endpoint?
Best Regards
Simon
How can I download a file from S3 to local PC or an EC2 instance using s3copy?
Thanks in advance
Program works great, except it seems to be limited on how much bandwidth it can consume. I have an open 10Mbps connection and it only uses about 20% of that, while other tools will use all available bandwidth. Any chance that could be corrected.
Well this is weird — I’m also called Max and have written a single-EXE S3 client for Windows! Maybe we are related. It does wildcards and a few other very simple things
Hi,
I am getting a funny message saying application failed to initialize properly 0x0000135. Any thoughts? I’m running windows server 2008 core.
Thanks,
Tom.
Tom,
I ran into the same problem. You need to download and install Microsoft .NET framework.
http://www.microsoft.com/downloads/details.aspx?familyid=333325FD-AE52-4E35-B531-508D977D32A6&displaylang=en
Nice. I was looking for a Linux command line or shell script and found this…works great for initial testing from my PC before I could move the solution to my Linux hosting servers 🙂
Thanks a lot.
Hi, I have a problem.
Credentials checked, pathes checked.
When I run from Windows Server 2008 (Amazon EC2 Instance) I get Access Denied message:
Copying file to s3… An exception occurred.
Message: Access Denied
HTTP Status Code: Forbidden
Error Code: AccessDenied
XML:
AccessDeniedAccess DeniedEA5FBF7D2889AC70TaqwfXXjs07al4ELyS147gjCz6qj/FMosLkxSJcZ6sxQ
0KnoqLw8UnmlVIefOpTV
Hi, again
I’ve found where was my fault. For bucket name I used word reserved for S3.
So use unique name.
How can I download a file from S3 to local PC or an EC2 instance using s3copy?
Thanks in advance
Is there any way of changing the default ACL with this? I would like the file to be publicly readable.
Would you be willing to post or share privately the source? There are a few tweaks I would love to make.
how about a way to move instead of copy?
Hello Phil,
Yes you can do this using my very own tool Bucket Explorer, You can use its commander version and set the bucket default permissions and meta-data as desired.
On uploading,copying the stuff in your S3 bucket it got automatically set for all the content being copied.
If you want to set Bucket Default structure and interested in 3rd party java based tool else please ignore
Thanks and Regards
Kirti
S3 Browser Freeware includes command line bulk uploader: s3browser-put.exe
Thanks! Just what I was looking for!
I’m wondered about some antivirus detecting malicious software in this file. I followed the link, clicked ‘Download’, got a copy of cnet2_s3copy.exe and uploaded it to Virustotal. 3 of 42 anti-viruses detected something malicious. Am I doing something wrong?
Hello,
Thanks for your great job!
could it be possible to have the source file ?
Best regards
I can’t get files to a bucket and they and they company I’m trying to get it to even created me my own bucket and I still can’t. I get
Message: The request signature we calculated does not match the signature you pr
ovided. Check your key and signing method.
HTTP Status Code: Forbidden
I don’t know what I’m doing wrong.
The bucket just starts with s3.//ThierCompany.name.import.prod.julie