S3 Authentication with IAM Role credentials

This is a technical post for users that are familiar with AWS EC2.

When you are launching an EC2 instance you can assign it with an IAM role with specific permissions to AWS resources. This enables you to use security credentials on EC2 to access S3 that are temporary and rotated automatically. Refer to IAM Roles for Amazon EC2.

We have now added a login option to the current snapshot builds of 4.7 available through a connection profile that allows to connect from EC2 with Cyberduck on Windows or the CLI version on Linux instances with IAM role credentials obtained from instance metadata.

The S3 (Temporary Credentials) profile can be installed from the wiki. See Connecting with temporary access credentials from EC2. Make sure to edit the role name in the profile to match your IAM configuration.

AWS Authentication with IAM Role on EC2

We think this will make deployments of the duck to EC2 a lot easier without the need to distribute credentials along with.

The cloud in numbers

Guest post from Qloudstat.

Cyberduck has for a long time supported access logging configuration for AWS S3  & CloudFront, Rackspace Cloudfiles CDN and recently Google Storage.

Qloudstat comes to the rescue when it comes to draw a conclusion from the thousands of log files written to your logging container every day plotting data in beautiful graphs.

Qloudstat is doing analytics for not only hits but other crucial metrics such as bandwidth and transfer costs. Data is plotted in an interactive chart, timeline, table or geographical map split up by different dimensions in a user friendly web interface. Gain insight of URIs and filetypes used. List referring sites and search keywords. Analyze HTTP user agents, operations and status codes. Compare CDN edge locations. Visualize requests on a country, region or city map. The analytics provided are not static but queries can be made for for any given daily time period with custom filters applied. Dynamic reports are rendered instantaneously not dependent on the time period chosen to be visualized.

Reports are updated continuously around the clock to give instant and always up to date statistics no matter of access volume or the number and size of log files. Qloudstat copes with the rapidly growing traffic using highly optimized scalable systems without any installation, administration, maintenance and infrastructure required by clients. During the 3 months private beta the service was already operational with customers facing up to 25 million hits per day.

Qloudstat integrates with the market leaders in cloud storage and content delivery networks (CDN) currently supporting AWS S3, AWS CloudFront, Google Cloud Storage and Rackspace CloudFiles (Akamai CDN).

Security is a pivotal requirement for cloud based applications. Qloudstat accesses log files in third party accounts using dedicated security credentials either using OAuth authentication for Google Cloud Storage or a user managed under the Identity and Access Management (IAM) for integration with Amazon Web Services. Log files are fetched using a TLS secured connection and the website to access reports is only available with HTTPS as well.

Pricing is based on raw log data volume with four different monthly subscription plans offered. Additionally a free plan allows to use the service at zero costs for sites with less than 100’000 hits per month. With data liberation in mind, export formats for further external processing are provided.

Give it a try, there is a zero cost plan to get you started.

Enable server side encryption for AWS S3

The latest snapshot build of Cyberduck for Mac & Windows allows the configuration of server side encryption for files in Amazon S3. Server side encryption provides you the ability to encrypt data stored in Amazon S3. You can encrypt data on upload simply by selecting AES256 as the default encryption algorithm for uploads to S3 in the Preferences or selecting the checkbox in the Info panel for data already uploaded. Decryption happens automatically when data is retrieved.

CloudFront for the rest of us with custom origins

Configuring a bucket in S3 to make its content available through the Amazon CloudFront CDN (Content distribution network) was always most straight forwarded in Cyberduck. Both distributions for static files and streaming can be configured using the Distribution tab of the Info panel when connected to your S3 account with a snap.

Because Amazon has now lifted the requirement to upload your content to S3 to make it available in CloudFront, their powerful CDN feature can now be used with any server used as the origin source for content. This is especially useful for content generated only on demand or where your workflow just doesn’t allow to upload content additionally to S3.

As of the latest beta and snapshot builds of Cyberduck, you can connect to any server using FTP or SFTP and configure it as the origin of a new Amazon CloudFront CDN distribution. We expect this to make CloudFront a lot more popular for users that are tied to existing server resources but want to make us of content distribution in the cloud.

Detailed usage instructions can be found in the help wiki. Give it a try!

Versioning

Amazon S3, the cloud storage service Cyberduck can connect to, supports a versioning feature. This allows to revert to previous revisions of a file replaced in S3. The latest snapshot build of Cyberduck supports versioning in S3 in the following way:

  • Toggle the versioning status using File → Info → S3
  • Display previous versions and deleted files in the browser using View → Show Hidden Files. Past revisions are displayed in a grey text colour and can be downloaded or permanently deleted like regular files.
  • Revert files to a previous version using File → Revert or the Browser context menu.
  • Optionally enabling Multi-Factor Authentication (MFA) Delete