Delighted to have received the CH Open Source Award 2015.

There have been requests since a long time that we should provide a way to mount remote server storage as a volume in the Finder.app and File Explorer on Windows, respectively.

We are excited to announce a private beta for Mountain Duck – “Cyberduck for mounting volumes in the file explorer”. It comes with a minimal user interface as a status bar application supporting all protocols available in Cyberduck. Mountain Duck mounts the remote storage as a volume to allow any application to access the files for reading and writing.

In contrary to other solutions available for the Mac, Mountain Duck requires no installation of kernel extensions or system libraries and runs sandboxed on OS X 10.8 or later.

Please request an invite to the private beta and let us know how it works for you. Mountain Duck will be available later this year in the Mac App Store and as a Windows Installer.

We have introduced support for the iRODS protocol (refer to our documentation) in collaboration with iPlant at the BIO5 Institute. Below two excerpts from the iPlant Collaborative Press Release.

(…)

iRODS is widely used by thousands globally to manage their data in a variety of disciplines, including hydrology, astronomy, engineering, and life sciences. Thus, the new Cyberduck plug-in effort led by iPlant will have a global impact to improve convenience of data transfer for scientists working with large-scale data.

(…)

Using Cyberduck, Barthelson was able to download a 2.8 Gigabyte file in 3 minutes, transferring the content at a rate of approximately 1 Gigabyte per minute into iPlant’s Data Store, which provides a reliable and redundant repository for storage, analysis and sharing of scientific data.

Cyberduck supports transfers between servers regardless of the protocol by simply opening two browser windows connected to different hosts and drag file and folders between.

With Cyberduck CLI, there is now also the option to automate such tasks with a script running on Mac, Windows or Linux to keep a current copy of files on a different host for failsafe access. Use the --copy option to do the same but allowing you to automate the task. Consider you want to copy files from Amazon S3 to Rackspace CloudFiles running OpenStack Swift, all you need is the command

duck --copy s3://<Access Key ID>@<bucket>/ rackspace://<Username>@<container>/

to copy all files. Use the --existing compare option for subsequent invocations to only transfer new and changed files.

Server side encryption for file uploads to S3 is not new and supported since version 4.2. Still we want to give short summary how to ensure all files uploaded to a bucket are protected.

• Create a new bucket in S3 choosing File → New Folder… in Cyberduck. Refer to Create a bucket.
• Enforce encryption by applying a bucket policy using the AWS S3 console. Choose Edit Bucket Policy and enter the policy

  {
   "Version": "2012-10-17",
   "Id": "PutObjPolicy",
   "Statement": [
   {
     "Sid": "DenyUnEncryptedObjectUploads",
     "Effect": "Deny",
     "Principal": "*",
     "Action": "s3:PutObject",
     "Resource": "arn:aws:s3:::bucketname/*",
     "Condition": {
       "StringNotEquals": {
         "s3:x-amz-server-side-encryption": "AES256"
       }
     }
    }
  ]
  }

• In Preferences… → S3 → Encryption choose AES256 as the default.

You can verify an object is encrypted in S3, by choosing ⌘-I for a object and choose the S3 tab. Ensure the Server Side Encryption checkbox is selected.

References

• Protecting Data Using Server-Side Encryption with Amazon S3-Managed Encryption Keys (SSE-S3

With Cyberduck CLI there are many new ways automate tasks that require to transfer files to a remote server with duck.. We have added a section to our wiki to show use cases.

One such is making use of fswatch, a file change monitor that receives notifications when the contents of the specified files or directories are modified. Used together with Cyberduck CLI, you can keep a remote directory in sync with local changes.

fswatch -0 ~/Sites/mywebsite/ | xargs -0 -I {} -t sh -c 'f="{}"; duck --upload ftps:///sandbox`basename "${f}"` "${f}" -existing overwrite'

The command will make fswatch listen for changes in directory ~/Sites/mywebsite/ and upload any changed file into the remote folder ftps:///sandbox using a FTP-TLS connection overwriting any existing files.

Version 4.7 of Cyberduck brings support to use multiple connections for downloads and uploads. When there are many small files this can double or triple transfer speeds because of the parallel execution. Enable Multiple Connections per default in Preferences → Transfers or when per bookmark. The maximum number of connections to open is limited with the control in the lower right of the Transfers window. Please refer to our documentation.

This feature is supported for all connection protocols for downloads & uploads. Additionally large single file uploads to OpenStack Swift (2GB threshold) and Amazon S3 (100MB threshold) will use multiple connections and upload the file in segments.

We have also worked on improving throughput for downloads with segmenting single large files into parts. This will split a download into multiple segments for concurrent transfer and assemble the file when complete. This feature is currently experimental and can be enabled manually. Refer to the documentation.

To use multiple connections for file transfers with Cyberduck CLI, use the --parallel option.

We have put some more love into duck, the CLI version of Cyberduck. There are now several new options to facilitate automation jobs.

• --retry <count> allows to repeat failed connections.
• -y will dismiss all prompts and warnings with Yes by default.
• --throttle <bytes/second> allows to throttle transfers.

plus the most interesting to improve throughput is

• --parallel <connections>

to open multiple concurrent connections for transfers (#1135). For downloads, larger files will be segmented into multiple parts (#4498) to transfer with multiple connections and assembled again when complete. This feature will also find its way into version 4.7 of Cyberduck.

We have readied a command line interface (CLI) version of Cyberduck for power users. It is named duck and runs in your shell on Linux and OS X or your Windows command line prompt.

The CLI version includes the power features of Cyberduck ready to be used in scripts and automated deployments. Edit files on remote servers, download, upload and copy between servers with FTP, SFTP or WebDAV plus support for cloud storage Amazon S3 & OpenStack Swift deployments. Let’s looks at some examples what in particular duck is useful for.

Copy a file from an FTPS server to a Rackspace Cloud container

Note the URI syntax protocol://container/key used for OpenStack Swift, Microsoft Azure and Amazon S3.

osaka:~ dkocher$ duck --copy ftps://dkocher@cyberduck.io/update.cyberduck.io/Cyberduck-4.6.1.zip rackspace://sandbox/Cyberduck-4.6.1.zip


FTP-SSL connection opened…
Login cyberduck.io with username and password. No login credentials could be found in the Keychain.
Username (dkocher):
Login as dkocher
Password:
Cloud Files connection opened…
Login identity.api.rackspacecloud.com with username and password. No login credentials could be found in the Keychain.
Username (dkocher): dkocher
Login as dkocher
API Key:
[▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮ ] 37.4 MiB (39,239,680 bytes) of 55.1 MiB (67%, 694.9 KB/sec, 27 seconds remaining)


Upload a website

Recursively upload a folder to a remote location. An overwrite prompt will ask the action to take for existing files or can be specified using the --existing option.

osaka:duck.sh dkocher$ duck --username dkocher --upload ftps://cyberduck.io/duck.sh/ output/


FTP-SSL connection opened…
Login cyberduck.io with username and password. No login credentials could be found in the Keychain.
Username (dkocher):
Login as dkocher
Password:
Login successful…
The remote file duck.sh already exists. Choose what action to take:

Resume Append existing files (resume)
Compare Skip files that match size, modification date or checksum (compare)
Rename existing Rename existing files with timestamp appended to filename (rename)
Overwrite Overwrite all files (overwrite)
Cancel (cancel)
Skip Skip transfer of files that already exist (skip)
Rename Rename transferred files with a timestamp appended to the filename (similar)

Action [resume, compare, rename, overwrite, cancel, skip, similar]: compare
[▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮▮] 1.0 MiB (1,067,009 bytes) of 1.0 MiB (100%, 88.6 KB/sec)


Edit a remote file over SSH

Connect using public key authentication using --identity for the private key.

osaka:~ dkocher$ duck --username dkocher --identity ~/.ssh/cyberduck.io-rsa --edit sftp://cyberduck.io/etc/httpd/vhosts.d/duck.sh.conf


List all buckets in a given region in S3

Note the --region parameter that limits the output to containers that are in the Frankfurt (eu-central-1) AWS data center.

osaka:~ dkocher$ duck --username AKIAJOC4AH3QDQUB3C7Q --password ugQE+uj8tZyywd9hjoyZ5H5EaPcMuJ4Pk2C7jF85 --list s3:/// --region eu-central-1

List files in a Rackspace container

Using the --retry option to retry the task on a networking failure.

osaka:~ dkocher$ duck --list rackspace://sandbox/ --retry
Cloud Files connection opened…
Login identity.api.rackspacecloud.com with username and password. No login credentials could be found in the Keychain.
Username (dkocher): rackcloud
Login as rackcloud
API Key:
Login successful…
Cyberduck-4.6.1.zip


Check out the output of duck --help and the Cyberduck Wiki for more help about available commands and options.

To provide a seamless setup we provide Homebrew packages on OS X, package repositories for YUM and APT on Linux and a Chocolatey package on Windows. Besides there are regular installers for all platforms available if you use no package manager.