Eating our own dog food – how we automate file uploads to AWS S3

by | Feb 11, 2020

Early in my career, I learned the importance of “eating your own dog food.”  Product development teams tend to focus on the methodology du jour and geeky coding activities. But this loses site of the real reason for developing software in the first place: to solve problems for customers so that you create value in your solution and earn money for the company. It is by using your own software to solve real-world problems that your company faces that you truly understand what is important in your solution.

Here at Coviant, I am proud to say that we eat our own dog food. We use Coviant Diplomat MFT to automate and secure file transfers for numerous business processes, and I would like to share one such case that arose recently. I think this might help others to solve real world problems, too!

The Challenge – how to add automatic file uploads to S3 to our software delivery pipeline

We recently decided to shift our software installer downloads away from our web hosting environment and onto AWS S3. S3 provides cost-effective storage and fast, easy customer access to our numerous, large product installers.  But how do we get our finalized product installers automatically uploaded to S3?

We wanted to upload the product after the software delivery pipeline had finished its automated building and testing procedures. This would ensure that we were upload an verified version of the installer.  And automating this process is important for efficiency and to reduce the risk of human error.

The Answer:  Eat our own Diplomat MFT dog food!

Luckily – or, more appropriately, by intentional designCoviant Diplomat MFT provided the easy answer. It is simple to create a Transaction in Diplomat MFT which took matching files from a given network shared folder and uploaded to a given bucket in AWS S3. We enabled this transaction for execution by external invocation, and easily added a step in our software delivery pipeline to invoke the Diplomat Scripting Agent that triggered the transaction at the appropriate time.

Diplomat MFT Screenshot

Diplomat MFT Moving Files to AWS S3

Thus, in two very easy steps we were able to use Diplomat MFT in our software delivery pipeline to automate the process of uploading our final installers to AWS S3.  This also provides us with all the robustness inherent in Diplomat MFT, including file overwrite rules, error recovery logic, audit trails, and archiving of all uploads.  No crazy home-grown scripts or unsupported 3rd-party plugins (with requisite crossed fingers) for us!

Wrapping Up

By eating our own dog food, we better understand the challenges that our customers face, and end up feeling the same pain that our customers do. As a result, our solutions become better because as we solve those problems for ourselves, the value becomes intrinsic in the product itself.