Monitoring Your FTP File Transfers

Often you need to monitor your FTP file transfers on a regular basis. Maybe customers submit orders electronically for processing, or you need to get marketing data from several different systems to pull into your data warehouse.

You can set yourself reminders and keep logging in to check for newly uploaded files. Or, you can take advantage of technology to monitor and even automate some of your file transfer processes.

ExaVault has several tools you can use to monitor file transfer activity in your account. Get notification emails, set up polling, or use webhooks. Depending on what you want to monitor and your technical ability, each option has benefits and possible downsides.

Notification Emails

Notification emails let you monitor uploads and downloads to your file sharing account. This type of FTP file transfer monitoring is easy to set up. Notifications can be sent to multiple email addresses or sent as a group email. Because this is a low tech option, it is ideal for cases where files need to be manually reviewed by a person, or when more technical choices aren’t feasible.

Monitoring file transfers with notification emails is also handy when dealing with intermittent transfers, or when you don’t know how frequently an upload will occur. 

With notification emails, you don’t have to use FTP to pick up the files. Files uploaded to your ExaVault account via FTP are available to preview or download from the web interface. Using email notifications is a basic tool that provides a way to monitor the flow of files to your FTP service.

Monitoring file transfer with notification emails is a more manual process. There is no automation, and you will always need a person to go and pick up the files. Some people are not a fan of email. If a monitored folder gets heavy usage, such as multiple uploads daily, it can result in a lot of emails to sort through.

Polling

Polling is another way to monitor FTP file transfers. This type of monitoring allows automatic connections via scripted S/FTP at regular intervals to check for new files to download. 

However, polling requires more technical skill to set up, such as a developer or someone familiar with scripting. Generally, you need some sort of scheduled script on your local machine to reach out and open a connection to the server to facilitate the transfers.

WinSCP and other FTP clients have some built-in scripting options.

If you have the ability, or IT staff to do it, polling is an excellent monitoring tool. Once you’ve done the initial work, it’s generally set it and forget it. Polling can also be worked into other automated parts of the workflow. Perhaps an external company uploads files every hour, so you set your script to log in and look for new files to download each hour.

Downsides of Polling

Polling is trickier to troubleshoot compared to other monitoring options. If something goes wrong, there’s no immediate visibility. An incomplete upload or issue connecting to the server may happen with no indicator that a file should have been uploaded to your account. You just don’t see new files. 

Polling also uses a lot of overhead on the server. Polling limitations on servers generally allows checks every 5-15 minutes. Frequent polling can be a downside if your FTP service has transaction limits.

Webhooks

The third option for monitoring FTP file transfers is webhooks. Webhooks send a JSON message automatically to your application when files are uploaded (or downloaded.) Your application can then use the API to do the next step of the transfer process. For example, downloading a recently uploaded file.

Webhooks are the best choice for end-to-end automation. They provide real-time messaging, so there is no delay between the triggering action and the message sent to the waiting application. Webhooks also use less overhead than polling as messages get sent only when a triggering event occurs. There is less chance of setting off or exceeding account limits. 

Another benefit of setting up webhooks to automatically monitor file transfer is that it offers more flexible triggering events, not just upload or download. 

There is so much you can do with webhooks, but they do require more developer involvement than polling to both set up and maintain. To start, you’ll have to have developer access to your application. You’ll need to be able to modify your application code to handle webhooks messages and make API calls. 

On the downside, it is not uncommon for systems and FTP services to limit the number of webhooks or API calls allowed in a day. ExaVault doesn’t limit webhooks messages, but there is a limit for API calls.

Activity History

A final way to keep track of your FTP file transfers is by checking the activity history. ExaVault keeps a record of all actions on your account in the last 90 days. You can view all activity or filter to view just FTP file transfers.

As a backup to other monitoring tools that help with workflows, activity history can be a great spot check to verify particular actions. The logs can be sorted by type of action (find all uploads,) by user, even by IP address. Activity logs let you verify that all transferred files have been handled. You can look for trends, see who is using FTP vs. the web browser interface, and check that users are retrieving their files.

Start monitoring your FTP file transfers today with ExaVault!