For many organizations, B2B integration needs are (still) commonly addressed with periodic file transfers using (S)FTP protocols. Frequently these are implemented using custom scripting and a simple scheduling mechanism (e.g. cron). In larger organizations there may be multiple departments managing their own file transfers (MFT). With the growing number of integrations, organizations realize they have limited visibility and traceability of the files being transferred, the maintenance is resource-intensive, and adding new transfers is inefficient.
Organizations in this situation are therefore looking for a solution to help them manage file transfers. An enterprise service bus product, such as Oracle Service Bus, is frequently already available within the organization, and offers (S)FTP connectivity, monitoring and alerting out of the box. Here is why this is probably not a good idea.
1. Technically complex configuration for (S)FTP connectivity.
– Two variants for communication are offered through the Service Bus (the native transports, and the JCA FTP adapter) with overlapping feature sets. This might require you to use both, obviously complicating the maintenance and operations of the solution.
– Complex and undocumented configurations to get to work. There are dozens of configuration parameters, and making the integration work with an unknown FTP server can therefore take hours or days, trying out different combinations of settings. Sometimes tweaking settings like “ignoreFilePermissions” is required in order to make the file transfer work.
2. Involved configuration and deployment.
– There is no consolidated overview of the configuration. Developers and operators are required to make some configuration changes in multiple places: via the Weblogic Console (for the JCA connection factories), the JCA file that is deployed to the Service Bus, and in the flow itself (proxy service or business service).
– Unclear behavior upon configuration changes – sometimes changes take effect immediately, often a restart of the managed server is necessary. Unfortunately it is not clear when this is the case.
3. No scheduling functionality built-in.
– In order to run the job at fixed moments in the day, week or month, it still needs to be triggered using an external mechanism such as the existing “cron” schedule.
4. Limited monitoring and alerting.
– Only after a file is successfully retrieved, the reporting, logging, alerting and dashboard mechanisms of the Service Bus are available. Otherwise, the only visibility is an error in the server log file. An external mechanism for monitoring the log files, and sending alerts must therefore be implemented additionally.
– There is an overview as to which connections to partners are established, but not a history of when connections were attempted and established successfully.
– In our experiments, troubleshooting connectivity and (S)FTP protocol issues was needed due to different behavior of each (S)FTP server. This requires debug flags to be turned on and the server to be restarted, and in our case this means we needed time from the server administrators and the restarts were affecting other users of the environment.
5. No DMZ support.
– With strict security requirements for network access around the demilitarized zone (DMZ), setting up file transfers between external FTP sites and internal systems can be a bit more involved. Depending on the access rules between the green zone and the DMZ, you may end up needing two service bus deployments (one in the DMZ, one in the green zone), to get this to work.
Conclusion
By using the OSB to manage file transfers you may be able to address the centralization of file transfer management and monitoring. But the complexities in configuration, deployment and monitoring lead to high and unpredictable time for implementation and maintenance.
So what’s the alternative?
There are specialized products for managing file transfer. One of them is Oracle Managed File Transfer (MFT) product, which comes with the Fusion Middleware 12c product suite, and is supposed to solve almost all of the aforementioned issues. It also integrates with the Service Bus or SOA Suite (both as consumer and provider of a service) and as such clear separation of duties can be established.
