-
Notifications
You must be signed in to change notification settings - Fork 28
08) Integration Patterns
File Transfer - The File Transfer integration style is pretty simple, basically one application produces a file, a subsequent system(s) will pick up that file, transform the data and then load it into its system. Sometimes this integration style is challenging due to the fact that some systems cannot produce files in a standard format like JSON or XML, especially older legacy systems. There are also other drawbacks to this approach, such as timing and the fact that it's a snapshot of data in a system at a specific moment in time. In most scenarios it isn't a great idea to use this technique as it cannot hold up to current consumer expectations for receiving their data. That said there are still a one decent scenario for using this approach, replicating data from one system to another for reporting purposes when a system does not support API's and doesn't grant you access to it's data via any other forms.
Unified Data Store - This integration style basically has multiple applications rely on a single common database that they all interact with. In today's world this approach isn't really used and likely shouldn't be used, but there are places experimenting with it again due to the concept of data lakes. The problem with this approach is the dependencies between the applications and the unified database, it can also easily create bottlenecks and can be a very expensive solution since many applications are not inherently built to work with a database outside of their own.
Remote Procedure Invocation - Remote Procedure Invocation or RPI is a style that is used when you want to call a specific piece of logic housed in another system. Say you have two systems and system A is responsible for leads for new franchises and system B is responsible for generating and sending those leads documentation on how to become a franchise if they are qualified. System A would reach out to System B and just call the exposed method for generating and sending the documentation for those leads. This is most often done via web services.
Messaging - Messaging is an asynchronous integration style where two or more systems are tied together via a message bus, which has channels (virtual pipes) that connect the message sender to the message receiver. It uses message routers to determine how to navigate through the channels to deliver the message to the right receiving system. In this style the systems sending and receiving do not have to be up at the same moment in time, also due to the asynchronous nature of this integration style it lends itself to both scalability and flexibility.
https://resources.docs.salesforce.com/sfdc/pdf/integration_patterns_and_practices.pdf
Point to Point Integration - Disclaimer, this approach should NEVER BE PRESENTED IN A BOARD!! You should utilize middleware like Mulesoft always, but it's important to understand why you shouldn't use it, so let's go through that.
Problems with P2P Integration:
-
The amount of unique channels established between systems can become enormous. To find out how many P2P integrations may become necessary between systems use the formula N(N-1)/2 replacing N with the number of systems being integrated together with P2P integrations. With just 5 systems we get 10 integrations, with 10 systems we get 45 integrations, so you can see the number exponentially grows with each system in the P2P integration chain.
-
If a system becomes deprecated and another takes its place, it can be very hard to switch all P2P integrations to that new system. Each one could have varying business logic, outdated security standards that the new system won't support, etc. It can become very time consuming and costly.
-
Each P2P integration needs its own retry mechanisms and error handling. As opposed to a hub and spoke model (middleware) where you could just create this once for all integrations with the system being connected to.
-
Reliability is normally substantially lower for P2P integrations. For instance if you had a chain of systems integrated System A -> System B -> System C the reliability could be calculated by using System A's Reliability * System B's Reliability * System C's Reliability. So if Sys A's reliability was .7 and Sys B's reliability was .8 and Sys C's reliability was .65 then the reliability would be .7 * .8 * .65 = .364. Not super ideal. The hub and spoke model significantly increases reliability.
As you can see by the above points. It's not a great idea to go with a P2P integration solution, especially for a larger organization. It doesn't scale well with your business, it can be very unreliable and it can lead to some highly dependent applications that slows down technological change in your organization.
Extract, Transform, Load (ETL) - The ETL method of integration is basically when data is pulled from one or more data sources and then transformed into the format it needs to be to be loaded into one or more systems. After transformation it is then loaded into one or more systems. It's kinda like file transfer, but has some key differences. The basic principles of ETL are as follows:
- Access and then extract the data from one or many data sources.
- Transform the data by formatting the data, validating the data, cleansing the data, etc
- Access the system or systems you are loading the data into and then load the data into them.
Most ETL jobs are scheduled jobs that run on specified intervals, however, they can also be triggered using an HTTP listener in some cases as well.
Some of the more popular ETL tools available for SF are Informatica PowerCenter, Informatica Cloud, Talend, Jitterbit and MuleSoft.
Enterprise Service Bus (ESB) - ESB is a method of data integration where applications are integrated via a communication bus. All applications only communicate with the bus instead of each other.