E-what? A quick game of buzzword bingo
On this website we will introduce you to concepts that relate to the interactions between different parts of the IT landscape of an individual company, or of multiple related companies. By doing so, we hope to shed some light on the countless acronyms, keywords and marketing terms you will encounter. You can find more information about all of these concepts throughout the rest of this document.
EAI
EAI stands for Enterprise Application Integration. The word “Enterprise” refers to businesses or organisations. EAI involves integrating all the different applications within a business or organisation – or in other words, joining them together in a network. This is not done by arranging for each individual application to be able to talk to every other application; rather, it involves creating an intermediate layer – a kind of shared communications platform – which the different applications are connected to via communications interfaces or adapters, so that almost all these applications can communicate and collaborate with each other. This is also where the processes – or the rules of collaboration – are defined.
EDI
EDI stands for Electronic Data Interchange. It sounds banal, but it’s highly complex. In a sense, even something like sending a fax is a form of electronic data interchange – after all, the fax contains data and is transmitted electronically. However, this example has little in common with the EDI we are talking about here. EDI is intended to allow applications (by which we mean programs) to exchange data with each other largely automatically, and across the boundaries of the enterprise. This is very similar to EAI, but it takes place between two or more companies instead of within a single company. For example, when a customer’s software sends an order directly to a supplier in the form of a file and that order lands in the supplier’s ordering system without human intervention, that’s EDI.
ESB
ESB stands for Enterprise Service Bus, which is a somewhat vague idea. An ESB can be a concept, a technology, an infrastructure or a piece of software designed to support the integration of different applications within an enterprise, a department or a project. In other words, the ESB can form the foundation for EAI. It connects different applications via adapters, transports data between them, and helps to transform the data so that the systems can understand each other. However, it has no understanding of more complicated relationships and business processes, as these things once again fall under the category of EAI.
ETL/ELT
This acronym stands for Extract, Transform, Load and is synonymous with retrieving (or extracting) data from one or more systems, transforming them, and loading (or strictly speaking, saving them) to a target system – typically a database. ELT means the same thing, just in a different order. Here, the data are first pumped into the database before they are transformed. All this falls under the scope of the data warehouse. And the point of it all is to process data from different sources and systems in a standardised manner in order to make them available for statistics and analysis (among other things). This standardisation is brought about through transformation, which not only converts the different formats used in the source systems into a standard format for the database, but also improves the quality of the data at the same time (by eliminating duplicates and so on).
SOA
SOA stands for Service-Oriented Architecture. This refers to many different applications – which can even be spread throughout the world – providing an interface through which their services can be requested. In a general and global sense, this might involve calculating the distance between two coordinates, or in a highly specialised case within an individual company network, it might be asking when a particular lorry is available for a delivery. Both of these services may also be needed in combination in order to deliver goods, so there may also be an additional service that calls up the other two “under the bonnet”.
However, nobody needs to know how the service in question is implemented; they only need to know how to use it. For that reason, only the interface to the service needs to be known to the outside world. An example of this is web services for data exchange.
Is this something I need?
If you have read through the buzzword bingo, and ideally the detailed descriptions of the individual concepts too, then you will now have a rough idea of what terms such as EDI, EAIor SOS mean. The question now is whether you would need something like this yourself, and if so, what exactly you need.
Question 1: What does your IT landscape currently look like?
There are still companies whose IT systems consist of a few PCs with MS Office and an email program. Orders arrive over the phone or via fax, the warehouse manager has the inventory in his head, and delivery notes are filled out by hand. Larger-scale IT systems are nowhere to be found. If that works well for you and there aren’t any business partners demanding EDI processes, you can forget about the subject for now and put this manual to one side.
However, most companies nowadays no longer work like this, and once they reach a certain size (or work with big enough business partners), they have no choice but to digitalise one business process after another. And as mentioned in the description of EAI, that often happens bit by bit. Somewhere along the way, you end up with a diverse
assortment of IT systems, all of which have different data formats and interfaces and are unable to communicate with each other.
Perhaps you’ve already gone through this phase, and have spent the last few years cobbling together a mishmash of C programs, shell scripts or even Excel macros that you use to get data from one system to another. Plus, let’s not forget your major customers or suppliers who have recently begun lumbering you with data in a format specified by them. And now those customers are threatening you with penalties for not sending their data back to them in the same format, so your programmer is ready and waiting to rustle up a bunch of new programs and scripts to solve the problem – each of them will somehow carry out a particular task (though it will be a miracle if anyone still remembers how six monthslater). And if a business partner comes along with slightly different requirements then you’ll suddenly need another few thousand lines of code.
Please, pull the emergency cord! If you carry on this way then you’ll end up in a very sticky situation. And that’s not to mention what would happen if your programmer went missing – at least within your organisation. In that scenario, you would be left with endless different sources and absolutely no idea what is going on in the background or how you could ever make any changes to it.
Question 2: How precise are your future requirements?
What has been happening so far in the space between your individual IT systems, and between your business and the outside world? You should start by analysing this in depth. Has communication with the outside world virtually ground to a halt? Do data need to be transported and reconciled between your own systems in order for that to change? In that case, you are looking more at EAI, and possibly also ETL.
Do orders come in from outside your organisation, and do order confirmations, delivery notes and invoices need to be sent? Do you regularly send electronic catalogues to your customers? Do you have lorries on the roads whose locations can be tracked (via GPS services)? Whenever data flows to you from outside your company or vice versa, that falls under the category of EDI.
What kinds of systems do you have in your organisation? Databases? SAP? Older ERP systems that can only import and export proprietary file formats? All this will determine how and with whom a possible future EAI or EDI system needs to be able to communicate. And how does the outside world talk to you? Do you receive Excel files? Will that remain the case? Shouldn’t your system also be able to handle general standards in order to be future-proof? Standards such as EDIFACT, X12, XML, and so on? Does all your communication take place via email, or are secure transfer paths such as AS2 or SSH-based communication required?
These questions will determine what your future system needs to be capable of. And a lot of different things come together when you start thinking in these terms, don’t they? A specialised EDI or EAI system may no longer be adequate, as the tasks of both areas need to be covered. Oh, and let’s not forget about regular reconciliation between multiple internal databases, which falls under the scope of ETL. And then there are your customers, who would like to send you queries about your current inventory (for example). That problem can be solved brilliantly using web services. Then you can deploy either multiple specialised tools for each specific area, or a single tool that may not match the optimum performance of those specialist tools, but does bring everything together under one roof.
Question 3: What should each tool offer you?
Is it alright for the tool to require a specific system (Linux/Unix, Windows, or even AS400/iSeries)? Or do you expect it to be platform-independent? Or better yet: should it be installable in just a few clicks via a container-based method using Docker, with no programming required? What’s the operability like? Should you have to learn a sort of programming language before you can get the software working? Or should the focus be on professional matters, with the software operated more intuitively? With the latter, do bear in mind that EDI and EAI involve highly complex processes that even a professional can’t simply patch together in just a few clicks – so please manage your expectations when it comes to intuitiveness!
What role does licensing play in your business? Do you buy licences or rent services? The ideal solution would be highly flexible. In other words, on-premises (cloud provider), with no additional technical expense. For that, EDI experts recommend the option of a dongle-based installation.
What does the monitoring look like? Do you always have a clear overview of whether everything is working, or whether there are any errors that need to be fixed? Does the system actively notify you of any problems?
How important is transparency to you? Why not simply open your browser to display all important system information in a quick and straightforward way in modern HTML5? All under the motto “any place, any time, any device” – whether laptop, tablet or mobile phone.
Data security is also a crucial consideration for business-critical data. For example, can invoices be archived for the period required by law? Are all important data and logs kept available for inquiries at a later date?
Further specialised requirements may also come into play, especially in the EDI field. For example: in some countries, digital signatures are still required for certain types of data (especially invoices). That obligation has long since been abolished in Germany, for example, but you might need to be able to integrate a signature system in your country of business or when you do business with certain other countries.
On top of that, a standard tool often provides you with an endless list of functions – and yet Murphy’s law dictates that you’ll end up with a highly specific requirement for which your tool doesn’t provide an ‘out-of-the-box’ solution. Will there then be options to extend the system yourself? Not everyone has their own in-house programmer – but even if you do, the software you deploy should still offer interfaces that make it possible to manage things and add your own extensions. Or should the manufacturer at least offer to develop those extensions for you?
Last but not least: What’s the situation with support? How quickly does the manufacturer respond to problems? Do they also offer help with more straightforward questions? What do people say about them? Spending hours on hold for the customer hotline is a thing of the past. Why not receive direct support in just one click with the help of detailed explanatory videos or contextual documentation?
When you were reading question 1, did you realise that a tool like that wouldn’t actually be such a bad idea? Then you should think carefully about the points listed under questions 2 and 3 too. If you know what you expect the software you’re looking for to provide then take a look around on the market, have your key scenarios evaluated by different
manufacturers (how expensive are they, and will they even work?), and see which product meets the majority of your requirements. It might not be the cheapest one, but it’s better to make the right investment up-front than to regret buying the wrong program after endless rounds of breakdowns, extra work and lost time.
To round things off, you can also take a look around on the Internet or ask other companies in your industry whether anyone has any experience with your preferred product and find out what they have to say about the product, the manufacturer, the support etc. from their own experience.
The EDI experts at Lobster GmbH will also be happy to help answer any of your questions about EDI and EAI (info@lobster-uk.com).
What do we need by “beyond”?
At the start of this manual, we explained the terms EDI and EAI (Electronic Data Interchange and Enterprise Application Integration). Put very simply, these terms relate to the transportation of data between different software systems and the transformation of those data into different formats, with the difference between EDI and EAI being more organisational than technical in nature.
In classic EDI/EAI, the transformation is limited to syntactical changes. In other words, a file in XML, CSV or EDIFACT format is converted into CSV, fixed record, Excel, IDOC, etc. The field contents are either copied across unchanged or subject to formal transformation. Examples of formal transformations include different date formats, decimal points, fill characters, units of quantity, and so on.
However, as the interfaces grow more complex, so do the requirements placed on the internal logic of the interfaces. As a result, more and more internal functionality is shifted to the interfaces. This internal functionality might relate to typical business logic (item numbers, delivery blocks, checks on delivery dates, and so on).
But in other cases, a separate logic may be required that is used solely to return data to the person submitting the query. In addition, data may also be generated during conversion that add to or expand upon the logic previously available in the company.
We have listed a few examples below that go beyond the classic terminology of EDI/EAI and help clarify the explanation provided above:
- Checking reference data
Master data such as item numbers and custom numbers are checked for validity during conversion. If any values are invalid, the data might be rejected with an error message, or the incorrect data might be replaced with dummy values. During replacement with dummy values, the value received is either transported to the destination system in a text field, or forwarded via email. This solution is used when the destination system is unable to process errors in the master data as conveniently as required. - Notifying specialist departments
During conversion, a check is carried out to gauge whether the order is an urgent one. If it is, the specialist department is notified via email at the same time as the conversion is made in order to avoid losing any time. This solution is used when the destination system is unable to display urgent orders separately, quickly or clearly enough to users.
- Building a tracking system
When order data are received by a company with a heterogeneous software landscape, the orders pass through various different programs before the goods eventually leave the company. If there are questions about the status of those orders along the way, company employees will need to go to the trouble of looking them up. The effort involved in this can be reduced if the order status is written to a database during conversion between the different programs. This limits enquiries to a single system. It also offers the opportunity to enable even external partners to make their own enquiries via a web interface. - Temporary storage of partner reference values
When sending their data, the partner will include reference values that need to be included in any responses sent back to them. Yet these reference values aren’t used in the destination system, and there also aren’t any fields available for them. During conversion, the reference values are temporarily stored in a database from which they are then read when the response is sent. This solution allows you to avoid improper use of fields in the interface and the destination system. It also makes it possible to avoid having to customise the destination system. - Aggregating order items
During conversion, multiple items appearing in an order or multiple deliveries to the same customer (to take two examples) are aggregated within a single transfer. This is typical business logic that should only be implemented during conversion if the destination system is unable to deliver the required functionality. - Recording data for statistical purposes
During conversion, additional data relating to volumes, insured values, numbers of items etc. are written to a database. These data can then be statistically analysed at a later date using external programs.
- Generating supporting documents for EDI processes
During conversion, the data are used to generate additional documents for human readers in PDF or Excel format, before sending those documents to the partner. Documents of this kind are mandatory for some companies, as with summary invoices (for example).