An Industry Unplugged: Accelerating Trading by Addressing Manual Processes
From trade entry through to invoicing, the manual and often fragmented nature of processes has a negative impact on energy trading activities. We highlight several commonly observed inefficiencies across trading and post-trade operational processes, and compare automation rates in crude and refined products, gas, and the power industry to highlight potential remedies.
By Russell Meyer and Sudhir Devappa
TRADE ENTRY
To begin, let’s focus on an often-overlooked task: trade entry. Many trading companies lack solutions that are fully integrated with the exchanges and brokers that execute their trades. As a result, traders must manually enter exchange-traded transactions into the company’s trading system. It is the same for over the counter (OTC) or bilateral trades. It is also important to note that trade details may be needed into multiple systems. Based on our research, we found that trades exist in up to five systems within mid-sized energy firms, rising to as many as ten in the case of multinational energy companies.
Some energy trading organizations have created interfaces between systems to reduce manual data entry. However, even in these companies, it is still commonplace for users to input trades manually. This duplication of effort causes delays, data accuracy issues, and reconciliation problems. The growing number of identical transactions in different systems also complicates the reconciliation process. Identifying discrepancies late in the process makes resolutions more challenging and can significantly impact the respective systems. All this is of significant concern for the company’s risk analysis and downstream operational processes.
Trade Confirmations
Trade confirmations are a common source of inefficiency in the energy industry, particularly for OTC and bilateral trades executed outside of an exchange. The online Intercontinental Exchange eConfirm (ICE eConfirm) platform offers a centralized confirmation service, but energy companies still confirm many trades manually. For example, if one or both parties have not subscribed to ICE eConfirm, they must exchange a trade confirmation, usually in the form of a PDF. This process typically involves two-way communication, often in the form of an email with an attachment and a subsequent email for approval.
A trading company might receive confirmations from a hundred different counterparties daily, each using a different format. Every confirmation must be carefully reviewed and compared against a trade in the system. Assuming it matches, it is then approved with the counterparty and marked as confirmed in the system. The confirmation PDFs should be stored with or alongside the system for future reference.
Optical Character Recognition (OCR) and now AI tools are increasingly used to digitize files that support electronic matching with the transactional data in the customer’s Energy Trading and Risk Management (ETRM) solution. This helps reduce the manual stages in this process.
Scheduling and Logistics
Physical energy scheduling and logistics remain among the least standardized business processes in the energy trading industry. Even within a single commodity and mode of transportation, there are variations including region, market, or individual operators. All this requires significant human intervention adding to the cost of resources while slowing down critical processes.
ETRM systems offer generalized scheduling solutions for each commodity but often fail to overcome these differences. If not, a scheduler may bypass the ETRM system for scheduling and instead update it after the fact, primarily for accounting purposes. Scheduling for physical energy commodities is typically not performed at the individual trade level. Instead, schedulers are responsible for aggregating trades to create efficient delivery schedules bridging the gap between the ETRM system and physical operators.
Excel spreadsheets are most commonly used for scheduling. This enables every scheduler to create their own solution for each unique operator. However, this leads to multiple copies of transactional data and requires integration with the company’s internal systems and the operator. Unfortunately, these integrations frequently require manual intervention.
The natural gas industry offers electronic submission of nominations to pipelines through EDI and third-party service providers. But manual entry of nominations into pipeline bulletin boards is the predominant process, both daily and often intraday.
In the crude and refined products industry, there are fewer operators and less standardization. Some operators offer EDI or other electronic upload mechanisms, as well as web screens for manual entry. But compared with gas, there are more variations from operator to operator. Crude scheduling is organized around a monthly calendar with daily adjustments, whereas refined products scheduling is more dynamic with weekly or daily scheduling—although not as dynamic as natural gas scheduling.
The power industry has solved the physical scheduling problem out of necessity. It is impractical to input hourly or quarter-hourly power nominations manually. In addition, the power business requires electronic communication of operational bidding and scheduling processes. Therefore, market participants must use a solution that enables them to communicate in near real-time with the markets.
Actualization of Physical Logistics
Post-physical delivery, trading organizations must document the exact quantities of products that were produced, transported, and delivered or consumed. Depending upon the commodity, there might be a delay of hours, days or even weeks before this information is recorded.
Some companies invoice based on estimated actuals and then ‘true-up’ those invoices in subsequent months with prior period adjustments. Other companies follow a business process where invoicing tracks a month behind the ‘delivery’ month to allow sufficient actualization time and minimize the accounting complications associated with prior period adjustments.
As mentioned in the scheduling section, trade-level data is aggregated between the ETRM system and the operators when scheduling. The flip side of this occurs during actualization. The operators typically provide actuals at the same aggregated level as scheduled. This often requires an allocation process in order to distribute aggregated quantities at the trade level in the ETRM system. Allocation may be automated or manual, but it is a common source of discrepancies which require manual reconciliation and adjustment.
With physical power, the actuals are known and communicated electronically in near real-time. Again, with power, the hourly and quarter-hourly granularity of data necessitates an electronic solution and results in a shorter reconciliation process.
For natural gas, the pipelines provide scheduled quantities after the flow, along with estimated actuals on a one-to-two-day lag. However, these are often replaced by end-of-month actuals, followed by subsequent revisions. While there are various electronic methods to obtain this actualized data, such as EDI and third-party service providers, the actualization process is often managed manually, involving hand-keying of data. In terms of procedure, the gas industry usually waits until the end of the month to reconcile actuals, just before invoicing deadlines.
When it comes to crude and refined products, the actual quantities are usually known three to seven days after delivery, depending on the mode of transportation. There is less standardization in this industry, with each operator providing electronic data in different formats. Similar to the power industry, we see more automated, digitized solutions with truck ticketing systems for the physical delivery of crude and refined products. This is due to the higher volume of more specific transactions. However, even with truck transportation, the level of automation varies, with some companies still dominated by manual processes.
Invoicing
Invoicing has similar issues as confirmations. Each counterparty typically creates its own invoice template, while email exchange of invoice PDFs, reconciliation, and approval are still common. But where trade confirmation is for one trade, one invoice may contain dozens or even hundreds of trades, adding to the complexity. With physical commodities a trade may also have multiple locations, along with daily or hourly actual quantities.
A common problem is invoice aggregation, where two companies with different systems cannot generate an invoice (or the corresponding reconciliation statement) at the same scope. For example, separate invoices by transaction type, commodity, contract, location, and flexible netting rules. Some systems have more flexibility in controlling these aggregation options. Two counterparties with a hundred trades between them may not be able to exchange invoices with the same scope, making reconciliation and approval more complicated.
Similar to confirmations, a third party’s invoice PDF must be stored in or alongside the ETRM system. Some companies are digitizing this data for electronic reconciliation, but invoice reconciliation in the energy industry is still predominantly a manual task. Another common scenario is where counterparties interchange an invoice summary PDF and an Excel spreadsheet which contains transaction details to assist the manual reconciliation process.
Some companies conduct a preliminary review with their counterparties to identify and fix discrepancies before finalizing invoices. This avoids post-finalization corrections and prior period adjustments. The review process may involve previewing the invoice PDF or exchanging Excel files of data extracts, both of which require manual reconciliation. While some companies provide a web portal for customers to download invoices, the most common practice is to email an invoice. However, a few may still be sent via fax machine.
The power industry has made the most advances in this area thanks to the granularity of its transactional data. It simply isn’t practical to manually reconcile hourly and quarterly hourly data at the month’s end. The electronic reconciliation of power transactions between the markets and market participants and between counterparties has also progressed significantly. Not only does reconciliation occur electronically, but it also occurs daily throughout the month, which drastically reduces the manual month-end invoicing process. In contrast, crude and refined products and the natural gas industry, which operate at a monthly or daily granularity, still follow a predominantly manual process and defer most of that process until the month-end invoicing cycle.
CONCLUSION
These are a few examples where automation can streamline operations and enhance accuracy in key areas like scheduling, settlements, invoicing, and P&L reporting. The opportunities to automate and optimize processes across the entire energy value chain are vast.