Blog News Events Publications Directory Community Industry Voices Media

The Commodity Trading industry is on the cusp of transformation – and Digitization is at the heart of the change.

By Cyrus Dadachanji, Partner, Midstream CTRM Practice, Infosys Consulting The Commodity trading industry is no stranger to change. From the birth of the independent trader in the 1990’s, to more recent changes wrought by the wholesale departure of the investment banks from the commodity trading scene, industry participants have long had to adjust to shifts in the trading landscape.

Recent changes have seen the large independent traders acquire assets in a bid to move from pure-play financial traders to asset-backed traders, rivalling the once sole preserve of the industry majors. In addition to core trading participants, the Utilities industry too transformed. From traditional thermal energy consumers to renewable power generators having to change the merit order of the generation stack, the vertically integrated utilities are sophisticated trading organizations in their own right, and have had significant impact on the energy markets.

AdvertisingFendahl CTRM Technology

Now, the commodity trading industry is once again looking at a sea-change; this time not through a restructuring of industry participants, but through the application of new technologies.

The impact of technology is nothing new – the industry has long embraced the analytical possibilities for better optimization of physical assets such as power plants, gas storage and refineries. And the potential for digitization has also been talked about for some also nothing new

However, the transformative potential offered by digitization is likely to be of an entirely different order of magnitude, with applicability to almost every point of the trading and logistics value chain.


Digitization may be viewed as a combination of three key technology areas: Artificial Intelligence (AI), Machine Learning (ML), and Robotic Process Automation (RPA). A combination of these technologies can be applied to multiple points in the trading value chain, from pre-deal analytics through to contract settlement, as well as to numerous activities within the Logistics value chain.

AI and the predictive analytical technologies will see greatest usage in Front/Middle office and Logistics activities, while RPA is likely to find greater usage in the Middle and Back office functions.

In order to assess where Digital technologies may best be applied, we start by identifying “pain points” across the Commodity Trading and Risk Management value chain – those areas where excessive manual interventions remain, where a plethora of data sources and/or systems are used, and those where current analytical methods are insufficient to gain necessary insights.


Although point solutions may be developed to address specific issues, there will naturally be linkage between these solutions. In particular, digital solutions around data entry will naturally morph into wider solutions for operational risk, fraud detection and regulatory compliance.

We foresee companies using advanced analytical techniques such as AI in a number of key areas. Examples include:

1) In Logistics and Shipping optimization, the base use case for advanced analytics is to manage fleet voyages, taking into account weather patterns, port data and freight market intelligence. But this quickly extends to overlaying advanced analytics on large quantities of shipping and voyage data to develop a forward view of inventory levels of a specific commodity at a chosen port, giving traders additional market signals to make better trading decisions. Further, analytics can be used to identify correlations between ship and voyage data, and price movements on relevant marketplaces such as the Baltic Exchange.

2) In the risk space, advanced analytics may be used for both Credit and Market Risk Management. Taking credit risk as an example, analytics can be used to analyse large quantities of internal and external data of varying types. On the one hand, “formal” data such as from rating agencies, exchanges and investment bank analysis, all of which is publically available and readily importable; on the other, a variety of other, less easily visible data sources such as websites, magazines, social media platforms, etc., also publicly available, but more difficult to access and to interpret. And lastly, the information available from conversations on messaging platforms etc., which is yet harder to access. AI is able to access all these sources, filter and extract the relevant information, and recognize patterns of company failure.

3) For complex networks such as gas pipelines, advanced analytics will be used to optimize and balance the entire portfolio, including long and short positions, new transaction opportunities, and pipeline flow alternatives to provide higher value solutions.

In addition to still-maturing concepts such as AI, Robotic Process Automation already exists in implementable form, and is being considered for numerous middle and back office processes including demurrage claim management, invoicing, contract management, reporting and margining. Already in wider use in financial services institutions, the commodity trading industry is now also likely to see significant efficiency gains in coming years.

CostsSource: Infosys Consulting analysis

As well as the application of AI to enhance revenue generation, and of RPA to increase efficiency, trading companies are also looking at the potential to use advanced analytical tools to help them in their compliance efforts. In particular the use of AI to identify unusual trading patterns and potential market abuse is gaining ground.

Digital “workbenches” aimed at specific trading company personas (e.g., Operators, Risk Managers) bring together data from disparate sources, both internal and external, allowing for a single view of pertinent information. Customizable advanced analytics are then applied to give traders, schedulers and others the information they need to make better decisions.

To illustrate, take Operational Risk as an example. The issues facing the industry are clear and present:

trading sources

  • Companies are rarely able to point to a holistic view of the state of play in respect to operational errors;
  • P/L is not kept up-to-date, with a back-log of deals and deal-entry reconciliation required.
  • Uncoordinated controls reduce effectiveness – controls failures remain commonplace;
  • Companies face significant financial fines and reputational damage for non-compliance of regulations;
  • Genuine errors go unnoticed for longer.

Operational risk, compliance and regulatory reporting is increasingly high on commodity trading firms’ agendas, but organizations currently must rely on multiple tools such as trade reporting and voice recording, typically with each approach acting in a stand-alone manner.

Digitization solutions such as the one illustrated above provide a holistic approach to Operational Risk. Data can be collated from multiple sources including IMs, Voice Recordings, Spreadsheets, ETRM systems, HR data, Office entry / exit, providing real time analytical capability allowing stakeholders to interrogate datasets.

Automated alerts can be generated for suspicious activities, behaviour anomalies, and operational errors. The system will keep updated a list of trades it predicts has been executed, and the list may be updated based on machine reading of telephonic and electronic communications and monitoring of user-spreadsheets and / or blotters. The system is then able to compare its own list vs. the company’s ETRM system to highlight potential errors.

Why the slow take up, and how should companies approach implementation?

Despite violent agreement that advanced analytics and Digitization is here to stay, and will have a game-changing impact on the industry, the pace of adoption has been slow. Trading company CIOs have “Innovation” budget, but often lack a clear spending vision.

Often the difficulty is due to starting with a “chosen” AI technology platform, then asking how it can be utilized. Yet this misses the point that companies’ primary aim should be to improve their business and solve business problems, not simply aiming to use the latest gadget. A man with a hammer always sees the nail as part of the solution… Critical to successful implementation of any Digitization strategy is to approach the topic from the business viewpoint, and not from the technology side.

The starting point for a successful AI journey in ETRM is to identify the correct problem to solve. Companies often look internally at their own processes and operations. Instead, companies should apply Design Thinking, focusing on outside-in thinking and empathizing with the various “persona” in the commodity trading value chain.

There are numerous benefits of taking a design thinking approach, but in particular the lack of common understanding of the role of AI in the ETRM industry makes the approach all the more powerful. Further, although Innovation budgets more often than not sit with CIOs and IT departments, value will be judged by, often more cynical, business functions.

dSource: Institute of Design at Stamford

Design Thinking helps companies to find the “right” problem, often not the problem initially envisaged. Focus can then shift to identifying the solution which will address the core problem, and to finding the data which is required power the solution. Following data mapping, execution can be planned in the form of a Proof of Concept (POC). Such POC should use agile methodologies in which algorithms can be iteratively trained, tested and tuned.

More importantly even the POC is the Proof of Value (POV). i.e., even if the technology is proven to work, does it add value to the organization?  A rapid fast fail POC/POV allows companies to test the technology, assess business benefits, and make decisions around wider implementation.


More information and insights into achieving success with AI can be found at xxx: “Achieving Repeatable Success with AI”.

Impact on the structure of the CTRM Industry

Thus far changes in the commodity trading industry have been driven by traders seeking higher margin returns by broadening their portfolios, gaining access to physical assets, integrating their trading, origination and operations arms, increasing their trading sophistication, and increasing their coverage of the value chain, and taking advantage of scale. Thus the commodity trading industry is dominated by a relatively few, very large, global and sophisticated players.

Digitization will change this. Smaller, more nimble, digital players, able to extract information from vast reservoirs of data, will be the new entrants to the commodity trading market. Fewer people will be required, and the human factor will be to enhance and improve the algorithms that machines use to make trading decisions. Middle and Back Offices will be entirely transformed into much leaner and more efficient operations.

As more asset information becomes available to all, new, smaller entrants will find they have the information they need to be able to make trading decisions without the cost and operational risk of owning those assets. The asset-backed trading strategy advantage held by large trading companies through control of physical assets will be neutered.

The existing large, global commodity traders will need to change, and will likely need to lead the change in order to remain competitive. The changes will also benefit the global players – digitization will result in far smaller margins from vanilla trading, and more revenue will need to come from emerging markets and non-standard transactions.

These changes will impact all traded commodities, from energy to metals and agricultural products, though given the relative maturity of the oil and energy markets, these are likely to feel the changes the most.

Some market participants will adapt and others will fall by the wayside. The coming change in the commodity trading industry has the potential to be a truly seismic shift.