How Algorithms in CME Grain Futures Determine Which Trades are Filled
Prepared by G. A. (Art) Barnaby, Jr., (firstname.lastname@example.org) Professor, Department of Agricultural Economics, and Daniel O’Brien, (email@example.com), Extension Agricultural Economist, K-State Research and Extension, Kansas State University, Manhattan, KS 66506,
Kansas State University Department of Agricultural Economics – May 16, 2018
Electronic trading has replaced pit trading. So now the question is which orders are filled first? The computer decision rules or “algorithms” used to determine how orders are filled are intended to be completely objective – i.e., without any human bias or error being entered in the queuing for filling orders.
The function and use of these “decision rule based algorithms” for Chicago Mercantile Exchange (CME) grain futures contracts was a topic of discussion at the 2018 Agricultural Commodity Futures Conference on April 5-6 in Overland Park, Kansas.
This conference was co-sponsored by the Commodity Futures Trading Commission (CFTC) (https://www.cftc.gov/) and the Kansas State University Center for Risk Management Education and Research (www.k-state.edu/riskmanagment). Papers and power point slides from the conference are located at the following web address: http://www.k-state.edu/riskmanagement/conference.html
The objective of this paper is to clarify questions about how the CME fills orders via algorithms that were raised in an April 16th paper posted on AgManager titled “Fixing Arbitrage to Cause Convergence; No Consensus”. A link to the paper is at: https://www.agmanager.info/crop-insurance/risk-management-strategies/fixing-arbitrage-cause-convergence-no-consensus
Authors cited the following comments found in the April 16th AgManager paper mentioned above:
“CME has eliminated pit trading in favor of computerized matching of buy-sell orders. Surprisingly, it is not the oldest futures contract bid that is filled first. To the surprise of many participants, CME has an algorithm that determines which contracts are filled first. There was one very upset participant that stated his order was not filled, even though his bid was higher than CME’s posted close. His question was how was that possible? Answer, that is how the algorithms work. Some participants questioned the “equity” and “fairness” of a CME algorithm determined queueing order for filling contracts.”
Unfortunately, the agenda of the April 5-6 meeting did not allow enough time to fully discuss the order or the mechanism by which contracts are filled. Afterwards the conference and the follow-up April 16th article, it was brought to our attention that the comment by the participant who was quoted above was misleading and caused confusion among some grain futures contract users. Our objective in this article is to clarify the computer algorithm rules that determine the order used to fill grain futures contracts. This information taken from CME web-resources and from direct conversations with CME staff.
Matching algorithms: The CME’s stated purpose for these order-matching algorithms is “to provide the best possible execution of futures contract positions at the fairest price”. The function of a matching algorithm is to match an aggressor order with a resting order to complete the trade. Separate bid-ask prices occur and exist for each futures contract throughout the trading period. As long as the bid-ask prices are different, then both are classified as “resting orders”. In order for an actual trade to be completed, an aggressor order has to occur in which a new order decides to accept the ask price if she is a buyer, or the bid price if she is selling. In other words, an aggressor order is required to complete a trade. Otherwise the bid-ask prices remain apart and there is no trade.
Anecdotally, this process is like trading pickup trucks. At some point in time either you or the dealer has to accept the other persons’ offer or there is no trade. Like futures, the price of that pickup probably changes a lot from your original offer and the dealers’ original asking price.
The Algorithm Used by CME Grain Futures to Determine Order of Contract Fill:
The CME uses what it calls a “K”-type or Split FIFO/Pro-Rata algorithm to determine the ranking by which orders are filled on grain futures contracts. The CME’s algorithm uses “multiple rounds” of choice rules for determining the sequential order by which orders are to be filled. These multiple rounds of “allocation order-fill criteria” are described as follows:
Round 1: Top-Order Allocation: To begin the process, a top order allocation is given to the first incoming order that betters the market and is filled at 100% between a minimum of 1 lot and a maximum of 100 lots (note the maximum for KC wheat contracts is 50 lots). Top-Order Allocations typically make up a small proportion of the total amount of orders filled, i.e., 1% (range of 0% to 5%).
Top-order allocation is designed to reward a trader that either improves the best bid or the best offer, or rewards a trader who makes an aggressing order that is not completely filled. For example, if an order was made for 10 contracts, and only 2 contracts were filled, then the remaining 8 contracts will be first in line for the next aggressing order. However, the remaining unfilled order cannot be changed either in terms of price or the number of contracts involved – otherwise the order goes to the “back of the line” for consideration. With so many restrictions, it explains why so few contracts meet the criteria of being filled in the Top-Order Allocation. As a result, these initial Top-Order Allocations typically account for only about 1% of the trades.
Round 2: FIFO Allocation: After a typically small number of Top-Order fills in round #1, then the ranking order of fill is for the next 40% of the orders as determined using FIFO (First In-First Out). FIFO allocations are based on the timing of when the orders are initially placed, with the first orders placed time-wise being the first orders that are filled.
Round 3: Pro-Rata Allocation: Then following both rounds #1 & #2, 60% of the remaining volume is allocated via Pro-Rata Allocation, with order size and time being the variables for allocation. Pro-Rata allocations are calculated by dividing the quantity of contracts bid by an aggressing order by the total quantity of resting orders that are available at a particular price. These “pro-rata” percentages are then used to calculate the amount of allocated orders to be filled by individual resting orders among the total quantity of resting orders available.
In some cases, for outright grain buy/sell positions (excluding grain spread futures in most cases, except apparently for KC HRW Wheat), a small proportion of “aggressing” orders may still remain that need to be filled. Outright contracts include basic grain futures themselves, but exclude contract-to-contract spreads. These relatively few remaining orders are filled using a combination of Top-Leveling Allocation and FIFO Allocation methods. For grain spread futures orders, only FIFO allocations are used in a final “clean up” step.
Round 4: Top-Leveling Allocation: Any participant that did not receive an allocation via pro-rata allocation due to “percentage rounding” issues, receives a 1-lot allocation. If there is a volume of orders that remains to be filled. “Leveling” is an additional round of pro-rata that allocates only one (1) lot per order based on the percent of the remaining resting orders yet to be filled.
Round 5: FIFO Allocation: Any volume of orders that remain to be filled even after the top-leveling allocation then use a FIFO allocation among the remaining resting orders to complete the procedure.
During the time when pit trading was occurring, the function of “order filling” was carried out by the traders themselves on a face-to-face basis. Because of most market participants’ preference for electronic trading versus pit trading, the CME has fully adopted electronic trading. With this change, the function of “order filling” is now determined by computationally-based “allocating algorithms”. With pit trading, if a trader placed a bid for a position in grain futures that was not filled, it was possible to blame “human behavior” and/or “human error”.
Now with electronic futures being fully adapted by the CME, it is the job of these “allocating algorithms” to fairly and equitably allocate orders among resting positions for grain futures contracts. The authors don’t see any reason to believe this computer allocation of order filling is unfair to farmers – in fact it may be more fair.
In our original paper we included the Lead Market Maker (LMM) allocating algorithm component to be used in the multiple round order filling process before the FIFO and Pro-Rata allocations are calculated. However, the LMM allocation is not being used by the CME to allocate the order of contract fills in grain futures and option contracts. LMM has been used for some newly established contracts to help create a market in situations that were plagued with low volume or lightly traded conditions. However, none of these were grain futures contracts.
Finally, there is a remaining question: how is it possible for one’s buy bid to be higher than the CME closing price and not get contract fill? The answer is there is a difference between closing price and settlement price. CME calculates the “posted settlement price” as the volume weighted average price for trade between 1:14 p.m. and 1:15 p.m. of trading each day. However, the “closing price” is determined to be the price of the last trade at 1:20 p.m. when the market closes.
Therefore, if the higher bid was submitted during the 1:15 to 1:20 time frame, it is possible that the bid was above the CME posted settlement price which was calculated from 1:14 to 1:15 earlier that same day. As result, a higher bid submitted late in the final minutes of trade could exceed the calculated volume weighted average used to figure the “posted settlement price”, resulting with that particular higher priced bid not getting filled.