It is always interesting to get feedback from people about my blog articles. One that a few people have talked to me about is this article on message rates and system throughput. It covers the importance of throughput for market making and how it relates to system latency. If you have a very performant system then likely you will not have issues, however that is only one possible solution.
The aim of this article is to elaborate on the...
I have wanted to write a reading list for a while. I’ve been apprehensive because I didn’t want to include too much and wanted also to explain why the books were in the list. For example, if you trade crypto there is not much point reading the Hull interest rate models book. This list likely will be a work in progress so keep that in mind. As I write this I realise I have a problem with how many books...
I was re-reading some older posts, and I realised I owed some readers a follow up. Hopefully I will be forgiven that this took 2 years. This post will be short and the core of the ideas are a follow up to the original article here.
The key takeaways from the original article are:
Writing an OMS for an HFT platform is a really difficult task that is often taken for granted. It is made more difficult in crypto because the exchange infrastructure is unreliable. It is not uncommon to simply be told “go away and come back later” when trying to call api functions. There is also often a hidden danger when writing an OMS which I have not seen discussed. This article is about what can be the most dangerous part of...
There is a common debate when people are discussing code optimisation that relates to how fast code needs to be. A recent Twitter post about parsing binance BBA messages stated processing times of around 200ns. This is, in my admission, is very fast.
To put it into perspective, Serde is a common rust deserialization library and is incredibly easy to use. It is however, a lot slower than the optimised alternative taking around 2-3 micro seconds to do the...
It has been a while. I hope you have not missed me! It seems that Rust is gaining steam and more people are starting to use it for building trading platforms. I have been advising people in the most tactless way possible that their life would be better if they embrace the crustation. So here we are. I have about two years of Rust experience under my belt, Although time is not the only measure of ability. There are some...
Candlesticks are a common way to represent price and volume of an asset over a period of time. There are various common types of bars such as time, volume, tick bars, hieken-ashi, renko to name a few. There is a lot of information about the implementations of these on the internet so their details will not be covered here. The aim of this article is to share some tips for implementation and also an implementation of Marcos Lopez de Prado’s...
In this article, we’ll be discussing a fast way of logging in Rust and its application to high-frequency trading. The code presented here solves two problems, one is well-known, the latter less so. It is imperative to avoid using IO operations within the strategy thread, but logging operations can hide a lot of memory operations that should also be avoided. Logging from the strategy is useful for monitoring or debugging purposes. Logging can however be expensive if implemented inefficiently. I...
So the crypto markets on May 19th were fun!? If you have been a part of these markets for any time, this volatility is not that surprising. That said it is still pretty amazing that the price can drop 30% in a single day. This is maybe more surprising since it is to the backdrop of an increase in institutional investors and a narrative of “this time it’s different”. I was asked a question with regards to “how is this...
I love programming! There is something really satisfying about solving a complicated problem concisely. That said I see programming languages as a tool to solve a problem rather than purely coding for coding sake. I have used a lot of programming languages over the last 20 years namely Java, R, Matlab, Python, C++ and now Rust. It is pretty common to read articles about language wars and which one is best. Would you write a website in assembler? Try embedding...
I have been thinking a lot about risk lately. The liquidity injections from the FED are pushing risk assets higher and higher. There seem to be bubbles in nearly every speculative assets. The main concern long term would be rising rates at the same time as a falling dollar suggesting there is no longer a market for US debt. Recently however rates have been rising which is causing concern that the risk asset bubble might burst under the pressure of...
Back in 2008 when I worked at Deutsche Bank I became interested in algorithmic trading. At the time we were working on some automated pricing models for trading Euro Gov fixed income which at the time was quite novel. The issue then was that professional markets were inaccessible and there was little in the way of retail offerings. I have never heard of Interactive Brokers and cryptocurrency was not even on the horizon. So I turned to Betfair. The thing...
I have been very busy lately trying to navigate the world burning down. I have managed to come up for air in between waves and can write a little bit about what I have been up to. There has been quite a lot of interest in regards to cryptocurrency options and it would seem like as the market has matured the options have become useful for estimating market expectations about the future distribution of prices. I studied options academically but...
To forecast prices they first need to be differenced to reduce their order of integration. The issue with integrated series is the patterns learnt from one time period dont apply to the future. If the values of the integrated series are very different, likely the learnt patterns are of little to no use. The process of calculating returns from prices is to make the series stationary to solve this issue. Stationarity is defined as
a stochastic process whose unconditional...
I have been thinking a lot about reinforcement learning since it seems to be an interesting technique for creating trading strategies. Reinforcement learning is a simple idea and can be implemented in different ways from Q learning, actor-critic models and other deep learning techniques. The general idea is, given a set of actions A, there is a value function V which gives the quality of each action given the state S. A discrete action set is useful for micro-structure based...
In 2014, I went to talk by a collection of asset managers hosted by Bloomberg. This is one of the nice perks of having a Bloomberg terminal. The speakers were large asset managers and not short term traders; their conclusion was to make money you need to be looking at quarterly returns since at lower time frames the noise is too significant. They were trading billions of dollars so likely their opinion was skewed by practical issues associated with execution...
Today BTC finally broke the all-time high and seems to be holding well above 20k USD. At this point, all bets are off as to where the bull market will end. Technical analysis is not very useful when market profile or historic prices can not be used. The historical precedent is 10x gains which would mean targets of 100k are not unreasonable (some investment banks such as Morgan Stanley have price targets of 160-250k, Plan B has a S2F...
Given transaction costs are non-zero and funding rates are around 1 bps per 8 hours (depending on the exchange) it is a good idea to trade around times when the market is busy. I have done this analysis on other markets but have not applied it to cryptos. I was interested to know if volatility tends to cluster around certain days or hours. Before doing the analysis I had a suspicion that there tended to be a lot of activity...
I find bitcoin market microstructure very interesting since it is so different to any market I have looked at. I come from a fixed income HFT background, and a lot of the instruments I have previously traded tend to be deep markets. The 10Y UST T-Note future (ZN) contract on CME-Globex normally has about 5000 contracts of 100k USD value per contract each price level. Moving the price even a few ticks in one go requires very deep pockets. Bitcoin,...
The slippage saga is thankfully coming to an end. Even after all my best efforts, it seems that the issue was related to something I had not foreseen. The experience has helped confirm the use of this blog as a cardboard chewbacca. During research about hidden size on BYBIT and book cleaning, I was reading the API change logs and noticed that the open-api was in the process of being deprecated. I didn’t think too much of it other...
I ran some backtests overnight to try to work out if latency is likely an issue. If the slippage issue is due to latency in production and the simulation is accurate then we would expect to see the same negative effect of latency on pnl in the simulations.
Results from the simulation over the last 30 days.
Latency has an effect on pnl, but it is not that significant and doesn’t explain the amount of slippage...
It turns out I was wrong about there being no slippage between paper and live. There is slippage, and it is pretty bad! There was a bug related to multithreading that was fixed and now after analysing the new data the multi-threading issue was masking the fills with bad slippage. The good thing is that the orders with slippage are few (10%), the bad thing is quite how significant the slippage is in some cases. The execution strategy is designed...
I recently went live with a crypto mean reversion strategy on BYBIT. The reason for choosing to test the strategy on this exchange is mainly that they offer rebate fees of -0.025% if you are a market maker. This is much more attractive than the fees on many other exchanges. BYBIT in my experience is pretty much a carbon copy of BITMEX and seems to have a very stable API. BITMEX has its issues regarding becoming unresponsive in times of...
Joel Spolsky is a computer scientist and blogger who wrote many books about programming and software development. Some of his observation about programming are timeless, and his books are worth a read. The writing style is light and enjoyable making his articles both educational as well as easy to read. His blog joel on software was also compiled into a series of books. Some of the blog posts are pertinent today and even explain my interest in maintaining...
How do you know if a return distribution is skewed? The empiricist method is to fit the data to a normal and skew-normal distribution and to compare the AIC or BIC of each fit. I would like to present an alternative that is more powerful and is also useful for giving confidence intervals to estimate the quality of fit of each parameter. This is especially useful for computing correlation matrices since most techniques only give point estimations and not parameter...
Swing points are localised high or low prices in a time series. Swing point can be used to estimate optimal stop loss or take profit orders since they often become support/resistance at a later point in time. Collections of swing points can be used to identify technical patterns such as ranges, triangles, flags etc. They can also be used for identifying trends and return distributions. Having an automated technique to identify swing points reliably is quite a useful goal.
There are various kinds of smoothers and moving averages. One that I have used a lot and am a particular fan of is the savitsky-golay filter. The filter fits a local polynomial to the data sequentially. The main advantages of this are both that the fit is closer to the data but also the ability to fit a polynomial means that the derivatives as also easy to compute.
from scipy.signal import savgol_filter...
I recently came across tr8dr.github.io and specifically this post. I only recently found this site and its older version even though it has been around for a long time. I am a big fan of this blog and after reading the post was wondering if the technique could be used to train an online labeller using an HMM model. The idea is to use the labeller offline to generate 3 return classifications, trend up, down and no trend....
Moving averages are so prolific that even price action traders (a notoriously anti indicator group) often include a single moving average. Many other indicators such as MACDs use moving averages and moving averages of differences in moving averages. Even with its extensive usage, there is one major drawback to moving averages which is they lag the data. Even though lag is a commonly known issue there is little discussion of how much lag there is, it’s implications and how it...