The python of Wall Street Part 1
Or how how to create a trader bot
Disclaimer:
This project is a eight parts project that I’ll leverage to have the opportunity to implement different technologies I want to explore.
You are welcome to re-use any part of this script. But I would not advise using it on the stock market with your money. If you do, I am in no way responsible for whatever may result of it.
part 1: extracting data and computing trend indicators
part 2: creating an ETL pipeline for data quality and centralization
part 3: creating a classification model
part 4: automatic retraining of the model
part 5: create apis to access the data
part 6: data visualization
part 7: create docker container for micro services architecture
part 8: process automation
Part 1: Or what is indeed the stock market
Simply put, stocks are a way to build wealth. They are an investment that means you own a share in the company that issued the stock.
Stocks are how ordinary people invest in some of the most successful companies in the world. For companies, stocks are a way to raise money to fund growth, products and other initiatives.
What are stocks and why should you own them?
At some point in their developement, companies sell shares in their business to raise money. THis money can then be reinvested in the company, to fund new products, prducts lines, invest in growth, expand operations…
Those shares are caled stocks. Therefore, buying stocks means buying ownerships of parts of the company.
Stocks can then be kept in order to get some dividends or the stockholder can wait to re-sell those shares later and earn a profit if the stocks’ price goes up.
As a result, in the second case, you want to buy for the lowest price possible and sell for the highest price possible to make the biggest margin possible, maximizing your return on investment.
Predicting the price with good acuracy is a all field in itself and is an extremelly hard (possibly impossible) task. The evolution of stock prices being highly unpredictable and depending on a lot of external factors adding an unpredictable noise factor to the actual price, resulting in a very unreliable price prediction
But, the stock markets also tend to have a lot of inertia. Once a real movement, uptrend or downtrend is begining, the companys prices tend to follow this trend and some indicators can be leveraged to analyze the trend and make educated guess regarding the evolution of prices in the future.
Part 2: Or what is the trend
A trend is the general direction of the price of a market, asset, or metric.
Uptrends are marked by rising data points, such as higher swing highs and higher swing lows.
Downtrends are marked by falling data points, such as lower swing lows and lower swing highs.
Many traders opt to trade in the same direction as the trend, attempting to profit from a continuation of that trend.
https://www.investopedia.com/terms/t/trend.asp
The trend being a general indication of the direction of the market, it helps identifying when to hold, buy or sell the stocks.
But, the stocks are highly volatile and therefore actually identifying the trend can seem somewhat hard due to the short terms variation of the market. Is it going up, down, stagnating… ?
Therefore, denoising the data before any further analysis to extract the deep tendency of the market is a good first step.
Using the fourier transform to denoise the data
The fourier transform takes a signal as input data and separate it in its different components. Then, you can select the most representative components thaht contain the most information about this signal and use those components to recreate a funcion that can approximate the real behaviour of the signal. The resulting signal contains less information than the source signal, but as a result also contains less noise and caries mainly the actual information.
After some experiments, it seems that:
- 100 components cary most of the information of the signal and is almost identical to the source signal.
- 20 components is a good representation of the actual trend of the signal.
The resulting representation of the signal using 20 components will be used later to compute trend indicators as they are more representative of the real trend and will result in a more conclusive and reliable analysis.
A second way to evaluate the trend of a stock is to rely on the candlestick patterns.
A candlestick is a type of price chart used in technical analysis that displays the high, low, open, and closing prices of a security for a specific period. It originated from Japanese rice merchants and traders to track market prices and daily momentum hundreds of years before becoming popularized in the United States. The wide part of the candlestick is called the “real body” and tells investors whether the closing price was higher or lower than the opening price (black/red if the stock closed lower, white/green if the stock closed higher).
https://www.investopedia.com/terms/c/candlestick.asp
There are different patterns of candlestick that can give indications on the behaviour of the stock (more about this in part 2).
Part 3: Or what are indicators
Once you have some representative data that enable a more precise study of the trend, you can leverage different indicators that based on those data will give some indications on the current and near future trend of the market and enable educated guesses regarding the buying or selling behaviour to follow.
There are many more, but for this implementation, I decided to implement the following:
Bollinger band
Bollinger Bands are a type of chart indicator for technical analysis and have become widely used by traders in many markets, including stocks, futures, and currencies. Created by John Bollinger in the 1980s, the bands offer unique insights into price and volatility. In fact, there are a number of uses for Bollinger Bands, such as determining overbought and oversold levels, as a trend following tool, and for monitoring for breakouts.
https://www.investopedia.com/trading/using-bollinger-bands-to-gauge-trends/
This indicator helps identifying overselling and overbuying periods. When, for some reason the stockholders all decide to go on a selling or buying frenesia. This will result in spikes in the stock price.
The bollinger band being composed of two bands around the actual stock price, piercing the line above indicates that the price is going to sky rocket for a while as it is likely overbought and due to the inertia of the market should pull back. On the other hand, if it goes through the below line, it is likely oversold and should go back to normal as well.
When it happens, it is likely a good opportunity to sell for a high price or buy for a low price (this indicator is especially usefull in forex trading).
Moreover, the delta between the upper and lower band do inflate or contract with the volatility of the stock. It is called the squeeze.
The bands come coser together: the trend may go upand should result in a buying action.
The delta gets wider : the trend may go down and result in a selling action.
it is computed as follow (simple_moving _average; SMA, standard_deviation; STD)
- upper_band = 20-daySMA + 2xdaily_STD
- lower_band = 20-daySMA — 2xdaily_STD
Exponential moving average
An exponential moving average (EMA) is a type of moving average that places a greater weight and significance on the most recent data points. Therefore, it reacts significantly to recent price changes. It is used to produce buy and sell signals based on crossovers and divergences between diferent moving averages.
In this case, we’ll study cross-overs between 12 periods EMA and 26 periods EMA.
When the 12_ema is on top of the 26_ema and crosses it, it is a decrease signal, and can be interpreted as a sales opportunity.
WHen the 12_ema is under the 26_ema and crosses it, it is an increase signal and can be interpreted as a buying opportunity.
It is computed as follow:
EMA = (Close — previous EMA) x (2 / n+1) + previous EMA
where n is the number of periods to look at.
Moving average
A simple moving average (SMA) is an arithmetic moving average calculated by adding recent prices and then dividing that by the number of time periods in the calculation average. Short-term averages respond quickly to changes in the price of the underlying, while long-term averages are slower to react.
A simple moving average is customizable in that it can be calculated for a different number of time periods, simply by adding the closing price of the security for a number of time periods and then dividing this total by the number of time periods, which gives the average price of the security over the time period.
It smooths out volatility, and makes it easier to view the price trend of a stock. If the simple moving average points up, this means that the stocks’ price is increasing. If it is pointing down it means that the security’s price is decreasing.
The longer the time frame for the moving average, the smoother the simple moving average.
A shorter-term moving average is more volatile, but its reading is closer to the source data.
In this case, we’ll study the crossing between the 20 periods SMA and the 50 periods SMA.
When the 20_sma is on top of the 50_sma and crosses it, it is a decrease signal, and can be interpreted as a sales opportunity.
When the 20_sma is under the 50_sma and crosses it, it is an increase signal and can be interpreted as a buying opportunity.
SMA = sum(closing_prices_on_the_period)/n
where n is the number of prices on the considered number of periods.
Simple Moving Average vs Exponential Moving Average
The biggest difference between EMA and SMA lies in the weighted part of the EMA, giving more weight to recent data points and therefore more sensitivity.
The EMA is more reactive to the latest price changes than SMA.
Moving Averages and Bollinger Bands
Stochastic
This indicator is a momentum based indicator. It compares a closing price of a stock to a range of those prices over a given number of period.
It generates overbought or oversold trading signals giving a [0;100] range.
Usually, a %K for 14 periods and a %D for 3 periods are computed and the crossing are studied as follow:
- if the %K and %D are higher than 80 % it is a signal that the stock is oversold and is therefore a selling signal.
- if the %K and %D are lower than 20 % it is a signal that the stock is overbought and is therefore a buying signal.
- if you are in those ranges:
in the lower than 20% part, if the %K is piercing the %D line increasing, it is a signal that the price is going up (buying signal).
in the higher than 80% part, if the %K crosses the %D line decreasing, it is a signal that the price is going down (selling signal).
%K = ((current_closing — lowest_closing_14)/(highest_closing_14 — lowest_closing_14)) x 100
%D = ((current_closing — lowest_closing_3)/(highest_closing_3 — lowest_closing_3)) x 100
Relative Strength Index (RSI)
This indicator is a momentum based indicator. It measures the magnitude of recent prices changes to estimate oversold or overbought stocks. It results in an oscillator ranging in [0;100].
- if the RSI is higher than 70% (in this implementation 80%) the stock is overvalued and and overbought and may reverse.
- if the RSI is lower than 30 % (in this implementation 20%) the stock is undervalued and oversold and may reverse.
delta = average_gain/average_loss (on a given window)
RSI_step1 = 100 — (100/(1+delta))
Part 4: Or what is the strategy
A very basic strategy could be as follow
1: Denoise the data
2: Compute all indicators
3: If one indicator changes, change your position regarding this stock
4: In the case if this indicator signal changes but the ouptu goes against the trend, do not change the position. The market having some inertia and not all indicators having the same sensibility, it may be a false positive.
Bollinger Band
In this case, the bollinger bands were actually relevant when predicting when to buy and when to sell. However, only relying on them does not provide the best strategy has it results in a lot of transactions whereas fewer translation with better return on investement could be better
Exponential moving average
The ema is a very robust strategy as the crossing actually happen once the trend is clearly defined. Resulting in very safe predictions. But on a very volatile stock, it can result in decision that are taken once the best window of opportunity has passed due to its time delayed nature.
Stochastic
Stochastics are also very robust indicators and seem to be more reactive than the moving average.
RSI
The RSI indicator is a very greedy one and tends to over-predict and not manifest itself at the best moment.
Part 5: Or how to extract data from api
After some research regarding the best free api to get stock data from for intraday trading, I found alpha vantage and iexcloud. The two apis allow for free calls every minutes for some stocks data.
One the one hand, alpha vantage provides streaming data and also some indicators functions. This could have been an interesting all in one solution, but at some point, it just stopped providing real time data and I am still trying to figure out why, but as of now, all I can get is a dump of all the data from the day before the day of the api call. It can be interesting to generate a dataset, but not for real time trading.
On the other hand, the iexcloud api is still working as expected as of now and providing data at a minutely rate.
Conclusion
The stock market being highly unpredictable, actually knowing what the price will be is impossible. But, it is possible to leverage denoising systems and indicators to make an educated guess regarding the behaviour of the market and bet on the inertia of the market to buy and sell in order to maximize the return on investment.
The script for this project can be found here: https://github.com/elBichon/midas_project.github.io
Next Steps
The next step is to optimize the data pipeline and features extraction channel in order to extract some consolidated features to train predictive models on, teaching a model to buy low and sell high.
Références
https://www.investopedia.com/terms/c/candlestick.asp
https://school.stockcharts.com/doku.php?id=trading_strategies:bollinger_band_squeeze
https://www.abcbourse.com/
https://www.alphavantage.co/
https://iexcloud.io/
