- Gekko Trading Bot - Why Backtest is horribly inaccurate, and how you can fix it
- Rebalance your Crypto Index Portfolio
- Shrimpy Social Leaderboard
- Creating a Backtesting environment
- Installation procedure
- Operating system
- Anaconda installation
- Installation of specific packages
- Installation of zipline
- Backtesting a Sentiment Analysis Strategy for Bitcoin
- Test of our environment
- Shrimpy Insights
- Creation of a Bundle Blockchain
- Definition of CSV files of the assets to be included in the Bundle
- Select a Rebalance Period for your Portfolio
- Definition of the module responsible for creating the Bundle
- Defining an extension to register the Bundle
- Testing the Bundle Blockchain
- Download resources
Gekko Trading Bot - Why Backtest is horribly inaccurate, and how you can fix it
Nowadays, technology has become indispensable to the financial industry. It brings innovation, speed and competitiveness.
The need to issue high-frequency financial transactions, coupled with the necessity to process large volumes of data, has made technology one of the main drivers of finance.
These last few years, we have also witnessed two phenomena: artificial intelligence and cryptocurrencies.
Artificial intelligence plays a key role today in asset management companies. Artificial intelligence programs are used to effectively determine the best periods of trading (buying or selling), the assets to be considered, but also the amount of stock to buy or sell.
At the same time, many people hold cryptocurrencies (Bitcoin, Ethereum, Ripple …).
For most of them, their strategy is to hold their position hoping that their assets will gain value in the markets. However, based on the Bitcoin price in early 2018 (Figure 1), we realize that this strategy may not be the most effective.
Indeed, we observe a real increase in the price at the end of 2017, but it fell sharply from the beginning of 2018.
Today, the price of Bitcoin is becoming increasingly volatile. What is the right strategy to adopt in the face of market reactions?
The development of a trading strategy goes through several phases.
Rebalance your Crypto Index Portfolio
One in particular is the definition of the events which are likely to generate the signal. Once this is done, we must do some preliminary tests (Backtesting) on our computer, in order to optimize our strategy and to evaluate its performances and robustness.
Of course, all of this relies heavily on the underlying theory or conviction that any strategy that has worked well in the past will probably work as well in the future, and that any strategy that has worked badly in the past will work badly in the future.
All of this can quickly become very restrictive. In any case, it is crucial to have a stable working environment to allow us to focus on the development of a strategy.
We therefore propose to provide a solution for creating a robust work environment to efficiently explore (Backtest) Trading Strategies.
This environment is useful for anyone interested in the development of trading algorithms.
For that we offer in this tutorial to present the necessary configuration allowing anyone to obtain a local Trading environment on our post. This environment will allow testing strategies on assets of traditional markets (NYSE, NASDAQ, …).
Shrimpy Social Leaderboard
Then we will show how to integrate the data of different cryptocurrencies into our trading environment.
We caution any reader of this article because the development of a strategy requires knowledge in many areas: mathematics, statistics, software development, market microstructure and even psychology. We will not cover these topics in this article as we do not offer effective strategies to win in the markets. On the other hand, we offer to help you in the construction of an effective environment allowing you to focus and quickly backtest ideas that you may have in order to be more confident about the actions to take in the future.
Do not forget: invest only the money you are willing to lose.
Creating a Backtesting environment
The development of a Backtesting environment can be very restrictive and you have to be vigilant because it is easy to make mistakes in the implementation of this tool.
A Backtesting tool usually consists of the following essential components:
A DataHandler: an interface in charge of the management of market data.
A Strategy: an interface retrieving market data and generating signals.
A signal defines a direction (LONG or SHORT) on an asset (symbol).
A Portofolio: an interface to manage orders against our current positions for a given strategy.
A Risk Manager: who manages risk across the entire portfolio, including the size of the positions to be taken.
An Execution Handler: who sends orders to the broker and receives a signal indicating whether the stock has been bought or sold.
This module is used to manage the costs associated with transactions, including the costs related to the order itself, the commission of the broker and the costs related to the delays of execution (slippage) if it is important to model it.
In addition to these components, there are many more that we can add to our Backtester, depending on the complexity.
In addition, a Backtester must provide a wide range of performance metrics to evaluate a strategy, namely: volatility, return, Sharpe ratio, Sortino ratio, Drawdown and many more.
To overcome this complexity we recommend using a specialized API that provides all these components.
In this article, we use the Zipline API (http://www.zipline.io/appendix.html).
In the next section, we will give you the steps to create our Zipline API environment.
In this part we will configure a local trading environment to manipulate assets (AAPL, GOOG, FB, AMZN, …) traded on well-known stock exchanges (NYSE, NASDAQ, …).
We will use the Zipline API.
We will use specific versions of packages each time; we do not guarantee that this will work if you use newer or earlier versions.
We used UBUNTU 16.04 LTS as the operating system. You can get a free version via the following link: http://cdimage.ubuntu.com/netboot/16.04/.
For the installation you can refer to the procedure indicated here: https://doc.ubuntu-fr.org/installation
In order to create your Python work environment, install a version of anaconda.
We used version 5.1 which you can get via the following address: https://repo.continuum.io/archive/Anaconda3-5.1.0-Linux-x86_64.sh.
You can download it easily using the wget command (https://doc.ubuntu-fr.org/wget).
Once downloaded, proceed with the installation with the following command:
bash Anaconda3-5.1.0-Linux-x86_64.sh zz
Installation of specific packages
In order to facilitate the process of creating the environment and to manage potential dependency issues between the package versions you might encounter, we have created a copy of our working environment (downloadable file requirement.txt).
To use this file and thus clone our environment, enter the following command:
conda create –name zipline –file requirements.txt
This command will create an environment with the name zipline and will install specific versions of packages.
In order to activate the environment enter the following command:
source activate zipline
You can get a list of commands needed to manage Anaconda environments via the following link: https://conda.io/docs/user-guide/tasks/manage-environments.html.
Then enter the following two commands:
sudo apt-get update
sudo apt-get install build-essential
These commands provide the necessary libraries to compile a Debian package.
They will allow us, for example, to obtain the gcc and g++ compilers, and other utilities.
Installation of zipline
Zipline ( http://www.zipline.io/index.html ) is an algorithmic trading library.
Backtesting a Sentiment Analysis Strategy for Bitcoin
It is an event-based system for Backtesting (https://www.quantstart.com/articles/Event-Driven-Backtesting-with-Python-Part-I ).
Zipline is used in production as a Backtesting and Trading engine on Quantopian (https://www.quantopian.com/home) which is a free platform for building and executing trading strategies.
It’s an easy-to-use library that allows us to focus on the development of algorithms.
We used version 1.2.0 of zipline.
In order to install it, enter the following command:
pip install zipline==1.2.0
Zipline uses the notion of Bundle (http://www.zipline.io/bundles.html).
A Bundle is a database containing historical asset data in different markets. Bundles allow us to preload all the data we need to run backtests.
The first step in using a bundle is to download the data on the Quandl platform (https://www.quandl.com/).
This step may take some time as it involves downloading and processing a large amount of data. We need an API key Quandl (https://docs.quandl.com/docs#section-authentication) for this operation.
In order to get data from the quantopian-quandl bundle, enter the following command:
QUANDL_API_KEY=<your_key> zipline ingest -b quantopian-quandl
In the second part of the article we will show how to create our Blockchain Bundle that relies on crypto-currency data.
In order to have the list of bundles, you can type the command:
You should see the qauntopian-quandl Bundle appear with the creation date.
Test of our environment
Our environment is ready for a test.
We will use the buyapple.py code provided here. You can get code samples via the following address: https://github.com/quantopian/zipline/blob/master/zipline/examples/
For an understanding of each of the instructions, we invite you to look at the official documentation (http://www.zipline.io/appendix.html).
The code in buyapple.py is simply to buy APPLE shares and keep the position.
In order to execute this code, enter the following command:
zipline run -f buy_and_hold.py –capital-base 500 –bundle quantopian-quandl -s 2018-1-1 -e 2018-4-30
The different options (http://www.zipline.io/beginner-tutorial.html#running-the-algorithm) provided above allow respectively:
-f: specify the file to execute
-capital-base: indicate the initial capital for the simulation
-bundle: indicate the database to consider
-s: indicate the start date of the simulation
-e: indicate the end date of the simulation
After running the code, you should get the following figure:
The first figure shows the value of our portfolio (cash + total asset value) and the following figure shows the unit price of a share.
It is therefore normal for our portfolio to behave similarly to the market as we have taken a long position.
In the next section we will show how to create a Blockchain Bundle that uses data from the main crypto-currencies.
Then we can use this Bundle to create our strategies in the cryptocurrency markets.
Creation of a Bundle Blockchain
In the first part of the article, we showed how to use traditional financial asset data, APPLE for the example.
In this section we will show how to create a Bundle using cryptocurrency data.
The different steps to follow are described below.
This section is based on the following works: http://www.prokopyshen.com/create-custom-zipline-data-bundle.
You can also consult the official documentation online: http://www.zipline.io/bundles.html#writing-a-new-bundle.
Definition of CSV files of the assets to be included in the Bundle
Zipline uses the open-high-low-close-volume (OHLC) format to store asset data. For this you need to create CSV files in this format for each cryptocurrency you want to embed in the Bundle.
You can use the two files provided here; they contain the Bitcoin and Ethereum price history for the year 2018.
IMPORTANT: The days defined in the files are those during which the NYSE market is open because it is the default calendar used in Zipline (http://www.zipline.io/trading-calendars.html).
Select a Rebalance Period for your Portfolio
In order to define a new calendar, for example to buy or sell assets every day of the year, please refer to the following page: http://www.zipline.io/trading-calendars.html.
Definition of the module responsible for creating the Bundle
The code to create a Bundle from a csv file set is provided in the load_csv.py file.
For a detailed explanation please consult the documentation at the following address: http://www.zipline.io/appendix.html#bundles.
Copy the load_csv.py file into zipline/data/bundles/.
In order to have a trace of execution, you must set the variable boDebug to True.
To complete this step, copy the Bitcoin.csv and Ethereum.csv files to the ~/notebooks/csv/ directory.
You can change this path directly in the load_csv.py file.
Defining an extension to register the Bundle
In order to create the Blockchain Bundle, we need to define an extension that will load the data.
For this, the file extension.py provides the necessary code to register our Bundle with data from Bitcoin and Ethereum using the NYSE calendar.
Copy the extension.py file to the ~/.zipline/ folder using the following command:
We can actually create our Blockchain Bundle by running the following command:
zipline ingest -b Blockchain
You should have the following output on your console if you left the boDebug variable to True in the load_csv.py file.
Testing the Bundle Blockchain
To verify that our Bundle Blockchain has been registered, we can use the following command:
We realize that the Bundle Blockchain appears on the console:
Finally, we will test our Blockchain Bundle with the recorded Bitcoin data.
The code defined in the buy_bitcoin.py file is simply to buy 1 Bitcoin and keep this position.
In order to execute the code, enter the following command:
zipline run -f buy_bitcoin.py -bundle Blockchain -s 2018-1-1 -e 2018-1-28
We added a new option:
-bundle Blockchain: Indicates which Bundle to use
After running the code, you should get the following figure:
The first figure shows the value of our portfolio (cash + total value of assets) and the following figure the unit price of one share.
The portfolio has the same price as the market, which is normal because we bought a bitcoin and we kept that position.
In this article we started by giving a set of steps to create a local trading environment using traditional market data through the Zipline API.
Subsequently we gave the procedure to follow to integrate cryptocurrency data in this environment.