Back-testing with Metatrader4 can be a bit tricky for the unwary as there are some quirks that you need to be aware of and if you’re not careful you can end up back-testing with random data rather than the good quality data that you thought you were. In this blog post I go through how to ensure that your back-test is what you think it is. I also walk you through the necessary steps for importing and converting your data to different time-frames. This is particularly relevant if you want to use good quality back-test data of your own and are worried about it being “contaminated” by data from other sources. Incidentally, you may wish to take a look at the Snapdragon Historic Data Service as a good source of clean historic data for back-testing.
Part 1 – Clearing Out Old History Data
The first thing to be aware of is that the data that is used in the Mt4 back-test is stored in the history folder. You can look at this folder by going to the File | Open Data Folder menu option which will bring up the Windows Explorer open at the root folder for your installation. If you then open the history folder and select your broker folder you will be able to see all the history files which will have names like EURUSD1.hst. These are the same files that you can see when you select the File | Open offline menu option instead, but in that instance they are all listed in a dialog box and you can’t delete them. So that’s where the data is stored. What you also need to know is that when Mt4 closes down, it writes out all the history data to this folder. So if you want to clear all your history data for example then you need to do it in the following way:
1. Open up the folder in Windows Explorer by going to File | Open Data Folder
2. Now close down Mt4 (but leave Windows Explorer open)
3. Now to delete all the data for EURUSD say, navigate down to the appropriate files and delete all the EURUSD*.hst files.
If you’re importing your own data for a back-test then I recommend that you first clear out the old data using this method.
Part 2 – Importing Your Own Data
To import some data of your own into the history folder then this is done using the History Center dialog box, which is brought up using the Tools | History Center menu option (or just the F2 key). Once you bring up this dialog box you will see something like this
You can see down the left-hand side are listed all the markets and by drilling down into them you can see sub-folders for each of the available time-frames. One important thing to note is that in order to select one of these sub-folders you need to double-click on it – just selecting it is not enough. If you look in the title bar of the dialog box it shows you which folder is currently active.
A really important fact to note is that when you double-click on a folder Mt4 “helpfully” goes and populates it with some data of its own. So even after you’ve deleted all the history data for a pair as described in Part 1 it will still go and put some more in that it’s downloaded from its servers. The best way around this issue is to double-click on the M1 folder, let it populate the list with its own data and then to delete it again. To do this, highlight the top element, hold down the shift key and then scroll to the bottom and click on the last element. This should now highlight all the elements in the list. Now click on the Delete button to delete them all. This deletes all the entries in the list.
The next thing to do is to import your own data. You do this by clicking the Import button and then navigating to the file that you want to import. Once you’ve imported your own data, Mt4 won’t overwrite it, but it will still add stuff beyond the end of your data. There’s not a lot you can do about this but as long as you know what date your data goes up to and are careful only to test up to that point then it should be OK. You should now have your own imported data in the list.
You should now see your data in the dialog box. Finally to make sure that it’s saved to the history folder you should close down Mt4.
Part 3 – Populating Different Time-frames
Now, when running back-tests, the Strategy Tester sometimes pulls in data from different time-frames in order to construct the bar data that you are back-testing. The best way to ensure that you are only using your own data is to populate all the time-frames with your own data. Even if you’re only back-testing one time-frame it is still recommended that you populate all the others just to be on the safe side. Now you may only have 1 minute data so you how do you create M5, M15 etc from this? The answer is to use the Period Converter script. Below are the necessary steps for doing this
1. The first thing that you need to do is to change the Max bars in history and Max bars in chart options in your Mt4 to ensure that you can load in all your data without any problems. Go to Tools | Options and then select the Charts tab from the dialog box. Set the Max bars in history and the Max bars in chart options to as big a number as you can fit in there: I just keep putting 9’s in there until it won’t take any more.
2. Next you need to go to the File | Open offline menu option which will bring up the dialog box showing all the history data. You should see the file for the symbol that you’ve imported already, so your EURUSD,M1 file for example. Select this and click on the Open button. This will bring up a chart and because of your Max bars settings it will load all that it can into the chart.
3. Next we will need to generate the other time-frames. To do this we can use the Period Converter script that comes with your Mt4 installation. Drop the script into the chart and set the Period multiplier factor to 5 first of all to generate the 5 minute data. Note that you won’t see anything happen on the chart but in the Experts tab it will say X records written.
4. Now repeat this process for all the time-frames that you are likely to need, so 15, 30, 60, 240 etc. I usually don’t bother with the daily if I’m not testing a daily strategy but will do all the intraday time-frames.
5. Once you’ve done this then if you bring up the Offline Chart dialog (File | Open offline) then you should see all the different time-frames now listed there.
6. Finally, remember to set your two Max bars settings back to something moderate, otherwise it will slow your Mt4 right down.
7. The last step is to close down Mt4. This will write out all the history data to the folder so it’s now backed up.
Part 4 – Using Your Data
That should be it! You have now created all the history data that you want populated with your own data source and you can go ahead and do your back-tests with the Strategy Tester. Just make sure that you specify the end date from the test so it stops at or before your own data does. Otherwise you’ll be using some data that Mt4 has provided.