High frequency quantized trading platform event drive principle

【The first step in developing a trading system is to understand the working principle of the system after the transaction interface is established. This article is a summary of the company’s driver platform documentation I’ve seen. The company was modified from open source vn. py, but the driver engine is the same.

Classification of computer programs

All computer programs can be broadly divided into two types: scripted (single run) and continuous (until the user actively exits).

1、Scripting type

Scripting programs include the earliest batch files and the use of Python to do transaction strategy testing and so on. The characteristics of such programs are that after the user starts up, they will run step by step according to the programmed steps, and all the steps will automatically exit after running.

2、Continuous running type

Continuous running programs include the operating system and most of the software we use everyday, etc. These programs run continuously in an infinite loop after startup until the user actively exits.

Two. Continuous running program

The transaction system we want to develop is a continuous running program, which can be roughly divided according to the operation mechanism of its computing logic.Time driven and event drivenTwo.

1、Time driven

Time-driven program logic is relatively easy to design, simply to let the computer do something automatically at intervals. The thing itself can be complex and involve many steps, but these steps are linear, step by step.

The following code shows a very simple time driven Python program.

from time import sleep

def demo():
    print u'Time driven programs run demo functions every 1 second.While 1:Demo ()Sleep (1)

A time-driven program essentially runs a script (the demo function in the code above) at regular intervals. Although the script itself can be long and contain a lot of steps, we can see that the mechanism of this program is relatively simple and easy to understand.

Give some examples of quantified transactions:

  1. Every five minutes, through the open API of Sina Finance Web page, we read the prices of Shanghai and Shenzhen 300 stocks, sort them according to the daily increase and output them to the computer screen.
  2. Every one second, check the latest received TICK data, update the K-line and other technical indicators, check whether the trend strategy to meet the order conditions, if satisfied, then execute the order.

For quantified transactions requiring high speed (intra day CTA strategy, high frequency strategy, etc.),Time driven programs have a very big disadvantage.：The processing delay of data information in response operation. In the example, the program does not respond to new data information (quotations, transaction pushes, etc.) in the first second after each logical script runs, and the relevant count is only made when the script runs again after the wait time has ended.Calculate processing. The direct consequence of dealing with delays in quantified transactions is that the market price slips and the limit order misses the price that could have been traded.

Time-driven programs have other drawbacks in quantifying transactions, such as waste of CPU computing resources, high complexity of implementing asynchronous logic, and so on.

2、event driven

Corresponding to the time-driven is the event-driven program: when a new event is pushed into the program (such as API pushing new quotes, transactions), the program immediately calls the corresponding processing function of the event for related operations.

The event-driven version of the above example: The trading program monitors the TICK data of the stock index, and when no new quotations come in, the program remains monitored without doing anything; when new data is received, the data processing function immediately updates the K-line and other technical indicators, and checks whether the next item meets the trend strategy.Execute the order.

For simple programs, we can use the scheme in the above test code directly.APICallback functionThe corresponding logic is written in. But as the complexity of the program increases, such a scheme will become increasingly infeasible. Suppose we have a quantitative trading system with a graphical interface. The system receives the stock index futures data pushed by the API at some point. The data system needs to be processed as follows:

  1. Update the graph of the K-line shown on the chart (drawing).
  2. Update the market index of Stock Index Futures (form update)
  3. Policy 1 requires an internal algorithm to run once to check whether the data triggers the policy to place an order (operation, order)
  4. Policy 2 also needs to run an internal algorithm once to check whether the data triggers the policy to place an order (operation, order)
  5. The wind control system needs to check whether the latest quotation price will cause the overall risk of the account to exceed the limit, if the limit needs to be alarmed (operation, alarm)

Writing all of the above into a callback function becomes a very bad solution, too long code is error-prone, and scalability is poor. Every addition of a policy or function requires modification of the previous source code (experienced readers will know that frequent modification of production code is very dangerous)Operation management method.

Summary: Although there is no graphical interface on our trading platform, because this is only an auxiliary function, not the core function of production. However, there may be other signals or events that we need to deal with, so when to deal with them? How many resources are allocated? So we should reduce coupling, in order to solve this problem.We need to use an event-driven engine to manage event listeners for different events and perform all event-driven operations.

Three, event driven engine principle

vn.pyThe vn.event module in the framework contains an extensible event driven engine. The implementation of the whole engine is not complicated. After removing annotations and blank lines, there are probably about 100 lines of code.

# encoding: UTF-8

# System moduleFrom Queue import Queue, EmptyFrom threading import ThreadThird party moduleFrom PyQt4.QtCore import QTimerThe module developed by ourselvesFrom eventType import *It is very popular in the world.It is very popular.Class EventEngine:"" ""Event driven engineAll variables in the event driven engine are set to private, in order to prevent carelessness.These are modified from the outside.The value or state of a variable causes bug.Variable description__queue: private variable, event queue__active: private variable, event engine switch.__thread: private variable, event processing thread__timer: private variable, timer__handlers: private variable, event handler dictionaryMethod description__run: private method, event processing thread running continuously.__process:The private method handles events and calls the listener function registered in the engine._u onTimer: Private method that stores timer events into the event queue after the timer fires at a fixed event intervalStart: common method, start engineStopPublic method, stop engineRegister: public method, register listener function to engine.Unregister: public method to logoff the listener function to the engine.Put: public method to store new events into event queues.Event listener functions must be defined as input parameters that are only one event object.functionDef func (event)...Object methodDef method (SEL)F, event)..."" ""#---------------------------------------------------------------- ----Def __init__ (self):"Initialize event engine" ""Incident queueSelf.__queue = Queue ()Event engine switchSelf.__active = FalseThread handling threadSelf.__thread = Thread (TAR)Get = self.__run)The timer is used to trigger timer events.Self.__timer = QTimer ()Self.__timer.timeOut.connect (self.__onTimer)__handlers is a dictionary that stores the corresponding event call relationship.Each key corresponds to a list.Saved the function function of monitoring the event.Self.__handlers = {}What's more?----------------------Def __run (self):"" engine run "" "While self.__active = = True:Try:Event = self.__queue.get (block = True, timeout = 1). The blocking time of the acquisition event is 1.secondSelf.__process (event)Except Empty:Pass- -------------------------------------------------------------------Def __process (self, event):"Handling" events ""Is there a processing function for monitoring the event?If event.type_ in self.__handlers:If it exists, the event is passed sequentially to the execution of the processing function.[handler (event) for handler in self.__handlers[event.type_]]The above sentence is the way to parse the Python list.#for handler in self.__handlers[event.type_]:#handler (event)What's more?- ----Def __onTimer (self):"" store timer events in event queues "" "Create a timer eventEvent = EvEnt (type_=EVENT_TIMER)To store timer events in queues.Self.put (event)It is just like that.----------------------------------------------Def start (self):"" engine start "" "The engineSet up to startSelf.__active = TrueStart processing threadSelf.__thread.start ()Start timerThe timer interval is set to 1 second by default.Self.__timer.start (1000)What's more?-------------------------Def stop (self):"" stop engine "" "The engine is set to stop.Self.__acTive = FalseStop timerSelf.__timer.stop ()Waits for event processing thread exitSelf.__threaD.join ()What's more?Def registEr (self, type_, handler):"" registered event handler function listens "" "Try to get the list of processing functions corresponding to the event type, if not, create a list of processing functions.Try:HandlerList = self.__handlers[type_]Except KeyError:HandlerList = []Self.__handlers[type_] = handlerListIf you want to register a processor that is not in the processor list of the event, register the event.IfHandler not in handlerList:HandlerList.append (handler)It is just like that.---------------------------------------------Def unregister (self, type_, handler):"" "note"Pin event handler listensIt tries to get the list of processing functions corresponding to the event type, and if no, it ignores the cancellation request.Try:HandlerList = self.haNdlers[type_]If the function exists in the list, remove the function.If handler in handlerList:HandlerList.remove (handler)If the function list is empty, remove the event type from the engine.If not handlerList:Del self.handlers[type_]Except KeyError:PassIt is ---------------------------------------------------------Def put (self, event):"To" event queueStore event "" "Self.__queue.put (event)

1、Initialization

When event driven engine objects are created, initialization function __init__ creates the following private variables:

  • __queue：Queue for saving events
  • __active：Switch for controlling engine start and stop
  • __thread：Threads responsible for handling events and performing specific operations
  • __timer：A timer for triggering timing events at regular intervals.
  • __handlers：A dictionary for event handling functions corresponding to different types of events.

2、Registered event handling function

The engine provides the register method to register the event handler function with the engine, and the incoming parameters are

  1. type_：Constant strings representing event types, defined by the user, note that there is no duplication between different event types
  2. handler：When this type of event is triggered, the user expects the event handler to do the corresponding operation, and the definition method of the function refers to the comments in the code

When the user calls the register method to register the event handler, the engine attempts to get a list of handlers corresponding to the event type in the handlers dictionary (if not an empty list) and adds the event handler to the list. Using PythonList objects make it easy for users to control the order in which multiple event handlers work under the same event type, so some complex algorithms involving multistep operations can be guaranteed to execute in the right order, which is best compared to some system 0 messaging mechanisms, such as Signal / Slot for QtBig advantage.

If the target market changes, the high frequency arbitrage algorithm of the option needs to perform the following operations:

  1. First use the pricing engine to calculate the new option theoretical price and Greek value.
  2. Use the risk control engine to summarize the current risk and calculate the middle price.
  3. The arbitrage engine is used to calculate the specific price and make the order based on the parameters such as the predetermined price difference and the number of orders.

The above three steps can ensure the correct order of operation by registering and listening to the market events of the subject matter when the trading system starts.

The unregister method, which corresponds to register, is used to listen for logout event handlers with the same incoming parameters. See the source code for details. In practical applications, users can dynamically combine register and unregister methods.You need to listen to some events and listen to the monitor, so as to save CPU resources.

Here let the author Tucao some domestic C++ platform (of course not all), each strategy listens to all the order returns in the system, if it is related to its own processing, irrelevant PASS. This way of writing is just how much unnecessary judgment to judge whether or not it is related to itself.How much CPU resources are wasted, and as the number of policies increases, waste tends to increase linearly, and this platform is still clamoring for high frequencies, alas…

3、Trigger event

Users can put events into the event queue_u queue through the engine’s put method and wait for the event processing thread to process them. The implementation of the event class is as follows:

########################################################################
class Event:
    """Event object"""

    #----------------------------------------------------------------------
    def __init__(self, type_=None):
        """Constructor"""
        self.type_ = type_      # Event typeSelf.dict_= {}         # Dictionary is used to store specific event data.

When the object is created, the user can select the incoming event type string type_ as the parameter. The dict_ dictionary is used to store specific event related data information for operation of event handlers.

4、Continuous operation of event processing thread

The function that runs continuously in the event processing thread thread of the event engine is run: When the switch activity of the event engine is not turned off, the engine tries to read the latest event from the event queue, and if the read is successful, the process function is called immediatelyWhen this event is handled, if it cannot be read (the queue is empty), it enters a blocking state to save CPU resources, and enters the above loop again when the blocking time (default is 1 second) ends.

__processWhen the function works, it first checks whether the event type of the event object exists in the handlers dictionary. If it exists (indicating that an event handler is listening for the event), it calls the event handler in the listener list in the order of registration to do the relevant operation.

5、timer

The u timer in the event engine is a QTimer object in PyQt, which provides a very simple function: the function u onTimer is automatically run at intervals (set by the user). The __onTimer function creates a type of EVENT_TIMER.Event objects (defined in the eventType. py file) are stored in the event queue by calling the put method of the engine.

Sensitive readers may have realized that this timer is essentially a time-driven function. Although we mentioned the importance of event-driven in the development of quantitative trading platforms earlier, it is undeniable that the implementation of some trading functions must be time-driven, for example, if two seconds after placing an order is not traded.Cancel the order immediately and save the transaction record every 5 minutes to the database. This type of functionality is implemented with the option of using an event handler to listen for timer events of type EVENT_TIMER (see the example in the next section, Use of Event Driven Engines).

6、Start and stop

Users can start and stop the event-driven engine through start and stop, the principle is very simple readers can directly refer to the source code.

When the timer is started, the event interval is set by default to 1 second (1000 milliseconds), which the user can adjust to his or her needs. Assuming that the user uses a time-driven function with an interval of minutes, you can choose to set the parameter to 60 seconds (600,000 milliseconds), and so on..

Four, event driven engine usage

Also in eventEngine.py, a test code test function is included to show how event-driven engines are used:

#----------------------------------------------------------------------
def test():
    """Test function"""
    import sys
    from datetime import datetime
    from PyQt4.QtCore import QCoreApplication

    def simpletest(event):
        print u'Handling timer events triggered per second:%s' % str(datetime.now())

    app = QCoreApplication(sys.argv)

    ee = EventEngine()
    ee.register(EVENT_TIMER, simpletest)
    ee.start()

    app.exec_()


# Direct run script can be tested.if __name__ == '__main__':
    test()

testThe function as a whole contains these steps.

  1. Import the relevant packages (sys, datetime, PyQt4), and note that since the EventEngine implementation uses the PyQt4 QTimer class, the entire program must run in a Qt event loop, even with QCoreApplicaThe exec_() method of tion (or QApplication in PyQt4. QtGui) starts the event loop in the main thread of the program.
  2. Define a simple function simpletest that contains an input parameter event object that prints a character and the current time after the function is called
  3. Create QCoreApplication object app
  4. Create event driven engine EventEngine object EE
  5. Register the simpletest function to the engine to monitor the timer event EVENT_TIMER.
  6. Start event driven engine
  7. Start the Qt event loop

On the whole, when users develop their own programs, they need to modify only second steps and fifth steps:Create your own event handler and register these functions on the corresponding event type.。

summary

With the API interface and event-driven engine, we can start developing our own platform, and the next few articles will show you step by step the development of a simple LTS trading platform.

Leave a Reply

Your email address will not be published. Required fields are marked *