The Secrets Behind Qlibs Strategy Evaluation Process#

Common Challenges in Strategy Evaluation#

• Overfitting: When your strategy performs brilliantly in the past but fails to generalize to future market conditions.
• Data Snooping: Using in-sample data repeatedly can lead to inadvertently tailoring your strategy to the noise in historical data.
• Incomplete Risk Assessment: Focusing solely on returns without adequately evaluating volatility, drawdowns, and correlation to other assets.
• Ignored Market Realities: Omitting considerations like transaction costs, slippage, liquidity, and execution delays can make an otherwise promising strategy unprofitable.

Qlib addresses these challenges by providing clear workflows for data ingestion, pre-processing, backtesting, and performance measurement, letting you incorporate real-world nuances.

Step 1: A Simple Signal#

A straightforward example might be a moving average crossover strategy on daily stock data. Suppose we define:

• A short moving average (SMA) of 10 periods
• A long moving average (LMA) of 60 periods

When the SMA crosses above the LMA, we generate a buy signal; when it crosses below, we generate a sell signal. Below is an illustrative code snippet:

1
import qlib
2
from qlib.data import D
3
from qlib.data.dataset import DatasetD, TSDatasetH
4
from qlib.contrib.strategy.signal_strategy import BaseSignalStrategy
5

6
# Initialize Qlib
7
qlib.init(provider_uri="~/.qlib/qlib_data/cn_data")
8

9
class MovingAverageCrossover(BaseSignalStrategy):
10
    def __init__(self, short_period=10, long_period=60, **kwargs):
11
        super().__init__(**kwargs)
12
        self.short_period = short_period
13
        self.long_period = long_period
14

15
    def generate_signal(self, instrument, start_time=None, end_time=None, freq='day'):
16
        # Load price data
17
        df = D.features([instrument], ['$close'],
18
                        start_time=start_time,
19
                        end_time=end_time, freq=freq)
20
        df['SMA'] = df['$close'].rolling(self.short_period).mean()
21
        df['LMA'] = df['$close'].rolling(self.long_period).mean()
22
        df['Signal'] = 0
23
        df['Signal'][df['SMA'] > df['LMA']] = 1
24
        df['Signal'][df['SMA'] < df['LMA']] = -1
25
        return df['Signal']

In this snippet:

• We fetch historical close prices for a given instrument.
• Calculate short- and long-term moving averages.
• Assign a positive signal (1) when the short average is above the long average, and a negative signal (-1) when it is below.

Step 2: Backtesting the Strategy#

Next, we define how these signals turn into trades. Qlib provides an Executor to simulate the process. One of the more commonly used approaches involves specifying parameters like transaction cost and making sure signals are converted into trades for each day (or bar).

1
from qlib.backtest import backtest, executor
2
from qlib.contrib.strategy.strategy import WeightStrategyBase
3
from qlib.constant import REG_CN
4

5
# WeightStrategy: Convert signals to portfolio weight
6
class SimpleWeightStrategy(WeightStrategyBase):
7
    def __init__(self, cash_budget=1e6, **kwargs):
8
        super().__init__(**kwargs)
9
        self.cash_budget = cash_budget
10

11
    def generate_weight_position(self, score, current, trade_start_time, current_time):
12
        # score is the signal: 1 or -1
13
        # If 1, invest entire budget. If -1, short entire budget.
14
        weight = {}
15
        for instrument in score.index:
16
            signal = score.loc[instrument]
17
            # Let's keep it simpler by either going fully long or fully short
18
            weight[instrument] = signal
19
        return weight
20

21
# Combine the above
22
def run_moving_average_crossover_backtest(instrument="SH600000", start_time='2019-01-01', end_time='2020-12-31'):
23
    mac_strategy = MovingAverageCrossover()
24
    weight_strategy = SimpleWeightStrategy()
25

26
    # Qlib's executor can handle trades daily
27
    trade_executor = executor.SimulatorExecutor()
28

29
    # Perform the backtest
30
    bt_result = backtest(
31
        start_time=start_time,
32
        end_time=end_time,
33
        strategy=mac_strategy,
34
        trade_strategy=weight_strategy,
35
        executor=trade_executor
36
    )
37
    return bt_result
38

39
# Example usage
40
result = run_moving_average_crossover_backtest()
41
analysis_df = result["analysis"]["portfolio"]
42
print(analysis_df.head())

Here, backtest coordinates everything:

• start_time: The beginning date for historical simulation.
• end_time: The ending date for historical simulation.
• strategy: The instance that provides signals.
• trade_strategy: Determines how signals translate into position sizes.
• executor: Simulates the mechanics of order execution.

Reviewing Results#

analysis_df will return a DataFrame with daily portfolio metrics, such as daily returns, net value, and other stats. From there, you can generate performance metrics or produce charts to see how the strategy performed over time.

1. Hyperparameter Tuning#

Often, you do not know the optimal lookback periods for your moving averages. Qlib supports typical model selection patterns. For example, if you want to experiment with different short and long window sizes:

1
short_windows = [5, 10, 20]
2
long_windows = [30, 60, 120]
3
best_config = None
4
best_sharpe = float('-inf')
5

6
for sw in short_windows:
7
    for lw in long_windows:
8
        if sw >= lw:
9
            continue
10
        strategy = MovingAverageCrossover(short_period=sw, long_period=lw)
11
        weight_strat = SimpleWeightStrategy()
12
        trade_exec = executor.SimulatorExecutor()
13

14
        result = backtest(
15
            start_time='2019-01-01',
16
            end_time='2020-12-31',
17
            strategy=strategy,
18
            trade_strategy=weight_strat,
19
            executor=trade_exec
20
        )
21
        # Evaluate performance
22
        df_metrics = result['analysis']['excess_return_without_cost']
23
        ann_ret = (1 + df_metrics['return'].cumsum()[-1])**(252/len(df_metrics)) - 1
24
        vol = df_metrics['return'].std() * np.sqrt(252)
25
        sharpe = ann_ret / vol if vol != 0 else 0
26

27
        if sharpe > best_sharpe:
28
            best_sharpe = sharpe
29
            best_config = (sw, lw)
30

31
print("Best Config:", best_config, "with Sharpe Ratio:", best_sharpe)

2. Transaction Costs and Slippage#

Transaction costs: Real trading involves costs such as broker commissions and spread. Qlibs backtesting engine accommodates a cost parameter or a more elaborate cost model.
Slippage: The difference between a trades expected fill price and the actual fill price. This can occur when the strategys order is large relative to the market volume, or when markets move quickly. You can incorporate slippage assumptions via Qlibs trade_strategy or executor arguments.

Heres a snippet that includes a simple cost assumption:

1
trade_executor = executor.SimulatorExecutor(
2
    trade_type=executor.TradeType.SINGLE_POSITION,
3
    closing_price='adj',  # Use adjusted closing price
4
    trade_cost=0.001      # 0.1% transaction cost
5
)

3. Walk-Forward / Rolling Backtesting#

To avoid overfitting to one period, it helps to continuously re-train or re-select parameters in out-of-sample windows. A walk-forward test splits data into multiple segments, training on one period and testing on the subsequent period, then rolling forward. This is more advanced but vital for robust evaluation:

1. In-sample period: Use historical data to optimize parameters.
2. Out-of-sample period: Test the optimized strategy on the next block of data.
3. Roll forward: Shift the window and repeat, collecting metrics across all out-of-sample segments.

1. Risk Management Techniques#

• Position Sizing: Instead of going all-in, you might risk a fraction of capital based on volatility.
• Stop-Loss: Close your position if the loss hits a certain threshold.
• Hedging: Use derivatives or offsetting positions to reduce market exposure.

Below is a simplistic example showing a stop-loss approach within a custom Executor:

1
class StopLossExecutor(executor.SimulatorExecutor):
2
    def __init__(self, stop_loss_pct=0.05, *args, **kwargs):
3
        super().__init__(*args, **kwargs)
4
        self.stop_loss_pct = stop_loss_pct
5
        self.entry_price = {}
6

7
    def generate_trade_decision(self, score, current, trade_start_time, current_time):
8
        # We do everything as the normal executor does
9
        trade_decision = super().generate_trade_decision(score, current, trade_start_time, current_time)
10

11
        # Then we adjust for stop loss
12
        for trade_order in trade_decision.trade_order_list:
13
            inst = trade_order.symbol
14
            if inst not in self.entry_price:
15
                self.entry_price[inst] = trade_order.last_price  # Track entry price
16

17
            current_loss = (trade_order.last_price - self.entry_price[inst]) / self.entry_price[inst]
18
            if trade_order.position.direction > 0 and current_loss < -self.stop_loss_pct:
19
                # Liquidate long if below stop loss
20
                trade_order.position.close()
21
            elif trade_order.position.direction < 0 and current_loss > self.stop_loss_pct:
22
                # Liquidate short if loss is too high
23
                trade_order.position.close()
24

25
        return trade_decision

2. Factor Analysis#

If you are building alpha factors (predictive signals) using Qlibs dataset structure, you might conduct a factor analysis to determine how each factor contributes to your strategy. This typically involves:

1. Computing factor returns using cross-sectional regressions.
2. Checking factor exposures and correlations.
3. Evaluating the factors performance stability over time.

Example pseudocode for factor return computation:

1
from qlib.contrib.analysis.factor_analysis import FactorReturnAnalysis
2

3
factors = ['factor_mom', 'factor_value']
4
analysis = FactorReturnAnalysis(
5
    instruments=['SH600000', 'SH600519'],
6
    factors=factors,
7
    start_time='2019-01-01',
8
    end_time='2021-01-01'
9
)
10
factor_returns = analysis.compute_factor_returns()
11
print(factor_returns.head())

1. Automated Pipeline Integration#

Instead of one-off?scripts, you might incorporate Qlib into a data pipeline that continuously:

• Fetches new price and fundamental data from an API.
• Updates signals or re-trains predictive models.
• Automatically runs backtests or paper trading environments.
• Sends signals or orders to a broker interface.

This is often achieved using Docker containers, cloud services, scheduling frameworks (e.g., Airflow, cron jobs), and database solutions for robust data management.

2. Multi-Asset and Multi-Strategy Portfolios#

Institutional portfolios may combine multiple strategies across stocks, bonds, commodities, and other markets. Qlibs modular design encourages you to set up multiple strategies and unify them under a single portfolio or to evaluate them individually and compare performance. You might schedule meta-strategies?that allocate capital among sub-strategies based on recent performance or market conditions, thus layering your risk.

3. Alternative Data and Feature Engineering#

Going professional often involves data beyond price/volume:

• News sentiment (e.g., from web scraping or aggregator APIs).
• Social media feeds.
• Satellite imagery (e.g., for agricultural estimates).
• Supply chain data.

In Qlib, you can integrate these data sources by designing custom Dataset objects or hooking third-party APIs. Your factor generation can become more sophisticated, potentially using text analytics and deep learning to develop alpha factors.

4. High-Frequency Strategies and Real-Time Execution#

While many examples center on daily bars, advanced practitioners may trade intraday or high-frequency data. Qlib is capable of handling high-frequency data. You must carefully manage:

• Latency: Strategies that require fast updates.
• Queueing: Handling large volumes of tick data.
• Execution constraints: High frequency trades must consider exchange rules, microstructure effects, and significantly higher transaction costs.

5. Robust Stress Testing#

Professional context demands stress tests to see how strategies perform under extreme volatility or during market crashes. Qlib can be adapted to replay historical crisis periods or artificially manipulate data (e.g., price shock simulations). Stress testing helps confirm if your strategy can survive tail risks without catastrophic drawdowns.