Building a Real-Time Market Data Generator for Testing Algo Trading Systems

When building algorithmic trading systems, the hardest part to test reliably is the real-time market feed. Live markets are unpredictable, noisy, and expensive to test against. A lightweight, deterministic market data generator that simulates realistic tick-level patterns lets you validate strategies, risk controls, and order routing without paying exchange fees or risking real money.

Why Build a Market Data Generator?

• Deterministic testing — replayable market scenarios for consistent regression testing.
• Edge case simulation — bursts, gaps, halts, and illiquid periods.
• Rapid iteration — run thousands of “market minutes” in seconds.
• Offline CI integration — validate trading logic in automation pipelines.

Core Design Requirements

1. Generate tick-level events: price, size, symbol, timestamp.
2. Support multiple instruments: equities, options, indices.
3. Inject scenarios: volatility spikes, news shocks, gaps.
4. Produce realistic microstructure: bid/ask, trade sizes, spreads.
5. Adjustable time scale: real-time or accelerated playback.

System Architecture

The generator consists of five lightweight components:

• Instrument Catalog — a JSON list defining symbols, strikes, expiries, and lot sizes.
• Market Clock — drives the tick loop in real-time or accelerated mode.
• Tick Generator — produces stochastic price updates per instrument.
• Scenario Engine — injects external events such as volatility spikes or news.
• Publisher — streams events to subscribers via WebSocket or Server-Sent Events (SSE).

  ┌───────────┐      ┌────────────────────────┐      ┌────────────────┐
  │ Generator │─────▶│ FastAPI / Redis Stream │─────▶│ Trading Engine │
  └───────────┘      └────────────────────────┘      └────────────────┘
                                                              │
                                                              ▼
                                                     ┌────────────────┐
                                                     │ UI Dashboard   │
                                                     └────────────────┘
          

Tick Generation Model

A simple yet realistic price evolution model is a stochastic random walk with drift and volatility. Each tick is computed as:

p_next = p_prev * (1 + drift*dt + sigma * sqrt(dt) * N(0,1) + jump)

• drift — directional bias representing market sentiment.
• sigma — volatility factor controlling randomness.
• jump — a sudden change drawn from a Poisson process (for “news” events).

You can further enhance realism by rounding prices to tick size, generating heavy-tailed trade volumes, and simulating spreads between bid and ask quotes.

Implementation Example (Python)

import asyncio, random, math, json, datetime

async def tick_generator(symbol, base_price=100.0, drift=0.0001, sigma=0.001):
    p = base_price
    while True:
        dt = 1/100  # seconds between ticks
        jump = 0
        if random.random() < 0.0005:  # rare news event
            jump = random.uniform(-0.03, 0.05)
        p *= (1 + drift*dt + sigma*math.sqrt(dt)*random.gauss(0,1) + jump)
        yield {
            "symbol": symbol,
            "price": round(p, 2),
            "volume": random.randint(10, 1000),
            "timestamp": datetime.datetime.utcnow().isoformat() + "Z"
        }
        await asyncio.sleep(dt)

This coroutine continuously yields ticks that you can broadcast via FastAPI using WebSockets or Server-Sent Events.

Publishing with FastAPI + SSE

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio, json

app = FastAPI()

async def event_stream():
    gen = tick_generator("RELIANCE")
    async for tick in gen:
        yield f"data: {json.dumps(tick)}\n\n"

@app.get("/stream")
async def stream():
    return StreamingResponse(event_stream(), media_type="text/event-stream")

Your frontend can connect using JavaScript’s EventSource:

const es = new EventSource("/stream");
es.onmessage = (e) => {
  const tick = JSON.parse(e.data);
  console.log("Tick:", tick.symbol, tick.price);
};

Scenario Engine

To make simulations meaningful, add a “scenario engine” that manipulates volatility and drift. For instance:

• Volatility Spike — increase sigma by 5× for 2 minutes.
• Market Crash — apply negative drift and trigger price gaps.
• News Event — random positive/negative jump with increased volume.

Integrating into CI/CD

You can embed the generator into automated test pipelines:

• Run replay tests with stored random seeds.
• Assert P&L invariants: no negative quantities, logical fills, stop-loss execution.
• Collect metrics from simulated runs to track strategy performance trends.

Closing Thoughts

A reliable market data generator is an indispensable tool for any algorithmic trading system. It bridges the gap between theory and production by providing controlled realism. Start with a single-symbol generator and incrementally add features — volatility modeling, bursts, option greeks, or multi-exchange feeds.

For Growth Quantix, my goal is to use this generator to validate latency, signal accuracy, and trade execution logic before connecting to the real Upstox live feed. The same system can even run on weekends for regression testing or strategy dry-runs.