Testing

Backtesting

A backtest replays previously recorded market data to drive a simulated market. Your strategy connects to that simulation, receives market updates, and places orders. The orders match against the simulated market and produce trades, all in a controlled environment.

Unlike production, a backtest is fully deterministic: the same backtest run twice produces identical results. CryptoStruct records all data from every supported market, and once you have the files locally, backtests run on your machine without any external services.

Why backtest

Risk-free testing. Validate config or algorithm changes without risking capital.
Market analysis. Use data from markets you don't yet trade to evaluate opportunities and characteristics.
Latency simulation. Configure per-instrument latency separately, which matters when a strategy hits instruments in different markets or regions.
Faster than real time. Backtests run much faster than wall clock, so months of data play back in minutes or hours.
No special builds. The exact same strategy and configuration files used in production run in the backtest.
Debuggable. Run the backtest under your IDE's debugger (IntelliJ, Eclipse), set breakpoints, step through. Simulation timing is unaffected.
Data collection. Strategies can write extra data for offline analysis with no impact on simulated trading performance.
Profilable. Attach a profiler and look for hotspots; the backtest doesn't care.

Tip

Backtest often. It's the cheapest way to validate a change before it reaches production.

Limitations

Warning

Backtests have inherent limitations:

Static market. Recorded data is fixed; your strategy can't move the market in any meaningful way.
Heuristic matching. Exchanges typically aggregate market data (per level, per time interval), so the backtest applies heuristics to decide how individual orders match.
Single instance. Only one strategy instance runs per backtest.
Feature parity. Not every production feature is supported yet. This is improving over time.

The trade-offs are real, but the backtest is the easiest, fastest, risk-free environment for iterating on a strategy.

Backtest configuration

Required inputs

A backtest is configured through a single JSON file. The file describes:

Strategy instance configuration

The same shape as in production:

Strategy class name to instantiate.
Instance name.
Instance parameters (strategy-specific).

Strategy artifact

Path to the compiled JAR.

Market data files

Per instrument:

Market data file. The last 3 months are available for direct download. For older data, contact CryptoStruct support.
Latency (microseconds). One-way latency applied to order requests on the way to the exchange and to responses and market updates on the way back.
Time-priority heuristic. When the simulation can't decide exactly how a level update affects orders at the same level, this picks the policy:
- REMOVE_SUPERIOR: prefer removing volume in front of your order (raises fill probability).
- REMOVE_INFERIOR: prefer removing volume behind your order.
- REMOVE_EVENLY: prefer removing volume evenly around your order.

Optional: master data files

Underlyings and instruments. Recent market-data files include this data, so the explicit configuration is usually unnecessary.

Optional: initial account state

Wallet balances.
Existing positions.

Optional: message-channel configuration

Output directory for strategy messages.
List of input files containing messages to forward to the strategy.

Configuration example

{
  "strategyFile": "./target/my-strategy.jar",
  "strategyConfig": {
    "instanceName": "instance123",
    "strategyName": "MyStrategy",
    "parameters": {
      "tradingInstrument": {
        "id": 22,
        "accountId": 15,
        "maxOrderCount": {
          "bid": 1,
          "ask": 1
        },
        "maxPosition": {
          "bid": "200",
          "ask": "200"
        }
      },
      "dataInstrument": {
        "id": 67824
      },
      "myParameter1": "123.123",
      "myParameter2": 123
    }
  },
  "instruments": [
    {
      "instrumentId": 22,
      "marketDataFile": "2022-08-17_instrument_22.zst",
      "latencyUs": 10000,
      "timePrioHeuristic": "REMOVE_SUPERIOR"
    },
    {
      "instrumentId": 67824,
      "marketDataFile": "2022-08-17_instrument_67824.zst",
      "latencyUs": 10000,
      "timePrioHeuristic": "REMOVE_SUPERIOR"
    }
  ],
  "accounts": [
    {
      "accountId": 15,
      "wallet": [
        {
          "underlyingId": 100,
          "marginBalance": "0"
        },
        {
          "underlyingId": 103,
          "marginBalance": "0"
        }
      ],
      "positions": [
        {
          "instrumentId": 22,
          "position": "500"
        }
      ]
    }
  ],
  "messageChannels": {
    "outputDirectory": "output",
    "inputFiles": ["input1.txt", "input2.txt"]
  }
}

Field reference:

strategyFile: path to the compiled strategy JAR.
strategyConfig: strategy instance configuration.
- instanceName: unique identifier for this instance.
- strategyName: name of the strategy class to instantiate.
- parameters: strategy-specific configuration parameters.
instruments: instruments to simulate.
- instrumentId: instrument identifier.
- marketDataFile: path to the market-data file.
- latencyUs: one-way latency in microseconds.
- timePrioHeuristic: time-priority heuristic (see above).
accounts: account state.
- accountId: account identifier.
- wallet: wallet balances per underlying.
- positions: existing positions per instrument.
messageChannels: message-channel configuration.
- outputDirectory: directory for strategy message output.
- inputFiles: message input files.

Running a backtest

Basic execution

Once the configuration is ready, run the backtest tool with the config file:

java -jar backtest.jar backtest-configuration.json

Command-line arguments

In addition to the config file, the tool accepts:

--eventlog <file>: write all order requests and trades to a file.
--loglevel <DEBUG|INFO|WARN|ERROR>: set the strategy's log level.
--message-channel-directory <directory>: write all data published on channels by the strategy to files.

Example:

java -jar backtest.jar backtest-configuration.json \
  --eventlog trades.log \
  --loglevel DEBUG \
  --message-channel-directory ./output

Message input files

A set of messages can be read from files and forwarded to the strategy during the backtest run. Useful for testing how the strategy reacts to external events or signals.

File format

One message per line.
Files can be plain .txt or compressed as .gz or .zst.
Messages are forwarded only to channels opened with type String. Messages targeted at channels with other types are dropped.

Message structure

{
  "time": "1970-01-01T00:00:00.000Z",
  "origin": "myOrigin",
  "channel": "myChannel",
  "body": "myBody"
}

Fields:

time: ISO 8601 timestamp. This is the delivery time of the event. The strategy receives the message when backtest time reaches this timestamp. Backtest time is bounded by the time range of the referenced market-data files, so timestamps need to fall inside that range.
origin: the message origin.
channel: the channel to publish on.
body: the message content. A string, a JSON object, or an array. Objects and arrays are escaped and converted to a string.

Note

Messages are delivered on backtest simulation time, not wall-clock time. Make sure your timestamps line up with the data range.

Script testing

Warning

Script testing is still under development. Features may change.

Script testing uses the same market simulation as backtesting, but instead of replaying recorded data, every market update is driven by explicit script commands. You can also write expectations (e.g. on order operations) that the strategy must meet.

Script file structure

A script file has three sections:

[masterdata]

A JSON structure with all master data: instruments, underlyings, accounts.

[strategies]

A JSON structure with the strategy configuration.

[script]

The script itself, one command or expectation per line.

Script commands

Each command starts with a time advance, in milliseconds.

Example:

+100 update TEST1 BID 10 99.9

This advances simulation time by 100ms and updates instrument TEST1 by setting the size of the bid level at price 99.9 to 10.

Expectations

Expectations are checked before the next update fires. The strategy must meet the expectation within the specified window.

Complete example:

+100 update TEST1 BID 10 99.9
expect order-place TEST1 BID 5 99.7 --name order1
+100 update TEST1 BID 15 99.9

This script:

Updates instrument TEST1 with a bid level.
Expects the strategy to place a bid order with size 5 at price 99.7 (referenced by name order1).
Verifies the expectation before the next update (the order has to land within 100ms of the first update).

Note

Script testing simulates a fixed latency of 10ms between strategy and exchange. Keep that in mind when writing very small time steps.

Execution

Two ways to run a script test.

Command-line tool:

java -jar scripttest.jar myscript.script

The tool also documents the available script commands and expectations. Run java -jar scripttest.jar --help to print them.

Unit test:

Use the strategy-test library to run script tests as unit tests in your build pipeline.

Development and feature requests

The backtest is used heavily by both customers and the internal team. Development is driven by feature requests from both sides.

Tip

If something is missing from backtesting or script testing for your use case, contact CryptoStruct support with the requirement.