# Testing ## Backtesting A **backtest** replays previously recorded market data to drive a simulated market. Your strategy connects to that simulation, receives market updates, and places orders. The orders match against the simulated market and produce trades, all in a controlled environment. Unlike production, a backtest is fully deterministic: the same backtest run twice produces identical results. **CryptoStruct** records all data from every supported market, and once you have the files locally, backtests run on your machine without any external services. ### Why backtest - **Risk-free testing.** Validate config or algorithm changes without risking capital. - **Market analysis.** Use data from markets you don't yet trade to evaluate opportunities and characteristics. - **Latency simulation.** Configure per-instrument latency separately, which matters when a strategy hits instruments in different markets or regions. - **Faster than real time.** Backtests run much faster than wall clock, so months of data play back in minutes or hours. - **No special builds.** The exact same strategy and configuration files used in production run in the backtest. - **Debuggable.** Run the backtest under your IDE's debugger (IntelliJ, Eclipse), set breakpoints, step through. Simulation timing is unaffected. - **Data collection.** Strategies can write extra data for offline analysis with no impact on simulated trading performance. - **Profilable.** Attach a profiler and look for hotspots; the backtest doesn't care. > [!TIP] > Backtest often. It's the cheapest way to validate a change before it > reaches production. ### Limitations > [!WARNING] > Backtests have inherent limitations: - **Static market.** Recorded data is fixed; your strategy can't move the market in any meaningful way. - **Heuristic matching.** Exchanges typically aggregate market data (per level, per time interval), so the backtest applies heuristics to decide how individual orders match. - **Single instance.** Only one strategy instance runs per backtest. - **Feature parity.** Not every production feature is supported yet. This is improving over time. The trade-offs are real, but the backtest is the easiest, fastest, risk-free environment for iterating on a strategy. --- ## Backtest configuration ### Required inputs A backtest is configured through a single JSON file. The file describes: **Strategy instance configuration** The same shape as in production: - Strategy class name to instantiate. - Instance name. - Instance parameters (strategy-specific). **Strategy artifact** - Path to the compiled JAR. **Market data files** Per instrument: - **Market data file.** The last 3 months are available for direct download. For older data, contact CryptoStruct support. - **Latency (microseconds).** One-way latency applied to order requests on the way to the exchange and to responses and market updates on the way back. - **Time-priority heuristic.** When the simulation can't decide exactly how a level update affects orders at the same level, this picks the policy: - `REMOVE_SUPERIOR`: prefer removing volume in front of your order (raises fill probability). - `REMOVE_INFERIOR`: prefer removing volume behind your order. - `REMOVE_EVENLY`: prefer removing volume evenly around your order. **Optional: master data files** - Underlyings and instruments. Recent market-data files include this data, so the explicit configuration is usually unnecessary. **Optional: initial account state** - Wallet balances. - Existing positions. **Optional: message-channel configuration** - Output directory for strategy messages. - List of input files containing messages to forward to the strategy. ### Configuration example ```json { "strategyFile": "./target/my-strategy.jar", "strategyConfig": { "instanceName": "instance123", "strategyName": "MyStrategy", "parameters": { "tradingInstrument": { "id": 22, "accountId": 15, "maxOrderCount": { "bid": 1, "ask": 1 }, "maxPosition": { "bid": "200", "ask": "200" } }, "dataInstrument": { "id": 67824 }, "myParameter1": "123.123", "myParameter2": 123 } }, "instruments": [ { "instrumentId": 22, "marketDataFile": "2022-08-17_instrument_22.zst", "latencyUs": 10000, "timePrioHeuristic": "REMOVE_SUPERIOR" }, { "instrumentId": 67824, "marketDataFile": "2022-08-17_instrument_67824.zst", "latencyUs": 10000, "timePrioHeuristic": "REMOVE_SUPERIOR" } ], "accounts": [ { "accountId": 15, "wallet": [ { "underlyingId": 100, "marginBalance": "0" }, { "underlyingId": 103, "marginBalance": "0" } ], "positions": [ { "instrumentId": 22, "position": "500" } ] } ], "messageChannels": { "outputDirectory": "output", "inputFiles": ["input1.txt", "input2.txt"] } } ``` **Field reference:** - `strategyFile`: path to the compiled strategy JAR. - `strategyConfig`: strategy instance configuration. - `instanceName`: unique identifier for this instance. - `strategyName`: name of the strategy class to instantiate. - `parameters`: strategy-specific configuration parameters. - `instruments`: instruments to simulate. - `instrumentId`: instrument identifier. - `marketDataFile`: path to the market-data file. - `latencyUs`: one-way latency in microseconds. - `timePrioHeuristic`: time-priority heuristic (see above). - `accounts`: account state. - `accountId`: account identifier. - `wallet`: wallet balances per underlying. - `positions`: existing positions per instrument. - `messageChannels`: message-channel configuration. - `outputDirectory`: directory for strategy message output. - `inputFiles`: message input files. --- ## Running a backtest ### Basic execution Once the configuration is ready, run the backtest tool with the config file: ```bash java -jar backtest.jar backtest-configuration.json ``` ### Command-line arguments In addition to the config file, the tool accepts: - `--eventlog `: write all order requests and trades to a file. - `--loglevel `: set the strategy's log level. - `--message-channel-directory `: write all data published on channels by the strategy to files. Example: ```bash java -jar backtest.jar backtest-configuration.json \ --eventlog trades.log \ --loglevel DEBUG \ --message-channel-directory ./output ``` --- ## Message input files A set of messages can be read from files and forwarded to the strategy during the backtest run. Useful for testing how the strategy reacts to external events or signals. ### File format - One message per line. - Files can be plain `.txt` or compressed as `.gz` or `.zst`. - Messages are forwarded only to channels opened with type `String`. Messages targeted at channels with other types are dropped. ### Message structure ```json { "time": "1970-01-01T00:00:00.000Z", "origin": "myOrigin", "channel": "myChannel", "body": "myBody" } ``` Fields: - `time`: ISO 8601 timestamp. This is the **delivery time** of the event. The strategy receives the message when backtest time reaches this timestamp. Backtest time is bounded by the time range of the referenced market-data files, so timestamps need to fall inside that range. - `origin`: the message origin. - `channel`: the channel to publish on. - `body`: the message content. A string, a JSON object, or an array. Objects and arrays are escaped and converted to a string. > [!NOTE] > Messages are delivered on backtest simulation time, not wall-clock > time. Make sure your timestamps line up with the data range. --- ## Script testing > [!WARNING] > **Script testing** is still under development. Features may change. **Script testing** uses the same market simulation as backtesting, but instead of replaying recorded data, every market update is driven by explicit script commands. You can also write **expectations** (e.g. on order operations) that the strategy must meet. ### Script file structure A script file has three sections: **`[masterdata]`** A JSON structure with all master data: instruments, underlyings, accounts. **`[strategies]`** A JSON structure with the strategy configuration. **`[script]`** The script itself, one command or expectation per line. ### Script commands Each command starts with a time advance, in milliseconds. Example: ``` +100 update TEST1 BID 10 99.9 ``` This advances simulation time by 100ms and updates instrument `TEST1` by setting the size of the bid level at price `99.9` to `10`. ### Expectations Expectations are checked before the next update fires. The strategy must meet the expectation within the specified window. Complete example: ``` +100 update TEST1 BID 10 99.9 expect order-place TEST1 BID 5 99.7 --name order1 +100 update TEST1 BID 15 99.9 ``` This script: 1. Updates instrument `TEST1` with a bid level. 2. Expects the strategy to place a bid order with size `5` at price `99.7` (referenced by name `order1`). 3. Verifies the expectation before the next update (the order has to land within 100ms of the first update). > [!NOTE] > Script testing simulates a fixed latency of 10ms between strategy and > exchange. Keep that in mind when writing very small time steps. ### Execution Two ways to run a script test. **Command-line tool:** ```bash java -jar scripttest.jar myscript.script ``` The tool also documents the available script commands and expectations. Run `java -jar scripttest.jar --help` to print them. **Unit test:** Use the `strategy-test` library to run script tests as unit tests in your build pipeline. --- ## Development and feature requests The backtest is used heavily by both customers and the internal team. Development is driven by feature requests from both sides. > [!TIP] > If something is missing from backtesting or script testing for your > use case, contact CryptoStruct support with the requirement.