ramseshk
a7f811eb81
New data/ module with: - data/store.py: Parquet-based raw message storage with background writer thread. Messages partitioned by channel/coin/date. Thread-safe queue. Supports pyarrow Parquet with zstd compression. Includes read_range() helper for replay. - data/collectors/hyperliquid.py: HL WebSocket + REST collector - WebSocket: l2Book (full book reconstruction), trades, allMids (mark prices) - REST pollers: funding rates, predicted funding, open interest, liquidations - Per-coin OrderBook class with snapshot/update reconstruction - Sequence gap detection with per-coin re-snapshot on gap - Latency tracking (exchange transport, signal, order, roundtrip) - Periodic stats reporter (book stats + latency summary every 60s) - CLI entrypoint: python -m data.collectors.hyperliquid --coins BTC ETH - data/normalizer.py: Timestamp normalization (ms, s, ISO strings from HL/Binance/Bybit/OKX/Coinbase/Deribit) + SequenceTracker with gap detection - data/latency.py: Rolling-window latency metrics (p50/p90/p95/p99) for transport, signal computation, order submission, and roundtrip - Added pyarrow + aiohttp to requirements.txt
27 lines
361 B
Plaintext
27 lines
361 B
Plaintext
# Core
|
|
nautilus-trader>=1.210.0
|
|
numpy>=1.24.0
|
|
pandas>=2.0.0
|
|
pyyaml>=6.0
|
|
requests>=2.28.0
|
|
|
|
# Framework
|
|
vectorbt>=1.0.0
|
|
hyperliquid-python-sdk>=0.20.0
|
|
websockets>=12.0
|
|
|
|
# Dashboard
|
|
fastapi>=0.109.0
|
|
uvicorn[standard]>=0.27.0
|
|
|
|
# Data storage
|
|
pyarrow>=12.0.0
|
|
aiohttp>=3.9.0
|
|
|
|
# Visualization
|
|
matplotlib>=3.7.0
|
|
seaborn>=0.12.0
|
|
|
|
# Optional: dashboard
|
|
streamlit>=1.28.0
|