Browse Source

Add Docker multi-service setup, README, and externalize DB password

- Extract DB_PASSWORD into .env variable, update .env.example and
  DATABASE_URL to use interpolation via dotenv-expand
- Expand docker-compose.yml with bot and dashboard services
- Add multi-stage Dockerfile and .dockerignore
- Add project README.md with setup and usage docs
- Add dotenv-expand to dashboard for ${VAR} support in next.config.mjs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Horace Zhang 7 months ago
parent
commit
cb77e1654a

+ 9 - 0
.dockerignore

@@ -0,0 +1,9 @@
+node_modules
+dist
+.next
+.turbo
+.env
+.env.local
+.git
+.claude
+*.md

+ 2 - 1
.env.example

@@ -1,5 +1,6 @@
 # === Database ===
-DATABASE_URL=postgresql://bot:botpass@localhost:5432/weather_bot
+DB_PASSWORD=botpass
+DATABASE_URL=postgresql://bot:${DB_PASSWORD}@localhost:5432/weather_bot
 REDIS_URL=redis://localhost:6379
 
 # === Polymarket Wallet ===

+ 39 - 0
Dockerfile

@@ -0,0 +1,39 @@
+FROM node:20-alpine AS base
+RUN corepack enable && corepack prepare pnpm@9.15.4 --activate
+WORKDIR /app
+
+# --- Install dependencies ---
+FROM base AS deps
+COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
+COPY packages/core/package.json packages/core/
+COPY packages/weather-engine/package.json packages/weather-engine/
+COPY packages/market-engine/package.json packages/market-engine/
+COPY packages/strategy/package.json packages/strategy/
+COPY packages/trading-engine/package.json packages/trading-engine/
+COPY packages/dashboard/package.json packages/dashboard/
+RUN pnpm install --frozen-lockfile
+
+# --- Copy source and generate Prisma client ---
+FROM deps AS source
+COPY . .
+RUN pnpm db:generate
+
+# --- Bot: build TS packages (no DB needed) ---
+FROM source AS bot-build
+RUN pnpm --filter @poly-weather/core build && \
+    pnpm --filter @poly-weather/weather-engine build && \
+    pnpm --filter @poly-weather/market-engine build && \
+    pnpm --filter @poly-weather/strategy build && \
+    pnpm --filter @poly-weather/trading-engine build
+
+# --- Bot runtime ---
+FROM bot-build AS bot
+ENV NODE_ENV=production
+EXPOSE 3001
+CMD ["node", "packages/trading-engine/dist/main.js"]
+
+# --- Dashboard runtime (builds at startup since it needs DB access) ---
+FROM source AS dashboard
+ENV NODE_ENV=production
+EXPOSE 3000
+CMD ["sh", "-c", "pnpm --filter @poly-weather/core build && pnpm --filter @poly-weather/dashboard build && pnpm --filter @poly-weather/dashboard start"]

+ 163 - 0
README.md

@@ -0,0 +1,163 @@
+# Poly Weather
+
+Polymarket 天气预测市场套利机器人。通过比较专业气象模型(GFS / ECMWF / ICON 集合预报)的概率与 Polymarket 每日温度市场的隐含概率,自动发现并交易定价偏差。
+
+## 策略
+
+| 策略 | 说明 |
+|------|------|
+| **TAIL_NO** | 在尾部区间买入 NO — 当模型概率 < 5% 但市场 YES 价格 > 3¢ 时 |
+| **LATENCY_ARB** | 新气象数据使概率偏移 > 5%,市场尚未重新定价时交易 |
+| **PARITY_ARB** | 当某事件所有 YES 价格之和偏离 100¢ 时的纯套利 |
+
+## 架构
+
+TypeScript monorepo,使用 Turborepo + pnpm workspaces。
+
+```
+packages/
+  core/             — 共享类型、工具函数、Prisma schema、常量
+  weather-engine/   — GFS/ECMWF/ICON 预报获取(Open-Meteo)、集合概率计算
+  market-engine/    — Polymarket Gamma API 扫描、CLOB 订单簿读取、认证下单
+  strategy/         — 信号生成(Tail NO / Latency Arb / Parity Arb)、风控、Kelly 仓位
+  trading-engine/   — 机器人主循环、流水线编排、健康检查服务
+  dashboard/        — Next.js App Router 仪表盘,实时市场数据、信号、手动下单
+```
+
+### 依赖关系
+
+```
+core (叶子节点 — 无内部依赖)
+  ↑
+weather-engine, market-engine (依赖 core)
+  ↑
+strategy (依赖 core + weather-engine + market-engine)
+  ↑
+trading-engine (依赖以上所有)
+dashboard (依赖 core,直接读取数据库)
+```
+
+## 快速开始
+
+### 前置条件
+
+- Node.js >= 20
+- pnpm >= 9
+- Docker & Docker Compose
+
+### 1. 克隆并安装
+
+```bash
+git clone <repo-url> && cd poly-weather
+pnpm install
+```
+
+### 2. 配置环境变量
+
+```bash
+cp .env.example .env
+# 编辑 .env,填入你的 Polymarket 钱包私钥和地址
+```
+
+### 3. 启动基础服务
+
+```bash
+# 仅启动 PostgreSQL + Redis(本地开发)
+docker compose up -d postgres redis
+
+# 或者启动全部服务(包括 bot + dashboard)
+docker compose up -d
+```
+
+### 4. 初始化数据库
+
+```bash
+pnpm db:generate
+pnpm db:push
+```
+
+### 5. 派生 L2 API 密钥
+
+```bash
+pnpm derive-keys
+# 将输出的 key/secret/passphrase 填入 .env
+```
+
+### 6. 启动
+
+```bash
+# 方式一:Docker 全栈启动
+docker compose up -d
+
+# 方式二:本地开发
+pnpm i
+pnpm build
+pnpm bot:start        # 启动交易机器人
+pnpm dashboard:dev    # 启动仪表盘 http://localhost:3000
+```
+
+## Docker 服务
+
+| 服务 | 端口 | 说明 |
+|------|------|------|
+| `postgres` | 5432 | PostgreSQL 16 数据库 |
+| `redis` | 6379 | Redis 7 缓存 |
+| `bot` | 3001 | 交易机器人(健康检查端口) |
+| `dashboard` | 3000 | Next.js 仪表盘 |
+
+```bash
+docker compose up -d                # 启动全部
+docker compose up -d postgres redis # 仅基础设施
+docker compose logs -f bot          # 查看 bot 日志
+docker compose down                 # 停止全部
+```
+
+## 常用命令
+
+```bash
+pnpm install              # 安装所有依赖
+pnpm build                # 构建所有包
+pnpm dev                  # 开发模式(全部包)
+pnpm test                 # 运行测试
+pnpm db:generate          # 生成 Prisma Client
+pnpm db:push              # 推送 schema 到数据库
+pnpm db:studio            # 打开 Prisma Studio
+pnpm bot:start            # 启动交易机器人
+pnpm dashboard:dev        # 启动仪表盘开发服务器
+pnpm derive-keys          # 从 PRIVATE_KEY 派生 L2 API 密钥
+```
+
+## 技术栈
+
+- **运行时**: Node.js 20+, TypeScript (ES2022, NodeNext, strict)
+- **Monorepo**: pnpm workspaces + Turborepo
+- **数据库**: PostgreSQL 16 (Prisma ORM) + Redis 7
+- **仪表盘**: Next.js 14 (App Router), Tailwind CSS, next-themes, Radix UI
+- **市场 API**: Polymarket Gamma API + CLOB API
+- **气象 API**: Open-Meteo (GFS, ECMWF, ICON)
+- **交易**: @polymarket/clob-client, ethers
+
+## 环境变量
+
+参见 `.env.example`,关键变量:
+
+| 变量 | 说明 |
+|------|------|
+| `DB_PASSWORD` | PostgreSQL 密码 |
+| `DATABASE_URL` | PostgreSQL 连接串(引用 `${DB_PASSWORD}`) |
+| `REDIS_URL` | Redis 连接地址 |
+| `PRIVATE_KEY` | Polymarket 钱包私钥 |
+| `FUNDER_ADDRESS` | Polymarket 代理钱包地址 |
+| `DRY_RUN` | `true`(默认)模拟交易,`false` 真实下单 |
+| `AUTO_EXECUTE` | `false`(默认)仅检测信号,不自动下单 |
+
+## 注意事项
+
+- **DRY_RUN=true 为默认值** — 所有信号检测和数据库记录正常工作,但不会真正下单
+- **温度单位** — 美国城市使用 °F,其他城市使用 °C
+- **仅 Maker 订单** — 在 Polymarket 上使用限价单(GTC)以免手续费
+- **时间戳均为 UTC** — 本地时区仅在仪表盘显示中使用
+
+## License
+
+Private

+ 37 - 4
docker-compose.yml

@@ -1,17 +1,18 @@
 version: '3.8'
+
 services:
   postgres:
     image: postgres:16-alpine
     environment:
-      POSTGRES_DB: weather_bot
-      POSTGRES_USER: bot
-      POSTGRES_PASSWORD: botpass
+      POSTGRES_DB: polybot
+      POSTGRES_USER: polybot
+      POSTGRES_PASSWORD: ${DB_PASSWORD:-botpass}
     volumes:
       - pgdata:/var/lib/postgresql/data
     ports:
       - '5432:5432'
     healthcheck:
-      test: ['CMD-SHELL', 'pg_isready -U bot -d weather_bot']
+      test: ['CMD-SHELL', 'pg_isready -U polybot -d polybot']
       interval: 10s
       timeout: 5s
       retries: 5
@@ -28,6 +29,38 @@ services:
       timeout: 5s
       retries: 5
 
+  bot:
+    build:
+      context: .
+      target: bot
+    env_file: .env
+    environment:
+      DATABASE_URL: postgresql://polybot:${DB_PASSWORD:-botpass}@postgres:5432/polybot
+      REDIS_URL: redis://redis:6379
+    ports:
+      - '${BOT_HEALTH_PORT:-3001}:3001'
+    depends_on:
+      postgres:
+        condition: service_healthy
+      redis:
+        condition: service_healthy
+    restart: unless-stopped
+
+  dashboard:
+    build:
+      context: .
+      target: dashboard
+    env_file: .env
+    environment:
+      DATABASE_URL: postgresql://polybot:${DB_PASSWORD:-botpass}@postgres:5432/polybot
+      REDIS_URL: redis://redis:6379
+    ports:
+      - '${DASHBOARD_PORT:-3000}:3000'
+    depends_on:
+      postgres:
+        condition: service_healthy
+    restart: unless-stopped
+
 volumes:
   pgdata:
   redisdata:

+ 3 - 1
packages/dashboard/next.config.mjs

@@ -1,11 +1,13 @@
 import { config } from 'dotenv';
+import { expand } from 'dotenv-expand';
 import { resolve, dirname } from 'path';
 import { fileURLToPath } from 'url';
 
 const __dirname = dirname(fileURLToPath(import.meta.url));
 
 // Load root .env so all packages share the same env vars (override any existing)
-config({ path: resolve(__dirname, '../../.env'), override: true });
+// Use dotenv-expand to support ${VAR} references (e.g. DATABASE_URL referencing DB_PASSWORD)
+expand(config({ path: resolve(__dirname, '../../.env'), override: true }));
 
 /** @type {import('next').NextConfig} */
 const nextConfig = {

+ 1 - 0
packages/dashboard/package.json

@@ -20,6 +20,7 @@
     "class-variance-authority": "^0.7.1",
     "clsx": "^2.1.1",
     "dotenv": "^17.3.1",
+    "dotenv-expand": "^12.0.3",
     "lucide-react": "^0.577.0",
     "next": "^14.2.0",
     "next-themes": "^0.4.6",

+ 13 - 12
packages/dashboard/src/lib/queries.ts

@@ -46,8 +46,8 @@ export async function getDashboardSummary(): Promise<DashboardSummary> {
     }),
   ]);
 
-  const todayPnl = todayTrades.reduce((sum, t) => sum + (t.pnl ?? 0), 0);
-  const wins = last30Trades.filter((t) => (t.pnl ?? 0) > 0).length;
+  const todayPnl = todayTrades.reduce((sum: number, t: { pnl: number | null }) => sum + (t.pnl ?? 0), 0);
+  const wins = last30Trades.filter((t: { pnl: number | null }) => (t.pnl ?? 0) > 0).length;
   const winRate = last30Trades.length > 0 ? wins / last30Trades.length : 0;
 
   return {
@@ -220,7 +220,7 @@ export async function getRecentTrades(limit = 50): Promise<TradeRecord[]> {
     },
   });
 
-  return rows.map((t) => {
+  return rows.map((t: typeof rows[number]) => {
     const sig = t.signals[0];
     return {
       id: t.id,
@@ -261,7 +261,7 @@ export async function getMarkets(): Promise<WeatherMarket[]> {
     }
     // Only keep latest per bucket
     const existing = grouped.get(key)!;
-    const existingBucket = existing.find((s) => s.bucketLabel === snap.bucketLabel);
+    const existingBucket = existing.find((s: typeof snap) => s.bucketLabel === snap.bucketLabel);
     if (!existingBucket) {
       existing.push(snap);
     }
@@ -274,9 +274,9 @@ export async function getMarkets(): Promise<WeatherMarket[]> {
   for (const [key, snaps] of grouped) {
     const [city, date] = key.split('|');
     const tempUnit = ['New York', 'Chicago', 'Miami', 'Houston', 'Denver', 'Los Angeles', 'Phoenix', 'Seattle'].includes(city) ? 'F' : 'C';
-    const totalVolume = snaps.reduce((s, b) => s + b.volume24h, 0);
+    const totalVolume = snaps.reduce((s: number, b: typeof snapshots[number]) => s + b.volume24h, 0);
 
-    const buckets: MarketBucket[] = snaps.map((s) => ({
+    const buckets: MarketBucket[] = snaps.map((s: typeof snapshots[number]) => ({
       label: s.bucketLabel,
       modelProb: 0, // Will be filled when forecasts are available
       marketPrice: s.yesPrice,
@@ -306,7 +306,8 @@ export async function getPnLData(): Promise<{ data: PnLDataPoint[]; metrics: PnL
 
   // Build cumulative PnL series
   let cumulative = 0;
-  const data: PnLDataPoint[] = dailyMetrics.map((dm) => {
+  type DM = typeof dailyMetrics[number];
+  const data: PnLDataPoint[] = dailyMetrics.map((dm: DM) => {
     cumulative += dm.netPnl;
     return {
       date: dm.date.toISOString().slice(0, 10),
@@ -316,16 +317,16 @@ export async function getPnLData(): Promise<{ data: PnLDataPoint[]; metrics: PnL
   });
 
   // Aggregate metrics
-  const totalTrades = dailyMetrics.reduce((s, d) => s + d.totalTrades, 0);
-  const winningTrades = dailyMetrics.reduce((s, d) => s + d.winningTrades, 0);
+  const totalTrades = dailyMetrics.reduce((s: number, d: DM) => s + d.totalTrades, 0);
+  const winningTrades = dailyMetrics.reduce((s: number, d: DM) => s + d.winningTrades, 0);
   const totalPnl = cumulative;
   const winRate = totalTrades > 0 ? winningTrades / totalTrades : 0;
 
   // Sharpe ratio (annualized from daily)
-  const dailyReturns = dailyMetrics.map((d) => d.netPnl);
-  const avgReturn = dailyReturns.length > 0 ? dailyReturns.reduce((s, r) => s + r, 0) / dailyReturns.length : 0;
+  const dailyReturns = dailyMetrics.map((d: DM) => d.netPnl);
+  const avgReturn = dailyReturns.length > 0 ? dailyReturns.reduce((s: number, r: number) => s + r, 0) / dailyReturns.length : 0;
   const variance = dailyReturns.length > 1
-    ? dailyReturns.reduce((s, r) => s + (r - avgReturn) ** 2, 0) / (dailyReturns.length - 1)
+    ? dailyReturns.reduce((s: number, r: number) => s + (r - avgReturn) ** 2, 0) / (dailyReturns.length - 1)
     : 0;
   const stdDev = Math.sqrt(variance);
   const sharpe = stdDev > 0 ? (avgReturn / stdDev) * Math.sqrt(252) : 0;

+ 3 - 0
pnpm-lock.yaml

@@ -78,6 +78,9 @@ importers:
       dotenv:
         specifier: ^17.3.1
         version: 17.3.1
+      dotenv-expand:
+        specifier: ^12.0.3
+        version: 12.0.3
       lucide-react:
         specifier: ^0.577.0
         version: 0.577.0(react@18.3.1)