6 Commits

Author SHA1 Message Date
ABAPPLO
9697b5890d feat: 任务发布到企微客户群(企业群发)与资源库单条新增
- 任务中心新增「发布到企微群」:同步客户群清单(wecom_group_chats)、
  按群主分组创建企业群发任务(add_msg_template)、发送记录落库
  wecom_group_pushes,迁移 mysql/0009;lib/wecom-client.ts 补
  listCustomerGroupChats/createGroupMsgTemplate
- KOC 资源库支持单条新增:app/api/resources-insert + lib/resource-write,
  拆出资源写入公共逻辑供导入复用;0008 补合作方外部联系人字段
- 环境变量示例补企微凭证与 SEED_DEMO_DATA;next.config 增加
  allowedDevOrigins;CLAUDE.md 补充项目说明
- .gitignore 排除 .codegraph/ 与 .ipynb_checkpoints/
2026-08-20 15:16:18 +08:00
8f7ea0558d Merge pull request 'feat: 接入企业微信通知(临期催办 + 群机器人汇总)' (#3) from feat/account-tags into main
Reviewed-on: #3
2026-08-18 09:12:59 +00:00
ABAPPLO
f2ac751c4c feat: 接入企业微信通知(临期催办 + 群机器人汇总)
新增 lib/wecom-client.ts 与 lib/wecom-notifier-service.ts,支持
群机器人 webhook 与应用消息双通道;scheduler 加入临期 N 天催办
与每日管理员汇总;action 路由补 bind_wecom_external_id 与
send_test_wecom 两个管理端动作,配套测试。
2026-08-18 17:05:16 +08:00
巫凤萍
cac6c5e83b fix: 修复回填更新与截图刷新 2026-08-16 22:22:50 +08:00
巫凤萍
74671a9b9f docs: 更新 main 私有化部署指南 2026-08-15 03:57:50 +08:00
巫凤萍
f37d05dd88 feat: 完善视频任务与 KOC 资源库 2026-08-15 03:53:09 +08:00
85 changed files with 10939 additions and 844 deletions

View File

@@ -11,3 +11,14 @@ AI_TOOL_CENTER_MCP_KEY=replace-with-mcp-key
# Feishu custom app credentials. Keep the secret out of source control.
FEISHU_APP_ID=cli_xxxxxxxxxxxxxxxxx
FEISHU_APP_SECRET=replace-with-feishu-app-secret
# 企业微信 · 全部选填;不填则临期催办与汇总都跳过,不影响其他功能
WECOM_CORP_ID=
WECOM_AGENT_ID=
WECOM_SECRET=
WECOM_ROBOT_WEBHOOK=
WECOM_NOTIFY_DUE_DAYS=3
WECOM_NOTIFY_ENABLED=true
# 本地测试用:让 /api/bootstrap 自动 seed 测试 partner + distribution上线设为 false
SEED_DEMO_DATA=false

View File

@@ -24,6 +24,15 @@ FEISHU_APP_SECRET=
AI_TOOL_CENTER_MCP_URL=
AI_TOOL_CENTER_MCP_KEY=
# 企业微信通知(临期催办 + 管理员汇总)。群机器人只需 webhookKOC 侧催办还需 corp/agent/secret
# 并在后台 partners 编辑里把 wecom_external_user_id 填好。
WECOM_ROBOT_WEBHOOK=
WECOM_CORP_ID=
WECOM_AGENT_ID=
WECOM_SECRET=
WECOM_NOTIFY_DUE_DAYS=3
WECOM_NOTIFY_ENABLED=true
# 每天北京时间 09:00 自动执行采集计划。
ENABLE_SCHEDULER=true
SEED_DEMO_DATA=false

3
.gitignore vendored
View File

@@ -22,6 +22,8 @@
# misc
.DS_Store
*.pem
.codegraph/
.ipynb_checkpoints/
# debug
npm-debug.log*
@@ -39,6 +41,7 @@ yarn-error.log*
# typescript
next-env.d.ts
*.tsbuildinfo
/dist/
/.wrangler/
/outputs/

132
CLAUDE.md Normal file
View File

@@ -0,0 +1,132 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## What This Project Does
KOC LOOP is a content distribution & data collection platform for KOC (Key Opinion Consumer) operations teams. It runs on **vinext** (Vite-based Next.js 16 on Cloudflare Workers + Pages) with Cloudflare D1 (SQLite) and R2 storage.
The platform manages: task creation from Feishu (飞书) spreadsheets, content distribution to KOC partners, publishing to Xiaohongshu (小红书), and automated data collection (likes/comments/collects) via MCP-based metrics scraping.
## Two Applications
### 1. Admin App (`app/`) — Main dashboard
- Server-rendered admin UI at root (`/`), protected by ChatGPT Sign-In + admin email check
- Heavy client-side SPA in `app/admin-app.tsx` (a single ~59KB component with all dashboard state/views)
- API routes under `app/api/` for CRUD, Feishu import, data collection, image upload
### 2. KOC Portal (`koc-portal/`) — External task portal
- Independent Next.js app (separate `package.json`) for external KOC collaborators
- KOCs can claim tasks, view assigned content, and submit publish URLs/screenshots
- Does NOT connect directly to the database — uses the admin app's API
## Tech Stack
- **Framework**: Next.js 16 + React 19 + TypeScript
- **Build/Runtime**: vinext 0.0.50 (Vite 8 plugin → Cloudflare Workers/Pages)
- **Database**: Cloudflare D1 (SQLite via Drizzle ORM 0.45)
- **Storage**: Cloudflare R2 for uploads/screenshots
- **Styling**: Tailwind CSS 4
- **Cron**: Cloudflare Workers cron (daily 02:00 UTC / 10:00 CST)
## Database Schema (7 tables in `db/schema.ts`)
| Table | Purpose |
|-------|---------|
| `partners` | KOC partners/groups (name, owner, stats) |
| `tasks` | Campaign tasks (brand, quantity, due date, Feishu source) |
| `contents` | Content items (title, body, images, linked to task) |
| `accounts` | Xiaohongshu accounts scraped from publish links |
| `claims` | Partner claims on task content |
| `delegation_bundles` | Delegation bundles with share tokens |
| `distributions` | Content-to-partner assignments (publish URL, metrics, collection status) |
| `collection_runs` | Scheduled metrics collection run log |
Schema is maintained both via Drizzle (`db/schema.ts`) and imperative migrations in `lib/mvp-db.ts:ensureSchema()`. The imperative path is the source of truth for production — Drizzle migrations are optional.
## Key Libraries
- `lib/mvp-db.ts` — Raw D1 helpers, schema bootstrapping, seed data, `getDashboardData()`
- `lib/collection-service.ts` — Scheduled metrics collection orchestration
- `lib/mcp-collection-client.ts` — MCP-based client for scraping Xiaohongshu public metrics
- `lib/feishu-client.ts` — Feishu API client (spreadsheet/wiki/doc reading)
- `lib/account-enrichment-service.ts` — Backfill account profiles from Xiaohongshu
- `lib/admin-auth.ts` — Admin email/token authentication
- `lib/partner-utils.ts` / `lib/publish-url.ts` — URL extraction helpers
- `lib/date-utils.ts` — Date formatting (Shanghai timezone)
## Worker (`worker/index.ts`)
The Cloudflare Worker entry point handles:
1. `fetch` — Image optimization proxy at `/_vinext/image`, delegates everything else to vinext app router
2. `scheduled` — Daily cron: ensures DB schema, runs scheduled collections, backfills account profiles
> Note: 在私有化部署Node + MySQL里实际调度走 `lib/scheduler.ts` 的 `node-cron`,每天 09:00 Asia/Shanghai 跑同一套 `runDailyJob`(采集 + 账号资料补全 + 企业微信催办。Cloudflare Worker 入口仅用于原线上版本。
## API Routes (`app/api/`)
- `action/route.ts` — Central admin action endpoint: create task from Feishu, dashboard data, collect metrics, batch operations, account backfill, 企业微信测试 / 状态查询 / 绑定外部联系人 / 客户群同步与企业群发(半自动,群主确认后送达)
- `partner/route.ts` — Partner-facing API (claim, delegation, distribution)
- `bootstrap/route.ts` — Seed database with demo data
- `upload/route.ts` — Generic file upload to R2
- `partner-upload/route.ts` — Partner screenshot upload (CORS-enabled)
- `partner-image/route.ts` — Partner image serving (CORS-enabled)
- `content-image-upload/route.ts` — Content image upload
- `creator-screenshot/route.ts` — Creator screenshot upload
- `resources-import/route.ts` — 批量导入 KOC 账号资源Excel/CSV
- `resources-insert/route.ts` — 单条新增 KOC 账号资源(表单)
## Environment Variables (`.dev.vars`)
```
KOC_PORTAL_URL — URL of the koc-portal app
ADMIN_ALLOWED_EMAIL — ChatGPT email allowed for admin access
ADMIN_INTERNAL_TOKEN — Shared secret for API-to-API auth
AI_TOOL_CENTER_MCP_URL — MCP endpoint for XHS data collection
AI_TOOL_CENTER_MCP_KEY — MCP API key
FEISHU_APP_ID — Feishu custom app credentials
FEISHU_APP_SECRET — Feishu app secret
WECOM_CORP_ID — 企业微信企业 ID自建应用消息推送用可选
WECOM_AGENT_ID — 企业微信自建应用 AgentId可选
WECOM_SECRET — 企业微信自建应用 Secret可选
WECOM_ROBOT_WEBHOOK — 企业微信群机器人 webhook当日催办汇总可选
WECOM_NOTIFY_DUE_DAYS — 临期阈值,默认 3取值 130可选
WECOM_NOTIFY_ENABLED — 企业微信通知总开关,默认 true可选
```
## Commands
```bash
# Admin app (root of repo)
npm run dev # Start local dev server
npm run build # Build for production
npm test # Build + run integration tests
npm run lint # ESLint check
npm run db:generate # Generate Drizzle SQL migration
# KOC Portal (koc-portal/)
cd koc-portal && npm run dev -- --port 3000 # Start on port 3000
cd koc-portal && npm run build
cd koc-portal && npm test
# Tests use `node --test` (Node built-in test runner), live in tests/
node --test tests/date-utils.test.mjs # Run a single test
```
## Build System
- `vinext` replaces the Next.js build pipeline with Vite
- `@cloudflare/vite-plugin` provides D1/R2/cron local bindings
- Vite config at root resolves bindings from `.openai/hosting.json`
- Custom `build/sites-vite-plugin.ts` copies `.openai/` and `drizzle/` into `dist/` for deployment
- The `koc-portal/` subdirectory has its own identical build setup (separate `vite.config.ts`, `build/sites-vite-plugin.ts`)
<!-- BEGIN:nextjs-agent-rules -->
# This is NOT the Next.js you know
This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in `node_modules/next/dist/docs/` (resolved from this file's directory; in monorepos the `next` package may not be visible from the repo root) before writing any code. Heed deprecation notices.
This block is written and re-added by `next dev` — verify at `node_modules/next/dist/server/lib/generate-agent-files.js`. Removing it from a diff only re-creates the uncommitted change; committing it with your work keeps the tree clean.
<!-- END:nextjs-agent-rules -->

View File

@@ -1,6 +1,6 @@
# KOC LOOP
KOC 内容分发与数据回收闭环。当前私有化分支运行于标准 Next.js Node.js、MySQL 8、Nginx 和本地持久化文件存储。
KOC 内容分发与数据回收闭环。`main` 分支运行于标准 Next.js Node.js、MySQL 8、Nginx 和本地持久化文件存储。
## Prerequisites
@@ -45,13 +45,21 @@ npm run build
## KOC 资源导入
超级管理员和管理员可在“KOC资源”页面下载标准 Excel 模板,批量导入已有资源。模板只需填写小红书账号主页,合作来源可选填;上传后系统自动解析账号名称、小红书号、IP属地粉丝数。
超级管理员和管理员可在“KOC资源”页面下载标准 Excel 模板,批量导入已有资源。模板只需填写账号主页;账号名称、账号 ID、IP 属地粉丝数、性别、简介、标签和合作来源均可选填。多个标签使用逗号分隔,每个账号最多 5 个标签
- 单次最多导入 100 个账号,支持 `.xlsx``.csv`,文件不超过 5MB。
- 当前自动解析支持小红书账号主页;上传后先展示新增、更新和异常数据,存在异常时不会写入数据库
- 单次最多导入 10,000 个账号,支持 `.xlsx``.csv`,文件不超过 20MB。
- 当前自动解析支持小红书账号主页;上传后先展示新增、更新和异常数据。异常行会跳过,其余有效账号可以正常导入
- 按“平台 + 账号主页”去重;解析出相同小红书号时也会更新已有账号。
- 重复账号更新公开资料和合作来源,不产生两份资源。
- 导入的合作来源会进入现有资源搜索、筛选和导出结果
- 大批量导入会先写入资源库,再在后台逐步补全缺失的公开资料
- KOC 使用手机号或微信号领取任务后,系统会把该值写入“当前联系人”;原“合作来源”继续保留渠道信息。
- 导入的标签、当前联系人和合作来源会进入资源搜索或导出结果。
## KOC 批量回填 Excel
KOC 领取端支持导出和上传批量回填表。视频任务只生成“序号、标题、笔记内容、视频、发布链接、笔记截图、数据分析截图”列,不生成“图片”列。视频链接通过当前公网域名生成,下载接口返回可播放的 `.mp4` 附件。
反向代理部署必须正确传递 `Host``X-Forwarded-Host``X-Forwarded-Proto`,并把 `APP_ORIGIN` 配置为实际公网地址;不要填写 `localhost` 或容器内部地址。
## Agent MCP

File diff suppressed because it is too large Load Diff

View File

@@ -2,7 +2,10 @@ import { getRuntimeEnv } from "../../../lib/runtime-env";
import { runInBackground } from "../../../lib/background";
const env = getRuntimeEnv();
import { backfillAccountProfiles } from "../../../lib/account-enrichment-service";
import {
backfillAccountProfiles,
enrichDistributionAccount,
} from "../../../lib/account-enrichment-service";
import {
ensureSchema,
getDashboardData,
@@ -34,7 +37,23 @@ import {
DistributionReleaseError,
releaseUnfinishedDistribution,
} from "../../../lib/distribution-release-service";
import {
resolveWecomConfig,
sendWecomAppMessage,
sendWecomRobotMessage,
listExternalContacts,
WecomClientError,
type WecomBindings,
} from "../../../lib/wecom-client";
import { runDueSoonWecomNotifications } from "../../../lib/wecom-notifier-service";
import {
listGroupChatRows,
listGroupPushes,
pushTaskToGroupChats,
syncGroupChats,
} from "../../../lib/wecom-group-push-service";
import { isManagerRequest } from "../../../lib/user-auth";
import { extractPublishUrl } from "../../../lib/publish-url";
type ActionBody = {
action?: string;
@@ -63,6 +82,14 @@ export async function POST(request: Request) {
sheetName: source.sheetName,
syncedAt: source.syncedAt,
rowCount: source.rows.length,
imageCount: source.rows.reduce(
(total, row) => total + row.images.length,
0,
),
videoCount: source.rows.reduce(
(total, row) => total + row.videos.length,
0,
),
columns: source.columns,
preview: source.rows.slice(0, 3),
});
@@ -72,6 +99,8 @@ export async function POST(request: Request) {
const name = String(body.name ?? "").trim();
const brand = String(body.brand ?? "").trim();
const dueAt = String(body.dueAt ?? "").trim();
const platform = body.platform === "抖音" ? "抖音" : "小红书";
const contentFormat = body.contentFormat === "video" ? "video" : "image_text";
if (!name || !brand || !dueAt) {
return Response.json(
{ error: "请补全任务名称、品牌和截止日期" },
@@ -84,6 +113,8 @@ export async function POST(request: Request) {
name,
brand,
dueAt,
platform,
contentFormat,
},
env as unknown as FeishuBindings,
);
@@ -146,6 +177,159 @@ export async function POST(request: Request) {
db,
String(body.distributionId ?? "").trim(),
);
} else if (body.action === "update_distribution_publish_url") {
if (!(await isManagerRequest(request))) return adminForbidden();
const distributionId = String(body.distributionId ?? "").trim();
if (!distributionId) {
return Response.json(
{ error: "作品记录不存在" },
{ status: 400 },
);
}
const current = await db
.prepare(
`SELECT d.id, d.task_id, d.partner_id, d.publish_url,
t.task_type, t.platform, t.collection_start_date, t.collection_days,
COALESCE(a.nickname, '待识别账号') AS account_nickname
FROM distributions d
JOIN tasks t ON t.id = d.task_id
LEFT JOIN accounts a ON a.id = d.account_id
WHERE d.id = ?`,
)
.bind(distributionId)
.first<{
id: string;
task_id: string;
partner_id: string;
publish_url: string | null;
task_type?: string | null;
platform: string;
collection_start_date: string | null;
collection_days: string;
account_nickname: string;
}>();
if (!current) {
return Response.json({ error: "作品记录不存在" }, { status: 404 });
}
if (current.task_type === "screenshot_collect") {
return Response.json(
{ error: "截图回收任务不需要填写发布链接" },
{ status: 400 },
);
}
const platform = current.platform === "抖音" ? "抖音" : "小红书";
const publishUrl = extractPublishUrl(
String(body.publishUrl ?? "").trim(),
platform,
);
if (!publishUrl) {
return Response.json(
{ error: `请填写包含${platform}作品链接的发布内容` },
{ status: 400 },
);
}
if (current.publish_url === publishUrl) {
return Response.json(await getDashboardData());
}
let collectionDays: number[] = [];
try {
const parsed = JSON.parse(current.collection_days || "[]");
if (Array.isArray(parsed)) {
collectionDays = [...new Set(parsed.map(Number))]
.filter(
(day) =>
Number.isInteger(day) && day >= 1 && day <= 7,
)
.sort((a, b) => a - b);
}
} catch {
collectionDays = [];
}
const isScheduled = Boolean(
current.collection_start_date && collectionDays.length > 0,
);
const statements = [
db
.prepare(
`UPDATE distributions SET
publish_url = ?,
publish_time = CURRENT_TIMESTAMP,
status = 'published',
d2_likes = NULL,
d2_comments = NULL,
d2_collects = NULL,
d5_likes = NULL,
d5_comments = NULL,
d5_collects = NULL,
d7_likes = NULL,
d7_comments = NULL,
d7_collects = NULL,
latest_likes = NULL,
latest_comments = NULL,
latest_collects = NULL,
latest_shares = NULL,
collection_status = ?,
collection_status_description = ?,
collection_updated_at = NULL,
last_collection_day = NULL,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?`,
)
.bind(
publishUrl,
isScheduled ? "scheduled" : "pending",
isScheduled
? `管理员已更新链接,等待${collectionDays.length}个采集日`
: "管理员已更新链接,等待设置采集计划",
distributionId,
),
db
.prepare("DELETE FROM collection_runs WHERE distribution_id = ?")
.bind(distributionId),
];
if (!current.publish_url) {
statements.push(
db
.prepare(
`UPDATE partners SET completed_total = completed_total + 1
WHERE id = ?`,
)
.bind(current.partner_id),
);
}
await db.batch(statements);
if (isScheduled && current.collection_start_date) {
await createCollectionRunTasks(
db,
current.task_id,
current.collection_start_date,
collectionDays,
);
runInBackground(
runDueScheduledCollections(
db,
Date.now(),
resolveCollectionMcpConfig(
env as unknown as CollectionMcpBindings,
),
"catchup",
current.task_id,
).catch(() => undefined),
"collection catchup after publish URL update",
);
}
runInBackground(
enrichDistributionAccount(
db,
distributionId,
publishUrl,
current.account_nickname,
resolveCollectionMcpConfig(
env as unknown as CollectionMcpBindings,
),
).catch(() => undefined),
"account enrichment after publish URL update",
);
} else if (body.action === "save_collection_schedule") {
const taskId = String(body.taskId ?? "").trim();
const startDate = String(body.startDate ?? "").trim();
@@ -365,6 +549,143 @@ export async function POST(request: Request) {
)
.bind(exposure, views, distributionId)
.run();
} else if (body.action === "bind_wecom_external_id") {
const partnerId = String(body.partnerId ?? "").trim().slice(0, 80);
const externalId = String(body.wecomExternalUserId ?? "")
.trim()
.slice(0, 128);
const wecomName = String(body.wecomName ?? "").trim().slice(0, 120);
if (!partnerId) {
return Response.json(
{ error: "缺少 partnerId" },
{ status: 400 },
);
}
if (wecomName) {
await db
.prepare(
"UPDATE partners SET wecom_external_user_id = ?, wecom_name = ? WHERE id = ?",
)
.bind(externalId || null, wecomName, partnerId)
.run();
} else {
await db
.prepare(
"UPDATE partners SET wecom_external_user_id = ? WHERE id = ?",
)
.bind(externalId || null, partnerId)
.run();
}
return Response.json({
partnerId,
wecomExternalUserId: externalId || null,
wecomName,
});
} else if (body.action === "send_test_wecom") {
const wecomConfig = resolveWecomConfig(
env as unknown as WecomBindings,
);
const partnerId = String(body.partnerId ?? "").trim();
let partnerExternalId: string | null = null;
if (partnerId) {
const row = await db
.prepare(
"SELECT wecom_external_user_id FROM partners WHERE id = ?",
)
.bind(partnerId)
.first<{ wecom_external_user_id: string | null }>();
partnerExternalId = row?.wecom_external_user_id ?? null;
}
const testContent = `[KOC LOOP 测试] 群机器人连通性正常,时间 ${new Date().toISOString()}`;
let robotStatus: "ok" | "skipped" = "skipped";
if (wecomConfig.robotWebhook) {
await sendWecomRobotMessage(testContent, wecomConfig);
robotStatus = "ok";
}
let appStatus: "ok" | "skipped" | "failed" = "skipped";
if (
partnerExternalId &&
wecomConfig.corpId &&
wecomConfig.agentId &&
wecomConfig.secret
) {
const result = await sendWecomAppMessage(
[partnerExternalId],
testContent,
wecomConfig,
);
appStatus = result.failed > 0 ? "failed" : "ok";
}
return Response.json({
robot: robotStatus,
app: appStatus,
});
} else if (body.action === "wecom_status") {
const wecomConfig = resolveWecomConfig(
env as unknown as WecomBindings,
);
return Response.json({
robotConfigured: Boolean(wecomConfig.robotWebhook),
appConfigured: Boolean(
wecomConfig.corpId && wecomConfig.agentId && wecomConfig.secret,
),
dueDays: wecomConfig.dueDays,
});
} else if (body.action === "trigger_wecom_due_soon") {
const wecomConfig = resolveWecomConfig(
env as unknown as WecomBindings,
);
const summary = await runDueSoonWecomNotifications(
db,
wecomConfig,
);
return Response.json(summary);
} else if (body.action === "wecom_list_external_contacts") {
const wecomConfig = resolveWecomConfig(
env as unknown as WecomBindings,
);
const contacts = await listExternalContacts(wecomConfig);
return Response.json({ contacts });
} else if (body.action === "wecom_sync_group_chats") {
const wecomConfig = resolveWecomConfig(
env as unknown as WecomBindings,
);
const groups = await syncGroupChats(db, wecomConfig);
return Response.json({ groups, syncedCount: groups.length });
} else if (body.action === "wecom_push_task_to_groups") {
const wecomConfig = resolveWecomConfig(
env as unknown as WecomBindings,
);
const summary = await pushTaskToGroupChats(
db,
{
taskId: String(body.taskId ?? ""),
chatIds: body.chatIds,
text: body.text,
},
wecomConfig,
);
const failCount = summary.results.reduce(
(total, item) => total + item.failList.length,
0,
);
const groupCount = summary.results.reduce(
(total, item) => total + item.chatCount,
0,
);
return Response.json({
results: summary.results,
groupCount,
failCount,
hint: "群发任务已创建,群主需在企微客户端「群发助手」点击发送后,消息才会送达客户群",
});
} else if (body.action === "wecom_task_group_pushes") {
const taskId = String(body.taskId ?? "").trim();
const pushes = await listGroupPushes(db, taskId || undefined);
return Response.json({ pushes });
} else if (body.action === "wecom_group_chats") {
const groups = await listGroupChatRows(db);
return Response.json({ groups });
} else {
return Response.json({ error: "不支持的操作" }, { status: 400 });
}
@@ -376,7 +697,8 @@ export async function POST(request: Request) {
{
status:
error instanceof FeishuSourceError ||
error instanceof DistributionReleaseError
error instanceof DistributionReleaseError ||
error instanceof WecomClientError
? error.status
: 500,
},

View File

@@ -32,6 +32,8 @@ const toolOutputSchema = z.object({
due_date: z.string(),
sheet_name: z.string(),
note_count: z.number().int().nonnegative(),
platform: z.enum(["小红书", "抖音"]),
content_format: z.enum(["image_text", "video"]),
claim_url: z.string().url(),
});
@@ -57,7 +59,7 @@ function createServer(context: McpRequestContext) {
{
title: "创建 KOC 分发任务",
description:
"读取飞书电子表格中的标题、正文和配图,在 KOC LOOP 创建分发任务,并返回可直接发给 KOC 的领取链接。飞书表格若有多个工作表,链接必须包含目标 sheet 参数。",
"读取飞书电子表格中的标题、正文及图片或视频,在 KOC LOOP 创建小红书/抖音分发任务,并返回可直接发给 KOC 的领取链接。飞书表格若有多个工作表,链接必须包含目标 sheet 参数。",
inputSchema: z.object({
feishu_url: z
.string()
@@ -74,6 +76,14 @@ function createServer(context: McpRequestContext) {
.max(100)
.optional()
.describe("品牌或项目名称;未提供时系统记录为“未设置项目”"),
platform: z
.enum(["小红书", "抖音"])
.optional()
.describe("发布平台,默认小红书"),
content_format: z
.enum(["image_text", "video"])
.optional()
.describe("内容形式image_text 图文video 视频;默认图文"),
}),
outputSchema: toolOutputSchema,
annotations: {
@@ -83,7 +93,7 @@ function createServer(context: McpRequestContext) {
openWorldHint: true,
},
},
async ({ feishu_url, task_name, due_date, brand_project }) => {
async ({ feishu_url, task_name, due_date, brand_project, platform, content_format }) => {
try {
const bindings = getBindings();
const portalUrl = String(bindings.KOC_PORTAL_URL ?? "").trim();
@@ -96,6 +106,8 @@ function createServer(context: McpRequestContext) {
name: task_name,
brand: brand_project?.trim() || "未设置项目",
dueAt: due_date,
platform: platform ?? "小红书",
contentFormat: content_format ?? "image_text",
},
bindings,
{ deduplicate: true },
@@ -108,6 +120,8 @@ function createServer(context: McpRequestContext) {
due_date: result.dueAt,
sheet_name: result.sheetName,
note_count: result.noteCount,
platform: result.platform,
content_format: result.contentFormat,
claim_url: buildClaimUrl(portalUrl, result.shareToken),
};
const actionText = result.created ? "已创建" : "已找到相同任务";
@@ -115,7 +129,7 @@ function createServer(context: McpRequestContext) {
content: [
{
type: "text",
text: `${actionText}${result.name}”,共 ${result.noteCount}笔记。领取链接:${output.claim_url}`,
text: `${actionText}${result.name}”,平台:${result.platform},内容形式:${result.contentFormat === "video" ? "视频" : "图文"}${result.noteCount} 篇。领取链接:${output.claim_url}`,
},
],
structuredContent: output,

View File

@@ -0,0 +1,750 @@
import { getRuntimeEnv } from "../../../lib/runtime-env";
import { runInBackground } from "../../../lib/background";
import type { DatabaseStatement } from "../../../lib/database";
import {
downloadFeishuMedia,
type FeishuBindings,
} from "../../../lib/feishu-client";
import {
ensureSchema,
getRawDb,
getUploadBucket,
hashText,
uid,
} from "../../../lib/mvp-db";
import {
PARTNER_BATCH_MAX_BYTES,
buildPartnerBatchWorkbookColumns,
parsePartnerBatchWorkbook,
resolvePartnerWorkbookOrigin,
} from "../../../lib/partner-batch-workbook";
import {
accountFromPublishLink,
} from "../../../lib/partner-utils";
import { extractPublishUrl } from "../../../lib/publish-url";
import {
partnerOptions,
withPartnerCors,
} from "../../../lib/partner-cors";
import {
buildRecoveryWorkbook,
type RecoveryWorkbookImage,
type RecoveryWorkbookRow,
} from "../../../lib/recovery-workbook";
import {
resolveCollectionMcpConfig,
type CollectionMcpBindings,
} from "../../../lib/mcp-collection-client";
import { enrichDistributionAccount } from "../../../lib/account-enrichment-service";
import {
createCollectionRunTasks,
runDueScheduledCollections,
} from "../../../lib/collection-service";
import { normalizeWorkbookImage } from "../../../lib/workbook-image";
const env = getRuntimeEnv();
type StoredAsset = {
index: number;
key: string;
fileToken?: string;
width?: number | null;
height?: number | null;
};
type BatchRow = {
distribution_id: string;
title: string;
body: string;
source_row: number | null;
image_assets: string;
video_assets: string;
publish_url: string | null;
publish_screenshot_key: string | null;
screenshot_key: string | null;
partner_id: string;
account_id: string | null;
claimant_name: string;
};
type TaskRow = {
id: string;
name: string;
task_type: string;
collection_start_date: string | null;
collection_days: string;
platform: "小红书" | "抖音";
content_format: "image_text" | "video";
};
type BatchAccess = {
task: TaskRow;
rows: BatchRow[];
};
function textValue(value: string | null, maxLength = 100) {
return String(value ?? "").trim().slice(0, maxLength);
}
function safeFileName(value: string) {
return value.replace(/[\\/:*?"<>|]/g, " ").trim().slice(0, 60) || "领取笔记";
}
function exactArrayBuffer(bytes: Uint8Array) {
const copy = new Uint8Array(bytes.byteLength);
copy.set(bytes);
return copy.buffer;
}
function contentTypeFromObject(object: { writeHttpMetadata(headers: Headers): void }) {
const headers = new Headers();
object.writeHttpMetadata(headers);
return headers.get("Content-Type") || "application/octet-stream";
}
function parseAssets(
value: string,
prefixes = ["content-assets/", "task-assets/"],
) {
try {
const assets = JSON.parse(value || "[]") as StoredAsset[];
return Array.isArray(assets)
? assets
.filter(
(asset) =>
Number.isInteger(Number(asset.index)) &&
Number(asset.index) > 0 &&
typeof asset.key === "string" &&
prefixes.some((prefix) => asset.key.startsWith(prefix)),
)
.map((asset) => ({ ...asset, index: Number(asset.index) }))
.sort((left, right) => left.index - right.index)
: [];
} catch {
return [];
}
}
async function loadImage(
key: string,
description: string,
fileToken?: string,
width?: number | null,
height?: number | null,
compactSource = false,
) {
const bucket = getUploadBucket();
let object = await bucket.get(key);
if (!object && fileToken) {
const media = await downloadFeishuMedia(
fileToken,
env as unknown as FeishuBindings,
);
await bucket.put(key, media.bytes, {
httpMetadata: { contentType: media.contentType },
customMetadata: { source: "feishu-api" },
});
object = await bucket.get(key);
}
if (!object) return null;
return normalizeWorkbookImage(
{
bytes: new Uint8Array(await object.arrayBuffer()),
contentType: contentTypeFromObject(object),
description,
width,
height,
} satisfies RecoveryWorkbookImage,
compactSource
? { maxDimension: 1600, outputFormat: "jpeg", jpegQuality: 82 }
: undefined,
);
}
async function findAccess(
taskToken: string,
claimToken: string,
delegationToken: string,
): Promise<BatchAccess | null> {
const db = getRawDb();
const task = delegationToken
? await db
.prepare(
`SELECT t.id, t.name, t.task_type, t.collection_start_date, t.collection_days,
t.platform, t.content_format
FROM tasks t
JOIN delegation_bundles b ON b.task_id = t.id
WHERE b.share_token = ? AND b.status = 'active'`,
)
.bind(delegationToken)
.first<TaskRow>()
: await db
.prepare(
`SELECT t.id, t.name, t.task_type, t.collection_start_date, t.collection_days,
t.platform, t.content_format
FROM tasks t
JOIN claims cl ON cl.task_id = t.id
WHERE t.share_token = ? AND cl.claim_token = ?`,
)
.bind(taskToken, claimToken)
.first<TaskRow>();
if (!task) return null;
const rows = delegationToken
? await db
.prepare(
`SELECT d.id AS distribution_id, d.partner_id, d.account_id,
d.publish_url, d.publish_screenshot_key, d.screenshot_key,
c.title, c.body, c.source_row, c.image_assets, c.video_assets,
cl.claimant_name
FROM distributions d
JOIN contents c ON c.id = d.content_id
JOIN delegation_bundles b ON b.id = d.delegation_bundle_id
JOIN claims cl ON cl.id = d.claim_id
WHERE b.share_token = ? AND b.task_id = ? AND b.status = 'active'
ORDER BY d.claimed_at, d.id`,
)
.bind(delegationToken, task.id)
.all<BatchRow>()
: await db
.prepare(
`SELECT d.id AS distribution_id, d.partner_id, d.account_id,
d.publish_url, d.publish_screenshot_key, d.screenshot_key,
c.title, c.body, c.source_row, c.image_assets, c.video_assets,
cl.claimant_name
FROM distributions d
JOIN contents c ON c.id = d.content_id
JOIN claims cl ON cl.id = d.claim_id
WHERE cl.claim_token = ? AND cl.task_id = ?
ORDER BY d.claimed_at, d.id`,
)
.bind(claimToken, task.id)
.all<BatchRow>();
return { task, rows: rows.results };
}
async function handleGet(request: Request) {
try {
await ensureSchema();
const url = new URL(request.url);
const taskToken = textValue(url.searchParams.get("task"));
const claimToken = textValue(url.searchParams.get("claim"));
const delegationToken = textValue(url.searchParams.get("share"));
if (!delegationToken && (!taskToken || !claimToken)) {
return Response.json({ error: "领取凭证不完整" }, { status: 400 });
}
const access = await findAccess(taskToken, claimToken, delegationToken);
if (!access) return Response.json({ error: "领取凭证无效" }, { status: 403 });
if (access.task.task_type !== "content_publish") {
return Response.json({ error: "当前任务不支持笔记批量回填" }, { status: 400 });
}
const maxSourceImages = Math.max(
0,
...access.rows.map((row) => parseAssets(row.image_assets).length),
);
const maxSourceVideos = Math.max(
0,
...access.rows.map(
(row) => parseAssets(row.video_assets, ["content-videos/"]).length,
),
);
const columns = buildPartnerBatchWorkbookColumns({
contentFormat: access.task.content_format,
maxSourceImages,
maxSourceVideos,
});
const downloadOrigin = resolvePartnerWorkbookOrigin(
request,
env.APP_ORIGIN,
);
const workbookRows: RecoveryWorkbookRow[] = [];
for (let rowIndex = 0; rowIndex < access.rows.length; rowIndex += 1) {
const row = access.rows[rowIndex];
const images: RecoveryWorkbookRow["images"] = [];
const hyperlinks: NonNullable<RecoveryWorkbookRow["hyperlinks"]> = [];
const assets =
columns.sourceImageCount > 0 ? parseAssets(row.image_assets) : [];
for (let index = 0; index < assets.length; index += 1) {
const asset = assets[index];
const image = await loadImage(
asset.key,
`${row.title} 原图${asset.index}`,
asset.fileToken,
asset.width,
asset.height,
true,
);
if (image) {
images.push({
column: columns.sourceImageStartColumn + index,
image,
maxWidth: 160,
maxHeight: 118,
});
}
}
if (row.publish_screenshot_key) {
const image = await loadImage(
row.publish_screenshot_key,
`${row.title} 笔记截图`,
);
if (image) {
images.push({ column: columns.publishScreenshotColumn, image });
}
}
if (row.screenshot_key) {
const image = await loadImage(
row.screenshot_key,
`${row.title} 数据分析截图`,
);
if (image) {
images.push({ column: columns.creatorScreenshotColumn, image });
}
}
const videoAssets = parseAssets(row.video_assets, ["content-videos/"]);
for (let index = 0; index < videoAssets.length; index += 1) {
const params = new URLSearchParams({
distribution: row.distribution_id,
index: String(videoAssets[index].index),
kind: "video",
download: "1",
});
if (delegationToken) params.set("share", delegationToken);
else {
params.set("task", taskToken);
params.set("claim", claimToken);
}
hyperlinks.push({
column: columns.sourceVideoStartColumn + index,
url: `${downloadOrigin}/api/partner-image?${params}`,
});
}
workbookRows.push({
cells: [
rowIndex + 1,
row.title,
row.body,
...Array.from({ length: columns.sourceImageCount }, () => ""),
...Array.from(
{ length: columns.sourceVideoCount },
(_, index) => (index < videoAssets.length ? `下载视频${index + 1}` : ""),
),
row.publish_url || "",
"",
"",
row.distribution_id,
row.publish_screenshot_key || "",
row.screenshot_key || "",
],
images,
hyperlinks: [
...hyperlinks,
...(row.publish_url
? [{ column: columns.publishUrlColumn, url: row.publish_url }]
: []),
],
});
}
const workbook = buildRecoveryWorkbook({
sheetName: "批量回填",
headers: columns.headers,
columnWidths: columns.columnWidths,
rows: workbookRows,
hiddenColumns: [
columns.systemColumn,
columns.systemColumn + 1,
columns.systemColumn + 2,
],
});
const fileName = `${safeFileName(access.task.name)}-批量回填.xlsx`;
return new Response(exactArrayBuffer(workbook), {
headers: {
"Cache-Control": "private, no-store",
"Content-Type":
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"Content-Disposition": `attachment; filename="koc-batch.xlsx"; filename*=UTF-8''${encodeURIComponent(fileName)}`,
"X-Content-Type-Options": "nosniff",
},
});
} catch (error) {
return Response.json(
{ error: error instanceof Error ? error.message : "导出失败" },
{ status: 500 },
);
}
}
function extensionForImage(image: { fileName: string; contentType: string }) {
const fromName = image.fileName.split(".").at(-1)?.replace(/[^a-zA-Z0-9]/g, "");
if (fromName) return fromName.toLowerCase() === "jpeg" ? "jpg" : fromName;
return image.contentType.includes("png")
? "png"
: image.contentType.includes("webp")
? "webp"
: image.contentType.includes("gif")
? "gif"
: "jpg";
}
async function storeImportedImage(
kind: "publish" | "creator",
distributionId: string,
image: { bytes: Uint8Array; contentType: string; fileName: string },
) {
const prefix = kind === "publish" ? "publish-evidence" : "creator-center";
const key = `${prefix}/${distributionId}/${uid("sheet")}.${extensionForImage(image)}`;
await getUploadBucket().put(key, image.bytes, {
httpMetadata: { contentType: image.contentType },
customMetadata: { source: "partner-batch-workbook" },
});
return key;
}
async function isDifferentFromStoredImage(
existingKey: string | null,
image: { bytes: Uint8Array },
) {
if (!existingKey) return true;
const stored = await getUploadBucket().get(existingKey);
if (!stored) return true;
const storedBytes = new Uint8Array(await stored.arrayBuffer());
if (storedBytes.byteLength !== image.bytes.byteLength) return true;
for (let index = 0; index < storedBytes.byteLength; index += 1) {
if (storedBytes[index] !== image.bytes[index]) return true;
}
return false;
}
async function handlePost(request: Request) {
try {
await ensureSchema();
const form = await request.formData();
const file = form.get("file");
if (!(file instanceof File) || file.size === 0) {
return Response.json({ error: "请选择填写完成的Excel表" }, { status: 400 });
}
if (file.size > PARTNER_BATCH_MAX_BYTES) {
return Response.json({ error: "批量回填表不能超过80MB" }, { status: 400 });
}
if (!/\.xlsx$/i.test(file.name)) {
return Response.json({ error: "仅支持系统导出的 .xlsx 表格" }, { status: 400 });
}
const taskToken = textValue(String(form.get("taskToken") ?? ""));
const claimToken = textValue(String(form.get("claimToken") ?? ""));
const delegationToken = textValue(String(form.get("delegationToken") ?? ""));
const access = await findAccess(taskToken, claimToken, delegationToken);
if (!access) return Response.json({ error: "领取凭证无效" }, { status: 403 });
if (access.task.task_type !== "content_publish") {
return Response.json({ error: "当前任务不支持笔记批量回填" }, { status: 400 });
}
const importedRows = parsePartnerBatchWorkbook(await file.arrayBuffer());
const assignmentById = new Map(
access.rows.map((row) => [row.distribution_id, row]),
);
const seen = new Set<string>();
const errors: string[] = [];
const prepared = importedRows.map((row) => {
const assignment = assignmentById.get(row.distributionId);
if (!assignment) {
errors.push(`${row.spreadsheetRow}行不属于当前领取批次,请重新导出表格`);
} else if (seen.has(row.distributionId)) {
errors.push(`${row.spreadsheetRow}行笔记重复`);
} else if (row.title && row.title !== assignment.title) {
errors.push(`${row.spreadsheetRow}行标题已被修改,请重新导出表格`);
}
seen.add(row.distributionId);
const publishUrl = row.publishUrl
? extractPublishUrl(row.publishUrl, access.task.platform)
: assignment?.publish_url || "";
if (row.publishUrl && !publishUrl) {
errors.push(`${row.spreadsheetRow}行发布链接不是有效的${access.task.platform}作品链接`);
}
const hasPublishScreenshot = Boolean(
assignment?.publish_screenshot_key || row.publishScreenshot,
);
if (publishUrl && !hasPublishScreenshot) {
errors.push(`${row.spreadsheetRow}行填写了发布链接,请同时插入笔记截图`);
}
if (row.publishScreenshot && !publishUrl) {
errors.push(`${row.spreadsheetRow}行插入了笔记截图,请同时填写发布链接`);
}
if (row.creatorScreenshot && !publishUrl) {
errors.push(`${row.spreadsheetRow}行需先回填发布链接,再补数据分析截图`);
}
return {
imported: row,
assignment,
publishUrl,
};
});
if (errors.length > 0) {
return Response.json(
{ error: errors.slice(0, 8).join(""), errors },
{ status: 400 },
);
}
const db = getRawDb();
let updatedRows = 0;
let publishedCount = 0;
let analysisScreenshotCount = 0;
let noteScreenshotCount = 0;
let publishUrlChangedCount = 0;
const enrichments: Array<{ id: string; url: string; nickname: string }> = [];
for (const item of prepared) {
const assignment = item.assignment!;
let publishScreenshotKey = assignment.publish_screenshot_key;
let creatorScreenshotKey = assignment.screenshot_key;
const hasNewPublishScreenshot = item.imported.publishScreenshot
? await isDifferentFromStoredImage(
assignment.publish_screenshot_key,
item.imported.publishScreenshot,
)
: false;
const hasNewCreatorScreenshot = item.imported.creatorScreenshot
? await isDifferentFromStoredImage(
assignment.screenshot_key,
item.imported.creatorScreenshot,
)
: false;
if (hasNewPublishScreenshot && item.imported.publishScreenshot) {
publishScreenshotKey = await storeImportedImage(
"publish",
assignment.distribution_id,
item.imported.publishScreenshot,
);
noteScreenshotCount += 1;
}
if (hasNewCreatorScreenshot && item.imported.creatorScreenshot) {
creatorScreenshotKey = await storeImportedImage(
"creator",
assignment.distribution_id,
item.imported.creatorScreenshot,
);
analysisScreenshotCount += 1;
}
const statements: DatabaseStatement[] = [];
if (publishScreenshotKey !== assignment.publish_screenshot_key) {
statements.push(
db
.prepare(
`UPDATE distributions SET publish_screenshot_key = ?, updated_at = CURRENT_TIMESTAMP WHERE id = ?`,
)
.bind(publishScreenshotKey, assignment.distribution_id),
);
}
if (creatorScreenshotKey !== assignment.screenshot_key) {
statements.push(
db
.prepare(
`UPDATE distributions SET screenshot_key = ?,
ocr_status = CASE WHEN exposure IS NOT NULL AND views IS NOT NULL THEN ocr_status ELSE 'uploaded' END,
updated_at = CURRENT_TIMESTAMP WHERE id = ?`,
)
.bind(creatorScreenshotKey, assignment.distribution_id),
);
}
if (item.publishUrl && item.publishUrl !== assignment.publish_url) {
const isReplacement = Boolean(assignment.publish_url);
const account = accountFromPublishLink(
item.publishUrl,
access.task.platform,
);
if (!account) {
return Response.json(
{ error: `${item.imported.spreadsheetRow}行发布链接格式不正确` },
{ status: 400 },
);
}
const matchedAccount = await db
.prepare(
`SELECT id FROM accounts
WHERE platform = ? AND platform_uid = ?
LIMIT 1`,
)
.bind(account.platform, account.platformUid)
.first<{ id: string }>();
const accountId =
matchedAccount?.id ||
`account-${hashText(`${account.platform}:${account.platformUid}`)}`;
statements.push(
...(isReplacement
? [
db
.prepare(
`UPDATE distributions SET
d2_likes = NULL, d2_comments = NULL, d2_collects = NULL,
d5_likes = NULL, d5_comments = NULL, d5_collects = NULL,
d7_likes = NULL, d7_comments = NULL, d7_collects = NULL,
latest_likes = NULL, latest_comments = NULL,
latest_collects = NULL, latest_shares = NULL,
collection_status = 'pending',
collection_status_description = '批量回填已更新链接,等待重新采集',
collection_updated_at = NULL, last_collection_day = NULL,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?`,
)
.bind(assignment.distribution_id),
db
.prepare("DELETE FROM collection_runs WHERE distribution_id = ?")
.bind(assignment.distribution_id),
]
: []),
db
.prepare(
`INSERT INTO accounts
(id, platform, platform_uid, nickname, profile_url,
current_contact, post_count)
VALUES (?, ?, ?, ?, ?, ?, 1)
ON CONFLICT(platform, platform_uid) DO UPDATE SET
nickname = excluded.nickname,
profile_url = excluded.profile_url,
current_contact = CASE
WHEN excluded.current_contact != ''
THEN excluded.current_contact
ELSE accounts.current_contact
END,
last_seen_at = CURRENT_TIMESTAMP`,
)
.bind(
accountId,
account.platform,
account.platformUid,
account.nickname,
account.profileUrl,
assignment.claimant_name,
),
db
.prepare(
`UPDATE distributions SET account_id = ?, publish_url = ?,
publish_time = COALESCE(publish_time, CURRENT_TIMESTAMP),
status = 'published', updated_at = CURRENT_TIMESTAMP
WHERE id = ?`,
)
.bind(accountId, item.publishUrl, assignment.distribution_id),
);
statements.push(
db
.prepare(
`UPDATE accounts SET post_count = (
SELECT COUNT(*) FROM distributions WHERE account_id = ?
) WHERE id = ?`,
)
.bind(accountId, accountId),
);
if (assignment.account_id && assignment.account_id !== accountId) {
statements.push(
db
.prepare(
`UPDATE accounts SET post_count = (
SELECT COUNT(*) FROM distributions WHERE account_id = ?
) WHERE id = ?`,
)
.bind(assignment.account_id, assignment.account_id),
);
}
if (isReplacement) publishUrlChangedCount += 1;
if (!assignment.publish_url) {
statements.push(
db
.prepare(
`UPDATE partners SET completed_total = completed_total + 1 WHERE id = ?`,
)
.bind(assignment.partner_id),
);
publishedCount += 1;
}
enrichments.push({
id: assignment.distribution_id,
url: item.publishUrl,
nickname: account.nickname,
});
}
if (statements.length > 0) {
await db.batch(statements);
updatedRows += 1;
}
}
if (
(publishedCount > 0 || publishUrlChangedCount > 0) &&
access.task.collection_start_date &&
access.task.collection_days !== "[]"
) {
let collectionDays: number[] = [];
try {
const parsed = JSON.parse(access.task.collection_days);
if (Array.isArray(parsed)) collectionDays = parsed.map(Number);
} catch {
collectionDays = [];
}
if (collectionDays.length > 0) {
await db
.prepare(
`UPDATE distributions SET collection_status = 'scheduled',
collection_status_description = ?
WHERE task_id = ? AND publish_url IS NOT NULL AND publish_url != ''`,
)
.bind(
`已安排${collectionDays.length}个采集日每日09:00执行`,
access.task.id,
)
.run();
await createCollectionRunTasks(
db,
access.task.id,
access.task.collection_start_date,
collectionDays,
);
runInBackground(
runDueScheduledCollections(
db,
Date.now(),
resolveCollectionMcpConfig(
env as unknown as CollectionMcpBindings,
),
"catchup",
access.task.id,
).catch(() => undefined),
"collection catchup after batch publish update",
);
}
}
for (const enrichment of enrichments) {
runInBackground(
enrichDistributionAccount(
db,
enrichment.id,
enrichment.url,
enrichment.nickname,
resolveCollectionMcpConfig(env as unknown as CollectionMcpBindings),
).catch(() => undefined),
"batch distribution account enrichment",
);
}
return Response.json({
imported: true,
updatedRows,
publishedCount,
publishUrlChangedCount,
noteScreenshotCount,
analysisScreenshotCount,
skippedRows: importedRows.length - updatedRows,
});
} catch (error) {
return Response.json(
{ error: error instanceof Error ? error.message : "批量回填失败" },
{ status: 500 },
);
}
}
export async function GET(request: Request) {
return withPartnerCors(request, await handleGet(request));
}
export async function POST(request: Request) {
return withPartnerCors(request, await handlePost(request));
}
export async function OPTIONS(request: Request) {
return partnerOptions(request);
}

View File

@@ -13,6 +13,7 @@ import {
withPartnerCors,
} from "../../../lib/partner-cors";
import { parseResultScreenshotKeys } from "../../../lib/result-screenshots";
import { hasMp4FileSignature } from "../../../lib/video-file";
const env = getRuntimeEnv();
@@ -26,7 +27,11 @@ function textValue(value: string | null, maxLength = 100) {
return String(value ?? "").trim().slice(0, maxLength);
}
function findAsset(value: string, imageIndex: number) {
function findAsset(
value: string,
imageIndex: number,
prefixes = ["content-assets/", "task-assets/"],
) {
try {
const assets = JSON.parse(value) as StoredAsset[];
return Array.isArray(assets)
@@ -34,8 +39,7 @@ function findAsset(value: string, imageIndex: number) {
(asset) =>
asset.index === imageIndex &&
typeof asset.key === "string" &&
(asset.key.startsWith("content-assets/") ||
asset.key.startsWith("task-assets/")) &&
prefixes.some((prefix) => asset.key.startsWith(prefix)) &&
(asset.fileToken === undefined ||
typeof asset.fileToken === "string"),
)
@@ -55,18 +59,20 @@ async function handleGet(request: Request) {
const distributionId = textValue(url.searchParams.get("distribution"));
const imageIndex = Number(url.searchParams.get("index"));
const imageKind = textValue(url.searchParams.get("kind"), 20);
const downloadRequested = url.searchParams.get("download") === "1";
if (
(!delegationToken && (!taskToken || !claimToken)) ||
!distributionId ||
!Number.isInteger(imageIndex) ||
imageIndex < 1
) {
return Response.json({ error: "图片链接不完整" }, { status: 400 });
return Response.json({ error: "素材链接不完整" }, { status: 400 });
}
const row = delegationToken
? await getRawDb()
.prepare(
`SELECT c.image_assets,
c.video_assets,
d.result_screenshot_key,
d.publish_screenshot_key,
d.screenshot_key
@@ -80,6 +86,7 @@ async function handleGet(request: Request) {
.bind(distributionId, delegationToken)
.first<{
image_assets: string;
video_assets: string;
result_screenshot_key: string | null;
publish_screenshot_key: string | null;
screenshot_key: string | null;
@@ -87,6 +94,7 @@ async function handleGet(request: Request) {
: await getRawDb()
.prepare(
`SELECT c.image_assets,
c.video_assets,
d.result_screenshot_key,
d.publish_screenshot_key,
d.screenshot_key
@@ -102,6 +110,7 @@ async function handleGet(request: Request) {
.bind(distributionId, claimToken, taskToken)
.first<{
image_assets: string;
video_assets: string;
result_screenshot_key: string | null;
publish_screenshot_key: string | null;
screenshot_key: string | null;
@@ -124,36 +133,82 @@ async function handleGet(request: Request) {
? row?.screenshot_key?.startsWith("creator-center/")
? { index: 1, key: row.screenshot_key }
: undefined
: imageKind === "video"
? row
? findAsset(row.video_assets, imageIndex, ["content-videos/"])
: undefined
: row
? findAsset(row.image_assets, imageIndex)
: undefined;
if (!asset?.key) {
return Response.json({ error: "没有找到这张图片" }, { status: 404 });
return Response.json({ error: "没有找到这个素材" }, { status: 404 });
}
const bucket = getUploadBucket();
let object = await bucket.get(asset.key);
if (!object && asset.fileToken) {
let objectBytes = object ? await object.arrayBuffer() : null;
const invalidStoredVideo =
imageKind === "video" &&
objectBytes !== null &&
!hasMp4FileSignature(objectBytes);
if ((!object || invalidStoredVideo) && asset.fileToken) {
const media = await downloadFeishuMedia(
asset.fileToken,
env as unknown as FeishuBindings,
fetch,
{
maxBytes: imageKind === "video" ? 500 * 1024 * 1024 : undefined,
label: imageKind === "video" ? "视频" : "图片",
},
);
if (imageKind === "video" && !hasMp4FileSignature(media.bytes)) {
return Response.json(
{ error: "源视频不是可下载的 MP4 文件,请重新上传 MP4 视频" },
{ status: 422 },
);
}
await bucket.put(asset.key, media.bytes, {
httpMetadata: { contentType: media.contentType },
httpMetadata: {
contentType: imageKind === "video" ? "video/mp4" : media.contentType,
},
customMetadata: { source: "feishu-api" },
});
object = await bucket.get(asset.key);
objectBytes = object ? await object.arrayBuffer() : media.bytes;
}
if (!object) {
return Response.json({ error: "图片同步失败,请稍后重试" }, { status: 404 });
if (!object || !objectBytes) {
return Response.json({ error: "素材同步失败,请稍后重试" }, { status: 404 });
}
if (imageKind === "video" && !hasMp4FileSignature(objectBytes)) {
return Response.json(
{ error: "已存储的视频不是有效的 MP4 文件,请联系运营重新同步" },
{ status: 422 },
);
}
const headers = new Headers();
object.writeHttpMetadata(headers);
headers.set("Cache-Control", "private, max-age=3600");
headers.set("Content-Disposition", `inline; filename="note-${imageIndex}"`);
return new Response(await object.arrayBuffer(), { headers });
const isMutableEvidence =
imageKind === "publish" || imageKind === "creator";
headers.set(
"Cache-Control",
isMutableEvidence ? "private, no-store" : "private, max-age=3600",
);
if (imageKind === "video") {
headers.set("Content-Type", "video/mp4");
}
headers.set("Content-Length", String(objectBytes.byteLength));
headers.set("X-Content-Type-Options", "nosniff");
const fileName =
imageKind === "video"
? `video-${imageIndex}.mp4`
: `image-${imageIndex}`;
headers.set(
"Content-Disposition",
`${downloadRequested ? "attachment" : "inline"}; filename="${fileName}"; filename*=UTF-8''${encodeURIComponent(fileName)}`,
);
return new Response(objectBytes, { headers });
} catch (error) {
return Response.json(
{ error: error instanceof Error ? error.message : "图片读取失败" },
{ error: error instanceof Error ? error.message : "素材读取失败" },
{ status: 500 },
);
}

View File

@@ -4,7 +4,10 @@ import type { DatabaseStatement } from "../../../lib/database";
const env = getRuntimeEnv();
import { enrichDistributionAccount } from "../../../lib/account-enrichment-service";
import { createCollectionRunTasks } from "../../../lib/collection-service";
import {
createCollectionRunTasks,
runDueScheduledCollections,
} from "../../../lib/collection-service";
import {
resolveCollectionMcpConfig,
type CollectionMcpBindings,
@@ -17,8 +20,8 @@ import {
} from "../../../lib/mvp-db";
import {
accountFromPublishLink,
extractXhsPublishUrl,
} from "../../../lib/partner-utils";
import { extractPublishUrl } from "../../../lib/publish-url";
import { parseClaimantIdentifier } from "../../../lib/claimant-identifier";
import {
partnerOptions,
@@ -101,23 +104,28 @@ function publicImageAssets(value: unknown): ImageAsset[] {
}
}
type PartnerTask = {
id: string;
name: string;
brand: string;
quantity: number;
claimed_quantity: number;
due_at: string;
status: string;
task_type: string;
platform: string;
content_format: string;
};
async function findTask(taskToken: string) {
return getRawDb()
.prepare(
`SELECT id, name, brand, quantity, claimed_quantity, due_at, status, task_type
`SELECT id, name, brand, quantity, claimed_quantity, due_at, status,
task_type, platform, content_format
FROM tasks WHERE share_token = ?`,
)
.bind(taskToken)
.first<{
id: string;
name: string;
brand: string;
quantity: number;
claimed_quantity: number;
due_at: string;
status: string;
task_type: string;
}>();
.first<PartnerTask>();
}
async function findDelegationAccess(delegationToken: string) {
@@ -132,6 +140,8 @@ async function findDelegationAccess(delegationToken: string) {
t.due_at,
t.status,
t.task_type,
t.platform,
t.content_format,
b.id AS bundle_id,
b.label AS bundle_label,
b.quantity AS bundle_quantity,
@@ -141,15 +151,7 @@ async function findDelegationAccess(delegationToken: string) {
WHERE b.share_token = ? AND b.status = 'active'`,
)
.bind(delegationToken)
.first<{
id: string;
name: string;
brand: string;
quantity: number;
claimed_quantity: number;
due_at: string;
status: string;
task_type: string;
.first<PartnerTask & {
bundle_id: string;
bundle_label: string;
bundle_quantity: number;
@@ -172,13 +174,15 @@ async function findAccessibleAssignment(
d.publish_screenshot_key,
d.screenshot_key,
d.result_screenshot_key,
d.result_submitted_at`;
d.result_submitted_at,
c.claimant_name`;
if (delegationToken) {
return db
.prepare(
`${select}
FROM distributions d
JOIN delegation_bundles b ON b.id = d.delegation_bundle_id
JOIN claims c ON c.id = d.claim_id
WHERE d.id = ?
AND b.share_token = ?
AND b.task_id = ?
@@ -194,6 +198,7 @@ async function findAccessibleAssignment(
screenshot_key: string | null;
result_screenshot_key: string | null;
result_submitted_at: string | null;
claimant_name: string;
}>();
}
if (!claimToken) return null;
@@ -214,6 +219,7 @@ async function findAccessibleAssignment(
screenshot_key: string | null;
result_screenshot_key: string | null;
result_submitted_at: string | null;
claimant_name: string;
}>();
}
@@ -284,6 +290,7 @@ async function handleGet(request: Request) {
c.body,
c.source_row,
c.image_assets,
c.video_assets,
a.nickname AS account_nickname,
b.id AS delegation_bundle_id,
b.label AS delegation_label
@@ -326,7 +333,9 @@ async function handleGet(request: Request) {
assignments: assignments.results.map((assignment) => ({
...assignment,
images: publicImageAssets(assignment.image_assets),
videos: publicImageAssets(assignment.video_assets),
image_assets: undefined,
video_assets: undefined,
})),
delegations: delegations.results,
};
@@ -349,6 +358,7 @@ async function handleGet(request: Request) {
c.body,
c.source_row,
c.image_assets,
c.video_assets,
a.nickname AS account_nickname
FROM distributions d
JOIN contents c ON c.id = d.content_id
@@ -366,7 +376,9 @@ async function handleGet(request: Request) {
assignments: assignments.results.map((assignment) => ({
...assignment,
images: publicImageAssets(assignment.image_assets),
videos: publicImageAssets(assignment.video_assets),
image_assets: undefined,
video_assets: undefined,
})),
};
}
@@ -379,6 +391,8 @@ async function handleGet(request: Request) {
dueAt: task.due_at,
status: task.status,
type: task.task_type,
platform: task.platform,
contentFormat: task.content_format,
}
: {
name: task.name,
@@ -388,6 +402,8 @@ async function handleGet(request: Request) {
dueAt: task.due_at,
status: task.status,
type: task.task_type,
platform: task.platform,
contentFormat: task.content_format,
availableQuantity: available?.count ?? 0,
},
claim,
@@ -798,14 +814,15 @@ async function handlePost(request: Request) {
{ status: 400 },
);
}
const publishUrl = extractXhsPublishUrl(publishInput);
const platform = task.platform === "抖音" ? "抖音" : "小红书";
const publishUrl = extractPublishUrl(publishInput, platform);
if (!publishUrl) {
return Response.json(
{ error: "请粘贴包含小红书长链或短链的分享内容" },
{ error: `请粘贴包含${platform}作品链接的分享内容` },
{ status: 400 },
);
}
const account = accountFromPublishLink(publishUrl);
const account = accountFromPublishLink(publishUrl, platform);
if (!account) {
return Response.json({ error: "发布链接格式不正确" }, { status: 400 });
}
@@ -816,28 +833,69 @@ async function handlePost(request: Request) {
delegationToken,
);
if (!assignment) {
return Response.json({ error: "笔记与领取凭证不匹配" }, { status: 403 });
return Response.json({ error: "作品与领取凭证不匹配" }, { status: 403 });
}
if (!assignment.publish_screenshot_key) {
return Response.json({ error: "请先上传发布截图" }, { status: 400 });
}
const reuseExistingAccount =
assignment.publish_url === publishUrl && assignment.account_id;
const matchedAccount = reuseExistingAccount
? null
: await db
.prepare(
`SELECT id FROM accounts
WHERE platform = ? AND platform_uid = ?
LIMIT 1`,
)
.bind(account.platform, account.platformUid)
.first<{ id: string }>();
const accountId =
reuseExistingAccount ||
matchedAccount?.id ||
`account-${hashText(`${account.platform}:${account.platformUid}`)}`;
const statements: DatabaseStatement[] = [];
const publishUrlChanged = Boolean(
assignment.publish_url && assignment.publish_url !== publishUrl,
);
if (publishUrlChanged) {
statements.push(
db
.prepare(
`UPDATE distributions SET
d2_likes = NULL, d2_comments = NULL, d2_collects = NULL,
d5_likes = NULL, d5_comments = NULL, d5_collects = NULL,
d7_likes = NULL, d7_comments = NULL, d7_collects = NULL,
latest_likes = NULL, latest_comments = NULL,
latest_collects = NULL, latest_shares = NULL,
collection_status = 'pending',
collection_status_description = '发布链接已更新,等待重新采集',
collection_updated_at = NULL, last_collection_day = NULL,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?`,
)
.bind(assignment.id),
db
.prepare("DELETE FROM collection_runs WHERE distribution_id = ?")
.bind(assignment.id),
);
}
if (!reuseExistingAccount) {
statements.push(
db
.prepare(
`INSERT INTO accounts
(id, platform, platform_uid, nickname, profile_url, post_count)
VALUES (?, ?, ?, ?, ?, 1)
(id, platform, platform_uid, nickname, profile_url,
current_contact, post_count)
VALUES (?, ?, ?, ?, ?, ?, 1)
ON CONFLICT(platform, platform_uid) DO UPDATE SET
nickname = excluded.nickname,
profile_url = excluded.profile_url,
post_count = accounts.post_count + ?,
current_contact = CASE
WHEN excluded.current_contact != ''
THEN excluded.current_contact
ELSE accounts.current_contact
END,
last_seen_at = CURRENT_TIMESTAMP`,
)
.bind(
@@ -846,7 +904,7 @@ async function handlePost(request: Request) {
account.platformUid,
account.nickname,
account.profileUrl,
assignment.publish_url ? 0 : 1,
assignment.claimant_name,
),
);
}
@@ -863,6 +921,26 @@ async function handlePost(request: Request) {
)
.bind(accountId, publishUrl, assignment.id),
);
statements.push(
db
.prepare(
`UPDATE accounts SET post_count = (
SELECT COUNT(*) FROM distributions WHERE account_id = ?
) WHERE id = ?`,
)
.bind(accountId, accountId),
);
if (assignment.account_id && assignment.account_id !== accountId) {
statements.push(
db
.prepare(
`UPDATE accounts SET post_count = (
SELECT COUNT(*) FROM distributions WHERE account_id = ?
) WHERE id = ?`,
)
.bind(assignment.account_id, assignment.account_id),
);
}
if (!assignment.publish_url) {
statements.push(
db
@@ -896,26 +974,46 @@ async function handlePost(request: Request) {
collectionDays = [];
}
if (collectionDays.length > 0) {
await db
.prepare(
`UPDATE distributions SET collection_status = 'scheduled',
collection_status_description = ? WHERE id = ?`,
)
.bind(
`已安排${collectionDays.length}个采集日每日09:00执行`,
assignment.id,
)
.run();
await createCollectionRunTasks(
db,
task.id,
collectionSchedule.collection_start_date,
collectionDays,
);
runInBackground(
runDueScheduledCollections(
db,
Date.now(),
resolveCollectionMcpConfig(
env as unknown as CollectionMcpBindings,
),
"catchup",
task.id,
).catch(() => undefined),
"collection catchup after partner publish update",
);
}
}
if (account.platform === "小红书") {
const enrichment = enrichDistributionAccount(
db,
assignment.id,
publishUrl,
account.nickname,
resolveCollectionMcpConfig(
env as unknown as CollectionMcpBindings,
),
).catch(() => undefined);
runInBackground(enrichment, "distribution account enrichment");
}
const enrichment = enrichDistributionAccount(
db,
assignment.id,
publishUrl,
account.nickname,
resolveCollectionMcpConfig(
env as unknown as CollectionMcpBindings,
),
).catch(() => undefined);
runInBackground(enrichment, "distribution account enrichment");
return Response.json({ ok: true });
}

View File

@@ -17,6 +17,7 @@ import {
type RecoveryWorkbookRow,
} from "../../../lib/recovery-workbook";
import { consumeMcpExportToken } from "../../../lib/mcp-export-token";
import { normalizeWorkbookImage } from "../../../lib/workbook-image";
const env = getRuntimeEnv();
@@ -53,6 +54,7 @@ type ExportRow = {
latest_likes: number | null;
latest_comments: number | null;
latest_collects: number | null;
latest_shares: number | null;
collection_status: string | null;
collection_status_description: string | null;
collection_updated_at: string | null;
@@ -121,12 +123,16 @@ function latestMetrics(row: ExportRow) {
const likes = row.latest_likes ?? legacyLikes;
const comments = row.latest_comments ?? legacyComments;
const collects = row.latest_collects ?? legacyCollects;
const shares = row.latest_shares;
return {
likes,
comments,
collects,
shares,
total:
likes === null ? null : likes + (comments ?? 0) + (collects ?? 0),
likes === null
? null
: likes + (comments ?? 0) + (collects ?? 0) + (shares ?? 0),
};
}
@@ -181,13 +187,13 @@ async function loadImage(reference: ImageReference) {
object = await bucket.get(reference.key);
}
if (!object) return null;
return {
return normalizeWorkbookImage({
bytes: new Uint8Array(await object.arrayBuffer()),
contentType: contentTypeFromObject(object),
width: reference.width,
height: reference.height,
description: reference.description,
} satisfies RecoveryWorkbookImage;
} satisfies RecoveryWorkbookImage);
}
async function loadImages(references: ImageReference[]) {
@@ -236,9 +242,9 @@ export async function GET(request: Request) {
}
const db = getRawDb();
const task = await db
.prepare("SELECT id, name, brand FROM tasks WHERE id = ?")
.prepare("SELECT id, name, brand, platform FROM tasks WHERE id = ?")
.bind(taskId)
.first<{ id: string; name: string; brand: string }>();
.first<{ id: string; name: string; brand: string; platform: string }>();
if (!task) {
return Response.json({ error: "没有找到这个任务" }, { status: 404 });
}
@@ -269,6 +275,7 @@ export async function GET(request: Request) {
d.latest_likes,
d.latest_comments,
d.latest_collects,
d.latest_shares,
d.collection_status,
d.collection_status_description,
d.collection_updated_at,
@@ -318,18 +325,19 @@ export async function GET(request: Request) {
}
});
const loadedImages = await loadImages(references);
const isDouyin = task.platform === "抖音";
const metricHeaders = isDouyin
? ["点赞", "收藏", "转发", "评论", "总互动"]
: ["点赞", "收藏", "评论", "总互动"];
const headers = [
"序号(不能改)",
"标题",
"笔记内容(正文+话题)",
...Array.from({ length: maxContentImages }, (_, index) => `图片${index + 1}`),
"小红书昵称",
`${task.platform}昵称`,
"发布链接",
"发布时间",
"点赞",
"收藏",
"评论",
"总互动",
...metricHeaders,
"曝光量-实际第7天",
"阅读量-实际第7天",
"数据分析截图(单篇笔记数据分析截图)",
@@ -341,7 +349,7 @@ export async function GET(request: Request) {
];
const originalImageStart = 3;
const accountColumn = originalImageStart + maxContentImages;
const creatorScreenshotColumn = accountColumn + 9;
const creatorScreenshotColumn = accountColumn + (isDouyin ? 10 : 9);
const publishScreenshotColumn = creatorScreenshotColumn + 1;
const workbookRows: RecoveryWorkbookRow[] = rows.map((row, rowIndex) => {
const metrics = latestMetrics(row);
@@ -350,7 +358,7 @@ export async function GET(request: Request) {
{ length: maxContentImages },
(_, index) => {
const asset = contentAssets.find((item) => item.index === index + 1);
return asset && loadedImages.get(asset.key) ? "见图" : asset ? "图片读取失败" : "";
return "";
},
);
const creatorImage = row.screenshot_key
@@ -369,12 +377,13 @@ export async function GET(request: Request) {
formatExportDate(row.publish_time),
metrics.likes,
metrics.collects,
...(isDouyin ? [metrics.shares] : []),
metrics.comments,
metrics.total,
row.exposure,
row.views,
creatorImage ? "见图" : row.screenshot_key ? "截图读取失败" : "",
publishImage ? "见图" : row.publish_screenshot_key ? "截图读取失败" : "",
"",
"",
formatExportDate(row.collection_updated_at || row.updated_at),
row.distribution_id ? collectionLabel(row) : "未领取",
row.partner_name || "",
@@ -403,6 +412,7 @@ export async function GET(request: Request) {
20,
11,
11,
...(isDouyin ? [11] : []),
11,
11,
18,

View File

@@ -15,8 +15,12 @@ type AccountRow = {
profile_url: string;
ip_location: string;
followers: number;
gender: string;
bio: string;
tags: string;
post_count: number;
cooperation_source: string;
current_contact: string;
first_seen_at: string;
last_seen_at: string;
};
@@ -24,6 +28,7 @@ type AccountRow = {
type CooperationRow = {
account_id: string;
partner_name: string;
claimant_name: string | null;
delegation_bundle_id: string | null;
};
@@ -74,9 +79,11 @@ async function exportAccounts(accountIds: string[]) {
`SELECT
d.account_id,
p.name AS partner_name,
cl.claimant_name,
d.delegation_bundle_id
FROM distributions d
JOIN partners p ON p.id = d.partner_id
LEFT JOIN claims cl ON cl.id = d.claim_id
WHERE d.account_id IS NOT NULL`,
)
.all<CooperationRow>(),
@@ -109,8 +116,12 @@ async function exportAccounts(accountIds: string[]) {
"账号主页",
"IP地",
"粉丝数",
"性别",
"简介",
"标签",
"合作发布数",
"历史合作来源",
"当前联系人",
"资源归属",
"首次合作时间",
"最近合作时间",
@@ -119,7 +130,13 @@ async function exportAccounts(accountIds: string[]) {
const cooperation = cooperationByAccount.get(account.id) ?? [];
const sources = [
...new Set([
...cooperation.map((item) => item.partner_name),
...cooperation
.filter(
(item) =>
!item.claimant_name ||
item.partner_name !== item.claimant_name,
)
.map((item) => item.partner_name),
...(account.cooperation_source || "")
.split(/[、,;|]/)
.map((item) => item.trim())
@@ -138,8 +155,12 @@ async function exportAccounts(accountIds: string[]) {
account.profile_url || "",
account.ip_location || "待识别",
account.followers,
account.gender || "",
account.bio || "",
account.tags || "",
account.post_count,
sources.join("、"),
account.current_contact || "",
partnerManagedOnly ? "合作社资源 · 不可直联" : "可直联",
formatExportDate(account.first_seen_at),
formatExportDate(account.last_seen_at),
@@ -161,9 +182,13 @@ async function exportAccounts(accountIds: string[]) {
44,
14,
14,
10,
36,
32,
14,
32,
22,
22,
21,
21,
],

View File

@@ -1,214 +1,24 @@
import { isManagerRequest, managerForbidden } from "../../../lib/user-auth";
import { ensureSchema, getRawDb } from "../../../lib/mvp-db";
import { runInBackground } from "../../../lib/background";
import {
resolveCollectionMcpConfig,
resolveXhsProfileDetailsFromMcp,
resolveXhsPublicAccountDetails,
type CollectionMcpBindings,
} from "../../../lib/mcp-collection-client";
import { getRuntimeEnv } from "../../../lib/runtime-env";
import {
mergeCooperationSources,
normalizeProfileUrl,
parseResourceImportFile,
RESOURCE_IMPORT_MAX_BYTES,
resourcePlatformUid,
RESOURCE_IMPORT_MAX_ROWS,
resourceImportMissingFields,
type ResourceImportRow,
} from "../../../lib/resource-import";
import {
analyzeRows,
enrichRows,
loadAccounts,
mergeExistingFields,
writeAnalyzedRows,
RESOURCE_IMPORT_SYNC_ENRICH_ROWS,
type AnalyzedRow,
type AccountRow,
} from "../../../lib/resource-write";
type AccountRow = {
id: string;
platform: string;
platform_uid: string;
public_account_id: string;
nickname: string;
profile_url: string;
ip_location: string;
followers: number;
cooperation_source: string;
};
type AnalyzedRow = ResourceImportRow & {
action: "create" | "update" | "error";
accountId: string;
platformUid: string;
cooperationSource: string;
};
function identityKey(platform: string, value: string) {
return `${platform.trim().toLocaleLowerCase("zh-CN")}|${value
.trim()
.toLocaleLowerCase("zh-CN")}`;
}
async function loadAccounts() {
return getRawDb()
.prepare(
`SELECT id, platform, platform_uid, public_account_id, nickname,
profile_url, ip_location, followers, cooperation_source
FROM accounts`,
)
.all<AccountRow>();
}
async function mapConcurrent<T, R>(
items: T[],
limit: number,
worker: (item: T) => Promise<R>,
) {
const results = new Array<R>(items.length);
let cursor = 0;
await Promise.all(
Array.from({ length: Math.min(limit, items.length) }, async () => {
while (cursor < items.length) {
const index = cursor;
cursor += 1;
results[index] = await worker(items[index]);
}
}),
);
return results;
}
async function enrichRows(rows: ResourceImportRow[], accounts: AccountRow[]) {
const existingByProfile = new Map<string, AccountRow>();
for (const account of accounts) {
const profileUrl = normalizeProfileUrl(account.profile_url || "");
if (profileUrl) {
existingByProfile.set(identityKey(account.platform, profileUrl), account);
}
}
const mcpConfig = resolveCollectionMcpConfig(
getRuntimeEnv() as unknown as CollectionMcpBindings,
);
return mapConcurrent(rows, 4, async (row) => {
if (row.errors.length > 0 || !row.profileUrl || row.platform !== "小红书") {
return row;
}
const existing = existingByProfile.get(identityKey(row.platform, row.profileUrl));
const existingIpLocation =
existing?.ip_location && existing.ip_location !== "待识别"
? existing.ip_location
: "";
const baseline: ResourceImportRow = {
...row,
nickname: row.nickname || existing?.nickname || "",
publicAccountId: row.publicAccountId || existing?.public_account_id || "",
ipLocation: row.ipLocation || existingIpLocation,
followers: row.followersResolved
? row.followers
: Number(existing?.followers || 0),
followersResolved:
row.followersResolved || Number(existing?.followers || 0) > 0,
};
if (resourceImportMissingFields(baseline).length === 0) {
return baseline;
}
let details: {
nickname: string | null;
redId: string | null;
followers: number | null;
ipLocation: string | null;
} = await resolveXhsProfileDetailsFromMcp(row.profileUrl, mcpConfig).catch(
() => ({ nickname: null, redId: null, followers: null, ipLocation: null }),
);
const mcpResult = {
nickname: baseline.nickname || details.nickname?.trim() || "",
publicAccountId: baseline.publicAccountId || details.redId?.trim() || "",
ipLocation: baseline.ipLocation || details.ipLocation?.trim() || "",
followersResolved: baseline.followersResolved || details.followers !== null,
};
if (resourceImportMissingFields(mcpResult).length > 0) {
const publicDetails = await resolveXhsPublicAccountDetails(row.profileUrl).catch(
() => ({ nickname: null, redId: null, followers: null, ipLocation: null }),
);
details = {
nickname: details.nickname || publicDetails.nickname,
redId: details.redId || publicDetails.redId,
followers: details.followers ?? publicDetails.followers,
ipLocation: details.ipLocation || publicDetails.ipLocation,
};
}
return {
...baseline,
nickname: baseline.nickname || details.nickname?.trim() || "待识别账号",
publicAccountId: baseline.publicAccountId || details.redId?.trim() || "",
ipLocation: baseline.ipLocation || details.ipLocation?.trim() || "待识别",
followers: baseline.followersResolved
? baseline.followers
: (details.followers ?? 0),
followersResolved:
baseline.followersResolved || details.followers !== null,
};
});
}
function analyzeRows(rows: ResourceImportRow[], accountRows: AccountRow[]) {
const profileMap = new Map<string, AccountRow>();
const publicIdMap = new Map<string, AccountRow>();
const platformUidMap = new Map<string, AccountRow>();
for (const account of accountRows) {
const profileUrl = normalizeProfileUrl(account.profile_url || "");
if (profileUrl) profileMap.set(identityKey(account.platform, profileUrl), account);
if (account.public_account_id) {
publicIdMap.set(identityKey(account.platform, account.public_account_id), account);
}
platformUidMap.set(identityKey(account.platform, account.platform_uid), account);
}
return rows.map<AnalyzedRow>((row) => {
const platformUid = resourcePlatformUid(row);
const profileMatch = row.profileUrl
? profileMap.get(identityKey(row.platform, row.profileUrl))
: undefined;
const publicIdMatch = row.publicAccountId
? publicIdMap.get(identityKey(row.platform, row.publicAccountId))
: undefined;
const uidMatch = platformUidMap.get(identityKey(row.platform, platformUid));
const matches = [profileMatch, publicIdMatch, uidMatch].filter(
(account): account is AccountRow => Boolean(account),
);
const matchedIds = [...new Set(matches.map((account) => account.id))];
const errors = [...row.errors];
if (matchedIds.length > 1) {
errors.push("账号主页和账号ID匹配到不同的现有账号请先核对");
}
const existing = matchedIds.length === 1 ? matches[0] : undefined;
const accountId = existing?.id ?? `account-${crypto.randomUUID().slice(0, 12)}`;
const analyzed: AnalyzedRow = {
...row,
errors,
action: errors.length > 0 ? "error" : existing ? "update" : "create",
accountId,
platformUid: existing?.platform_uid ?? platformUid,
cooperationSource: mergeCooperationSources(
existing?.cooperation_source ?? "",
row.cooperationSource,
),
};
if (analyzed.action !== "error") {
const virtual: AccountRow = {
id: accountId,
platform: row.platform,
platform_uid: analyzed.platformUid,
public_account_id: row.publicAccountId || existing?.public_account_id || "",
nickname: row.nickname,
profile_url: row.profileUrl || existing?.profile_url || "",
ip_location: row.ipLocation || existing?.ip_location || "待识别",
followers: row.followers || existing?.followers || 0,
cooperation_source: analyzed.cooperationSource,
};
if (row.profileUrl) profileMap.set(identityKey(row.platform, row.profileUrl), virtual);
if (row.publicAccountId) {
publicIdMap.set(identityKey(row.platform, row.publicAccountId), virtual);
}
platformUidMap.set(identityKey(row.platform, analyzed.platformUid), virtual);
}
return analyzed;
});
}
const RESOURCE_IMPORT_PREVIEW_ROWS = 100;
function summarize(rows: AnalyzedRow[]) {
return {
@@ -219,10 +29,51 @@ function summarize(rows: AnalyzedRow[]) {
};
}
function previewAnalyzedRows(rows: AnalyzedRow[]) {
const errorRows = rows.filter((row) => row.action === "error");
if (errorRows.length === 0) {
return rows.slice(0, RESOURCE_IMPORT_PREVIEW_ROWS);
}
const importableRows = rows.filter((row) => row.action !== "error");
return [
...errorRows.slice(0, RESOURCE_IMPORT_PREVIEW_ROWS),
...importableRows.slice(
0,
Math.max(0, RESOURCE_IMPORT_PREVIEW_ROWS - errorRows.length),
),
];
}
function deferredEnrichmentCount(rows: ResourceImportRow[]) {
return rows.filter(
(row) =>
row.errors.length === 0 &&
row.profileUrl &&
resourceImportMissingFields(row).length > 0,
).length;
}
async function enrichImportedRowsInBackground(rows: ResourceImportRow[]) {
const accounts = await loadAccounts();
const baselineRows = mergeExistingFields(rows, accounts.results);
const missingRows = baselineRows.filter(
(row) =>
row.errors.length === 0 &&
row.profileUrl &&
resourceImportMissingFields(row).length > 0,
);
if (missingRows.length === 0) return;
const enriched = await enrichRows(missingRows, accounts.results);
const latestAccounts = await loadAccounts();
const analyzed = analyzeRows(enriched, latestAccounts.results).filter(
(row) => row.action !== "error",
);
await writeAnalyzedRows(analyzed);
}
export async function POST(request: Request) {
if (!(await isManagerRequest(request))) return managerForbidden();
try {
await ensureSchema();
const form = await request.formData();
const file = form.get("file");
const mode = String(form.get("mode") ?? "preview");
@@ -230,17 +81,31 @@ export async function POST(request: Request) {
return Response.json({ error: "请选择需要导入的 Excel 文件" }, { status: 400 });
}
if (file.size <= 0 || file.size > RESOURCE_IMPORT_MAX_BYTES) {
return Response.json({ error: "文件不能为空,且不能超过 5MB" }, { status: 400 });
return Response.json({ error: "文件不能为空,且不能超过 20MB" }, { status: 400 });
}
const rows = parseResourceImportFile(file.name, new Uint8Array(await file.arrayBuffer()));
const accounts = await loadAccounts();
const enriched = await enrichRows(rows, accounts.results);
const analyzed = analyzeRows(enriched, accounts.results);
const accountRows: AccountRow[] = accounts.results;
const shouldEnrichSynchronously = rows.length <= RESOURCE_IMPORT_SYNC_ENRICH_ROWS;
const preparedRows = shouldEnrichSynchronously
? await enrichRows(rows, accountRows)
: mergeExistingFields(rows, accountRows);
const analyzed = analyzeRows(preparedRows, accountRows);
const summary = summarize(analyzed);
const importableRows = analyzed.filter((row) => row.action !== "error");
const importableRowNumbers = new Set(
importableRows.map((row) => row.rowNumber),
);
const importablePreparedRows = preparedRows.filter((row) =>
importableRowNumbers.has(row.rowNumber),
);
const deferredEnrichment = shouldEnrichSynchronously
? 0
: deferredEnrichmentCount(importablePreparedRows);
if (mode !== "commit") {
return Response.json({
summary,
rows: analyzed.slice(0, 100).map((row) => ({
rows: previewAnalyzedRows(analyzed).map((row) => ({
rowNumber: row.rowNumber,
platform: row.platform,
nickname: row.nickname,
@@ -248,74 +113,44 @@ export async function POST(request: Request) {
profileUrl: row.profileUrl,
ipLocation: row.ipLocation,
followers: row.followers,
gender: row.gender,
bio: row.bio,
tags: row.tags,
cooperationSource: row.cooperationSource,
action: row.action,
errors: row.errors,
})),
truncated: analyzed.length > 100,
truncated: analyzed.length > RESOURCE_IMPORT_PREVIEW_ROWS,
deferredEnrichment,
maxRows: RESOURCE_IMPORT_MAX_ROWS,
});
}
if (summary.error > 0) {
if (importableRows.length === 0) {
return Response.json(
{ error: `${summary.error} 行数据未通过校验,请修正后重新上传`, summary },
{ error: "没有可导入的有效账号,请修正异常数据后重新上传", summary },
{ status: 400 },
);
}
const db = getRawDb();
const statements = analyzed.map((row) =>
row.action === "update"
? db
.prepare(
`UPDATE accounts SET
nickname = ?,
public_account_id = CASE WHEN ? != '' THEN ? ELSE public_account_id END,
profile_url = CASE WHEN ? != '' THEN ? ELSE profile_url END,
ip_location = CASE
WHEN ? != '' AND ? != '待识别' THEN ? ELSE ip_location END,
followers = CASE WHEN ? = 1 THEN ? ELSE followers END,
cooperation_source = ?,
last_seen_at = CURRENT_TIMESTAMP
WHERE id = ?`,
)
.bind(
row.nickname,
row.publicAccountId,
row.publicAccountId,
row.profileUrl,
row.profileUrl,
row.ipLocation,
row.ipLocation,
row.ipLocation,
row.followersResolved ? 1 : 0,
row.followers,
row.cooperationSource,
row.accountId,
)
: db
.prepare(
`INSERT INTO accounts
(id, platform, platform_uid, public_account_id, nickname,
profile_url, ip_location, followers, post_count, avg_views,
cooperation_source)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, 0, 0, ?)`,
)
.bind(
row.accountId,
row.platform,
row.platformUid,
row.publicAccountId,
row.nickname,
row.profileUrl,
row.ipLocation,
row.followers,
row.cooperationSource,
),
);
if (statements.length > 0) await db.batch(statements);
await writeAnalyzedRows(importableRows);
if (deferredEnrichment > 0) {
const importableSourceRows = rows.filter((row) =>
importableRowNumbers.has(row.rowNumber),
);
runInBackground(
enrichImportedRowsInBackground(importableSourceRows),
"bulk resource profile enrichment",
);
}
return Response.json({
summary,
message: `已导入 ${summary.create} 个新账号,更新 ${summary.update} 个已有账号`,
deferredEnrichment,
message: `已导入 ${summary.create} 个新账号,更新 ${summary.update} 个已有账号${
summary.error > 0 ? `;跳过 ${summary.error} 条异常数据` : ""
}${
deferredEnrichment > 0
? `${deferredEnrichment} 个账号的缺失公开资料将在后台补全`
: ""
}`,
});
} catch (error) {
const message = error instanceof Error ? error.message : "导入失败";

View File

@@ -0,0 +1,64 @@
import { isManagerRequest, managerForbidden } from "../../../lib/user-auth";
import { buildManualRow, type ManualResourceInput } from "../../../lib/resource-import";
import {
analyzeRows,
enrichRows,
loadAccounts,
writeAnalyzedRows,
} from "../../../lib/resource-write";
type InsertResult = {
action: "create" | "update";
message: string;
};
export async function POST(request: Request) {
if (!(await isManagerRequest(request))) return managerForbidden();
try {
const payload = (await request.json()) as Partial<ManualResourceInput> & {
mode?: string;
};
const row = buildManualRow({
profileUrl: payload.profileUrl ?? "",
nickname: payload.nickname ?? "",
publicAccountId: payload.publicAccountId ?? "",
ipLocation: payload.ipLocation ?? "",
followers: payload.followers ?? "",
gender: payload.gender ?? "",
bio: payload.bio ?? "",
tags: payload.tags ?? "",
cooperationSource: payload.cooperationSource ?? "",
});
if (row.errors.length > 0) {
return Response.json(
{ error: row.errors[0], errors: row.errors },
{ status: 400 },
);
}
const accounts = await loadAccounts();
const enriched = await enrichRows([row], accounts.results);
const analyzed = analyzeRows(enriched, accounts.results).filter(
(item) => item.action !== "error",
);
if (analyzed.length === 0) {
return Response.json(
{ error: "账号数据校验未通过,无法保存" },
{ status: 400 },
);
}
await writeAnalyzedRows(analyzed);
const result = analyzed[0];
const outcome: InsertResult = {
action: result.action === "update" ? "update" : "create",
message:
result.action === "update"
? `已更新账号 ${row.nickname || row.publicAccountId || "资料"}`
: `已新增账号 ${row.nickname || row.publicAccountId || "资料"}`,
};
return Response.json(outcome);
} catch (error) {
const message = error instanceof Error ? error.message : "保存失败";
return Response.json({ error: message }, { status: 400 });
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -21,6 +21,7 @@ export const partners = mysqlTable("partners", {
owner: varchar("owner", { length: 255 }).notNull().default("运营组"),
claimedTotal: int("claimed_total").notNull().default(0),
completedTotal: int("completed_total").notNull().default(0),
wecomExternalUserId: varchar("wecom_external_user_id", { length: 128 }),
createdAt: timestamp("created_at"),
});
@@ -37,6 +38,10 @@ export const tasks = mysqlTable(
taskType: varchar("task_type", { length: 32 })
.notNull()
.default("content_publish"),
platform: varchar("platform", { length: 32 }).notNull().default("小红书"),
contentFormat: varchar("content_format", { length: 32 })
.notNull()
.default("image_text"),
sourceUrl: text("source_url").notNull(),
sourceSheetId: varchar("source_sheet_id", { length: 255 }).notNull().default(""),
sourceSheetName: varchar("source_sheet_name", { length: 255 }).notNull().default(""),
@@ -59,6 +64,7 @@ export const contents = mysqlTable("contents", {
title: text("title").notNull(),
body: text("body").notNull(),
imageAssets: text("image_assets").notNull(),
videoAssets: text("video_assets").notNull(),
status: varchar("status", { length: 32 }).notNull().default("available"),
source: varchar("source", { length: 255 }).notNull().default("飞书内容表"),
sourceRow: int("source_row"),
@@ -76,11 +82,17 @@ export const accounts = mysqlTable(
profileUrl: text("profile_url").notNull(),
ipLocation: varchar("ip_location", { length: 255 }).notNull().default("待识别"),
followers: int("followers").notNull().default(0),
gender: varchar("gender", { length: 16 }).notNull().default(""),
bio: text("bio").notNull().default(""),
tags: varchar("tags", { length: 500 }).notNull().default(""),
postCount: int("post_count").notNull().default(0),
avgViews: int("avg_views").notNull().default(0),
cooperationSource: varchar("cooperation_source", { length: 500 })
.notNull()
.default(""),
currentContact: varchar("current_contact", { length: 255 })
.notNull()
.default(""),
firstSeenAt: timestamp("first_seen_at"),
lastSeenAt: timestamp("last_seen_at"),
},
@@ -162,6 +174,7 @@ export const distributions = mysqlTable("distributions", {
latestLikes: int("latest_likes"),
latestComments: int("latest_comments"),
latestCollects: int("latest_collects"),
latestShares: int("latest_shares"),
collectionStatus: varchar("collection_status", { length: 32 })
.notNull()
.default("pending"),
@@ -184,6 +197,7 @@ export const collectionRuns = mysqlTable(
likes: int("likes"),
comments: int("comments"),
collects: int("collects"),
shares: int("shares"),
statusDescription: text("status_description"),
startedAt: datetime("started_at", { mode: "string", fsp: 3 }),
completedAt: datetime("completed_at", { mode: "string", fsp: 3 }),
@@ -239,6 +253,35 @@ export const mcpExportTokens = mysqlTable(
(table) => [index("mcp_export_tokens_expires_at_idx").on(table.expiresAt)],
);
export const wecomGroupChats = mysqlTable(
"wecom_group_chats",
{
chatId: varchar("chat_id", { length: 128 }).primaryKey(),
name: varchar("name", { length: 512 }).notNull().default(""),
ownerUserId: varchar("owner_user_id", { length: 128 }).notNull(),
memberCount: int("member_count").notNull().default(0),
status: int("status").notNull().default(0),
syncedAt: timestamp("synced_at"),
},
(table) => [index("wecom_group_chats_owner_idx").on(table.ownerUserId)],
);
export const wecomGroupPushes = mysqlTable(
"wecom_group_pushes",
{
id: varchar("id", { length: 64 }).primaryKey(),
taskId: varchar("task_id", { length: 64 }).notNull(),
msgid: varchar("msgid", { length: 128 }).notNull().default(""),
sender: varchar("sender", { length: 128 }).notNull(),
chatIds: text("chat_ids").notNull(),
textContent: text("text_content").notNull(),
linkUrl: text("link_url").notNull(),
failList: text("fail_list").notNull(),
createdAt: timestamp("created_at"),
},
(table) => [index("wecom_group_pushes_task_idx").on(table.taskId)],
);
export const backgroundJobs = mysqlTable(
"background_jobs",
{

View File

@@ -2,7 +2,8 @@ server {
listen 80;
server_name _;
client_max_body_size 10m;
# 批量回填 Excel 会内嵌多篇笔记原图和截图。
client_max_body_size 85m;
location = /koc {
return 301 /koc/$is_args$args;
@@ -21,7 +22,8 @@ server {
proxy_buffering off;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
proxy_set_header Host $host;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
@@ -30,7 +32,8 @@ server {
location / {
proxy_pass http://app:3000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

BIN
design-qa-comparison.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 326 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 397 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 376 KiB

BIN
design-qa-inline-filter.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 633 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 228 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 280 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

View File

@@ -45,3 +45,249 @@
3. 重新构建后,浏览器识别到 1 条可点击标题;点击实际打开对应小红书页面,未回填行仍不可点击。
final result: passed
---
# KOC LOOP 任务筛选常驻搜索框设计 QA
## 验证对象
- 用户目标截图:`/var/folders/0g/8yrmts9s7v5g_xc60sxx0__80000gn/T/codex-clipboard-67b0b080-8ea8-4504-82eb-348e8cf4ec19.png`
- 浏览器实现截图:`/private/tmp/koc-distributions-direct-search.png`
- CSS 视口842 × 778设备像素比 2
- 本地页面:`http://localhost:8080/?nav=distributions`
## 调整结果
1. 品牌/项目、内容类型、平台不再使用“先点下拉、再输入搜索”的两层结构,筛选栏直接展示三个可输入的搜索框。
2. 输入内容时立即进行模糊筛选;聚焦或输入后,下方仅展示匹配候选项,不再重复显示第二个搜索框。
3. 宽屏下四个搜索框保持同一行;当前窄视口下任务名称独占一行,三个分类搜索保持在下一行,不互相遮挡。
4. 保留键盘与无障碍语义,三个分类搜索均使用 `combobox`,候选项使用 `listbox` / `option`
## 功能验证
- 默认展示 6 个任务。
- 品牌/项目直接输入“美团”后,候选项仅显示“美团”和“美团医美”,任务结果同步缩小为 2 个。
- 清空输入后恢复 6 个任务。
- 静态验收测试 14 项全部通过,正式构建通过;本地 Docker 应用已重新构建并加载新页面。
final result: passed
---
# KOC LOOP 任务筛选下拉遮挡设计 QA
## 验证对象
- 用户问题截图:`/var/folders/0g/8yrmts9s7v5g_xc60sxx0__80000gn/T/codex-clipboard-bd2d14b7-472f-43a5-b997-1b6ff9528ba1.png`
- 浏览器实现截图:`/Users/wufp/Documents/koc-loop/design-qa-inline-filter.png`
- 修复前后并排对照:`/Users/wufp/Documents/koc-loop/design-qa-inline-filter-comparison.png`
- CSS 视口1280 × 720设备像素比 2
- 本地页面:`http://localhost:8080/?nav=distributions`
## 问题与调整
1. 之前仍使用浏览器原生 `select`,系统级下拉层会悬浮在页面上,因此即使搜索框本身布局正常,展开菜单仍会遮住搜索区或任务卡片。
2. 品牌/项目、内容类型、平台三个筛选器改为页面内选择面板,面板使用正常文档流布局,展开时把任务卡片向下推。
3. 同一时间只展开一个筛选器;选择选项后自动收起,并保留键盘焦点、`aria-expanded``listbox``option` 语义。
## 布局与功能验证
- 内容类型面板展开后:面板 `position: static`,搜索框底部为 379px面板顶部为 388px两者无重叠。
- 第一张任务卡片顶部为 480px面板底部为 452px任务卡片位于面板下方未被覆盖。
- 选择“视频”后面板自动收起,任务数由 6 个缩小为 1 个;清空筛选后恢复 6 个任务。
- 浏览器实测搜索框、三个筛选按钮和任务卡片均可正常操作,交互过程中未出现页面报错。
- TypeScript 构建、静态验收及共 65 项测试全部通过;本地 Docker 服务已重新构建并通过健康检查。
final result: passed
---
# KOC LOOP 任务分发筛选栏设计 QA
## 验证对象
- 用户问题截图:`/var/folders/0g/8yrmts9s7v5g_xc60sxx0__80000gn/T/codex-clipboard-4c32c7e3-58e0-4262-b625-c02b20ec4490.png`
- 浏览器实现截图:`/Users/wufp/Documents/koc-loop/design-qa-implementation.png`
- 修复前后并排对照:`/Users/wufp/Documents/koc-loop/design-qa-comparison.png`
- CSS 视口1280 × 720
- 本地页面:`http://localhost:8080/?nav=distributions`
## 问题与调整
1. 全局 `label` 布局把搜索图标和输入框纵向堆叠,导致搜索框被拆成上下两层。
2. 搜索框现在显式使用横向布局,并清除继承的外边距;图标使用固定宽度居中对齐。
3. 品牌/项目下拉框加宽,选中值和下拉菜单锚点保持稳定,不再挤压搜索区域。
## 功能验证
- 输入“视频”后,结果从 6 个任务缩小为 1 个任务,筛选栏没有发生错位。
- 清空搜索后恢复展示 6 个任务。
- 浏览器控制台无错误。
- TypeScript、静态验收测试和正式构建均通过。
- 本地 Docker 服务已重建并通过健康检查。
final result: passed
---
# KOC LOOP 可搜索任务筛选浮层设计 QA
## 验证对象
- 参考截图:`/var/folders/0g/8yrmts9s7v5g_xc60sxx0__80000gn/T/codex-clipboard-fc7ce949-d8ea-4a3b-afa6-04a778fdb8dc.png`
- 浏览器实现截图:`/Users/wufp/Documents/koc-loop/design-qa-current-filter.png`
- 参考与实现并排对照:`/Users/wufp/Documents/koc-loop/design-qa-filter-comparison.png`
- 本地页面:`http://localhost:8080/?nav=distributions`
## 布局与交互验证
- 品牌/项目、内容类型、平台均改为紧凑的页面内浮层,不再使用系统原生下拉,也不会把任务卡片整体向下推。
- 浮层锚定在当前筛选按钮下方,宽度与筛选控件协调,未遮挡任务名称搜索框及其他筛选按钮。
- 浮层顶部支持输入模糊查询;输入“医”后只显示“美团医美”,可继续点击完成筛选。
- 同一时间只展开一个筛选器,支持点击页面其他位置或按 Esc 收起。
- 浏览器实测筛选栏、任务卡片和按钮均保持稳定,没有布局跳动。
## 数据修复验证
- 抖音作品重新采集后获得点赞 5,684、收藏 565、转发 6,878、评论 332。
- 抖音主页链接改用短链跳转携带的真实 `sec_uid`,不再用数字内部用户 ID 拼接主页。
- MCP 的公开主页解析对当前账号仍返回资源不存在,因此公开抖音号、粉丝数和 IP 暂时保持待识别,不再写入错误数据。
- 相关自动化测试共 67 项全部通过,本地 Docker 服务已重建并通过健康检查。
final result: passed
# KOC LOOP 任务分发横向搜索框设计 QA
## 验证对象
- 用户问题截图:`codex-clipboard-252bf483-c3d1-4d70-abfb-d54587a300a2.png`
- 浏览器实现截图:`/private/tmp/koc-loop-distribution-filters-fixed.png`
- 本地页面:`http://localhost:8080/?nav=distributions`
## 问题与调整
1. 品牌/项目、内容类型、平台三个筛选框使用通用表单标签,继承了纵向排列样式,导致图标和提示文字上下分离。
2. 筛选框改为独立容器,并显式使用横向排列和垂直居中。
3. 保留原有模糊搜索、筛选逻辑及整体视觉规范。
## 布局与功能验证
- 三个筛选框内图标、输入文字与容器的垂直中心误差均为 0px。
- 四个搜索框保持同排展示,无相互遮挡、无异常换行、无额外浮层。
- 项目构建通过;页面渲染测试 14/14 通过;本地 Docker 运行版本已更新。
最终结果:通过。
---
# KOC LOOP KOC资源卡片密度优化设计 QA
## 验证对象
- 参考卡片:`/var/folders/0g/8yrmts9s7v5g_xc60sxx0__80000gn/T/codex-clipboard-615400a6-0bd1-4110-b280-f7fdc29bf705.png`
- 原页面参考:`/var/folders/0g/8yrmts9s7v5g_xc60sxx0__80000gn/T/codex-clipboard-5bf2bfb5-a50e-4c08-a407-192fdcf20b58.png`
- 第一轮实现截图:`/Users/wufp/Documents/koc-loop/design-qa-resource-cards-v2.png`
- 最终浏览器截图:`/Users/wufp/Documents/koc-loop/design-qa-resource-cards-final.png`
- 聚焦卡片并排对照:`/Users/wufp/Documents/koc-loop/design-qa-resource-card-comparison.png`
- 本地页面:`http://localhost:8080/`KOC资源状态
## 环境与归一化
- CSS 视口1280 × 720设备像素比 2浏览器截图按 1280 × 720 CSS 像素输出。
- 参考卡片像素478 × 700最终完整页面截图1280 × 720。
- 聚焦对照将实现中的 304px 宽卡片等比放大到 478px并与参考卡片并排查看没有把两张独立截图当作同一对比证据。
- 桌面状态为三列卡片;同时验证 960px 两列、640px 一列。
## 完整画面对比
- 信息架构采用参考图的“头像 → 身份信息 → 简介 → 标签”顺序,同时保留 KOC LOOP 原有的粉丝、合作发布、合作来源和主页入口。
- 卡片高度由旧版的大块分区缩短到首屏实测约 274—278px三列宽度均为 304px页面没有横向溢出。
- 小红书使用粉红顶部识别线和柔和粉色标签,抖音使用深色顶部识别线,继续沿用现有平台标识。
## 聚焦区域检查
- 字体与层级:昵称 14px 并提高字重;账号号、属地、标签和数据从第一轮偏小的 8px 提升至 9—11px信息仍紧凑但可读性更好。
- 间距与布局:圆形头像、昵称与性别保持同一视觉组;账号号、平台和属地收进头像右侧;粉丝、合作发布、最近合作合并为一条浅底数据栏。
- 颜色与视觉标记:标签颜色与平台呼应,状态对比足够;卡片 hover 只使用轻微位移和阴影,不改变布局。
- 图片与资产:当前账号数据没有头像 URL因此保留现有首字母头像作为明确的数据缺失状态没有伪造真人头像平台标识继续使用项目已有资产。
- 文案与内容:真实简介最多展示三行;“未填写简介”“还没有简介”等占位内容统一折叠为“暂无简介”;无标签显示“待打标”;合作来源和主页入口合并到底部同一行。
## 交互与响应式验证
- 输入标签“美食探店”后,结果正确缩小为 3 个账号;清空后恢复完整列表。
- 主页入口保持可见并保留现有跳转目标;首屏六张卡片均检测到有效入口文案。
- 960px 视口为两列640px 视口为一列,两个断点均无横向溢出。
- 浏览器控制台无 error本地应用、MySQL、Nginx 均正常运行。
- 正式构建及完整自动化测试通过,共 82 项,无失败。
## 迭代记录
1. 第一轮已完成资料卡层级和高度压缩,但聚焦截图显示辅助文字偏小,无简介账号的底部留白仍较明显,记为 P2。
2. 第二轮提高辅助文字字号,移除固定最小高度,并把合作来源与主页入口合并为同一底栏。
3. 复查后卡片高度稳定在约 274—278px关键内容可读桌面与移动断点无溢出先前 P2 已解决。
## 结论
- 没有遗留 P0、P1 或 P2 问题。
- P3 后续项:如果 MCP 未来提供可靠头像 URL可将首字母头像替换成真实头像进一步接近参考图。
final result: passed
---
# KOC LOOP KOC资源卡片底栏对齐设计 QA
## 验证对象
- 用户标注截图:`/var/folders/0g/8yrmts9s7v5g_xc60sxx0__80000gn/T/codex-clipboard-fa2c031d-52ba-4ef5-bd70-2bf74adfa52d.png`
- 最终浏览器截图:`/Users/wufp/Documents/koc-loop/design-qa-resource-card-alignment-final.png`
- 归一化并排对照:`/Users/wufp/Documents/koc-loop/design-qa-resource-card-alignment-comparison.png`
- 本地页面:`http://localhost:8080/?nav=resources`
## 问题与调整
1. 不同账号的简介和标签数量不一致时,合作来源与主页入口会跟随前方内容上下浮动,导致同一排卡片的底部结构不齐。
2. 卡片保持纵向弹性布局,底栏改为自动占用剩余空间并贴住卡片底部;不增加固定高度,不改变数据或交互逻辑。
3. 三列桌面视口下实测前三张卡片高度均为 267.9px,底栏顶部均为 333.4px,底栏底部均为 365.9px,误差为 0px。
## 验证结果
- CSS 视口1280 × 720三列卡片状态。
- 不同简介、标签数量下,合作来源、来源标签和主页入口保持同一条水平基线。
- 浏览器控制台无 error页面 hover 位移不会改变静止状态的布局基线。
- 页面渲染测试与 TypeScript 检查通过;本地 Docker 已重建。
final result: passed
---
# KOC LOOP KOC资源卡片数据栏对齐设计 QA
## 验证对象
- 用户标注截图:`/var/folders/0g/8yrmts9s7v5g_xc60sxx0__80000gn/T/codex-clipboard-78335120-710a-4a7b-96f9-b1046191c788.png`
- 双列实现截图:`/Users/wufp/Documents/koc-loop/design-qa-resource-card-metrics-alignment-two-column.png`
- 三列实现截图:`/Users/wufp/Documents/koc-loop/design-qa-resource-card-metrics-alignment-final.png`
- 归一化并排对照:`/Users/wufp/Documents/koc-loop/design-qa-resource-card-metrics-alignment-comparison.png`
- 本地页面:`http://localhost:8080/?nav=resources`
## 环境与归一化
- 用户截图为 1674 × 1180px双列实现截图为 837 × 591px。
- 并排对照将用户截图归一化为 837 × 591px与实现截图使用同一双列宽度和页面状态进行聚焦比较。
- 同时在 1280 × 720 三列桌面视口复测,确认调整不依赖双列断点。
## 问题与调整
1. 先前仅把合作来源底栏贴底,数据栏仍跟随简介和标签数量上下浮动,形成 P2 对齐问题。
2. 将弹性留白移动到标签区之后,数据栏与合作来源底栏作为一个完整结构贴住卡片底部;数据内容、标签和交互均未改动。
## 验证结果
- 双列第一排两张卡片的数据栏顶部均为 233.4px、底部均为 267.9px;底栏顶部均为 277.9px、底部均为 310.4px。
- 双列第二排两张卡片的数据栏顶部均为 525.3px、底部均为 559.8px;底栏顶部均为 569.8px、底部均为 602.3px。
- 三列前三张卡片的数据栏与底栏上下边界误差均为 0px。
- 字体、颜色、图片资产和文案未发生变化;浏览器控制台无 error。
- 页面渲染测试 14/14、TypeScript 检查和 Docker 构建均通过。
final result: passed

32
dev.vars_exp Normal file
View File

@@ -0,0 +1,32 @@
KOC_PORTAL_URL=http://localhost:3000
ADMIN_ALLOWED_EMAIL=operator@example.com
ADMIN_INTERNAL_TOKEN=replace-with-a-random-secret
KOC_DEV=1
# Optional override. The production key must be stored as a runtime secret.
AI_TOOL_CENTER_MCP_URL=https://middle-aitool.gbotai.cn/mcp
AI_TOOL_CENTER_MCP_KEY=replace-with-mcp-key
# Feishu custom app credentials. Keep the secret out of source control.
FEISHU_APP_ID=cli_xxxxxxxxxxxxxxxxx
FEISHU_APP_SECRET=replace-with-feishu-app-secret
# 企业微信 · 全部选填;不填则临期催办与汇总都跳过,不影响其他功能
# 管理后台 https://work.weixin.qq.com →「我的企业 → 企业信息 → 企业 ID」
WECOM_CORP_ID=
# 管理后台 →「应用管理 → 自建」→ 创建应用「KOC LOOP 催办」后,应用详情页顶部显示
WECOM_AGENT_ID=
# 同一应用详情页 →「Secret」→ 点「发送」→ 企业微信里收到 64 位字符串
# 注意Secret 只能发送一次,收到立刻保存;丢了只能点「重置 Secret」重新生成
WECOM_SECRET=
# 测试群 → 群设置 → 群机器人 → 添加机器人 → 完整 webhook URL
# 形如 https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxxxxxxx
WECOM_ROBOT_WEBHOOK=
# 临期阈值,默认 3超过则不在催办。取值 1-30
WECOM_NOTIFY_DUE_DAYS=3
# 企业微信通知总开关,默认 true临时关闭设为 false
WECOM_NOTIFY_ENABLED=true
# 本地测试:让 /api/bootstrap 自动 seed 3 个测试 partner + 5 条待发布 distribution
# 上线后改为 false
SEED_DEMO_DATA=true

View File

@@ -1,6 +1,6 @@
# KOC LOOP 私有化部署指南
本文适用于 `codex/self-hosted-mysql` 分支。目标架构是运维提出的Nginx 代理 + KOC 服务 + MySQL 数据库,并让后台、外部 KOC 领取页和 Agent MCP 都能通过一个公网域名访问。
本文适用于 `main` 分支。目标架构是运维提出的Nginx 代理 + KOC 服务 + MySQL 数据库,并让后台、外部 KOC 领取页和 Agent MCP 都能通过一个公网域名访问。
## 1. 部署形态
@@ -35,7 +35,7 @@
```bash
git clone ssh://git@gta.gbotai.cn:42001/wufengping/koc-loop.git
cd koc-loop
git checkout codex/self-hosted-mysql
git checkout main
cp .env.self-hosted.example .env.self-hosted
```
@@ -54,9 +54,17 @@ cp .env.self-hosted.example .env.self-hosted
| `FEISHU_APP_ID` / `FEISHU_APP_SECRET` | 读取飞书内容表和配图 |
| `AI_TOOL_CENTER_MCP_URL` / `AI_TOOL_CENTER_MCP_KEY` | 小红书公开数据采集服务 |
| `ENABLE_SCHEDULER` | 是否启用每天 09:00 自动采集,生产保持 `true` |
| `WECOM_CORP_ID` / `WECOM_AGENT_ID` / `WECOM_SECRET` | 企业微信自建应用,用于临期催办的应用消息推送;三项需同时填写,全部留空则只走群机器人 |
| `WECOM_ROBOT_WEBHOOK` | 企业微信群机器人 webhook用于当日催办汇总 |
| `WECOM_NOTIFY_DUE_DAYS` | 临期阈值,默认 3超过则不在催办可填 130 |
| `WECOM_NOTIFY_ENABLED` | 企业微信通知总开关,默认 `true`;临时关闭设为 `false` |
企业微信变量全部选填:不填则临期催办与汇总都跳过,不影响其他功能。所有变量都可以在「合作方」页面顶部「企业微信连通性」面板查看就绪状态,并用「测试群机器人」「测试发送」按钮验证。
密钥必须由密码管理器生成,禁止提交到 Git、聊天、部署日志或 URL。三个业务密钥 `ADMIN_INTERNAL_TOKEN``KOC_MCP_API_KEY``AI_TOOL_CENTER_MCP_KEY` 不得复用。
`APP_ORIGIN` 必须填写用户实际访问的 HTTPS 公网地址,不能填写 `localhost``app:3000` 或其他容器内部地址。Excel 中的视频下载链接会优先使用这个地址;前置网关还必须把原始 `Host``X-Forwarded-Host``X-Forwarded-Proto` 传给仓库内的 Nginx。
## 4. 首次启动
```bash
@@ -88,7 +96,9 @@ curl -fsS http://127.0.0.1:${HTTP_PORT:-80}/api/health
3. 80 端口只做 301 跳转;
4. 保留 `/koc/` 静态规则、`/api/mcp` 长连接规则和 `/` 反向代理规则。
MCP 路由已经关闭代理缓冲并将超时时间延长到 1 小时,避免 Agent 的长调用被 Nginx 提前截断。
MCP 路由已经关闭代理缓冲并将超时时间延长到 1 小时,避免 Agent 的长调用被 Nginx 提前截断。仓库内 Nginx 同时将上传限制设为 85 MB用于接收最多 80 MB 的批量回填 Excel公司网关或负载均衡的请求体限制也必须不低于 85 MB。
视频下载接口必须经过 `/api/partner-image` 反向代理,正常响应应包含 `Content-Type: video/mp4` 和带 `.mp4` 文件名的 `Content-Disposition: attachment`。不要在网关层改写该响应类型或移除附件响应头。
## 6. 迁移原 Sites 数据
@@ -124,12 +134,13 @@ npm run db:import-json -- /backup/koc-d1-export.json
导入脚本按业务依赖顺序写入,并使用主键/唯一键安全更新已有记录。正式迁移前先在测试库演练并核对任务数、笔记数、领取数、发布数和账号数。
### 6.2 R2 图片导入
### 6.2 R2 媒体文件导入
把 R2 按原对象 key 导出到一个目录,目录层级必须保留,例如:
```text
content-assets/...
content-videos/...
publish-evidence/...
creator-center/...
```
@@ -141,7 +152,7 @@ UPLOAD_DIR=/data/koc/uploads \
npm run storage:import -- /backup/koc-r2-export
```
脚本会复制文件,并为缺少元数据的图片生成 Content-Type 元数据。容器部署时也可以在宿主机临时挂载 `upload_data` 卷后执行。
脚本会复制文件,并为缺少元数据的媒体文件生成 Content-Type 元数据。容器部署时也可以在宿主机临时挂载 `upload_data` 卷后执行。
## 7. 上线验收
@@ -155,9 +166,11 @@ UPLOAD_DIR=/data/koc/uploads \
6. 上传创作者截图并填写曝光量、阅读量;
7. 后台立即采集一篇笔记成功;
8. 保存次日采集计划,确认数据库产生 `collection_runs`
9. 导出的 Excel 内能直接看到原图和截图;
10. Agent 用 `KOC_MCP_API_KEY` 调用 `/api/mcp` 能发现全部工具
11. 重启全部容器后数据与图片不丢失。
9. 图文任务导出的 Excel 内能直接看到原图和截图;
10. 视频任务导出的 Excel 不含“图片”列,包含“视频”列,点击链接能下载扩展名为 `.mp4` 且可正常播放的文件
11. 批量回填 Excel 可以上传,发布链接、笔记截图和单篇笔记数据分析截图均能正确回写;
12. Agent 用 `KOC_MCP_API_KEY` 调用 `/api/mcp` 能发现全部工具;
13. 重启全部容器后数据、图片和视频不丢失。
## 8. 备份与恢复
@@ -180,6 +193,8 @@ docker compose --env-file .env.self-hosted \
-f docker-compose.self-hosted.yml up -d --build
```
应用容器每次启动都会按文件名顺序执行尚未应用的 `mysql/*.sql`。本次版本包含平台/视频字段、账号性别/简介/标签以及“当前联系人”字段的增量迁移;升级后应检查容器日志确认 `0005``0006``0007` 已执行或已被识别为历史迁移。
数据库迁移只允许向前追加新的 `mysql/*.sql` 文件,禁止修改已经在生产执行过的迁移。应用回滚到旧镜像前,要确认旧代码兼容当前数据库结构;涉及不可逆结构变化时,必须同时准备数据库恢复方案。
## 10. 运维排查
@@ -189,8 +204,65 @@ docker compose --env-file .env.self-hosted \
| `/api/health` 返回 503 | 查看 MySQL 容器健康状态与应用数据库变量 |
| 登录页可开但登录失败 | 确认迁移完成、超级管理员变量仅用于初始化 |
| 配图或截图 404 | 检查 `upload_data` 卷和 `UPLOAD_DIR=/data/koc/uploads` |
| Excel 视频链接出现 localhost 或无法访问 | 检查 `APP_ORIGIN`、公网域名和网关转发的 Host/Proto 请求头 |
| 视频下载后不是 MP4 或无法播放 | 检查 `/api/partner-image` 是否经过应用代理、文件是否完整,以及网关是否保留 Content-Type/Content-Disposition |
| 批量回填表上传返回 413 | 将公司网关、负载均衡和 Nginx 的请求体限制统一提高到至少 85 MB |
| 09:00 未自动采集 | 检查 `ENABLE_SCHEDULER=true`、服务器日志和采集 MCP 网络 |
| MCP 401 | 检查请求头是否为 `Authorization: Bearer <KOC_MCP_API_KEY>` |
| 飞书读取失败 | 检查应用权限、文档授权和服务器到飞书 OpenAPI 的网络 |
生产日志不得打印数据库密码、飞书 Secret、MCP key 或完整带 key 的采集服务 URL。
## 11. 企业微信自建应用与群机器人配置
KOC LOOP 的企业微信通知是**单向外发**:每天 09:00 与定时采集一同触发,给合作方绑定的外部联系人推送临期催办,再向群机器人发送当日汇总。不需要 OAuth 回调,也不需要拉通讯录。
### 11.1 群机器人(用于当日催办汇总)
1. 在企业微信里建立一个用于催办的群;
2. 群设置 → 群机器人 → 添加机器人 → 复制 webhook 地址,形如 `https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=...`
3. 把整个 webhook 写入 `WECOM_ROBOT_WEBHOOK`
只要这一项就绪,每天 09:00 的催办汇总就会推到该群。
### 11.2 自建应用(用于给单个合作方发应用消息)
1. 登录企业微信管理后台 → 应用管理 → 自建 → 创建应用,名称建议 `KOC LOOP 催办`
2. 应用详情页记下 `AgentId`,填入 `WECOM_AGENT_ID`
3. 我的企业 → 企业信息 → 企业 ID填入 `WECOM_CORP_ID`
4. 应用详情页 → Secret → 发送 Secret 到管理端,填入 `WECOM_SECRET`
5. 应用「可见范围」必须包含合作方对应的企业微信成员,否则推送时会返回 `invaliduser`
### 11.3 绑定外部联系人
企业微信应用消息推送需要 `external_userid`(外部联系人 IDKOC LOOP 不自动拉取,需要手动绑定:
1. 在企业微信「客户联系 → 外部联系人」里找到合作方对应的客户;
2. 复制其 external_userid形如 `woxxxxxx`
3. 在 KOC LOOP 后台「合作方」页面,找到对应合作方行,点「绑定」粘贴 ID → 「保存」;
4. 点「测试发送」验证;如果返回 `应用消息发送失败`,多半是应用可见范围未包含该外部联系人,或 external_userid 复制错了。
外部联系人 ID 不入库后不会自动同步企业微信端的变更;如果合作方的外部联系人在企业微信里被删除或转移,需要回来更新这一列。
### 11.4 验证
部署后到后台「合作方」页面,顶部「企业微信连通性」面板应显示三项就绪状态:群机器人、自建应用消息、临期阈值。逐项点「测试」:
- 「测试群机器人」:群内收到 `[KOC LOOP 测试] 群机器人连通性正常,时间 ...` 即配置成功;
- 合作方行的「测试发送」:对应外部联系人收到同样格式的测试消息即绑定正确。
如果面板显示「未配置」但已填环境变量,先确认容器加载了新的 `.env.self-hosted`(重启服务),再回到该页面点「刷新状态」。
### 11.5 任务发布到企微客户群(企业群发)
平台支持把任务以「企业群发」方式推送到多个外部客户群。**企业微信不允许外部群添加群机器人 webhook**,官方路径是半自动群发:平台创建群发任务 → 各群的群主在企业微信客户端「群发助手」里点击发送 → 消息才送达客户群。依赖 11.2 的同一个自建应用,无需新增环境变量;数据库表 `wecom_group_chats` / `wecom_group_pushes` 由迁移 `mysql/0009_wecom_group_push.sql` 创建(`npm run db:migrate` 自动执行)。
使用前需在企业微信管理后台完成三项配置:
1. **客户联系 → 配置 → 可调用应用**:把该自建应用加入可调用列表,否则客户群相关接口会报权限错误;
2. **应用可见范围**:必须包含所有目标群的群主(群主是群发任务的确认人);
3. **应用可信 IP**(应用详情页 → 开发者接口 → 企业可信 IP必须包含服务器出口 IP否则接口返回 `60020 not allow to access from your ip`
使用流程:后台「任务中心」→ 任务行「发布到企微群」→ 弹窗内「同步群列表」(拉取客户联系下的正常状态客户群)→ 勾选目标群、确认文案 → 「创建群发」。创建成功后平台返回 msgid 并落库 `wecom_group_pushes`,各群主在企微客户端收到群发助手提醒,**点击发送后**消息(含 KOC 领取链接)才出现在群里。
限制与注意:每个客户群每月最多接收「当月天数」条企业群发;群主使用的企业微信客户端需 ≥4.1.10 才能免选群直接发送;无效的 chat_id 会进入失败列表但不影响其他群。

View File

@@ -1,4 +1,6 @@
# KOC LOOP 部署指南
# KOC LOOP Sites 旧版部署指南
> 此文档仅适用于历史 `codex/sites-release-controls` 分支。`main` 已切换为 Next.js + MySQL + Nginx 私有化部署,正式部署请使用 [KOC LOOP 私有化部署指南](KOC%20LOOP%20私有化部署指南.md),不要按本文把 `main` 发布到 Sites。
KOC LOOP 由两个独立站点组成:

View File

@@ -0,0 +1,341 @@
import { strFromU8, unzipSync, zipSync } from "fflate";
export const PARTNER_BATCH_UPLOAD_MAX_BYTES = 80_000_000;
type WorkbookCell = {
reference: string;
row: number;
column: number;
attributes: string;
body: string;
value: string;
};
type ScreenshotColumns = {
headerRow: number;
columns: Set<number>;
};
function decodeXml(value: string) {
return value
.replace(/<[^>]+>/g, "")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&apos;/g, "'")
.replace(/&amp;/g, "&")
.replace(/&#(\d+);/g, (_, code) => String.fromCodePoint(Number(code)))
.replace(/&#x([0-9a-f]+);/gi, (_, code) =>
String.fromCodePoint(Number.parseInt(code, 16)),
);
}
function textNodes(xml: string) {
return [...xml.matchAll(/<t\b[^>]*>([\s\S]*?)<\/t>/g)]
.map((match) => decodeXml(match[1]))
.join("");
}
function columnIndex(reference: string) {
const letters = reference.match(/^[A-Z]+/i)?.[0]?.toUpperCase() ?? "";
let result = 0;
for (const letter of letters) result = result * 26 + letter.charCodeAt(0) - 64;
return Math.max(0, result - 1);
}
function normalizeHeader(value: string) {
return value.replace(/[\s_\-()]/g, "").toLocaleLowerCase("zh-CN");
}
function parseCells(worksheetXml: string, sharedStrings: string[]) {
const cells: WorkbookCell[] = [];
for (const match of worksheetXml.matchAll(/<c\b([^>]*)>([\s\S]*?)<\/c>/g)) {
const attributes = match[1];
const body = match[2];
const reference = attributes.match(/\br="([A-Z]+\d+)"/i)?.[1] ?? "";
if (!reference) continue;
const type = attributes.match(/\bt="([^"]+)"/)?.[1] ?? "";
const rawValue = body.match(/<v>([\s\S]*?)<\/v>/)?.[1] ?? "";
const value =
type === "s"
? sharedStrings[Number(rawValue)] ?? ""
: type === "inlineStr"
? textNodes(body)
: decodeXml(rawValue);
cells.push({
reference,
row: Number(reference.match(/\d+$/)?.[0] ?? 0),
column: columnIndex(reference),
attributes,
body,
value: value.trim(),
});
}
return cells;
}
function findScreenshotColumns(cells: WorkbookCell[]): ScreenshotColumns {
for (let row = 1; row <= 8; row += 1) {
const columns = new Set<number>();
for (const cell of cells) {
if (cell.row !== row) continue;
const header = normalizeHeader(cell.value);
if (
header === normalizeHeader("笔记截图") ||
header === normalizeHeader("发布截图") ||
header === normalizeHeader("数据分析截图") ||
header === normalizeHeader("数据分析截图(单篇笔记数据分析截图)") ||
header === normalizeHeader("创作者中心截图")
) {
columns.add(cell.column);
}
}
if (columns.size >= 2) return { headerRow: row, columns };
}
throw new Error("表格结构不正确,请使用本领取页面导出的批量回填表");
}
function relationshipMap(xml: string) {
const relationships = new Map<string, string>();
for (const match of xml.matchAll(/<Relationship\b([^>]*)\/?\s*>/g)) {
const id = match[1].match(/\bId="([^"]+)"/)?.[1] ?? "";
const target = match[1].match(/\bTarget="([^"]+)"/)?.[1] ?? "";
if (id && target) relationships.set(id, decodeXml(target));
}
return relationships;
}
function normalizeZipPath(value: string) {
const result: string[] = [];
for (const part of value.split("/")) {
if (!part || part === ".") continue;
if (part === "..") result.pop();
else result.push(part);
}
return result.join("/");
}
function resolveZipPath(base: string, target: string) {
const slash = base.lastIndexOf("/");
const directory = slash >= 0 ? base.slice(0, slash + 1) : "";
return normalizeZipPath(`${directory}${target}`);
}
function wpsScreenshotMedia(
entries: Record<string, Uint8Array>,
cells: WorkbookCell[],
screenshotColumns: ScreenshotColumns,
) {
const result = new Set<string>();
let expected = 0;
const screenshotIds = new Set<string>();
for (const cell of cells) {
if (
cell.row <= screenshotColumns.headerRow ||
!screenshotColumns.columns.has(cell.column)
) {
continue;
}
const id = decodeXml(cell.body).match(/DISPIMG\("([^"]+)"/i)?.[1];
if (id) {
expected += 1;
screenshotIds.add(id);
}
}
if (screenshotIds.size === 0) return { result, expected, resolved: 0 };
const cellImagesXml = entries["xl/cellimages.xml"]
? strFromU8(entries["xl/cellimages.xml"])
: "";
const relationships = relationshipMap(
entries["xl/_rels/cellimages.xml.rels"]
? strFromU8(entries["xl/_rels/cellimages.xml.rels"])
: "",
);
let resolved = 0;
for (const match of cellImagesXml.matchAll(
/<(?:etc:)?cellImage\b[^>]*>([\s\S]*?)<\/(?:etc:)?cellImage>/g,
)) {
const id = match[1].match(/<(?:xdr:)?cNvPr\b[^>]*\bname="([^"]+)"/)?.[1];
const relationshipId = match[1].match(
/<(?:a:)?blip\b[^>]*\br:embed="([^"]+)"/,
)?.[1];
if (!id || !relationshipId || !screenshotIds.has(id)) continue;
const target = relationships.get(relationshipId);
if (!target) continue;
result.add(resolveZipPath("xl/cellimages.xml", target));
resolved += 1;
}
return { result, expected, resolved };
}
function drawingScreenshotMedia(
entries: Record<string, Uint8Array>,
screenshotColumns: ScreenshotColumns,
) {
const result = new Set<string>();
let expected = 0;
let resolved = 0;
const worksheetXml = entries["xl/worksheets/sheet1.xml"]
? strFromU8(entries["xl/worksheets/sheet1.xml"])
: "";
const sheetRelationships = relationshipMap(
entries["xl/worksheets/_rels/sheet1.xml.rels"]
? strFromU8(entries["xl/worksheets/_rels/sheet1.xml.rels"])
: "",
);
const drawingId = worksheetXml.match(/<drawing\b[^>]*r:id="([^"]+)"/)?.[1];
const drawingTarget = drawingId ? sheetRelationships.get(drawingId) : undefined;
if (!drawingTarget) return { result, expected, resolved };
const drawingPath = resolveZipPath("xl/worksheets/sheet1.xml", drawingTarget);
const drawingXml = entries[drawingPath] ? strFromU8(entries[drawingPath]) : "";
const relationshipPath = `${drawingPath.slice(0, drawingPath.lastIndexOf("/") + 1)}_rels/${drawingPath.slice(drawingPath.lastIndexOf("/") + 1)}.rels`;
const drawingRelationships = relationshipMap(
entries[relationshipPath] ? strFromU8(entries[relationshipPath]) : "",
);
for (const anchor of drawingXml.matchAll(
/<xdr:(?:oneCellAnchor|twoCellAnchor)\b[^>]*>[\s\S]*?<xdr:from>([\s\S]*?)<\/xdr:from>[\s\S]*?<a:blip\b[^>]*r:embed="([^"]+)"[\s\S]*?<\/xdr:(?:oneCellAnchor|twoCellAnchor)>/g,
)) {
const column = Number(anchor[1].match(/<xdr:col>(\d+)<\/xdr:col>/)?.[1]);
const zeroBasedRow = Number(anchor[1].match(/<xdr:row>(\d+)<\/xdr:row>/)?.[1]);
if (
!Number.isInteger(column) ||
!Number.isInteger(zeroBasedRow) ||
zeroBasedRow + 1 <= screenshotColumns.headerRow ||
!screenshotColumns.columns.has(column)
) {
continue;
}
expected += 1;
const target = drawingRelationships.get(anchor[2]);
if (!target) continue;
result.add(resolveZipPath(drawingPath, target));
resolved += 1;
}
return { result, expected, resolved };
}
function richValueScreenshotMedia(
entries: Record<string, Uint8Array>,
cells: WorkbookCell[],
screenshotColumns: ScreenshotColumns,
) {
const result = new Set<string>();
let expected = 0;
let resolved = 0;
const metadataXml = entries["xl/metadata.xml"]
? strFromU8(entries["xl/metadata.xml"])
: "";
const richValueXml = entries["xl/richData/rdrichvalue.xml"]
? strFromU8(entries["xl/richData/rdrichvalue.xml"])
: "";
const richValueRelXml = entries["xl/richData/richValueRel.xml"]
? strFromU8(entries["xl/richData/richValueRel.xml"])
: "";
const relationships = relationshipMap(
entries["xl/richData/_rels/richValueRel.xml.rels"]
? strFromU8(entries["xl/richData/_rels/richValueRel.xml.rels"])
: "",
);
if (!metadataXml || !richValueXml || !richValueRelXml || !relationships.size) {
return { result, expected, resolved };
}
const valueMetadataXml =
metadataXml.match(/<valueMetadata\b[^>]*>([\s\S]*?)<\/valueMetadata>/)?.[1] ??
"";
const metadataToRichValue = [
...valueMetadataXml.matchAll(
/<bk\b[^>]*>[\s\S]*?<rc\b[^>]*\bv="(\d+)"[^>]*\/>[\s\S]*?<\/bk>/g,
),
].map((match) => Number(match[1]));
const richValueToRelationship = [
...richValueXml.matchAll(/<rv\b[^>]*>([\s\S]*?)<\/rv>/g),
].map((match) => Number(match[1].match(/<v>(\d+)<\/v>/)?.[1] ?? -1));
const relationshipIds = [
...richValueRelXml.matchAll(/<rel\b[^>]*\br:id="([^"]+)"[^>]*\/>/g),
].map((match) => match[1]);
for (const cell of cells) {
if (
cell.row <= screenshotColumns.headerRow ||
!screenshotColumns.columns.has(cell.column)
) {
continue;
}
const metadataIndex = Number(cell.attributes.match(/\bvm="(\d+)"/)?.[1] ?? 0);
if (!metadataIndex) continue;
expected += 1;
const richValueIndex = metadataToRichValue[metadataIndex - 1];
const relationshipIndex = richValueToRelationship[richValueIndex];
const relationshipId = relationshipIds[relationshipIndex];
const target = relationships.get(relationshipId);
if (!target) continue;
result.add(resolveZipPath("xl/richData/richValueRel.xml", target));
resolved += 1;
}
return { result, expected, resolved };
}
export type CompactedPartnerBatchWorkbook = {
bytes: Uint8Array;
removedMediaCount: number;
preservedScreenshotCount: number;
};
/**
* Oversized exports are usually caused by full-resolution source images. The
* upload only needs the two screenshot columns, so retain those image entries
* and omit source media from the temporary upload copy.
*/
export function compactPartnerBatchWorkbookForUpload(
input: Uint8Array,
): CompactedPartnerBatchWorkbook {
const isMediaFile = (name: string) =>
name.startsWith("xl/media/") && !name.endsWith("/");
const structure = unzipSync(input, {
filter: (file) => !isMediaFile(file.name),
});
const worksheetXml = structure["xl/worksheets/sheet1.xml"]
? strFromU8(structure["xl/worksheets/sheet1.xml"])
: "";
if (!worksheetXml) throw new Error("Excel 中没有可读取的批量回填工作表");
const sharedXml = structure["xl/sharedStrings.xml"]
? strFromU8(structure["xl/sharedStrings.xml"])
: "";
const sharedStrings = [
...sharedXml.matchAll(/<si\b[^>]*>([\s\S]*?)<\/si>/g),
].map((match) => textNodes(match[1]));
const cells = parseCells(worksheetXml, sharedStrings);
const screenshotColumns = findScreenshotColumns(cells);
const formats = [
wpsScreenshotMedia(structure, cells, screenshotColumns),
drawingScreenshotMedia(structure, screenshotColumns),
richValueScreenshotMedia(structure, cells, screenshotColumns),
];
const screenshotMedia = new Set<string>();
let expectedScreenshotCount = 0;
let resolvedScreenshotCount = 0;
for (const format of formats) {
expectedScreenshotCount += format.expected;
resolvedScreenshotCount += format.resolved;
for (const name of format.result) screenshotMedia.add(name);
}
if (resolvedScreenshotCount < expectedScreenshotCount) {
throw new Error("表格中的回填截图无法完整识别,请重新导出最新版回填表");
}
let mediaCount = 0;
const entries = unzipSync(input, {
filter: (file) => {
if (!isMediaFile(file.name)) return true;
mediaCount += 1;
return screenshotMedia.has(file.name);
},
});
const bytes = zipSync(entries, { level: 6 });
return {
bytes,
removedMediaCount: Math.max(0, mediaCount - screenshotMedia.size),
preservedScreenshotCount: screenshotMedia.size,
};
}

View File

@@ -48,6 +48,104 @@ button:disabled {
opacity: 0.5;
}
.platform-badge {
display: inline-flex !important;
width: auto !important;
height: 24px;
align-items: center;
flex: none;
gap: 5px;
margin: 0 !important;
padding: 2px 7px 2px 3px;
border: 1px solid #e1e7e4;
border-radius: 8px;
color: #52615c !important;
background: rgb(255 255 255 / 0.92);
font-size: 9px !important;
font-weight: 720;
line-height: 1 !important;
white-space: nowrap;
}
.platform-badge.compact {
height: 19px;
gap: 4px;
padding: 2px 5px 2px 2px;
border-radius: 6px;
font-size: 8px !important;
}
.platform-logo {
display: grid !important;
width: 18px !important;
height: 18px !important;
place-items: center;
flex: none;
overflow: hidden;
margin: 0 !important;
border-radius: 5px;
line-height: 1 !important;
}
.platform-badge.compact .platform-logo {
width: 14px !important;
height: 14px !important;
border-radius: 4px;
}
.platform-logo.xiaohongshu {
color: white !important;
background: #ff2442;
}
.platform-logo.xiaohongshu b {
color: inherit;
font-size: 5px;
font-weight: 900;
letter-spacing: -0.12em;
transform: translateX(-0.2px);
}
.platform-badge.compact .platform-logo.xiaohongshu b {
font-size: 4px;
}
.platform-logo.douyin {
background: #080b12;
}
.platform-logo.douyin svg {
width: 16px;
height: 16px;
}
.platform-badge.compact .platform-logo.douyin svg {
width: 13px;
height: 13px;
}
.platform-logo.douyin .douyin-cyan { fill: #25f4ee; transform: translate(-0.7px, 0.7px); }
.platform-logo.douyin .douyin-red { fill: #fe2c55; transform: translate(0.7px, -0.3px); }
.platform-logo.douyin .douyin-white { fill: #fff; }
.platform-meta-line,
.hero-platform-line {
display: inline-flex !important;
min-width: 0;
align-items: center;
flex-wrap: wrap;
gap: 6px;
}
.platform-meta-line > span,
.hero-platform-line > * {
margin: 0 !important;
}
.hero-platform-line {
margin-bottom: 10px;
}
.portal-shell {
width: min(100%, 1120px);
min-height: 100vh;
@@ -679,6 +777,15 @@ footer {
cursor: not-allowed;
}
.batch-workbook-input {
position: absolute;
width: 1px;
height: 1px;
overflow: hidden;
opacity: 0;
pointer-events: none;
}
.share-composer {
display: grid;
grid-template-columns: minmax(0, 1fr) minmax(190px, 0.45fr) auto;
@@ -843,6 +950,34 @@ footer {
text-align: center;
}
.note-thumb.platform-video-thumb {
display: grid;
place-items: center;
}
.platform-badge.logo-only {
width: 34px !important;
height: 34px;
padding: 0;
border: 0;
background: transparent;
}
.platform-badge.logo-only .platform-logo {
width: 34px !important;
height: 34px !important;
border-radius: 9px;
}
.platform-badge.logo-only .platform-logo.douyin svg {
width: 29px;
height: 29px;
}
.platform-badge.logo-only .platform-logo.xiaohongshu b {
font-size: 8px;
}
.note-index {
display: grid;
width: 34px;
@@ -1135,12 +1270,32 @@ footer {
white-space: pre-wrap;
}
.note-images {
.note-images,
.note-videos {
margin-top: 30px;
padding-top: 24px;
border-top: 1px solid #edf0ee;
}
.note-video-grid {
display: grid;
gap: 14px;
}
.note-video-card {
overflow: hidden;
border: 1px solid #e4e9e6;
border-radius: 12px;
background: #102a22;
}
.note-video-card video {
display: block;
width: 100%;
max-height: 680px;
background: #0b1f19;
}
.note-images-heading {
display: flex;
align-items: flex-end;
@@ -1234,7 +1389,8 @@ footer {
font-weight: 650;
}
.note-image-actions button {
.note-image-actions button,
.note-image-actions a {
height: 28px;
padding: 0 10px;
border: 1px solid #cfe1d9;
@@ -1243,9 +1399,12 @@ footer {
background: #f2f8f5;
font-size: 8px;
font-weight: 680;
line-height: 26px;
text-decoration: none;
}
.note-image-actions button:hover {
.note-image-actions button:hover,
.note-image-actions a:hover {
border-color: #9fc9b8;
background: #eaf5f0;
}
@@ -1633,6 +1792,11 @@ footer {
font-weight: 620;
}
.toast.success {
color: var(--green-deep);
font-weight: 720;
}
.loading-shell {
display: flex;
align-items: center;
@@ -1782,9 +1946,14 @@ footer {
.section-actions {
width: 100%;
flex-wrap: wrap;
justify-content: space-between;
}
.section-actions > span {
margin-right: auto;
}
.share-composer {
grid-template-columns: 1fr;
}

View File

@@ -6,6 +6,10 @@ import {
formatShanghaiDate as formatDate,
parseStoredDate,
} from "./date-utils";
import {
compactPartnerBatchWorkbookForUpload,
PARTNER_BATCH_UPLOAD_MAX_BYTES,
} from "./batch-workbook-upload";
type Assignment = {
id: string;
@@ -30,6 +34,11 @@ type Assignment = {
width: number | null;
height: number | null;
}>;
videos: Array<{
index: number;
width: number | null;
height: number | null;
}>;
};
type DelegationSummary = {
@@ -63,6 +72,8 @@ type TaskPayload = {
dueAt: string;
status: string;
type: "content_publish" | "screenshot_collect";
platform: "小红书" | "抖音";
contentFormat: "image_text" | "video";
};
claim: null | {
id: string;
@@ -92,7 +103,12 @@ function resolveAdminOrigin() {
return window.location.origin;
}
function partnerApi(path: "/api/partner" | "/api/partner-upload") {
function partnerApi(
path:
| "/api/partner"
| "/api/partner-upload"
| "/api/partner-batch-workbook",
) {
return `${resolveAdminOrigin()}${path}`;
}
@@ -111,6 +127,11 @@ function statusLabel(item: Assignment, taskType = "content_publish") {
}
const MAX_TASK_RESULT_SCREENSHOTS = 9;
const PUBLISH_BACKFILL_SUCCESS = "这篇笔记的发布记录回填成功啦~";
function toastClassName(message: string) {
return message === PUBLISH_BACKFILL_SUCCESS ? "toast success" : "toast";
}
function resultScreenshotKeys(value: string | null) {
const text = String(value ?? "").trim();
@@ -155,6 +176,36 @@ function safeFileBase(item: Assignment) {
);
}
function PlatformBadge({
platform,
compact = false,
logoOnly = false,
}: {
platform: "小红书" | "抖音";
compact?: boolean;
logoOnly?: boolean;
}) {
return (
<span
className={`platform-badge ${compact ? "compact" : ""} ${logoOnly ? "logo-only" : ""}`}
aria-label={logoOnly ? platform : undefined}
>
<span className={`platform-logo ${platform === "抖音" ? "douyin" : "xiaohongshu"}`} aria-hidden="true">
{platform === "抖音" ? (
<svg viewBox="0 0 24 24" focusable="false">
<path className="douyin-cyan" d="M14.2 3.2v9.5a4.7 4.7 0 1 1-3.4-4.5v2.7a2.1 2.1 0 1 0 .8 1.7V3.2h2.6Zm0 0c.4 2.4 2 3.8 4.3 4.1V10c-1.7-.1-3.2-.7-4.3-1.7V3.2Z" />
<path className="douyin-red" d="M15.2 2.5V12a4.7 4.7 0 1 1-3.4-4.5v2.7a2.1 2.1 0 1 0 .8 1.7V2.5h2.6Zm0 0c.4 2.4 2 3.8 4.3 4.1v2.7c-1.7-.1-3.2-.7-4.3-1.7V2.5Z" />
<path className="douyin-white" d="M14.7 2.9v9.5a4.7 4.7 0 1 1-3.4-4.5v2.7a2.1 2.1 0 1 0 .8 1.7V2.9h2.6Zm0 0c.4 2.4 2 3.8 4.3 4.1v2.7c-1.7-.1-3.2-.7-4.3-1.7V2.9Z" />
</svg>
) : (
<b></b>
)}
</span>
{!logoOnly && <span>{platform}</span>}
</span>
);
}
function exactArrayBuffer(bytes: Uint8Array) {
const copy = new Uint8Array(bytes.byteLength);
copy.set(bytes);
@@ -277,6 +328,7 @@ export default function Home() {
const [adminOrigin, setAdminOrigin] = useState("");
const [downloadingImage, setDownloadingImage] = useState<number | null>(null);
const [batchDownloading, setBatchDownloading] = useState(false);
const [batchWorkbookWorking, setBatchWorkbookWorking] = useState(false);
const [creatorWorking, setCreatorWorking] = useState(false);
const [creatorStage, setCreatorStage] = useState("");
const [selectedForShare, setSelectedForShare] = useState<string[]>([]);
@@ -289,6 +341,7 @@ export default function Home() {
} | null>(null);
const [noteContentCollapsed, setNoteContentCollapsed] = useState(false);
const submitCardRef = useRef<HTMLFormElement | null>(null);
const batchWorkbookInputRef = useRef<HTMLInputElement | null>(null);
const publishScreenshotPreview = useFilePreview(screenshot);
const creatorScreenshotPreview = useFilePreview(creatorScreenshot);
const taskResultPreviews = useMemo(
@@ -504,6 +557,25 @@ export default function Home() {
return `${adminOrigin}/api/partner-image?${params}`;
};
const noteVideoUrl = (
item: Assignment,
videoIndex: number,
download = false,
) => {
const params = new URLSearchParams({
distribution: item.id,
index: String(videoIndex),
kind: "video",
});
if (download) params.set("download", "1");
if (delegationToken) params.set("share", delegationToken);
else {
params.set("task", taskToken);
params.set("claim", claimToken);
}
return `${adminOrigin}/api/partner-image?${params}`;
};
const taskResultImageUrl = (item: Assignment, imageIndex: number) => {
const params = new URLSearchParams({
distribution: item.id,
@@ -527,6 +599,11 @@ export default function Home() {
index: "1",
kind,
});
const evidenceKey =
kind === "publish"
? item.publish_screenshot_key
: item.creator_screenshot_key;
if (evidenceKey) params.set("v", evidenceKey);
if (delegationToken) params.set("share", delegationToken);
else {
params.set("task", taskToken);
@@ -644,6 +721,112 @@ export default function Home() {
}
};
const batchWorkbookParams = () => {
const params = new URLSearchParams();
if (delegationToken) params.set("share", delegationToken);
else {
params.set("task", taskToken);
params.set("claim", claimToken);
}
return params;
};
const exportBatchWorkbook = async () => {
try {
setBatchWorkbookWorking(true);
const response = await fetch(
`${partnerApi("/api/partner-batch-workbook")}?${batchWorkbookParams()}`,
{ cache: "no-store" },
);
if (!response.ok) {
const result = (await response.json()) as { error?: string };
throw new Error(result.error || "Excel导出失败");
}
const disposition = response.headers.get("Content-Disposition") || "";
const encodedName = disposition.match(/filename\*=UTF-8''([^;]+)/i)?.[1];
const fileName = encodedName
? decodeURIComponent(encodedName)
: `${payload?.task.name || "领取笔记"}-批量回填.xlsx`;
downloadBlob(await response.blob(), fileName);
setToast("Excel已导出填写后从本页面上传即可批量回填");
} catch (reason) {
setToast(reason instanceof Error ? reason.message : "Excel导出失败");
} finally {
setBatchWorkbookWorking(false);
}
};
const importBatchWorkbook = async (event: ChangeEvent<HTMLInputElement>) => {
const file = event.target.files?.[0];
event.target.value = "";
if (!file) return;
try {
setBatchWorkbookWorking(true);
let uploadBody: Blob = file;
let compacted = false;
if (file.size > PARTNER_BATCH_UPLOAD_MAX_BYTES) {
setToast("文件较大,正在保留回填截图并精简原图…");
await new Promise<void>((resolve) => requestAnimationFrame(() => resolve()));
const compactedWorkbook = compactPartnerBatchWorkbookForUpload(
new Uint8Array(await file.arrayBuffer()),
);
if (compactedWorkbook.bytes.byteLength > PARTNER_BATCH_UPLOAD_MAX_BYTES) {
throw new Error("精简后的回填表仍超过80MB请重新导出最新版回填表");
}
uploadBody = new Blob([exactArrayBuffer(compactedWorkbook.bytes)], {
type: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
});
compacted = true;
}
const form = new FormData();
form.set("file", uploadBody, file.name);
form.set("taskToken", taskToken);
form.set("claimToken", claimToken);
form.set("delegationToken", delegationToken);
const response = await fetch(partnerApi("/api/partner-batch-workbook"), {
method: "POST",
body: form,
});
const contentType = response.headers.get("Content-Type") || "";
const result = (contentType.includes("application/json")
? await response.json()
: {
error:
response.status === 413
? "回填表超过上传限制,请重新导出最新版回填表"
: "批量回填服务暂时不可用,请稍后重试",
}) as {
error?: string;
updatedRows?: number;
publishedCount?: number;
noteScreenshotCount?: number;
analysisScreenshotCount?: number;
};
if (!response.ok) throw new Error(result.error || "批量回填失败");
await loadTask(taskToken, claimToken, delegationToken);
const details = [
result.publishedCount
? `${result.publishedCount}篇发布信息`
: "",
result.noteScreenshotCount
? `${result.noteScreenshotCount}张笔记截图`
: "",
result.analysisScreenshotCount
? `${result.analysisScreenshotCount}张数据分析截图`
: "",
].filter(Boolean);
setToast(
details.length > 0
? `${compacted ? "文件已自动精简," : ""}已更新${details.join("、")}`
: `${compacted ? "文件已自动精简," : ""}表格已读取,没有需要更新的数据`,
);
} catch (reason) {
setToast(reason instanceof Error ? reason.message : "批量回填失败");
} finally {
setBatchWorkbookWorking(false);
}
};
const prepareImage = async (item: Assignment, imageIndex: number) => {
const response = await fetch(noteImageUrl(item, imageIndex));
if (!response.ok) throw new Error("图片读取失败");
@@ -840,7 +1023,7 @@ export default function Home() {
});
}, 120);
}
setToast("这篇笔记已回填,不会与其他笔记错配");
setToast(PUBLISH_BACKFILL_SUCCESS);
} catch (reason) {
setToast(reason instanceof Error ? reason.message : "回填失败");
} finally {
@@ -987,7 +1170,10 @@ export default function Home() {
<span> {activeBatch.assignments.findIndex((item) => item.id === selected.id) + 1}</span>
<span></span>
</div>
<p className="document-label"></p>
<div className="document-label-row">
<p className="document-label"></p>
<PlatformBadge platform={payload.task.platform} compact />
</div>
<div className="note-title-row">
<h1>{selected.title}</h1>
<button
@@ -1107,7 +1293,7 @@ export default function Home() {
</form>
</div>
<ImageLightbox image={previewImage} onClose={() => setPreviewImage(null)} />
{toast && <div className="toast">{toast}</div>}
{toast && <div className={toastClassName(toast)}>{toast}</div>}
</main>
);
}
@@ -1136,7 +1322,9 @@ export default function Home() {
<span></span>
<strong>{selected.title}</strong>
<small>
{selected.images.length} ·
{selected.videos.length > 0
? `${selected.videos.length} 个视频`
: `${selected.images.length} 张配图`} ·
</small>
</div>
<button
@@ -1150,6 +1338,7 @@ export default function Home() {
<div className="note-document-content">
<div className="note-document-meta">
<span> {activeBatch.assignments.findIndex((item) => item.id === selected.id) + 1}</span>
<PlatformBadge platform={payload.task.platform} compact />
<span> {selected.source_row ?? "—"}</span>
</div>
{selected.publish_url && (
@@ -1237,6 +1426,38 @@ export default function Home() {
</div>
</section>
)}
{selected.videos.length > 0 && (
<section className="note-videos">
<div className="note-images-heading">
<div>
<p className="document-label"></p>
<span>使</span>
</div>
<b>{selected.videos.length} </b>
</div>
<div className="note-video-grid">
{selected.videos.map((video, index) => (
<div className="note-video-card" key={video.index}>
<video
controls
preload="metadata"
playsInline
src={noteVideoUrl(selected, video.index)}
/>
<div className="note-image-actions">
<span> {index + 1}</span>
<a
href={noteVideoUrl(selected, video.index, true)}
download={`${safeFileBase(selected)}-视频-${index + 1}.mp4`}
>
</a>
</div>
</div>
))}
</div>
</section>
)}
</div>
</article>
@@ -1261,7 +1482,7 @@ export default function Home() {
inputMode="url"
value={publishUrl}
onChange={(event) => setPublishUrl(event.target.value)}
placeholder="可粘贴小红书长链、短链或整段分享文案"
placeholder={`可粘贴${payload.task.platform}作品链接或整段分享文案`}
required
/>
<small className="field-hint">
@@ -1399,7 +1620,6 @@ export default function Home() {
value={creatorExposure}
onChange={(event) => setCreatorExposure(event.target.value)}
placeholder="填写截图中的曝光量"
required
/>
</label>
<label>
@@ -1413,7 +1633,6 @@ export default function Home() {
value={creatorViews}
onChange={(event) => setCreatorViews(event.target.value)}
placeholder="填写截图中的阅读量"
required
/>
</label>
</div>
@@ -1439,7 +1658,7 @@ export default function Home() {
</form>
</div>
<ImageLightbox image={previewImage} onClose={() => setPreviewImage(null)} />
{toast && <div className="toast">{toast}</div>}
{toast && <div className={toastClassName(toast)}>{toast}</div>}
</main>
);
}
@@ -1478,8 +1697,10 @@ export default function Home() {
: "合作社转派发布包"}
</p>
<h1>{payload.task.name}</h1>
<p>
{payload.task.brand} · {formatDate(payload.task.dueAt)}{isScreenshotTask ? "提交" : "发布"}
<p className="platform-meta-line">
<span>{payload.task.brand}</span>
<PlatformBadge platform={payload.task.platform} compact />
<span>{formatDate(payload.task.dueAt)}{isScreenshotTask ? "提交" : "发布"}</span>
{payload.delegation ? ` · ${payload.delegation.label}` : ""}
</p>
</div>
@@ -1500,7 +1721,7 @@ export default function Home() {
{isClaimOwner
? isScreenshotTask
? "打开一份查看关键词和要求完成后单独上传截图也可以转派给底层KOC"
: "打开一篇,查看内容并单独回填;也可以选择笔记转派给底层KOC"
: "可逐篇回填也可导出Excel填写后批量上传可以选择笔记转派给底层KOC"
: isScreenshotTask
? "打开任务查看搜索关键词和要求,完成后逐份上传截图"
: "打开一篇查看完整内容,发布后逐篇回填"}
@@ -1508,6 +1729,31 @@ export default function Home() {
</div>
<div className="section-actions">
<span>{activeBatch.assignments.length} {isScreenshotTask ? "份" : "篇"}</span>
{!isScreenshotTask && (
<>
<button
type="button"
disabled={batchWorkbookWorking}
onClick={() => void exportBatchWorkbook()}
>
{batchWorkbookWorking ? "处理中…" : "导出Excel"}
</button>
<button
type="button"
disabled={batchWorkbookWorking}
onClick={() => batchWorkbookInputRef.current?.click()}
>
</button>
<input
ref={batchWorkbookInputRef}
className="batch-workbook-input"
type="file"
accept=".xlsx,application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
onChange={(event) => void importBatchWorkbook(event)}
/>
</>
)}
{isClaimOwner && (
<button
type="button"
@@ -1609,6 +1855,10 @@ export default function Home() {
loading="lazy"
/>
</>
) : item.videos.length > 0 ? (
<span className="note-thumb platform-video-thumb">
<PlatformBadge platform={payload.task.platform} logoOnly />
</span>
) : (
<span className="note-thumb empty"></span>
)}
@@ -1617,7 +1867,11 @@ export default function Home() {
<p>
{isScreenshotTask
? `搜索关键词 · ${statusLabel(item, payload.task.type)}`
: `${item.images.length} 张配图 · 飞书源行 ${item.source_row ?? "—"} · ${statusLabel(item, payload.task.type)}`}
: <>
<span>{item.videos.length > 0 ? `${item.videos.length} 个视频` : `${item.images.length} 张配图`}</span>
<PlatformBadge platform={payload.task.platform} compact />
<span> {item.source_row ?? "—"} · {statusLabel(item, payload.task.type)}</span>
</>}
{!isScreenshotTask && item.creator_screenshot_key ? " · D7截图已交" : ""}
{item.delegation_label ? ` · 已转派给 ${item.delegation_label}` : ""}
</p>
@@ -1711,7 +1965,7 @@ export default function Home() {
? "请保存当前分享链接;完成任务后通过此链接上传截图即可。"
: "请保存当前分享链接发布后仍需通过此链接回填第7天截图、曝光量和阅读量。"}
</div>
{toast && <div className="toast">{toast}</div>}
{toast && <div className={toastClassName(toast)}>{toast}</div>}
</main>
);
}
@@ -1724,7 +1978,10 @@ export default function Home() {
</header>
<section className="task-hero">
<div className="hero-copy">
<p className="micro"> · {payload.task.brand}</p>
<div className="hero-platform-line">
<p className="micro"> · {payload.task.brand}</p>
<PlatformBadge platform={payload.task.platform} />
</div>
<h1>{payload.task.name}</h1>
<p>
{payload.task.type === "screenshot_collect"
@@ -1855,7 +2112,7 @@ export default function Home() {
<div><span>3</span><strong>{payload.task.type === "screenshot_collect" ? "上传截图" : "单篇回填"}</strong><p>{payload.task.type === "screenshot_collect" ? "每份任务与截图一一对应" : "昵称、链接、截图一一对应"}</p></div>
</section>
<footer> KOC LOOP </footer>
{toast && <div className="toast">{toast}</div>}
{toast && <div className={toastClassName(toast)}>{toast}</div>}
</main>
);
}

View File

@@ -37,14 +37,31 @@ test("keeps claiming minimal and backfill one-to-one", async () => {
assert.match(page, /发布链接/);
assert.match(page, /识别发布账号/);
assert.match(page, /inputMode="url"/);
assert.match(page, /长链、短链或整段分享文案/);
assert.match(page, /作品链接或整段分享文案/);
assert.doesNotMatch(page, /type="url"/);
assert.match(page, /发布截图/);
assert.match(page, /发布配图/);
assert.match(page, /复制标题/);
assert.match(page, /复制文案/);
assert.match(page, /下载原图/);
assert.match(page, /下载视频/);
assert.match(page, /download\s*=\s*false/);
assert.match(page, /params\.set\("download", "1"\)/);
assert.match(page, /视频-\$\{index \+ 1\}\.mp4/);
assert.match(page, /function PlatformBadge/);
assert.match(page, /platform-logo/);
assert.match(page, /logoOnly/);
assert.match(page, /platform-video-thumb/);
assert.match(styles, /\.platform-logo\.xiaohongshu/);
assert.match(styles, /\.platform-logo\.douyin/);
assert.match(styles, /\.platform-badge\.logo-only/);
assert.match(page, /批量保存图片/);
assert.match(page, /导出Excel/);
assert.match(page, /上传回填表/);
assert.match(page, /\/api\/partner-batch-workbook/);
assert.match(page, /compactPartnerBatchWorkbookForUpload/);
assert.match(page, /Content-Type/);
assert.match(page, /response\.status === 413/);
assert.match(page, /navigator\.share/);
assert.match(page, /zipSync/);
assert.match(page, /navigator\.clipboard\.writeText/);
@@ -53,7 +70,9 @@ test("keeps claiming minimal and backfill one-to-one", async () => {
assert.match(page, /找回领取记录/);
assert.match(page, /action:\s*"recover"/);
assert.match(page, /同一任务多次领取会分批展示/);
assert.match(page, /这篇笔记已回填,不会与其他笔记错配/);
assert.match(page, /这篇笔记的发布记录回填成功啦~/);
assert.match(page, /toastClassName\(toast\)/);
assert.match(styles, /\.toast\.success/);
assert.match(page, /笔记内容已收起/);
assert.match(page, /展开笔记内容/);
assert.match(page, /收起笔记内容/);
@@ -61,7 +80,7 @@ test("keeps claiming minimal and backfill one-to-one", async () => {
assert.match(page, /scrollIntoView/);
assert.match(page, /matchMedia\("\(max-width: 620px\)"\)/);
assert.match(styles, /\.note-document\.mobile-collapsed/);
assert.doesNotMatch(page, /批量回填/);
assert.match(page, /批量回填/);
assert.doesNotMatch(page, /复制标题和正文/);
assert.match(page, /CONFIGURED_ADMIN_ORIGIN/);
assert.match(page, /window\.location\.origin/);
@@ -75,6 +94,7 @@ test("keeps claiming minimal and backfill one-to-one", async () => {
assert.match(page, /creatorViews/);
assert.match(page, /截图仅用于运营核对不再自动OCR/);
assert.match(page, /evidenceImageUrl/);
assert.match(page, /params\.set\("v", evidenceKey\)/);
assert.match(page, /evidenceImageUrl\(selected, "publish"\)/);
assert.match(page, /evidenceImageUrl\(selected, "creator"\)/);
assert.match(page, /ImageLightbox/);
@@ -83,6 +103,8 @@ test("keeps claiming minimal and backfill one-to-one", async () => {
assert.match(page, /creatorScreenshotPreview/);
assert.match(page, /曝光量/);
assert.match(page, /阅读量/);
assert.match(page, /placeholder="填写截图中的曝光量"\s*\/>/);
assert.match(page, /placeholder="填写截图中的阅读量"\s*\/>/);
assert.doesNotMatch(page, /recognizeCreatorMetrics/);
assert.match(page, /note-index \$\{\(isScreenshotTask \? item\.result_submitted_at : item\.publish_url\) \? "done" : ""\}/);
assert.match(packageJson, /"fflate":\s*"0\.7\.4"/);

View File

@@ -1,7 +1,7 @@
import {
resolveXhsPublicAccountDetails,
resolveXhsAccountProfileFromMcp,
resolveXhsProfileDetailsFromMcp,
resolveAccountProfileFromMcp,
resolveProfileDetailsFromMcp,
type CollectionMcpConfig,
} from "./mcp-collection-client";
import { hashText } from "./mvp-db";
@@ -11,15 +11,20 @@ type DistributionAccountRow = {
id: string;
account_id: string | null;
publish_url: string | null;
platform: string;
claimant_contact: string | null;
};
type BackfillRow = DistributionAccountRow & {
resolved_account_id: string | null;
nickname: string | null;
platform: string | null;
platform_uid: string | null;
public_account_id: string | null;
profile_url: string | null;
followers: number | null;
gender: string | null;
bio: string | null;
tags: string | null;
};
function isVerifiedXhsProfileUrl(value: string | null) {
@@ -37,6 +42,23 @@ function isVerifiedXhsProfileUrl(value: string | null) {
}
}
function isVerifiedDouyinProfileUrl(value: string | null) {
if (!value) return false;
try {
const url = new URL(value);
const secUid = decodeURIComponent(url.pathname.match(/^\/user\/([^/?#]+)/)?.[1] ?? "");
return (
url.protocol === "https:" &&
(url.hostname === "douyin.com" ||
url.hostname.endsWith(".douyin.com")) &&
/^[A-Za-z0-9_-]{20,220}$/.test(secUid) &&
!/^\d+$/.test(secUid)
);
} catch {
return false;
}
}
export async function enrichDistributionAccount(
db: DatabaseClient,
distributionId: string,
@@ -44,27 +66,42 @@ export async function enrichDistributionAccount(
fallbackNickname: string,
mcpConfig: CollectionMcpConfig,
) {
const profile = await resolveXhsAccountProfileFromMcp(
publishUrl,
fallbackNickname,
mcpConfig,
);
const current = await db
.prepare(
`SELECT id, account_id, publish_url
FROM distributions
WHERE id = ?`,
`SELECT d.id, d.account_id, d.publish_url, t.platform,
cl.claimant_name AS claimant_contact
FROM distributions d
JOIN tasks t ON t.id = d.task_id
LEFT JOIN claims cl ON cl.id = d.claim_id
WHERE d.id = ?`,
)
.bind(distributionId)
.first<DistributionAccountRow>();
if (!current || current.publish_url !== publishUrl) {
return { updated: false, reason: "stale" as const };
}
const platform = current.platform === "抖音" ? "抖音" : "小红书";
const profile = await resolveAccountProfileFromMcp(
publishUrl,
fallbackNickname,
platform,
mcpConfig,
);
const canonicalAccountId = `account-${hashText(
`小红书:${profile.platformUid}`,
`${platform}:${profile.platformUid}`,
)}`;
if (current.account_id === canonicalAccountId) {
const existingAccount = await db
.prepare(
`SELECT id FROM accounts
WHERE platform = ? AND platform_uid = ?
LIMIT 1`,
)
.bind(platform, profile.platformUid)
.first<{ id: string }>();
const targetAccountId = existingAccount?.id || canonicalAccountId;
const currentContact = (current.claimant_contact || "").trim();
if (current.account_id === targetAccountId) {
await db
.prepare(
`UPDATE accounts SET
@@ -82,6 +119,12 @@ export async function enrichDistributionAccount(
WHEN ? IS NOT NULL AND (? > 0 OR followers = 0) THEN ?
ELSE followers
END,
gender = CASE WHEN ? != '' THEN ? ELSE gender END,
bio = CASE WHEN ? != '' THEN ? ELSE bio END,
current_contact = CASE WHEN ? != '' THEN ? ELSE current_contact END,
post_count = (
SELECT COUNT(*) FROM distributions WHERE account_id = ?
),
last_seen_at = CURRENT_TIMESTAMP
WHERE id = ?`,
)
@@ -96,12 +139,19 @@ export async function enrichDistributionAccount(
profile.followers,
profile.followers,
profile.followers,
canonicalAccountId,
profile.gender,
profile.gender,
profile.bio,
profile.bio,
currentContact,
currentContact,
targetAccountId,
targetAccountId,
)
.run();
return {
updated: true,
accountId: canonicalAccountId,
accountId: targetAccountId,
profileUrl: profile.profileUrl,
};
}
@@ -111,8 +161,9 @@ export async function enrichDistributionAccount(
db
.prepare(
`INSERT INTO accounts
(id, platform, platform_uid, public_account_id, nickname, profile_url, ip_location, followers, post_count)
VALUES (?, '小红书', ?, ?, ?, ?, ?, ?, 1)
(id, platform, platform_uid, public_account_id, nickname, profile_url,
ip_location, followers, gender, bio, current_contact, post_count)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1)
ON CONFLICT(platform, platform_uid) DO UPDATE SET
public_account_id = CASE
WHEN excluded.public_account_id != ''
@@ -131,17 +182,29 @@ export async function enrichDistributionAccount(
THEN excluded.followers
ELSE accounts.followers
END,
post_count = accounts.post_count + 1,
gender = CASE
WHEN excluded.gender != '' THEN excluded.gender ELSE accounts.gender END,
bio = CASE
WHEN excluded.bio != '' THEN excluded.bio ELSE accounts.bio END,
current_contact = CASE
WHEN excluded.current_contact != ''
THEN excluded.current_contact
ELSE accounts.current_contact
END,
last_seen_at = CURRENT_TIMESTAMP`,
)
.bind(
canonicalAccountId,
targetAccountId,
platform,
profile.platformUid,
profile.redId,
profile.nickname || fallbackNickname,
profile.profileUrl,
profile.ipLocation,
profile.followers ?? 0,
profile.gender,
profile.bio,
currentContact,
),
db
.prepare(
@@ -150,10 +213,25 @@ export async function enrichDistributionAccount(
updated_at = CURRENT_TIMESTAMP
WHERE id = ? AND publish_url = ?`,
)
.bind(canonicalAccountId, distributionId, publishUrl),
.bind(targetAccountId, distributionId, publishUrl),
db
.prepare(
`UPDATE accounts SET post_count = (
SELECT COUNT(*) FROM distributions WHERE account_id = ?
) WHERE id = ?`,
)
.bind(targetAccountId, targetAccountId),
]);
if (provisionalAccountId) {
if (provisionalAccountId && provisionalAccountId !== targetAccountId) {
await db
.prepare(
`UPDATE accounts SET post_count = (
SELECT COUNT(*) FROM distributions WHERE account_id = ?
) WHERE id = ?`,
)
.bind(provisionalAccountId, provisionalAccountId)
.run();
await db
.prepare(
`DELETE FROM accounts
@@ -165,14 +243,14 @@ export async function enrichDistributionAccount(
)
.bind(
provisionalAccountId,
canonicalAccountId,
targetAccountId,
provisionalAccountId,
)
.run();
}
return {
updated: true,
accountId: canonicalAccountId,
accountId: targetAccountId,
profileUrl: profile.profileUrl,
};
}
@@ -188,15 +266,22 @@ export async function backfillAccountProfiles(
d.id,
d.account_id,
d.publish_url,
a.id AS resolved_account_id,
a.nickname,
a.platform,
COALESCE(a.platform, t.platform) AS platform,
a.platform_uid,
a.public_account_id,
a.profile_url,
a.followers
a.followers,
a.gender,
a.bio,
a.tags,
cl.claimant_name AS claimant_contact
FROM distributions d
JOIN tasks t ON t.id = d.task_id
LEFT JOIN accounts a ON a.id = d.account_id
WHERE d.publish_url IS NOT NULL
LEFT JOIN claims cl ON cl.id = d.claim_id
WHERE d.publish_url IS NOT NULL
AND d.publish_url != ''
ORDER BY d.updated_at DESC
LIMIT 100`,
@@ -205,8 +290,34 @@ export async function backfillAccountProfiles(
let attempted = 0;
let updated = 0;
let failed = 0;
const backfilledAccounts = new Set<string>();
for (const row of rows.results) {
if (
row.resolved_account_id &&
!backfilledAccounts.has(row.resolved_account_id)
) {
backfilledAccounts.add(row.resolved_account_id);
const claimantContact = row.claimant_contact?.trim() || "";
await db
.prepare(
`UPDATE accounts
SET current_contact = CASE
WHEN ? != '' THEN ? ELSE current_contact
END,
post_count = (
SELECT COUNT(*) FROM distributions WHERE account_id = ?
)
WHERE id = ?`,
)
.bind(
claimantContact,
claimantContact,
row.resolved_account_id,
row.resolved_account_id,
)
.run();
}
if (attempted >= Math.max(1, Math.min(25, limit))) break;
const noteId = (() => {
try {
@@ -227,19 +338,31 @@ export async function backfillAccountProfiles(
row.platform === "小红书" &&
!isDemoAccount &&
isVerifiedXhsProfileUrl(row.profile_url) &&
(!row.public_account_id || Number(row.followers ?? 0) === 0)
(!row.public_account_id ||
Number(row.followers ?? 0) === 0 ||
!row.gender ||
!row.bio)
) {
attempted += 1;
attemptedThisRow = true;
const details = await resolveXhsProfileDetailsFromMcp(
const details = await resolveProfileDetailsFromMcp(
row.profile_url ?? "",
"小红书",
mcpConfig,
).catch(() =>
resolveXhsPublicAccountDetails(row.profile_url ?? ""),
);
).catch(async () => ({
...(await resolveXhsPublicAccountDetails(row.profile_url ?? "")),
gender: "" as const,
bio: "",
recentNoteTitles: [] as string[],
providerTags: [] as string[],
}));
if (
row.account_id &&
(details.redId || details.followers !== null)
(details.redId ||
details.followers !== null ||
details.gender ||
details.bio ||
details.recentNoteTitles.length > 0)
) {
await db
.prepare(
@@ -256,6 +379,8 @@ export async function backfillAccountProfiles(
WHEN ? != '' AND ? != '待识别' THEN ?
ELSE ip_location
END,
gender = CASE WHEN ? != '' THEN ? ELSE gender END,
bio = CASE WHEN ? != '' THEN ? ELSE bio END,
last_seen_at = CURRENT_TIMESTAMP
WHERE id = ?`,
)
@@ -268,6 +393,10 @@ export async function backfillAccountProfiles(
details.ipLocation ?? "",
details.ipLocation ?? "",
details.ipLocation ?? "",
details.gender,
details.gender,
details.bio,
details.bio,
row.account_id,
)
.run();
@@ -287,12 +416,17 @@ export async function backfillAccountProfiles(
}
const needsEnrichment =
!isDemoAccount &&
(!row.account_id ||
(!row.resolved_account_id ||
(row.platform === "小红书" &&
!isVerifiedXhsProfileUrl(row.profile_url)) ||
(row.platform === "抖音" &&
!isVerifiedDouyinProfileUrl(row.profile_url)) ||
(row.platform === "小红书" && !row.public_account_id) ||
(row.platform === "抖音" && !row.public_account_id) ||
(row.platform === "小红书" &&
Number(row.followers ?? 0) === 0) ||
(row.platform === "抖音" &&
Number(row.followers ?? 0) === 0) ||
row.platform_uid?.startsWith("pending-") ||
Boolean(noteId && row.platform_uid === noteId));
if (!needsEnrichment || !row.publish_url) {

View File

@@ -1,5 +1,5 @@
import {
collectXhsMetricsFromMcp,
collectMetricsFromMcp,
type CollectionMcpConfig,
} from "./mcp-collection-client";
import { uid } from "./mvp-db";
@@ -10,6 +10,7 @@ type DistributionForCollection = {
task_id: string;
publish_url: string | null;
ocr_status: string;
platform: string;
};
type ScheduledTask = {
@@ -165,11 +166,16 @@ export async function collectDistributionMetrics(
mcpConfig: CollectionMcpConfig,
) {
const current = await db
.prepare("SELECT * FROM distributions WHERE id = ?")
.prepare(
`SELECT d.*, t.platform
FROM distributions d
JOIN tasks t ON t.id = d.task_id
WHERE d.id = ?`,
)
.bind(distributionId)
.first<DistributionForCollection>();
if (!current) throw new Error("分发记录不存在");
if (!current.publish_url) throw new Error("笔记尚未回填发布链接");
if (!current.publish_url) throw new Error("作品尚未回填发布链接");
const scheduledAt = `${scheduledDate}T09:00:00+08:00`;
const runId = uid("run");
@@ -203,7 +209,12 @@ export async function collectDistributionMetrics(
.bind(distributionId, scheduledDate)
.first<{ id: string; status: string }>();
if (!run) throw new Error("采集任务创建失败");
if (run.status === "success") return { skipped: true };
// Scheduled jobs should remain idempotent, but an operator clicking
// “立即采集” is explicitly asking for a fresh snapshot. Reusing the same
// daily run lets us correct stale or previously mis-mapped platform data.
if (run.status === "success" && source !== "manual") {
return { skipped: true };
}
const collectingDescription =
source === "automatic"
@@ -235,8 +246,12 @@ export async function collectDistributionMetrics(
]);
try {
const { likes, comments, collects } =
await collectXhsMetricsFromMcp(current.publish_url, mcpConfig);
const { likes, comments, collects, shares } =
await collectMetricsFromMcp(
current.publish_url,
current.platform === "抖音" ? "抖音" : "小红书",
mcpConfig,
);
const dayWeight = scheduleDay ?? 1;
const successDescription =
source === "automatic"
@@ -253,6 +268,7 @@ export async function collectDistributionMetrics(
likes = ?,
comments = ?,
collects = ?,
shares = ?,
status_description = ?,
completed_at = CURRENT_TIMESTAMP
WHERE id = ?`,
@@ -261,6 +277,7 @@ export async function collectDistributionMetrics(
likes,
comments,
collects,
shares,
successDescription,
run.id,
),
@@ -270,6 +287,7 @@ export async function collectDistributionMetrics(
SET latest_likes = ?,
latest_comments = ?,
latest_collects = ?,
latest_shares = ?,
collection_status = 'success',
collection_status_description = ?,
collection_updated_at = CURRENT_TIMESTAMP,
@@ -285,12 +303,13 @@ export async function collectDistributionMetrics(
likes,
comments,
collects,
shares,
successDescription,
scheduleDay,
distributionId,
),
]);
return { skipped: false, likes, comments, collects };
return { skipped: false, likes, comments, collects, shares };
} catch (error) {
const message =
error instanceof Error ? error.message : "公开数据采集失败";

View File

@@ -2,7 +2,7 @@ const FEISHU_API_ORIGIN = "https://open.feishu.cn";
const MAX_SHEET_ROWS = 5_000;
const MAX_SHEET_COLUMNS = 100;
const MAX_CONTENT_ROWS = 1_000;
const MAX_MEDIA_BYTES = 20_000_000;
const DEFAULT_MAX_MEDIA_BYTES = 200_000_000;
export type FeishuBindings = {
FEISHU_APP_ID?: string;
@@ -16,11 +16,20 @@ export type FeishuSourceImage = {
height: number | null;
};
export type FeishuSourceVideo = {
index: number;
fileToken: string;
name: string;
mimeType: string;
size: number | null;
};
export type FeishuSourceRow = {
sourceRow: number;
title: string;
body: string;
images: FeishuSourceImage[];
videos: FeishuSourceVideo[];
};
export type FeishuSource = {
@@ -106,12 +115,47 @@ function cellText(value: unknown): string {
.filter(Boolean)
.join("");
}
if (!isRecord(value) || value.type === "embed-image") return "";
if (
!isRecord(value) ||
value.type === "embed-image" ||
value.type === "attachment"
) return "";
if (typeof value.text === "string") return value.text.trim();
if (typeof value.value === "string") return value.value.trim();
return "";
}
function extractVideos(value: unknown, output: FeishuSourceVideo[]) {
if (Array.isArray(value)) {
for (const item of value) extractVideos(item, output);
return;
}
if (!isRecord(value)) return;
const fileToken = bindingValue(value.fileToken ?? value.file_token);
const mimeType = bindingValue(value.mimeType ?? value.mime_type);
const name = bindingValue(value.text ?? value.name ?? value.file_name);
const isVideo =
value.type === "attachment" &&
(mimeType.startsWith("video/") || /\.(?:mp4|mov|m4v|webm)$/i.test(name));
if (isVideo && fileToken) {
output.push({
index: 0,
fileToken,
name: name || "视频",
mimeType: mimeType || "video/mp4",
size:
typeof value.size === "number" && Number.isFinite(value.size)
? value.size
: null,
});
}
for (const child of Object.values(value)) {
if (child !== value.fileToken && child !== value.file_token) {
extractVideos(child, output);
}
}
}
function extractImages(value: unknown, output: FeishuSourceImage[]) {
if (Array.isArray(value)) {
for (const item of value) extractImages(item, output);
@@ -167,6 +211,7 @@ function findHeader(values: unknown[][]) {
titleIndex: number;
bodyIndex: number;
tagsIndex: number;
videoIndex: number;
score: number;
}
| undefined;
@@ -185,9 +230,13 @@ function findHeader(values: unknown[][]) {
const tagsIndex = headers.findIndex((header) =>
headerMatches(header, [/标签/, /话题/, /^tags?$/]),
);
const videoIndex = headers.findIndex((header) =>
headerMatches(header, [/^视频\d*$/, /视频文件/, /视频素材/]),
);
const score =
(titleIndex >= 0 ? 5 : 0) +
(bodyIndex >= 0 ? 5 : 0) +
(videoIndex >= 0 ? 2 : 0) +
(idIndex >= 0 ? 1 : 0) +
(tagsIndex >= 0 ? 1 : 0);
if (!best || score > best.score) {
@@ -197,6 +246,7 @@ function findHeader(values: unknown[][]) {
titleIndex,
bodyIndex,
tagsIndex,
videoIndex,
score,
};
}
@@ -217,6 +267,7 @@ function parseRows(values: unknown[][]) {
const usedSourceRows = new Set<number>();
const rows: FeishuSourceRow[] = [];
let maxImageCount = 0;
let maxVideoCount = 0;
for (
let rowIndex = header.rowIndex + 1;
@@ -253,7 +304,20 @@ function parseRows(values: unknown[][]) {
})
.map((image, imageIndex) => ({ ...image, index: imageIndex + 1 }));
maxImageCount = Math.max(maxImageCount, images.length);
rows.push({ sourceRow, title, body, images });
const collectedVideos: FeishuSourceVideo[] = [];
if (header.videoIndex >= 0) {
extractVideos(row[header.videoIndex], collectedVideos);
}
const seenVideoTokens = new Set<string>();
const videos = collectedVideos
.filter((video) => {
if (seenVideoTokens.has(video.fileToken)) return false;
seenVideoTokens.add(video.fileToken);
return true;
})
.map((video, videoIndex) => ({ ...video, index: videoIndex + 1 }));
maxVideoCount = Math.max(maxVideoCount, videos.length);
rows.push({ sourceRow, title, body, images, videos });
}
if (rows.length === 0) {
@@ -266,6 +330,7 @@ function parseRows(values: unknown[][]) {
header.tagsIndex >= 0 ? cellText(headerRow[header.tagsIndex]) : "",
cellText(headerRow[header.bodyIndex]) || "正文",
...Array.from({ length: maxImageCount }, (_, index) => `图片${index + 1}`),
...Array.from({ length: maxVideoCount }, (_, index) => `视频${index + 1}`),
].filter(Boolean);
return {
@@ -515,34 +580,37 @@ export async function downloadFeishuMedia(
fileToken: string,
bindings: FeishuBindings,
fetchImpl: FetchLike = fetch,
options: { maxBytes?: number; label?: string; timeoutMs?: number } = {},
) {
const maxBytes = Math.max(1, options.maxBytes ?? DEFAULT_MAX_MEDIA_BYTES);
const label = bindingValue(options.label) || "素材";
const normalizedToken = bindingValue(fileToken);
if (!/^[A-Za-z0-9_-]{8,240}$/.test(normalizedToken)) {
throw new FeishuSourceError("飞书图片标识无效", 400);
throw new FeishuSourceError(`飞书${label}标识无效`, 400);
}
const token = await accessToken(bindings, fetchImpl);
const response = await fetchImpl(
`${FEISHU_API_ORIGIN}/open-apis/drive/v1/medias/${encodeURIComponent(normalizedToken)}/download`,
{
headers: { Authorization: `Bearer ${token}` },
signal: AbortSignal.timeout(20_000),
signal: AbortSignal.timeout(options.timeoutMs ?? 120_000),
},
);
const declaredSize = Number(response.headers.get("content-length"));
if (Number.isFinite(declaredSize) && declaredSize > MAX_MEDIA_BYTES) {
throw new FeishuSourceError("飞书图片超过20MB,无法同步", 413);
if (Number.isFinite(declaredSize) && declaredSize > maxBytes) {
throw new FeishuSourceError(`飞书${label}文件过大,无法同步`, 413);
}
if (!response.ok) {
throw new FeishuSourceError(
response.status === 403
? "飞书应用没有这张图片的下载权限"
: `下载飞书图片失败HTTP ${response.status}`,
? `飞书应用没有这${label}的下载权限`
: `下载飞书${label}失败HTTP ${response.status}`,
response.status === 403 ? 403 : 502,
);
}
const bytes = await response.arrayBuffer();
if (bytes.byteLength > MAX_MEDIA_BYTES) {
throw new FeishuSourceError("飞书图片超过20MB,无法同步", 413);
if (bytes.byteLength > maxBytes) {
throw new FeishuSourceError(`飞书${label}文件过大,无法同步`, 413);
}
return {
bytes,

View File

@@ -17,6 +17,7 @@ export type XhsPublicMetrics = {
likes: number;
comments: number;
collects: number;
shares: number;
};
export type XhsAccountProfile = {
@@ -26,6 +27,10 @@ export type XhsAccountProfile = {
redId: string;
ipLocation: string;
followers: number | null;
gender: "" | "男" | "女";
bio: string;
recentNoteTitles: string[];
providerTags: string[];
};
type JsonRpcEnvelope = {
@@ -341,16 +346,118 @@ function metricsFromToolResult(result: ToolResult): XhsPublicMetrics {
}
if (!data) throw new Error("采集结果缺少互动数据");
const count = (value: unknown, label: string) =>
value === null || value === undefined || value === ""
? 0
: metricValue(value, label);
return {
likes: metricValue(data.likes, "点赞数"),
comments: metricValue(data.comments, "评论数"),
collects: metricValue(
data.collects ?? data.favorites ?? data.favourites,
likes: count(
data.likes ??
data.liked_count ??
data.likedCount ??
data.like_count ??
data.likeCount ??
data.digg_count ??
data.diggCount,
"点赞数",
),
comments: count(
data.comments ?? data.comment_count ?? data.commentCount,
"评论数",
),
collects: count(
data.collects ??
data.collected_count ??
data.collectedCount ??
data.favorites ??
data.favourites ??
data.collect_count ??
data.collectCount,
"收藏数",
),
shares: count(
data.shares ??
data.share_count ??
data.shareCount ??
data.forwards ??
data.forward_count ??
data.forwardCount,
"转发数",
),
};
}
function usableDouyinSecUid(value: unknown) {
const candidate = stringValue(value);
return candidate && !/^\d+$/.test(candidate) && /^[A-Za-z0-9_-]{20,220}$/.test(candidate)
? candidate
: "";
}
function verifiedDouyinProfileUrl(value: unknown) {
const candidate = stringValue(value);
if (!candidate) return "";
try {
const parsed = new URL(candidate);
const secUid = decodeURIComponent(parsed.pathname.match(/^\/user\/([^/?#]+)/)?.[1] ?? "");
if (
parsed.protocol === "https:" &&
(parsed.hostname === "douyin.com" || parsed.hostname.endsWith(".douyin.com")) &&
usableDouyinSecUid(secUid)
) {
return parsed.toString();
}
} catch {
// The public redirect fallback below can still recover the profile URL.
}
return "";
}
async function douyinProfileFromPublicRedirect(
publishUrl: string,
fetchImpl: typeof fetch,
timeoutMs: number,
) {
let parsed: URL;
try {
parsed = new URL(publishUrl);
} catch {
return null;
}
if (
parsed.protocol !== "https:" ||
!(parsed.hostname === "douyin.com" || parsed.hostname.endsWith(".douyin.com"))
) {
return null;
}
try {
const response = await fetchImpl(parsed.toString(), {
method: "GET",
redirect: "manual",
signal: AbortSignal.timeout(Math.min(timeoutMs, 15_000)),
headers: {
"user-agent":
"Mozilla/5.0 (iPhone; CPU iPhone OS 18_0 like Mac OS X) AppleWebKit/605.1.15 Mobile/15E148",
},
});
const location = response.headers.get("location");
if (!location) return null;
const redirectUrl = new URL(location, parsed);
const secUid = usableDouyinSecUid(
redirectUrl.searchParams.get("sec_uid") ??
redirectUrl.searchParams.get("sec_user_id"),
);
return secUid
? {
platformUid: secUid,
profileUrl: `https://www.douyin.com/user/${encodeURIComponent(secUid)}`,
}
: null;
} catch {
return null;
}
}
function successfulToolData(result: ToolResult, fallbackMessage: string) {
const root = asRecord(result.payload);
const response = asRecord(root?.response) ?? root;
@@ -446,6 +553,64 @@ function findValueByKeys(
return undefined;
}
function profileGender(value: unknown): "" | "男" | "女" {
if (value === 1) return "男";
if (value === 2) return "女";
const normalized = String(value ?? "").trim().toLocaleLowerCase("zh-CN");
if (["男", "男性", "male", "m", "1"].includes(normalized)) return "男";
if (["女", "女性", "female", "f", "2"].includes(normalized)) return "女";
return "";
}
function recentNoteTitlesFromPayload(value: unknown) {
const titles: string[] = [];
const visit = (current: unknown, depth = 0) => {
if (depth > 12 || titles.length >= 20) return;
if (Array.isArray(current)) {
current.forEach((item) => visit(item, depth + 1));
return;
}
const record = asRecord(current);
if (!record) return;
const title = stringValue(record.title ?? record.note_title ?? record.noteTitle);
if (
title &&
(record.note_id || record.noteId || record.url || record.cover) &&
!titles.includes(title)
) {
titles.push(title);
}
Object.values(record).forEach((child) => visit(child, depth + 1));
};
visit(value);
return titles;
}
function providerTagsFromUser(value: unknown) {
const user = findRecord(value, (record) =>
Boolean(
record.gender !== undefined ||
record.desc !== undefined ||
record.signature !== undefined ||
record.fansCount !== undefined ||
record.fans_count !== undefined,
),
);
const raw = user?.tags;
if (!Array.isArray(raw)) return [];
return [
...new Set(
raw
.map((item) =>
typeof item === "string"
? item.trim()
: stringValue(asRecord(item)?.name ?? asRecord(item)?.title),
)
.filter(Boolean),
),
].slice(0, 5);
}
const FOLLOWER_KEYS = new Set([
"fans",
"fans_count",
@@ -529,10 +694,13 @@ async function xhsNoteIdFromShortLink(
) {
return { noteId: "", profile: null };
}
if (
url.hostname !== "xhslink.cn" &&
!url.hostname.endsWith(".xhslink.cn")
) {
const isShortLink =
url.hostname === "xhslink.cn" ||
url.hostname.endsWith(".xhslink.cn");
const isXhsPage =
url.hostname === "xiaohongshu.com" ||
url.hostname.endsWith(".xiaohongshu.com");
if (!isShortLink && !isXhsPage) {
return { noteId: "", profile: null };
}
@@ -608,6 +776,10 @@ function accountProfileFromPublicPage(
redId,
ipLocation: "待识别",
followers: null,
gender: "",
bio: "",
recentNoteTitles: [],
providerTags: [],
};
}
@@ -698,15 +870,45 @@ function profileDetailsFromToolResult(result: ToolResult) {
const redId =
findStringByKey(payload, "red_id") ||
findStringByKey(payload, "redId") ||
findStringByKey(payload, "unique_id") ||
findStringByKey(payload, "uniqueId") ||
findStringByKey(payload, "short_id") ||
findStringByKey(payload, "shortId") ||
findStringByKey(payload, "douyin_id") ||
findStringByKey(payload, "userId") ||
findStringByKey(payload, "user_id");
const ipLocation =
findStringByKey(payload, "ip_location") ||
findStringByKey(payload, "ipLocation");
if (followers === null && !nickname && !redId && !ipLocation) {
const gender = profileGender(findValueByKeys(payload, new Set(["gender", "sex"])));
const bio =
findStringByKey(payload, "desc") ||
findStringByKey(payload, "description") ||
findStringByKey(payload, "signature") ||
findStringByKey(payload, "bio");
const recentNoteTitles = recentNoteTitlesFromPayload(payload);
const providerTags = providerTagsFromUser(payload);
if (
followers === null &&
!nickname &&
!redId &&
!ipLocation &&
!gender &&
!bio &&
recentNoteTitles.length === 0
) {
throw new Error("账号主页采集结果缺少可用字段");
}
return { nickname, followers, redId, ipLocation };
return {
nickname,
followers,
redId,
ipLocation,
gender,
bio,
recentNoteTitles,
providerTags,
};
}
function accountProfileFromToolResult(
@@ -718,14 +920,18 @@ function accountProfileFromToolResult(
data,
(record) =>
Boolean(
stringValue(record.user_id ?? record.userid) &&
stringValue(record.user_id ?? record.userid ?? record.userId) &&
(stringValue(record.profile_url) ||
stringValue(record.nickname ?? record.name)),
),
);
if (!user) throw new Error("账号主页识别结果缺少作者信息");
const platformUid = stringValue(user.user_id ?? user.userid);
const candidateProfileUrl = stringValue(user.profile_url);
const platformUid = stringValue(
user.user_id ?? user.userid ?? user.userId,
);
const candidateProfileUrl = stringValue(
user.profile_url ?? user.profileUrl,
);
let profileUrl = "";
if (candidateProfileUrl) {
try {
@@ -755,6 +961,10 @@ function accountProfileFromToolResult(
redId: stringValue(user.red_id),
ipLocation: findStringByKey(data, "ip_location") || "待识别",
followers: followerCountFromPayload(data),
gender: "",
bio: "",
recentNoteTitles: [],
providerTags: [],
};
}
@@ -771,18 +981,7 @@ async function completeAccountProfile(
"parse_xhs_user_summary",
{ url: profile.profileUrl, use_proxy: true },
],
[
"fetch_user_detail",
{ link: profile.profileUrl, plant: "xhs" },
],
] as const) {
if (
completed.followers !== null &&
completed.redId &&
completed.ipLocation !== "待识别"
) {
break;
}
try {
const result = await callMcpTool(
fetchImpl,
@@ -801,6 +1000,16 @@ async function completeAccountProfile(
details.ipLocation && details.ipLocation !== "待识别"
? details.ipLocation
: completed.ipLocation,
gender: details.gender || completed.gender,
bio: details.bio || completed.bio,
recentNoteTitles:
details.recentNoteTitles.length > 0
? details.recentNoteTitles
: completed.recentNoteTitles,
providerTags:
details.providerTags.length > 0
? details.providerTags
: completed.providerTags,
};
} catch (error) {
if (error instanceof McpSessionLostError) throw error;
@@ -852,17 +1061,14 @@ async function resolveAccountInSession(
const endpoint = buildMcpUrl(config);
const timeoutMs = Math.max(5_000, config.timeoutMs ?? 30_000);
let noteId = xhsNoteIdFromUrl(publishUrl);
let publicPageProfile: XhsAccountProfile | null = null;
if (!noteId) {
const shortLink = await xhsNoteIdFromShortLink(
publishUrl,
fallbackNickname,
fetchImpl,
timeoutMs,
);
noteId = shortLink.noteId;
publicPageProfile = shortLink.profile;
}
const linkPage = await xhsNoteIdFromShortLink(
publishUrl,
fallbackNickname,
fetchImpl,
timeoutMs,
);
noteId = noteId || linkPage.noteId;
const publicPageProfile = linkPage.profile;
try {
const sessionId = await createMcpSession(
@@ -870,47 +1076,39 @@ async function resolveAccountInSession(
endpoint,
timeoutMs,
);
if (!noteId) {
const noteResult = await callMcpTool(
fetchImpl,
endpoint,
sessionId,
timeoutMs,
"fetch_content_detail",
{
link: publishUrl,
plant: "xhs",
include_comments: false,
auto_cookie: true,
},
);
const noteData = successfulToolData(
noteResult,
"无法识别小红书笔记",
);
noteId =
stringValue(noteData.noteId ?? noteData.note_id) ||
findStringByKey(noteData, "noteId") ||
findStringByKey(noteData, "note_id");
}
if (!noteId) throw new Error("无法从发布链接识别小红书笔记ID");
const authorResult = await callMcpTool(
const noteResult = await callMcpTool(
fetchImpl,
endpoint,
sessionId,
timeoutMs,
"collect_xhs_wen_note_detail",
"fetch_content_detail",
{
note_id: noteId,
need_desc: false,
include_raw: false,
link: publishUrl,
plant: "xhs",
include_comments: false,
auto_cookie: true,
},
);
const profile = accountProfileFromToolResult(
authorResult,
fallbackNickname,
const noteData = successfulToolData(
noteResult,
"无法识别小红书笔记",
);
noteId =
noteId ||
stringValue(noteData.noteId ?? noteData.note_id) ||
findStringByKey(noteData, "noteId") ||
findStringByKey(noteData, "note_id");
if (!noteId) throw new Error("无法从发布链接识别小红书笔记ID");
let profile: XhsAccountProfile;
try {
profile = accountProfileFromToolResult(
noteResult,
fallbackNickname,
);
} catch {
if (!publicPageProfile) throw new Error("笔记数据缺少公开作者主页");
profile = publicPageProfile;
}
return completeAccountProfile(
profile,
fetchImpl,
@@ -937,6 +1135,7 @@ async function resolveAccountInSession(
async function collectInSession(
publishUrl: string,
platform: "小红书" | "抖音",
config: CollectionMcpConfig,
fetchImpl: typeof fetch,
) {
@@ -952,28 +1151,12 @@ async function collectInSession(
"fetch_content_detail",
{
link: publishUrl,
plant: "xhs",
plant: platform === "抖音" ? "dy" : "xhs",
include_comments: false,
auto_cookie: true,
},
);
try {
return metricsFromToolResult(primary);
} catch {
const fallback = await callMcpTool(
fetchImpl,
endpoint,
sessionId,
timeoutMs,
"parse_xhs_note",
{
url: publishUrl,
include_comments: false,
auto_cookie: true,
},
);
return metricsFromToolResult(fallback);
}
return metricsFromToolResult(primary);
}
export function resolveCollectionMcpConfig(
@@ -1005,7 +1188,7 @@ export async function collectXhsMetricsFromMcp(
let lastError: unknown;
for (let attempt = 0; attempt < MAX_MCP_ATTEMPTS; attempt += 1) {
try {
return await collectInSession(parsed.toString(), config, fetchImpl);
return await collectInSession(parsed.toString(), "小红书", config, fetchImpl);
} catch (error) {
lastError = error;
if (!isRetryableTransportError(error)) throw error;
@@ -1022,6 +1205,211 @@ export async function collectXhsMetricsFromMcp(
: new Error("MCP采集服务暂时不可用");
}
export async function collectMetricsFromMcp(
publishUrl: string,
platform: "小红书" | "抖音",
config: CollectionMcpConfig,
fetchImpl: typeof fetch = fetch,
) {
if (platform === "小红书") {
return collectXhsMetricsFromMcp(publishUrl, config, fetchImpl);
}
let parsed: URL;
try {
parsed = new URL(publishUrl);
} catch {
throw new Error("发布链接无效");
}
if (!["http:", "https:"].includes(parsed.protocol)) {
throw new Error("发布链接无效");
}
let lastError: unknown;
for (let attempt = 0; attempt < MAX_MCP_ATTEMPTS; attempt += 1) {
try {
return await collectInSession(parsed.toString(), platform, config, fetchImpl);
} catch (error) {
lastError = error;
if (!isRetryableTransportError(error)) throw error;
}
}
throw lastError instanceof Error
? lastError
: new Error("MCP采集服务暂时不可用");
}
function douyinProfileFromToolResult(
result: ToolResult,
fallbackNickname: string,
): XhsAccountProfile {
const data = successfulToolData(result, "无法识别抖音作品");
const author = findRecord(data, (record) =>
Boolean(
stringValue(
record.sec_uid ?? record.secUid ?? record.uid ?? record.user_id ?? record.userId,
) && stringValue(record.nickname ?? record.name ?? record.unique_id ?? record.uniqueId),
),
);
if (!author) throw new Error("抖音作品数据缺少作者信息");
const verifiedSecUid = usableDouyinSecUid(author.sec_uid ?? author.secUid);
const fallbackUid = stringValue(author.uid ?? author.user_id ?? author.userId);
const platformUid = verifiedSecUid || fallbackUid;
const publicId = stringValue(
author.unique_id ?? author.uniqueId ?? author.short_id ?? author.shortId ?? author.douyin_id,
);
const candidateProfileUrl = verifiedDouyinProfileUrl(
author.profile_url ?? author.profileUrl,
);
const profileUrl = candidateProfileUrl ||
(verifiedSecUid
? `https://www.douyin.com/user/${encodeURIComponent(verifiedSecUid)}`
: "");
return {
platformUid,
nickname: stringValue(author.nickname ?? author.name) || fallbackNickname.trim(),
profileUrl,
redId: publicId,
ipLocation:
stringValue(author.ip_location ?? author.ipLocation) ||
findStringByKey(data, "ip_location") ||
findStringByKey(data, "ipLocation") ||
"待识别",
followers: followerCountFromPayload(author) ?? followerCountFromPayload(data),
gender: profileGender(author.gender ?? author.sex),
bio: stringValue(author.desc ?? author.description ?? author.signature ?? author.bio),
recentNoteTitles: recentNoteTitlesFromPayload(data),
providerTags: providerTagsFromUser(data),
};
}
async function resolveDouyinAccountInSession(
publishUrl: string,
fallbackNickname: string,
config: CollectionMcpConfig,
fetchImpl: typeof fetch,
) {
const endpoint = buildMcpUrl(config);
const timeoutMs = Math.max(5_000, config.timeoutMs ?? 30_000);
const sessionId = await createMcpSession(fetchImpl, endpoint, timeoutMs);
const result = await callMcpTool(
fetchImpl,
endpoint,
sessionId,
timeoutMs,
"fetch_content_detail",
{
link: publishUrl,
plant: "dy",
include_comments: false,
auto_cookie: true,
},
);
let profile = douyinProfileFromToolResult(result, fallbackNickname);
if (!verifiedDouyinProfileUrl(profile.profileUrl)) {
const resolved = await douyinProfileFromPublicRedirect(
publishUrl,
fetchImpl,
timeoutMs,
);
if (resolved) {
profile = {
...profile,
platformUid: resolved.platformUid,
profileUrl: resolved.profileUrl,
};
}
}
if (!verifiedDouyinProfileUrl(profile.profileUrl)) return profile;
try {
const detailsResult = await callMcpTool(
fetchImpl,
endpoint,
sessionId,
timeoutMs,
"parse_dy_user_summary",
{ url: profile.profileUrl },
);
const details = profileDetailsFromToolResult(detailsResult);
profile = {
...profile,
nickname: details.nickname || profile.nickname,
redId: details.redId || profile.redId,
followers: details.followers ?? profile.followers,
ipLocation:
details.ipLocation && details.ipLocation !== "待识别"
? details.ipLocation
: profile.ipLocation,
gender: details.gender || profile.gender,
bio: details.bio || profile.bio,
recentNoteTitles:
details.recentNoteTitles.length > 0
? details.recentNoteTitles
: profile.recentNoteTitles,
providerTags:
details.providerTags.length > 0
? details.providerTags
: profile.providerTags,
};
} catch (error) {
if (error instanceof McpSessionLostError) throw error;
}
return profile;
}
export async function resolveAccountProfileFromMcp(
publishUrl: string,
fallbackNickname: string,
platform: "小红书" | "抖音",
config: CollectionMcpConfig,
fetchImpl: typeof fetch = fetch,
) {
if (platform === "小红书") {
return resolveXhsAccountProfileFromMcp(
publishUrl,
fallbackNickname,
config,
fetchImpl,
);
}
let lastError: unknown;
for (let attempt = 0; attempt < MAX_MCP_ATTEMPTS; attempt += 1) {
try {
return await resolveDouyinAccountInSession(
publishUrl,
fallbackNickname,
config,
fetchImpl,
);
} catch (error) {
lastError = error;
if (!isRetryableTransportError(error)) throw error;
}
}
throw lastError instanceof Error ? lastError : new Error("抖音账号识别失败");
}
export async function resolveProfileDetailsFromMcp(
profileUrl: string,
platform: "小红书" | "抖音",
config: CollectionMcpConfig,
fetchImpl: typeof fetch = fetch,
) {
if (platform === "小红书") {
return resolveXhsProfileDetailsFromMcp(profileUrl, config, fetchImpl);
}
const endpoint = buildMcpUrl(config);
const timeoutMs = Math.max(5_000, config.timeoutMs ?? 30_000);
const sessionId = await createMcpSession(fetchImpl, endpoint, timeoutMs);
const result = await callMcpTool(
fetchImpl,
endpoint,
sessionId,
timeoutMs,
"parse_dy_user_summary",
{ url: profileUrl },
);
return profileDetailsFromToolResult(result);
}
export async function resolveXhsAccountProfileFromMcp(
publishUrl: string,
fallbackNickname: string,

View File

@@ -12,7 +12,7 @@ import {
type CollectionMcpBindings,
} from "./mcp-collection-client";
import { ensureSchema, getRawDb } from "./mvp-db";
import { extractXhsPublishUrl } from "./publish-url";
import { extractAnyPublishUrl } from "./publish-url";
import { buildClaimUrl } from "./task-service";
export type McpOperationBindings = CollectionMcpBindings & {
@@ -123,9 +123,9 @@ export async function taskGet(taskId: string, portalUrl: string) {
const [notes, claims, runs] = await Promise.all([
db
.prepare(
`SELECT c.id AS content_id, c.source_row, c.title, c.body, c.image_assets, c.status AS content_status,
`SELECT c.id AS content_id, c.source_row, c.title, c.body, c.image_assets, c.video_assets, c.status AS content_status,
d.id AS distribution_id, d.status AS distribution_status, d.publish_url, d.publish_time,
d.exposure, d.views, d.latest_likes, d.latest_comments, d.latest_collects,
d.exposure, d.views, d.latest_likes, d.latest_comments, d.latest_collects, d.latest_shares,
d.collection_status, d.collection_status_description, d.collection_updated_at,
a.id AS account_id, a.nickname AS account_nickname, a.profile_url, a.followers,
p.name AS cooperation_source, cl.claimant_name
@@ -172,12 +172,14 @@ export async function taskGet(taskId: string, portalUrl: string) {
notes: notes.results.map((row) => ({
...row,
image_assets: parseJsonArray(row.image_assets),
video_assets: parseJsonArray(row.video_assets),
total_interactions:
row.latest_likes == null
? null
: Number(row.latest_likes) +
Number(row.latest_comments ?? 0) +
Number(row.latest_collects ?? 0),
Number(row.latest_collects ?? 0) +
Number(row.latest_shares ?? 0),
})),
claims: claims.results,
collection_runs: runs.results,
@@ -220,7 +222,8 @@ export async function recoveryList(
const [rows, count] = await Promise.all([
db
.prepare(
`SELECT d.*, t.name AS task_name, c.source_row, c.title,
`SELECT d.*, t.name AS task_name, t.platform AS task_platform,
t.content_format, c.source_row, c.title,
a.nickname AS account_nickname, a.profile_url, p.name AS cooperation_source,
cl.claimant_name ${base}
ORDER BY COALESCE(d.publish_time, d.claimed_at) DESC LIMIT ${limit} OFFSET ${offset}`,
@@ -241,7 +244,7 @@ export async function recoveryList(
total_interactions:
row.latest_likes == null
? null
: Number(row.latest_likes) + Number(row.latest_comments ?? 0) + Number(row.latest_collects ?? 0),
: Number(row.latest_likes) + Number(row.latest_comments ?? 0) + Number(row.latest_collects ?? 0) + Number(row.latest_shares ?? 0),
})),
};
}
@@ -358,9 +361,11 @@ function resourceWhere(input: ResourceFilters) {
const conditions: string[] = [];
const bindings: unknown[] = [];
if (input.query?.trim()) {
conditions.push("(a.nickname LIKE ? ESCAPE '\\' OR a.public_account_id LIKE ? ESCAPE '\\')");
conditions.push(
"(a.nickname LIKE ? ESCAPE '\\' OR a.public_account_id LIKE ? ESCAPE '\\' OR a.current_contact LIKE ? ESCAPE '\\' OR a.tags LIKE ? ESCAPE '\\' OR a.bio LIKE ? ESCAPE '\\')",
);
const pattern = like(input.query.trim());
bindings.push(pattern, pattern);
bindings.push(pattern, pattern, pattern, pattern, pattern);
}
if (input.ipLocation?.trim()) {
conditions.push("a.ip_location LIKE ? ESCAPE '\\'");
@@ -369,8 +374,12 @@ function resourceWhere(input: ResourceFilters) {
if (input.cooperationSource?.trim()) {
conditions.push(
`(a.cooperation_source LIKE ? ESCAPE '\\' OR
EXISTS (SELECT 1 FROM distributions dx JOIN partners px ON px.id = dx.partner_id
WHERE dx.account_id = a.id AND px.name LIKE ? ESCAPE '\\'))`,
EXISTS (SELECT 1 FROM distributions dx
JOIN partners px ON px.id = dx.partner_id
LEFT JOIN claims cx ON cx.id = dx.claim_id
WHERE dx.account_id = a.id
AND (cx.claimant_name IS NULL OR px.name != cx.claimant_name)
AND px.name LIKE ? ESCAPE '\\'))`,
);
const pattern = like(input.cooperationSource.trim());
bindings.push(pattern, pattern);
@@ -389,7 +398,12 @@ export async function resourceSearch(input: ResourceFilters) {
const db = getRawDb();
const select = `SELECT a.*,
(SELECT COUNT(*) FROM distributions d WHERE d.account_id = a.id) AS cooperation_count,
(SELECT GROUP_CONCAT(DISTINCT p.name) FROM distributions d JOIN partners p ON p.id = d.partner_id WHERE d.account_id = a.id) AS cooperation_sources`;
(SELECT GROUP_CONCAT(DISTINCT p.name)
FROM distributions d
JOIN partners p ON p.id = d.partner_id
LEFT JOIN claims c ON c.id = d.claim_id
WHERE d.account_id = a.id
AND (c.claimant_name IS NULL OR p.name != c.claimant_name)) AS cooperation_sources`;
const [rows, count] = await Promise.all([
db.prepare(`${select} FROM accounts a ${where} ORDER BY a.last_seen_at DESC LIMIT ${limit} OFFSET ${offset}`)
.bind(...bindings).all<Record<string, unknown>>(),
@@ -422,7 +436,7 @@ export async function resourceGet(accountId: string) {
if (!account) throw new Error("账号不存在");
const history = await db.prepare(
`SELECT d.id AS distribution_id, d.task_id, t.name AS task_name, c.title,
d.publish_url, d.publish_time, d.latest_likes, d.latest_comments, d.latest_collects,
d.publish_url, d.publish_time, d.latest_likes, d.latest_comments, d.latest_collects, d.latest_shares,
d.exposure, d.views, p.name AS cooperation_source, cl.claimant_name
FROM distributions d JOIN tasks t ON t.id = d.task_id
JOIN contents c ON c.id = d.content_id JOIN partners p ON p.id = d.partner_id
@@ -438,7 +452,7 @@ export async function backfillResourceProfile(
) {
await ensureSchema();
const db = getRawDb();
const publishUrl = input.publishUrl ? extractXhsPublishUrl(input.publishUrl) : "";
const publishUrl = input.publishUrl ? extractAnyPublishUrl(input.publishUrl) : "";
const row = input.distributionId
? await db.prepare(
`SELECT d.id, d.publish_url, COALESCE(a.nickname, '') AS nickname

View File

@@ -52,7 +52,7 @@ const pagination = {
};
const resourceFilters = {
query: z.string().max(100).optional().describe("账号名称或小红书号/抖音号,支持模糊搜索"),
query: z.string().max(100).optional().describe("账号名称、账号号、简介或标签,支持模糊搜索"),
ip_location: z.string().max(100).optional().describe("IP地区关键词支持模糊搜索"),
cooperation_source: z.string().max(100).optional().describe("历史合作来源关键词,支持模糊搜索"),
platform: z.string().max(30).optional().describe("平台,例如小红书;不传表示全部"),
@@ -159,7 +159,7 @@ export function registerMcpOperationTools(server: McpServer, options: Options) {
"collection_collect_now",
{
title: "立即采集指定笔记",
description: "对指定分发记录立即采集点赞收藏评论数据。",
description: "对指定分发记录立即采集互动数据;小红书为点赞/收藏/评论,抖音另含转发。",
inputSchema: z.object({
distribution_id: z.string().min(1).describe("分发记录ID可从 task_get 或 recovery_list 获取"),
schedule_day: z.number().int().min(1).max(7).optional().describe("标记为第几天采集,可不传"),
@@ -190,7 +190,7 @@ export function registerMcpOperationTools(server: McpServer, options: Options) {
"resource_search",
{
title: "搜索 KOC 账号资源",
description: "按账号名称/账号号、IP地区、合作来源或平台搜索 KOC 资源。",
description: "按账号名称/账号号/标签、IP地区、合作来源或平台搜索 KOC 资源。",
inputSchema: z.object({ ...resourceFilters, ...pagination }),
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true },
},
@@ -204,7 +204,7 @@ export function registerMcpOperationTools(server: McpServer, options: Options) {
"resource_get",
{
title: "查看 KOC 账号详情",
description: "查看账号主页、账号号、粉丝数、IP地区以及全部合作记录。",
description: "查看账号主页、账号号、粉丝数、性别、简介、标签、IP地区以及全部合作记录。",
inputSchema: z.object({ account_id: z.string().min(1).describe("KOC LOOP 账号ID") }),
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true },
},
@@ -218,10 +218,10 @@ export function registerMcpOperationTools(server: McpServer, options: Options) {
"resource_backfill_profile",
{
title: "补全公开账号信息",
description: "根据已回填的发布链接补全账号主页、昵称、小红书号、IP地区和粉丝数。",
description: "根据已回填的小红书或抖音作品链接补全账号主页、昵称、账号号、IP地区和粉丝数。",
inputSchema: z.object({
distribution_id: z.string().optional().describe("分发记录ID和发布链接二选一"),
publish_url: z.string().optional().describe("小红书发布链接或包含链接的分享文案"),
publish_url: z.string().optional().describe("小红书或抖音作品链接,也可传包含链接的分享文案"),
}).refine((value) => Boolean(value.distribution_id || value.publish_url), "请提供分发记录ID或发布链接"),
annotations: { readOnlyHint: false, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},

View File

@@ -94,6 +94,8 @@ export async function ensureSchema(database?: DatabaseClient) {
due_at TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active',
task_type TEXT NOT NULL DEFAULT 'content_publish',
platform TEXT NOT NULL DEFAULT '小红书',
content_format TEXT NOT NULL DEFAULT 'image_text',
source_url TEXT NOT NULL DEFAULT '',
source_sheet_id TEXT NOT NULL DEFAULT '',
source_sheet_name TEXT NOT NULL DEFAULT '',
@@ -110,6 +112,7 @@ export async function ensureSchema(database?: DatabaseClient) {
title TEXT NOT NULL,
body TEXT NOT NULL DEFAULT '',
image_assets TEXT NOT NULL DEFAULT '[]',
video_assets TEXT NOT NULL DEFAULT '[]',
status TEXT NOT NULL DEFAULT 'available',
source TEXT NOT NULL DEFAULT '飞书内容表',
source_row INTEGER,
@@ -124,9 +127,13 @@ export async function ensureSchema(database?: DatabaseClient) {
profile_url TEXT NOT NULL DEFAULT '',
ip_location TEXT NOT NULL DEFAULT '待识别',
followers INTEGER NOT NULL DEFAULT 0,
gender TEXT NOT NULL DEFAULT '',
bio TEXT NOT NULL DEFAULT '',
tags TEXT NOT NULL DEFAULT '',
post_count INTEGER NOT NULL DEFAULT 0,
avg_views INTEGER NOT NULL DEFAULT 0,
cooperation_source TEXT NOT NULL DEFAULT '',
current_contact TEXT NOT NULL DEFAULT '',
first_seen_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
)`,
@@ -185,6 +192,7 @@ export async function ensureSchema(database?: DatabaseClient) {
latest_likes INTEGER,
latest_comments INTEGER,
latest_collects INTEGER,
latest_shares INTEGER,
collection_status TEXT NOT NULL DEFAULT 'pending',
collection_status_description TEXT,
collection_updated_at TEXT,
@@ -202,6 +210,7 @@ export async function ensureSchema(database?: DatabaseClient) {
likes INTEGER,
comments INTEGER,
collects INTEGER,
shares INTEGER,
status_description TEXT,
started_at TEXT,
completed_at TEXT,
@@ -293,6 +302,14 @@ export async function ensureSchema(database?: DatabaseClient) {
"public_account_id",
"public_account_id TEXT NOT NULL DEFAULT ''",
);
await ensureColumn("accounts", "gender", "gender TEXT NOT NULL DEFAULT ''");
await ensureColumn("accounts", "bio", "bio TEXT NOT NULL DEFAULT ''");
await ensureColumn("accounts", "tags", "tags TEXT NOT NULL DEFAULT ''");
await ensureColumn(
"accounts",
"current_contact",
"current_contact TEXT NOT NULL DEFAULT ''",
);
await ensureColumn("distributions", "claim_id", "claim_id TEXT");
await ensureColumn(
"distributions",
@@ -349,6 +366,11 @@ export async function ensureSchema(database?: DatabaseClient) {
"last_collection_day",
"last_collection_day INTEGER",
);
await ensureColumn(
"partners",
"wecom_external_user_id",
"wecom_external_user_id TEXT",
);
const tasksWithoutShare = await db
.prepare(
@@ -796,22 +818,44 @@ export async function getDashboardData() {
await Promise.all([
db.prepare("SELECT * FROM partners ORDER BY created_at DESC").all(),
db.prepare("SELECT * FROM tasks ORDER BY created_at DESC").all(),
db.prepare("SELECT * FROM accounts ORDER BY last_seen_at DESC").all(),
db
.prepare(
`SELECT
a.*,
COALESCE(
(
SELECT d.publish_url
FROM distributions d
WHERE d.account_id = a.id
AND TRIM(COALESCE(d.publish_url, '')) != ''
ORDER BY d.updated_at DESC, d.claimed_at DESC
LIMIT 1
),
''
) AS latest_publish_url
FROM accounts a
ORDER BY a.last_seen_at DESC`,
)
.all(),
db
.prepare(
`SELECT
d.*,
c.title AS content_title,
p.name AS partner_name,
cl.claimant_name AS claimant_name,
a.nickname AS account_nickname,
a.platform AS account_platform,
t.name AS task_name,
t.brand AS task_brand,
t.task_type AS task_type,
t.platform AS task_platform,
t.content_format AS content_format,
t.due_at AS due_at
FROM distributions d
JOIN contents c ON c.id = d.content_id
JOIN partners p ON p.id = d.partner_id
LEFT JOIN claims cl ON cl.id = d.claim_id
JOIN tasks t ON t.id = d.task_id
LEFT JOIN accounts a ON a.id = d.account_id
ORDER BY d.updated_at DESC, d.claimed_at DESC`,

View File

@@ -0,0 +1,547 @@
import path from "node:path";
import { strFromU8, unzipSync } from "fflate";
export const PARTNER_BATCH_HEADERS = [
"序号(不能改)",
"标题",
"笔记内容(正文+话题)",
"图片",
"发布链接",
"笔记截图",
"数据分析截图(单篇笔记数据分析截图)",
"_系统笔记ID",
"_原笔记截图",
"_原数据分析截图",
] as const;
export const PARTNER_BATCH_VISIBLE_COLUMN_COUNT = 7;
export const PARTNER_BATCH_MAX_BYTES = 80_000_000;
export type PartnerBatchWorkbookColumns = {
headers: string[];
columnWidths: number[];
sourceImageStartColumn: number;
sourceImageCount: number;
sourceVideoStartColumn: number;
sourceVideoCount: number;
publishUrlColumn: number;
publishScreenshotColumn: number;
creatorScreenshotColumn: number;
systemColumn: number;
};
function nonNegativeInteger(value: number) {
return Number.isFinite(value) ? Math.max(0, Math.floor(value)) : 0;
}
export function buildPartnerBatchWorkbookColumns(input: {
contentFormat: "image_text" | "video";
maxSourceImages: number;
maxSourceVideos: number;
}): PartnerBatchWorkbookColumns {
const sourceImageCount =
input.contentFormat === "video"
? 0
: Math.max(1, nonNegativeInteger(input.maxSourceImages));
const sourceVideoCount =
input.contentFormat === "video"
? Math.max(1, nonNegativeInteger(input.maxSourceVideos))
: 0;
const sourceImageStartColumn = 3;
const sourceVideoStartColumn = sourceImageStartColumn + sourceImageCount;
const publishUrlColumn = sourceVideoStartColumn + sourceVideoCount;
const publishScreenshotColumn = publishUrlColumn + 1;
const creatorScreenshotColumn = publishScreenshotColumn + 1;
const systemColumn = creatorScreenshotColumn + 1;
return {
headers: [
"序号(不能改)",
"标题",
"笔记内容(正文+话题)",
...Array.from(
{ length: sourceImageCount },
(_, index) => `图片${index + 1}`,
),
...Array.from(
{ length: sourceVideoCount },
(_, index) => `视频${index + 1}`,
),
"发布链接",
"笔记截图",
"数据分析截图(单篇笔记数据分析截图)",
"_系统笔记ID",
"_原笔记截图",
"_原数据分析截图",
],
columnWidths: [
14,
30,
62,
...Array.from({ length: sourceImageCount }, () => 24),
...Array.from({ length: sourceVideoCount }, () => 20),
45,
28,
32,
22,
22,
22,
],
sourceImageStartColumn,
sourceImageCount,
sourceVideoStartColumn,
sourceVideoCount,
publishUrlColumn,
publishScreenshotColumn,
creatorScreenshotColumn,
systemColumn,
};
}
function firstForwardedValue(value: string | null) {
return value?.split(",")[0]?.trim() ?? "";
}
function httpOrigin(value: string) {
try {
const url = new URL(value);
return url.protocol === "http:" || url.protocol === "https:"
? url.origin
: "";
} catch {
return "";
}
}
export function resolvePartnerWorkbookOrigin(
request: Request,
configuredOrigin = "",
) {
const requestUrl = new URL(request.url);
const host =
firstForwardedValue(request.headers.get("x-forwarded-host")) ||
firstForwardedValue(request.headers.get("host"));
const forwardedProtocol = firstForwardedValue(
request.headers.get("x-forwarded-proto"),
).toLowerCase();
const protocol = ["http", "https"].includes(forwardedProtocol)
? forwardedProtocol
: requestUrl.protocol.replace(":", "");
const proxyOrigin = host ? httpOrigin(`${protocol}://${host}`) : "";
return (
httpOrigin(configuredOrigin) ||
proxyOrigin ||
httpOrigin(requestUrl.origin) ||
requestUrl.origin
);
}
export type PartnerBatchImage = {
bytes: Uint8Array;
contentType: string;
fileName: string;
};
export type PartnerBatchImportRow = {
spreadsheetRow: number;
sequence: string;
title: string;
publishUrl: string;
distributionId: string;
originalPublishScreenshotKey: string;
originalCreatorScreenshotKey: string;
publishScreenshot: PartnerBatchImage | null;
creatorScreenshot: PartnerBatchImage | null;
};
function decodeXml(value: string) {
return value
.replace(/<[^>]+>/g, "")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&apos;/g, "'")
.replace(/&amp;/g, "&")
.replace(/&#(\d+);/g, (_, code) => String.fromCodePoint(Number(code)))
.replace(/&#x([0-9a-f]+);/gi, (_, code) =>
String.fromCodePoint(Number.parseInt(code, 16)),
);
}
function xmlAttribute(value: string) {
return decodeXml(value);
}
function textNodes(xml: string) {
return [...xml.matchAll(/<t\b[^>]*>([\s\S]*?)<\/t>/g)]
.map((match) => decodeXml(match[1]))
.join("");
}
function columnIndex(reference: string) {
const letters = reference.match(/^[A-Z]+/i)?.[0]?.toUpperCase() ?? "";
let result = 0;
for (const letter of letters) result = result * 26 + letter.charCodeAt(0) - 64;
return Math.max(0, result - 1);
}
function parseWorksheet(xml: string, sharedStrings: string[]) {
const rows: string[][] = [];
for (const rowMatch of xml.matchAll(/<row\b([^>]*)>([\s\S]*?)<\/row>/g)) {
const rowNumber = Number(
rowMatch[1].match(/\br="(\d+)"/)?.[1] ?? rows.length + 1,
);
const values: string[] = [];
for (const cellMatch of rowMatch[2].matchAll(
/<c\b([^>]*)>([\s\S]*?)<\/c>/g,
)) {
const attributes = cellMatch[1];
const body = cellMatch[2];
const reference = attributes.match(/\br="([A-Z]+\d+)"/i)?.[1] ?? "A1";
const type = attributes.match(/\bt="([^"]+)"/)?.[1] ?? "";
const rawValue = body.match(/<v>([\s\S]*?)<\/v>/)?.[1] ?? "";
let value = "";
if (type === "s") value = sharedStrings[Number(rawValue)] ?? "";
else if (type === "inlineStr") value = textNodes(body);
else if (type === "b") value = rawValue === "1" ? "TRUE" : "FALSE";
else value = decodeXml(rawValue);
values[columnIndex(reference)] = value.trim();
}
while (rows.length < rowNumber - 1) rows.push([]);
rows[rowNumber - 1] = values;
}
return rows;
}
function normalizeHeader(value: string) {
return value.replace(/[\s_\-()]/g, "").toLocaleLowerCase("zh-CN");
}
function isSourceImageHeader(value: string) {
return /^(?:图片|发布配图)\d*$/.test(normalizeHeader(value));
}
function headerAliases(header: (typeof PARTNER_BATCH_HEADERS)[number]) {
const aliases: Record<string, string[]> = {
"序号(不能改)": ["序号(不能改)", "序号"],
: ["标题"],
"笔记内容(正文+话题)": ["笔记内容(正文+话题)", "笔记内容"],
: ["图片", "发布配图"],
: ["发布链接"],
: ["笔记截图", "发布截图"],
"数据分析截图(单篇笔记数据分析截图)": [
"数据分析截图(单篇笔记数据分析截图)",
"数据分析截图",
"创作者中心截图",
],
_系统笔记ID: ["_系统笔记ID", "系统笔记ID"],
_原笔记截图: ["_原笔记截图"],
_原数据分析截图: ["_原数据分析截图"],
};
return aliases[header] ?? [header];
}
function findHeader(rows: string[][]) {
for (let rowIndex = 0; rowIndex < Math.min(rows.length, 8); rowIndex += 1) {
const mapping = new Map<(typeof PARTNER_BATCH_HEADERS)[number], number>();
rows[rowIndex].forEach((value, column) => {
for (const header of PARTNER_BATCH_HEADERS) {
if (header === "图片" && isSourceImageHeader(value)) {
if (!mapping.has(header)) mapping.set(header, column);
break;
}
if (
headerAliases(header).some(
(alias) => normalizeHeader(alias) === normalizeHeader(value),
)
) {
mapping.set(header, column);
break;
}
}
});
if (
mapping.has("序号(不能改)") &&
mapping.has("标题") &&
mapping.has("发布链接") &&
mapping.has("笔记截图") &&
mapping.has("数据分析截图(单篇笔记数据分析截图)") &&
mapping.has("_系统笔记ID")
) {
return { rowIndex, mapping };
}
}
return null;
}
function relationshipMap(xml: string) {
const relationships = new Map<string, string>();
for (const match of xml.matchAll(/<Relationship\b([^>]*)\/?\s*>/g)) {
const attributes = match[1];
const id = attributes.match(/\bId="([^"]+)"/)?.[1] ?? "";
const target = attributes.match(/\bTarget="([^"]+)"/)?.[1] ?? "";
if (id && target) relationships.set(id, xmlAttribute(target));
}
return relationships;
}
function contentType(bytes: Uint8Array, fileName: string) {
if (bytes[0] === 0x89 && bytes[1] === 0x50) return "image/png";
if (bytes[0] === 0xff && bytes[1] === 0xd8) return "image/jpeg";
if (bytes[0] === 0x47 && bytes[1] === 0x49 && bytes[2] === 0x46) {
return "image/gif";
}
if (String.fromCharCode(...bytes.slice(8, 12)) === "WEBP") return "image/webp";
const extension = path.extname(fileName).toLowerCase();
return extension === ".png"
? "image/png"
: extension === ".gif"
? "image/gif"
: extension === ".webp"
? "image/webp"
: "image/jpeg";
}
function resolveZipPath(base: string, target: string) {
return path.posix.normalize(path.posix.join(path.posix.dirname(base), target));
}
function parseDrawingImages(entries: Record<string, Uint8Array>) {
const images = new Map<string, PartnerBatchImage>();
const sheetRelationshipsXml = entries["xl/worksheets/_rels/sheet1.xml.rels"]
? strFromU8(entries["xl/worksheets/_rels/sheet1.xml.rels"])
: "";
const sheetRelationships = relationshipMap(sheetRelationshipsXml);
const sheetXml = entries["xl/worksheets/sheet1.xml"]
? strFromU8(entries["xl/worksheets/sheet1.xml"])
: "";
const drawingId = sheetXml.match(/<drawing\b[^>]*r:id="([^"]+)"/)?.[1] ?? "";
const drawingTarget = sheetRelationships.get(drawingId);
if (!drawingTarget) return images;
const drawingPath = resolveZipPath("xl/worksheets/sheet1.xml", drawingTarget);
const drawingXml = entries[drawingPath] ? strFromU8(entries[drawingPath]) : "";
const drawingRelationshipsPath = path.posix.join(
path.posix.dirname(drawingPath),
"_rels",
`${path.posix.basename(drawingPath)}.rels`,
);
const drawingRelationships = relationshipMap(
entries[drawingRelationshipsPath]
? strFromU8(entries[drawingRelationshipsPath])
: "",
);
for (const anchor of drawingXml.matchAll(
/<xdr:(?:oneCellAnchor|twoCellAnchor)\b[^>]*>[\s\S]*?<xdr:from>([\s\S]*?)<\/xdr:from>[\s\S]*?<a:blip\b[^>]*r:embed="([^"]+)"[\s\S]*?<\/xdr:(?:oneCellAnchor|twoCellAnchor)>/g,
)) {
const column = Number(anchor[1].match(/<xdr:col>(\d+)<\/xdr:col>/)?.[1]);
const row = Number(anchor[1].match(/<xdr:row>(\d+)<\/xdr:row>/)?.[1]);
const mediaTarget = drawingRelationships.get(anchor[2]);
if (!Number.isInteger(column) || !Number.isInteger(row) || !mediaTarget) continue;
const mediaPath = resolveZipPath(drawingPath, mediaTarget);
const bytes = entries[mediaPath];
if (!bytes) continue;
images.set(`${row + 1}:${column}`, {
bytes,
contentType: contentType(bytes, mediaPath),
fileName: path.posix.basename(mediaPath),
});
}
return images;
}
function parseRichValueImages(entries: Record<string, Uint8Array>) {
const images = new Map<string, PartnerBatchImage>();
const worksheetXml = entries["xl/worksheets/sheet1.xml"]
? strFromU8(entries["xl/worksheets/sheet1.xml"])
: "";
const metadataXml = entries["xl/metadata.xml"]
? strFromU8(entries["xl/metadata.xml"])
: "";
const richValueXml = entries["xl/richData/rdrichvalue.xml"]
? strFromU8(entries["xl/richData/rdrichvalue.xml"])
: "";
const richValueRelXml = entries["xl/richData/richValueRel.xml"]
? strFromU8(entries["xl/richData/richValueRel.xml"])
: "";
const richValueRelRelationships = relationshipMap(
entries["xl/richData/_rels/richValueRel.xml.rels"]
? strFromU8(entries["xl/richData/_rels/richValueRel.xml.rels"])
: "",
);
if (
!worksheetXml ||
!metadataXml ||
!richValueXml ||
!richValueRelXml ||
!richValueRelRelationships.size
) {
return images;
}
const valueMetadataXml =
metadataXml.match(/<valueMetadata\b[^>]*>([\s\S]*?)<\/valueMetadata>/)?.[1] ??
"";
const metadataToRichValue = [
...valueMetadataXml.matchAll(/<bk\b[^>]*>[\s\S]*?<rc\b[^>]*\bv="(\d+)"[^>]*\/>[\s\S]*?<\/bk>/g),
].map((match) => Number(match[1]));
const richValueToRelationship = [
...richValueXml.matchAll(/<rv\b[^>]*>([\s\S]*?)<\/rv>/g),
].map((match) => Number(match[1].match(/<v>(\d+)<\/v>/)?.[1] ?? -1));
const relationshipIds = [
...richValueRelXml.matchAll(/<rel\b[^>]*\br:id="([^"]+)"[^>]*\/>/g),
].map((match) => match[1]);
for (const cell of worksheetXml.matchAll(
/<c\b([^>]*)>[\s\S]*?<\/c>/g,
)) {
const reference = cell[1].match(/\br="([A-Z]+\d+)"/i)?.[1] ?? "";
const metadataIndex = Number(cell[1].match(/\bvm="(\d+)"/)?.[1] ?? 0);
const row = Number(reference.match(/\d+$/)?.[0] ?? 0);
const column = columnIndex(reference);
if (!reference || !metadataIndex || !row) continue;
const richValueIndex = metadataToRichValue[metadataIndex - 1];
const relationshipIndex = richValueToRelationship[richValueIndex];
const relationshipId = relationshipIds[relationshipIndex];
const mediaTarget = richValueRelRelationships.get(relationshipId);
if (!mediaTarget) continue;
const mediaPath = resolveZipPath(
"xl/richData/richValueRel.xml",
mediaTarget,
);
const bytes = entries[mediaPath];
if (!bytes) continue;
images.set(`${row}:${column}`, {
bytes,
contentType: contentType(bytes, mediaPath),
fileName: path.posix.basename(mediaPath),
});
}
return images;
}
function parseWpsCellImages(entries: Record<string, Uint8Array>) {
const images = new Map<string, PartnerBatchImage>();
const worksheetXml = entries["xl/worksheets/sheet1.xml"]
? strFromU8(entries["xl/worksheets/sheet1.xml"])
: "";
const cellImagesXml = entries["xl/cellimages.xml"]
? strFromU8(entries["xl/cellimages.xml"])
: "";
const relationships = relationshipMap(
entries["xl/_rels/cellimages.xml.rels"]
? strFromU8(entries["xl/_rels/cellimages.xml.rels"])
: "",
);
if (!worksheetXml || !cellImagesXml || !relationships.size) return images;
const imageIdToRelationship = new Map<string, string>();
for (const match of cellImagesXml.matchAll(
/<(?:etc:)?cellImage\b[^>]*>([\s\S]*?)<\/(?:etc:)?cellImage>/g,
)) {
const imageId = match[1].match(
/<(?:xdr:)?cNvPr\b[^>]*\bname="([^"]+)"/,
)?.[1];
const relationshipId = match[1].match(
/<(?:a:)?blip\b[^>]*\br:embed="([^"]+)"/,
)?.[1];
if (imageId && relationshipId) {
imageIdToRelationship.set(imageId, relationshipId);
}
}
for (const cell of worksheetXml.matchAll(/<c\b([^>]*)>([\s\S]*?)<\/c>/g)) {
const reference = cell[1].match(/\br="([A-Z]+\d+)"/i)?.[1] ?? "";
const imageId = decodeXml(cell[2]).match(/DISPIMG\("([^"]+)"/i)?.[1];
if (!reference || !imageId) continue;
const relationshipId = imageIdToRelationship.get(imageId);
const mediaTarget = relationshipId
? relationships.get(relationshipId)
: undefined;
if (!mediaTarget) continue;
const mediaPath = resolveZipPath("xl/cellimages.xml", mediaTarget);
const bytes = entries[mediaPath];
if (!bytes) continue;
const row = Number(reference.match(/\d+$/)?.[0] ?? 0);
if (!row) continue;
images.set(`${row}:${columnIndex(reference)}`, {
bytes,
contentType: contentType(bytes, mediaPath),
fileName: path.posix.basename(mediaPath),
});
}
return images;
}
function parseImages(entries: Record<string, Uint8Array>) {
const images = parseDrawingImages(entries);
for (const [cell, image] of parseRichValueImages(entries)) {
images.set(cell, image);
}
for (const [cell, image] of parseWpsCellImages(entries)) {
images.set(cell, image);
}
return images;
}
function valueAt(
row: string[],
mapping: Map<(typeof PARTNER_BATCH_HEADERS)[number], number>,
header: (typeof PARTNER_BATCH_HEADERS)[number],
) {
const column = mapping.get(header);
return column === undefined ? "" : String(row[column] ?? "").trim();
}
export function parsePartnerBatchWorkbook(input: ArrayBuffer | Uint8Array) {
const bytes = input instanceof Uint8Array ? input : new Uint8Array(input);
if (bytes.byteLength > PARTNER_BATCH_MAX_BYTES) {
throw new Error("批量回填表不能超过80MB");
}
const entries = unzipSync(bytes);
const worksheetBytes = entries["xl/worksheets/sheet1.xml"];
if (!worksheetBytes) throw new Error("Excel 中没有可读取的批量回填工作表");
const sharedXml = entries["xl/sharedStrings.xml"]
? strFromU8(entries["xl/sharedStrings.xml"])
: "";
const sharedStrings = [
...sharedXml.matchAll(/<si\b[^>]*>([\s\S]*?)<\/si>/g),
].map((match) => textNodes(match[1]));
const rows = parseWorksheet(strFromU8(worksheetBytes), sharedStrings);
const header = findHeader(rows);
if (!header) {
throw new Error("表格结构不正确,请使用本领取页面导出的批量回填表");
}
const images = parseImages(entries);
const publishScreenshotColumn = header.mapping.get("笔记截图")!;
const creatorScreenshotColumn = header.mapping.get(
"数据分析截图(单篇笔记数据分析截图)",
)!;
const result: PartnerBatchImportRow[] = [];
for (let index = header.rowIndex + 1; index < rows.length; index += 1) {
const row = rows[index];
const distributionId = valueAt(row, header.mapping, "_系统笔记ID");
if (!distributionId && !row.some((value) => String(value ?? "").trim())) continue;
result.push({
spreadsheetRow: index + 1,
sequence: valueAt(row, header.mapping, "序号(不能改)"),
title: valueAt(row, header.mapping, "标题"),
publishUrl: valueAt(row, header.mapping, "发布链接"),
distributionId,
originalPublishScreenshotKey: valueAt(
row,
header.mapping,
"_原笔记截图",
),
originalCreatorScreenshotKey: valueAt(
row,
header.mapping,
"_原数据分析截图",
),
publishScreenshot:
images.get(`${index + 1}:${publishScreenshotColumn}`) ?? null,
creatorScreenshot:
images.get(`${index + 1}:${creatorScreenshotColumn}`) ?? null,
});
}
if (result.length === 0) throw new Error("表格中没有可回填的笔记");
return result;
}

View File

@@ -1,22 +1,31 @@
import { hashText } from "./mvp-db";
import {
extractPublishUrl,
extractXhsPublishUrl,
platformFromPublishUrl,
safeHttpUrl,
type SupportedPlatform,
} from "./publish-url";
export { extractXhsPublishUrl } from "./publish-url";
export function accountFromPublishLink(input: string) {
const url = safeHttpUrl(extractXhsPublishUrl(input));
export function accountFromPublishLink(
input: string,
expectedPlatform?: SupportedPlatform,
) {
const extracted = expectedPlatform
? extractPublishUrl(input, expectedPlatform)
: extractXhsPublishUrl(input) || extractPublishUrl(input, "抖音");
const url = safeHttpUrl(extracted);
if (!url) return null;
const platform =
url.hostname.includes("xiaohongshu") || url.hostname.includes("xhslink")
? "小红书"
: "其他平台";
const platform = platformFromPublishUrl(url.toString());
if (!platform || (expectedPlatform && platform !== expectedPlatform)) return null;
const noteId =
url.pathname.match(
/\/(?:discovery\/item|explore)\/([A-Za-z0-9_-]{8,80})/,
)?.[1] ?? "";
platform === "抖音"
? url.pathname.match(/\/(?:video|note)\/([A-Za-z0-9_-]{8,80})/)?.[1] ?? ""
: url.pathname.match(
/\/(?:discovery\/item|explore)\/([A-Za-z0-9_-]{8,80})/,
)?.[1] ?? "";
const platformUid = `pending-${hashText(
noteId || `${url.origin}${url.pathname}`,
)}`;

View File

@@ -25,3 +25,44 @@ export function extractXhsPublishUrl(input: string) {
}
return "";
}
export type SupportedPlatform = "小红书" | "抖音";
function platformMatches(url: URL, platform: SupportedPlatform) {
const hostname = url.hostname.toLowerCase();
if (platform === "抖音") {
return hostname === "douyin.com" || hostname.endsWith(".douyin.com");
}
return (
hostname === "xiaohongshu.com" ||
hostname.endsWith(".xiaohongshu.com") ||
hostname === "xhslink.cn" ||
hostname.endsWith(".xhslink.cn")
);
}
export function extractPublishUrl(
input: string,
platform: SupportedPlatform = "小红书",
) {
const candidates =
input.match(/https?:\/\/[^\s<>"',。!?;:、【】()]+/gi) ?? [];
for (const candidate of candidates) {
const cleaned = candidate.replace(/[.,!?;:~)\]}]+$/g, "");
const url = safeHttpUrl(cleaned);
if (url && platformMatches(url, platform)) return url.toString();
}
return "";
}
export function extractAnyPublishUrl(input: string) {
return extractPublishUrl(input, "小红书") || extractPublishUrl(input, "抖音");
}
export function platformFromPublishUrl(input: string): SupportedPlatform | "" {
const url = safeHttpUrl(input);
if (!url) return "";
if (platformMatches(url, "小红书")) return "小红书";
if (platformMatches(url, "抖音")) return "抖音";
return "";
}

View File

@@ -13,6 +13,9 @@ export type RecoveryWorkbookRow = {
images: Array<{
column: number;
image: RecoveryWorkbookImage;
offsetX?: number;
maxWidth?: number;
maxHeight?: number;
}>;
hyperlinks?: Array<{
column: number;
@@ -25,6 +28,7 @@ type WorkbookOptions = {
headers: string[];
columnWidths: number[];
rows: RecoveryWorkbookRow[];
hiddenColumns?: number[];
};
const XML_HEADER = '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>';
@@ -130,9 +134,17 @@ function imageDimensions(image: RecoveryWorkbookImage) {
return { width: 4, height: 3 };
}
function imageDisplaySize(image: RecoveryWorkbookImage) {
function imageDisplaySize(
image: RecoveryWorkbookImage,
maxWidth = 160,
maxHeight = 150,
) {
const dimensions = imageDimensions(image);
const scale = Math.min(160 / dimensions.width, 150 / dimensions.height, 1);
const scale = Math.min(
maxWidth / dimensions.width,
maxHeight / dimensions.height,
1,
);
return {
width: Math.max(28, Math.round(dimensions.width * scale)),
height: Math.max(28, Math.round(dimensions.height * scale)),
@@ -152,6 +164,14 @@ export function buildRecoveryWorkbook(options: WorkbookOptions) {
const imageEntries = options.rows.flatMap((row, rowIndex) =>
row.images.map((item) => ({ ...item, row: rowIndex + 1 })),
);
const imageCells = new Set<string>();
imageEntries.forEach((entry) => {
const key = `${entry.row}:${entry.column}`;
if (imageCells.has(key)) {
throw new Error("Excel 单元格内只能嵌入一张图片,请为每张图片分配独立列");
}
imageCells.add(key);
});
const hyperlinkEntries = options.rows.flatMap((row, rowIndex) =>
(row.hyperlinks ?? [])
.map((item) => ({
@@ -178,7 +198,7 @@ export function buildRecoveryWorkbook(options: WorkbookOptions) {
const reference = `${columnName(columnIndex)}${number}`;
const value = row.cells[columnIndex] ?? "";
if (imageColumns.has(columnIndex)) {
return inlineCell(reference, value || "见图", 4);
return inlineCell(reference, value, 4);
}
return typeof value === "number"
? numberCell(reference, value, 3)
@@ -196,17 +216,22 @@ export function buildRecoveryWorkbook(options: WorkbookOptions) {
const columns = options.headers
.map((_, index) => {
const width = Math.min(70, Math.max(8, options.columnWidths[index] ?? 14));
return `<col min="${index + 1}" max="${index + 1}" width="${width}" customWidth="1"/>`;
const hidden = options.hiddenColumns?.includes(index) ? ' hidden="1"' : "";
return `<col min="${index + 1}" max="${index + 1}" width="${width}" customWidth="1"${hidden}/>`;
})
.join("");
const drawingXml = imageEntries.length
? `${XML_HEADER}<xdr:wsDr xmlns:xdr="http://schemas.openxmlformats.org/drawingml/2006/spreadsheetDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships">${imageEntries
.map((entry, index) => {
const size = imageDisplaySize(entry.image);
const width = size.width * 9525;
const height = size.height * 9525;
return `<xdr:oneCellAnchor><xdr:from><xdr:col>${entry.column}</xdr:col><xdr:colOff>57150</xdr:colOff><xdr:row>${entry.row}</xdr:row><xdr:rowOff>57150</xdr:rowOff></xdr:from><xdr:ext cx="${width}" cy="${height}"/><xdr:pic><xdr:nvPicPr><xdr:cNvPr id="${index + 1}" name="${cleanXmlText(entry.image.description || `图片${index + 1}`)}"/><xdr:cNvPicPr><a:picLocks noChangeAspect="1"/></xdr:cNvPicPr></xdr:nvPicPr><xdr:blipFill><a:blip r:embed="rId${index + 1}"/><a:stretch><a:fillRect/></a:stretch></xdr:blipFill><xdr:spPr><a:prstGeom prst="rect"><a:avLst/></a:prstGeom></xdr:spPr></xdr:pic><xdr:clientData/></xdr:oneCellAnchor>`;
const size = imageDisplaySize(
entry.image,
entry.maxWidth ?? 160,
entry.maxHeight ?? 150,
);
const offsetX = entry.offsetX ?? 6;
const offsetY = 6;
return `<xdr:twoCellAnchor editAs="twoCell"><xdr:from><xdr:col>${entry.column}</xdr:col><xdr:colOff>${offsetX * 9525}</xdr:colOff><xdr:row>${entry.row}</xdr:row><xdr:rowOff>${offsetY * 9525}</xdr:rowOff></xdr:from><xdr:to><xdr:col>${entry.column}</xdr:col><xdr:colOff>${(offsetX + size.width) * 9525}</xdr:colOff><xdr:row>${entry.row}</xdr:row><xdr:rowOff>${(offsetY + size.height) * 9525}</xdr:rowOff></xdr:to><xdr:pic><xdr:nvPicPr><xdr:cNvPr id="${index + 1}" name="${cleanXmlText(entry.image.description || `图片${index + 1}`)}"/><xdr:cNvPicPr><a:picLocks noChangeAspect="1"/></xdr:cNvPicPr></xdr:nvPicPr><xdr:blipFill><a:blip r:embed="rId${index + 1}"/><a:stretch><a:fillRect/></a:stretch></xdr:blipFill><xdr:spPr><a:prstGeom prst="rect"><a:avLst/></a:prstGeom></xdr:spPr></xdr:pic><xdr:clientData/></xdr:twoCellAnchor>`;
})
.join("")}</xdr:wsDr>`
: "";
@@ -227,7 +252,10 @@ export function buildRecoveryWorkbook(options: WorkbookOptions) {
const imageContentTypes = [...imageFormats.entries()]
.map(([extension, contentType]) => `<Default Extension="${extension}" ContentType="${contentType}"/>`)
.join("");
const contentTypes = `${XML_HEADER}<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="xml" ContentType="application/xml"/>${imageContentTypes}<Override PartName="/xl/workbook.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml"/><Override PartName="/xl/worksheets/sheet1.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml"/><Override PartName="/xl/styles.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.styles+xml"/>${imageEntries.length ? '<Override PartName="/xl/drawings/drawing1.xml" ContentType="application/vnd.openxmlformats-officedocument.drawing+xml"/>' : ""}<Override PartName="/docProps/core.xml" ContentType="application/vnd.openxmlformats-package.core-properties+xml"/><Override PartName="/docProps/app.xml" ContentType="application/vnd.openxmlformats-officedocument.extended-properties+xml"/></Types>`;
const drawingContentType = imageEntries.length
? '<Override PartName="/xl/drawings/drawing1.xml" ContentType="application/vnd.openxmlformats-officedocument.drawing+xml"/>'
: "";
const contentTypes = `${XML_HEADER}<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="xml" ContentType="application/xml"/>${imageContentTypes}<Override PartName="/xl/workbook.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml"/><Override PartName="/xl/worksheets/sheet1.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml"/><Override PartName="/xl/styles.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.styles+xml"/>${drawingContentType}<Override PartName="/docProps/core.xml" ContentType="application/vnd.openxmlformats-package.core-properties+xml"/><Override PartName="/docProps/app.xml" ContentType="application/vnd.openxmlformats-officedocument.extended-properties+xml"/></Types>`;
const hyperlinkRelationshipOffset = imageEntries.length ? 2 : 1;
const hyperlinksXml = hyperlinkEntries.length
? `<hyperlinks>${hyperlinkEntries
@@ -266,7 +294,9 @@ export function buildRecoveryWorkbook(options: WorkbookOptions) {
}
if (imageEntries.length) {
files["xl/drawings/drawing1.xml"] = strToU8(drawingXml);
files["xl/drawings/_rels/drawing1.xml.rels"] = strToU8(drawingRelationships);
files["xl/drawings/_rels/drawing1.xml.rels"] = strToU8(
drawingRelationships,
);
imageEntries.forEach((entry, index) => {
const format = imageFormat(entry.image.contentType, entry.image.bytes);
files[`xl/media/image${index + 1}.${format.extension}`] = entry.image.bytes;

View File

@@ -1,7 +1,7 @@
import { strFromU8, unzipSync } from "fflate";
export const RESOURCE_IMPORT_MAX_ROWS = 100;
export const RESOURCE_IMPORT_MAX_BYTES = 5 * 1024 * 1024;
export const RESOURCE_IMPORT_MAX_ROWS = 10_000;
export const RESOURCE_IMPORT_MAX_BYTES = 20 * 1024 * 1024;
export type ResourceImportRow = {
rowNumber: number;
@@ -12,6 +12,9 @@ export type ResourceImportRow = {
ipLocation: string;
followers: number;
followersResolved: boolean;
gender: "" | "男" | "女";
bio: string;
tags: string[];
cooperationSource: string;
errors: string[];
};
@@ -22,6 +25,9 @@ const HEADER_ALIASES = {
publicAccountId: ["账号ID", "账号号", "小红书号", "抖音号"],
ipLocation: ["IP属地", "IP地址", "IP地区", "IP所在地"],
followers: ["粉丝数", "粉丝", "粉丝量"],
gender: ["性别"],
bio: ["简介", "账号简介", "个人简介"],
tags: ["标签", "账号标签"],
cooperationSource: ["合作来源", "资源来源", "历史合作来源"],
} as const;
@@ -60,9 +66,11 @@ function parseWorksheet(xml: string, sharedStrings: string[]) {
const rowAttributes = rowMatch[1];
const rowNumber = Number(rowAttributes.match(/\br="(\d+)"/)?.[1] ?? rows.length + 1);
const values: string[] = [];
for (const cellMatch of rowMatch[2].matchAll(/<c\b([^>]*)>([\s\S]*?)<\/c>/g)) {
for (const cellMatch of rowMatch[2].matchAll(
/<c\b([^>]*?)(?:\/>|>([\s\S]*?)<\/c>)/g,
)) {
const attributes = cellMatch[1];
const body = cellMatch[2];
const body = cellMatch[2] ?? "";
const reference = attributes.match(/\br="([A-Z]+\d+)"/i)?.[1] ?? "A1";
const type = attributes.match(/\bt="([^"]+)"/)?.[1] ?? "";
const rawValue = body.match(/<v>([\s\S]*?)<\/v>/)?.[1] ?? "";
@@ -183,6 +191,15 @@ export function platformFromProfileUrl(profileUrl: string) {
) {
return "小红书";
}
if (
((url.hostname === "douyin.com" || url.hostname.endsWith(".douyin.com")) &&
/^\/user\/[^/]+/i.test(url.pathname)) ||
((url.hostname === "iesdouyin.com" ||
url.hostname.endsWith(".iesdouyin.com")) &&
/^\/share\/user\/[^/]+/i.test(url.pathname))
) {
return "抖音";
}
} catch {
// URL validation is reported by normalizeRows.
}
@@ -212,10 +229,42 @@ export function parseResourceFollowers(value: string) {
};
}
export function normalizeResourceGender(value: string) {
const normalized = value.trim().toLocaleLowerCase("zh-CN");
if (!normalized || ["未知", "未填写", "待识别", "unknown"].includes(normalized)) {
return { value: "" as const, valid: true };
}
if (["男", "男性", "male", "m"].includes(normalized)) {
return { value: "男" as const, valid: true };
}
if (["女", "女性", "female", "f"].includes(normalized)) {
return { value: "女" as const, valid: true };
}
return { value: "" as const, valid: false };
}
export function normalizeResourceTags(value: string | string[]) {
const source = Array.isArray(value) ? value.join(",") : value;
return [
...new Set(
source
.split(/[,,、;|]/)
.map((item) => item.trim().replace(/^#+/, ""))
.filter(Boolean),
),
];
}
export function resourceImportMissingFields(
row: Pick<
ResourceImportRow,
"nickname" | "publicAccountId" | "ipLocation" | "followersResolved"
| "nickname"
| "publicAccountId"
| "ipLocation"
| "followersResolved"
| "gender"
| "bio"
| "tags"
>,
) {
const missing: string[] = [];
@@ -225,6 +274,8 @@ export function resourceImportMissingFields(
missing.push("ipLocation");
}
if (!row.followersResolved) missing.push("followers");
if (!row.gender) missing.push("gender");
if (!row.bio.trim()) missing.push("bio");
return missing;
}
@@ -242,22 +293,39 @@ function normalizeRows(rows: string[][]) {
const platform = platformFromProfileUrl(profileUrl);
const rawFollowers = valueAt(source, header.mapping, "followers");
const parsedFollowers = parseResourceFollowers(rawFollowers);
const parsedGender = normalizeResourceGender(
valueAt(source, header.mapping, "gender"),
);
const tags = normalizeResourceTags(valueAt(source, header.mapping, "tags"));
const ipLocation = valueAt(source, header.mapping, "ipLocation");
const errors: string[] = [];
if (!rawProfileUrl) errors.push("账号主页不能为空");
else if (!profileUrl) errors.push("账号主页链接格式不正确");
else if (!platform) errors.push("当前自动解析仅支持小红书账号主页");
else if (!platform) errors.push("当前自动解析仅支持小红书或抖音账号主页");
if (!parsedFollowers.valid) {
errors.push("粉丝数格式不正确,请填写数字或如 1.3万、10+");
}
if (!parsedGender.valid) {
errors.push("性别格式不正确,请填写男、女或留空");
}
if (tags.length > 5) {
errors.push("单个账号最多填写 5 个标签,请用逗号分隔");
}
if (/^\d+$/.test(ipLocation)) {
errors.push("IP属地格式不正确请填写省份、地区或国家名称");
}
result.push({
rowNumber: index + 1,
platform,
nickname: valueAt(source, header.mapping, "nickname"),
publicAccountId: valueAt(source, header.mapping, "publicAccountId"),
profileUrl,
ipLocation: valueAt(source, header.mapping, "ipLocation"),
ipLocation,
followers: parsedFollowers.value,
followersResolved: parsedFollowers.resolved,
gender: parsedGender.value,
bio: valueAt(source, header.mapping, "bio"),
tags: tags.slice(0, 5),
cooperationSource: valueAt(source, header.mapping, "cooperationSource"),
errors,
});
@@ -269,6 +337,59 @@ function normalizeRows(rows: string[][]) {
return result;
}
export type ManualResourceInput = {
profileUrl: string;
nickname?: string;
publicAccountId?: string;
ipLocation?: string;
followers?: string;
gender?: string;
bio?: string;
tags?: string | string[];
cooperationSource?: string;
};
export function buildManualRow(input: ManualResourceInput): ResourceImportRow {
const rawProfileUrl = (input.profileUrl ?? "").trim();
const profileUrl = normalizeProfileUrl(rawProfileUrl);
const platform = platformFromProfileUrl(profileUrl);
const parsedFollowers = parseResourceFollowers(input.followers ?? "");
const parsedGender = normalizeResourceGender(input.gender ?? "");
const tags = normalizeResourceTags(input.tags ?? "");
const ipLocation = (input.ipLocation ?? "").trim();
const errors: string[] = [];
if (!rawProfileUrl) errors.push("账号主页不能为空");
else if (!profileUrl) errors.push("账号主页链接格式不正确");
else if (!platform) errors.push("当前自动解析仅支持小红书或抖音账号主页");
if (!parsedFollowers.valid) {
errors.push("粉丝数格式不正确,请填写数字或如 1.3万、10+");
}
if (!parsedGender.valid) {
errors.push("性别格式不正确,请填写男、女或留空");
}
if (tags.length > 5) {
errors.push("单个账号最多填写 5 个标签,请用逗号分隔");
}
if (ipLocation && /^\d+$/.test(ipLocation)) {
errors.push("IP属地格式不正确请填写省份、地区或国家名称");
}
return {
rowNumber: 1,
platform,
nickname: (input.nickname ?? "").trim(),
publicAccountId: (input.publicAccountId ?? "").trim(),
profileUrl,
ipLocation,
followers: parsedFollowers.value,
followersResolved: parsedFollowers.resolved,
gender: parsedGender.value,
bio: (input.bio ?? "").trim(),
tags: tags.slice(0, 5),
cooperationSource: (input.cooperationSource ?? "").trim(),
errors,
};
}
export function parseResourceImportFile(fileName: string, bytes: Uint8Array) {
const extension = fileName.toLocaleLowerCase().split(".").pop();
const workbooks =

366
lib/resource-write.ts Normal file
View File

@@ -0,0 +1,366 @@
import { ensureSchema, getRawDb } from "./mvp-db";
import {
resolveCollectionMcpConfig,
resolveProfileDetailsFromMcp,
resolveXhsPublicAccountDetails,
type CollectionMcpBindings,
} from "./mcp-collection-client";
import { getRuntimeEnv } from "./runtime-env";
import {
mergeCooperationSources,
normalizeProfileUrl,
resourceImportMissingFields,
resourcePlatformUid,
type ResourceImportRow,
} from "./resource-import";
export type AccountRow = {
id: string;
platform: string;
platform_uid: string;
public_account_id: string;
nickname: string;
profile_url: string;
ip_location: string;
followers: number;
gender: string;
bio: string;
tags: string;
cooperation_source: string;
};
export type AnalyzedRow = ResourceImportRow & {
action: "create" | "update" | "error";
accountId: string;
platformUid: string;
cooperationSource: string;
};
export const RESOURCE_IMPORT_SYNC_ENRICH_ROWS = 100;
const RESOURCE_IMPORT_DB_BATCH_SIZE = 100;
export function identityKey(platform: string, value: string) {
return `${platform.trim().toLocaleLowerCase("zh-CN")}|${value
.trim()
.toLocaleLowerCase("zh-CN")}`;
}
export async function loadAccounts() {
await ensureSchema();
return getRawDb()
.prepare(
`SELECT id, platform, platform_uid, public_account_id, nickname,
profile_url, ip_location, followers, gender, bio, tags,
cooperation_source
FROM accounts`,
)
.all<AccountRow>();
}
async function mapConcurrent<T, R>(
items: T[],
limit: number,
worker: (item: T) => Promise<R>,
) {
const results = new Array<R>(items.length);
let cursor = 0;
await Promise.all(
Array.from({ length: Math.min(limit, items.length) }, async () => {
while (cursor < items.length) {
const index = cursor;
cursor += 1;
results[index] = await worker(items[index]);
}
}),
);
return results;
}
export function mergeExistingFields(
rows: ResourceImportRow[],
accounts: AccountRow[],
) {
const existingByProfile = new Map<string, AccountRow>();
for (const account of accounts) {
const profileUrl = normalizeProfileUrl(account.profile_url || "");
if (profileUrl) {
existingByProfile.set(identityKey(account.platform, profileUrl), account);
}
}
return rows.map((row) => {
if (
row.errors.length > 0 ||
!row.profileUrl ||
!["小红书", "抖音"].includes(row.platform)
) {
return row;
}
const existing = existingByProfile.get(identityKey(row.platform, row.profileUrl));
const existingNickname =
existing?.nickname && existing.nickname !== "待识别账号"
? existing.nickname
: "";
const existingIpLocation =
existing?.ip_location && existing.ip_location !== "待识别"
? existing.ip_location
: "";
const existingGender: ResourceImportRow["gender"] =
existing?.gender === "男" || existing?.gender === "女"
? existing.gender
: "";
return {
...row,
nickname: row.nickname || existingNickname,
publicAccountId: row.publicAccountId || existing?.public_account_id || "",
ipLocation: row.ipLocation || existingIpLocation,
followers: row.followersResolved
? row.followers
: Number(existing?.followers || 0),
followersResolved:
row.followersResolved || Number(existing?.followers || 0) > 0,
gender: row.gender || existingGender,
bio: row.bio || existing?.bio || "",
tags: row.tags.length > 0
? row.tags
: (existing?.tags || "")
.split(/[,,、;|]/)
.map((item) => item.trim())
.filter(Boolean)
.slice(0, 5),
};
});
}
export async function enrichRows(
rows: ResourceImportRow[],
accounts: AccountRow[],
) {
const mcpConfig = resolveCollectionMcpConfig(
getRuntimeEnv() as unknown as CollectionMcpBindings,
);
const baselineRows = mergeExistingFields(rows, accounts);
return mapConcurrent(baselineRows, 4, async (baseline) => {
const row = baseline;
if (
row.errors.length > 0 ||
!row.profileUrl ||
!["小红书", "抖音"].includes(row.platform)
) {
return row;
}
if (resourceImportMissingFields(baseline).length === 0) {
return baseline;
}
let details: {
nickname: string | null;
redId: string | null;
followers: number | null;
ipLocation: string | null;
gender: "" | "男" | "女";
bio: string;
recentNoteTitles: string[];
providerTags: string[];
} = await resolveProfileDetailsFromMcp(
row.profileUrl,
row.platform === "抖音" ? "抖音" : "小红书",
mcpConfig,
).catch(() => ({
nickname: null,
redId: null,
followers: null,
ipLocation: null,
gender: "" as const,
bio: "",
recentNoteTitles: [],
providerTags: [],
}));
const mcpResult = {
nickname: baseline.nickname || details.nickname?.trim() || "",
publicAccountId: baseline.publicAccountId || details.redId?.trim() || "",
ipLocation: baseline.ipLocation || details.ipLocation?.trim() || "",
followersResolved: baseline.followersResolved || details.followers !== null,
gender: baseline.gender || details.gender,
bio: baseline.bio || details.bio,
tags: baseline.tags,
};
if (
row.platform === "小红书" &&
resourceImportMissingFields(mcpResult).length > 0
) {
const publicDetails = await resolveXhsPublicAccountDetails(row.profileUrl).catch(
() => ({ nickname: null, redId: null, followers: null, ipLocation: null }),
);
details = {
...details,
nickname: details.nickname || publicDetails.nickname,
redId: details.redId || publicDetails.redId,
followers: details.followers ?? publicDetails.followers,
ipLocation: details.ipLocation || publicDetails.ipLocation,
};
}
return {
...baseline,
nickname: baseline.nickname || details.nickname?.trim() || "待识别账号",
publicAccountId: baseline.publicAccountId || details.redId?.trim() || "",
ipLocation: baseline.ipLocation || details.ipLocation?.trim() || "待识别",
followers: baseline.followersResolved
? baseline.followers
: (details.followers ?? 0),
followersResolved:
baseline.followersResolved || details.followers !== null,
gender: baseline.gender || details.gender,
bio: baseline.bio || details.bio,
tags: baseline.tags,
};
});
}
export function analyzeRows(
rows: ResourceImportRow[],
accountRows: AccountRow[],
) {
const profileMap = new Map<string, AccountRow>();
const publicIdMap = new Map<string, AccountRow>();
const platformUidMap = new Map<string, AccountRow>();
for (const account of accountRows) {
const profileUrl = normalizeProfileUrl(account.profile_url || "");
if (profileUrl) profileMap.set(identityKey(account.platform, profileUrl), account);
if (account.public_account_id) {
publicIdMap.set(identityKey(account.platform, account.public_account_id), account);
}
platformUidMap.set(identityKey(account.platform, account.platform_uid), account);
}
return rows.map<AnalyzedRow>((row) => {
const platformUid = resourcePlatformUid(row);
const profileMatch = row.profileUrl
? profileMap.get(identityKey(row.platform, row.profileUrl))
: undefined;
const publicIdMatch = row.publicAccountId
? publicIdMap.get(identityKey(row.platform, row.publicAccountId))
: undefined;
const uidMatch = platformUidMap.get(identityKey(row.platform, platformUid));
const matches = [profileMatch, publicIdMatch, uidMatch].filter(
(account): account is AccountRow => Boolean(account),
);
const matchedIds = [...new Set(matches.map((account) => account.id))];
const errors = [...row.errors];
if (matchedIds.length > 1) {
errors.push("账号主页和账号ID匹配到不同的现有账号请先核对");
}
const existing = matchedIds.length === 1 ? matches[0] : undefined;
const accountId = existing?.id ?? `account-${crypto.randomUUID().slice(0, 12)}`;
const analyzed: AnalyzedRow = {
...row,
errors,
action: errors.length > 0 ? "error" : existing ? "update" : "create",
accountId,
platformUid: existing?.platform_uid ?? platformUid,
cooperationSource: mergeCooperationSources(
existing?.cooperation_source ?? "",
row.cooperationSource,
),
};
if (analyzed.action !== "error") {
const virtual: AccountRow = {
id: accountId,
platform: row.platform,
platform_uid: analyzed.platformUid,
public_account_id: row.publicAccountId || existing?.public_account_id || "",
nickname: row.nickname,
profile_url: row.profileUrl || existing?.profile_url || "",
ip_location: row.ipLocation || existing?.ip_location || "待识别",
followers: row.followers || existing?.followers || 0,
gender: row.gender || existing?.gender || "",
bio: row.bio || existing?.bio || "",
tags: (row.tags.length > 0
? row.tags
: (existing?.tags || "").split(/[,,、;|]/).filter(Boolean)
).slice(0, 5).join(","),
cooperation_source: analyzed.cooperationSource,
};
if (row.profileUrl) profileMap.set(identityKey(row.platform, row.profileUrl), virtual);
if (row.publicAccountId) {
publicIdMap.set(identityKey(row.platform, row.publicAccountId), virtual);
}
platformUidMap.set(identityKey(row.platform, analyzed.platformUid), virtual);
}
return analyzed;
});
}
function statementForAnalyzedRow(
db: ReturnType<typeof getRawDb>,
row: AnalyzedRow,
) {
return row.action === "update"
? db
.prepare(
`UPDATE accounts SET
nickname = ?,
public_account_id = CASE WHEN ? != '' THEN ? ELSE public_account_id END,
profile_url = CASE WHEN ? != '' THEN ? ELSE profile_url END,
ip_location = CASE
WHEN ? != '' AND ? != '待识别' THEN ? ELSE ip_location END,
followers = CASE WHEN ? = 1 THEN ? ELSE followers END,
gender = CASE WHEN ? != '' THEN ? ELSE gender END,
bio = CASE WHEN ? != '' THEN ? ELSE bio END,
tags = CASE WHEN ? != '' THEN ? ELSE tags END,
cooperation_source = ?,
last_seen_at = CURRENT_TIMESTAMP
WHERE id = ?`,
)
.bind(
row.nickname || "待识别账号",
row.publicAccountId,
row.publicAccountId,
row.profileUrl,
row.profileUrl,
row.ipLocation,
row.ipLocation,
row.ipLocation,
row.followersResolved ? 1 : 0,
row.followers,
row.gender,
row.gender,
row.bio,
row.bio,
row.tags.join(","),
row.tags.join(","),
row.cooperationSource,
row.accountId,
)
: db
.prepare(
`INSERT INTO accounts
(id, platform, platform_uid, public_account_id, nickname,
profile_url, ip_location, followers, post_count, avg_views,
gender, bio, tags, cooperation_source)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, 0, 0, ?, ?, ?, ?)`,
)
.bind(
row.accountId,
row.platform,
row.platformUid,
row.publicAccountId,
row.nickname || "待识别账号",
row.profileUrl,
row.ipLocation || "待识别",
row.followers,
row.gender,
row.bio,
row.tags.join(","),
row.cooperationSource,
);
}
export async function writeAnalyzedRows(rows: AnalyzedRow[]) {
const db = getRawDb();
const statements = rows
.filter((row) => row.action !== "error")
.map((row) => statementForAnalyzedRow(db, row));
for (let index = 0; index < statements.length; index += RESOURCE_IMPORT_DB_BATCH_SIZE) {
await db.batch(statements.slice(index, index + RESOURCE_IMPORT_DB_BATCH_SIZE));
}
}

View File

@@ -20,6 +20,12 @@ export type RuntimeEnv = {
AI_TOOL_CENTER_MCP_KEY?: string;
COLLECTION_MCP_URL?: string;
COLLECTION_MCP_KEY?: string;
WECOM_CORP_ID?: string;
WECOM_AGENT_ID?: string;
WECOM_SECRET?: string;
WECOM_ROBOT_WEBHOOK?: string;
WECOM_NOTIFY_DUE_DAYS?: string;
WECOM_NOTIFY_ENABLED?: string;
SEED_DEMO_DATA?: string;
ENABLE_SCHEDULER?: string;
};

View File

@@ -8,6 +8,11 @@ import {
} from "./mcp-collection-client";
import { ensureSchema, getRawDb } from "./mvp-db";
import { getRuntimeEnv, isEnabled } from "./runtime-env";
import {
resolveWecomConfig,
type WecomBindings,
} from "./wecom-client";
import { runDueSoonWecomNotifications } from "./wecom-notifier-service";
declare global {
var __kocLoopScheduler: ScheduledTask | undefined;
@@ -17,14 +22,31 @@ async function runDailyJob() {
await withDatabaseLock("koc-loop-daily-collection", 0, async () => {
await ensureSchema();
const db = getRawDb();
const env = getRuntimeEnv();
const config = resolveCollectionMcpConfig(
getRuntimeEnv() as unknown as CollectionMcpBindings,
env as unknown as CollectionMcpBindings,
);
const collections = await runScheduledCollections(db, Date.now(), config);
const accounts = await backfillAccountProfiles(db, config, 10);
let wecom: Awaited<ReturnType<typeof runDueSoonWecomNotifications>> | null =
null;
if (isEnabled(env.WECOM_NOTIFY_ENABLED, true)) {
const wecomConfig = resolveWecomConfig(env as unknown as WecomBindings);
if (
wecomConfig.robotWebhook ||
(wecomConfig.corpId && wecomConfig.agentId && wecomConfig.secret)
) {
try {
wecom = await runDueSoonWecomNotifications(db, wecomConfig);
} catch (error) {
console.error("[KOC LOOP] wecom notify failed", error);
}
}
}
console.info("[KOC LOOP] daily scheduler completed", {
collections,
accounts,
wecom,
});
});
}

View File

@@ -10,6 +10,8 @@ export type CreateDistributionTaskInput = {
name: string;
brand: string;
dueAt: string;
platform?: "小红书" | "抖音";
contentFormat?: "image_text" | "video";
};
export type CreateScreenshotTaskInput = {
@@ -33,6 +35,8 @@ export type DistributionTaskCreation = {
sheetId: string;
sheetName: string;
sourceUrl: string;
platform: "小红书" | "抖音";
contentFormat: "image_text" | "video";
};
export type ScreenshotTaskCreation = {
@@ -55,6 +59,8 @@ type TaskRow = {
source_url: string;
source_sheet_id: string;
source_sheet_name: string;
platform: "小红书" | "抖音";
content_format: "image_text" | "video";
};
function normalizedValue(value: string) {
@@ -99,17 +105,27 @@ async function findExistingTask(
return db
.prepare(
`SELECT id, share_token, name, brand, due_at, quantity,
source_url, source_sheet_id, source_sheet_name
source_url, source_sheet_id, source_sheet_name,
platform, content_format
FROM tasks
WHERE name = ?
AND brand = ?
AND due_at = ?
AND source_url = ?
AND platform = ?
AND content_format = ?
AND status IN ('active', 'importing')
ORDER BY created_at DESC
LIMIT 1`,
)
.bind(input.name, input.brand, input.dueAt, sourceUrl)
.bind(
input.name,
input.brand,
input.dueAt,
sourceUrl,
input.platform ?? "小红书",
input.contentFormat ?? "image_text",
)
.first<TaskRow>();
}
@@ -124,9 +140,10 @@ async function insertTaskFromSource(
.prepare(
`INSERT INTO tasks
(id, name, brand, quantity, claimed_quantity, due_at, status,
platform, content_format,
source_url, source_sheet_id, source_sheet_name, source_synced_at,
share_token)
VALUES (?, ?, ?, ?, 0, ?, 'importing', ?, ?, ?, ?, ?)`,
VALUES (?, ?, ?, ?, 0, ?, 'importing', ?, ?, ?, ?, ?, ?, ?)`,
)
.bind(
taskId,
@@ -134,6 +151,8 @@ async function insertTaskFromSource(
input.brand,
source.rows.length,
input.dueAt,
input.platform ?? "小红书",
input.contentFormat ?? "image_text",
source.url,
source.sheetId,
source.sheetName,
@@ -149,11 +168,15 @@ async function insertTaskFromSource(
...image,
key: `content-assets/${taskId}/${contentId}/${image.index}`,
}));
const videoAssets = row.videos.map((video) => ({
...video,
key: `content-videos/${taskId}/${contentId}/${video.index}`,
}));
return db
.prepare(
`INSERT INTO contents
(id, task_id, title, body, image_assets, status, source, source_row)
VALUES (?, ?, ?, ?, ?, 'available', ?, ?)`,
(id, task_id, title, body, image_assets, video_assets, status, source, source_row)
VALUES (?, ?, ?, ?, ?, ?, 'available', ?, ?)`,
)
.bind(
contentId,
@@ -161,6 +184,7 @@ async function insertTaskFromSource(
row.title,
row.body,
JSON.stringify(imageAssets),
JSON.stringify(videoAssets),
`飞书 · ${source.sheetName}`,
row.sourceRow,
);
@@ -189,11 +213,13 @@ export async function createDistributionTask(
options: { deduplicate?: boolean } = {},
): Promise<DistributionTaskCreation> {
await ensureSchema();
const input = {
const input: Required<CreateDistributionTaskInput> = {
feishuUrl: normalizedFeishuUrl(rawInput.feishuUrl),
name: normalizedValue(rawInput.name),
brand: normalizedValue(rawInput.brand),
dueAt: normalizedDueDate(rawInput.dueAt),
platform: rawInput.platform === "抖音" ? "抖音" : "小红书",
contentFormat: rawInput.contentFormat === "video" ? "video" : "image_text",
};
if (!input.feishuUrl || !input.name || !input.brand) {
throw new Error("请补全飞书链接、任务名称和品牌/项目");
@@ -213,11 +239,19 @@ export async function createDistributionTask(
sheetId: existing.source_sheet_id,
sheetName: existing.source_sheet_name,
sourceUrl: existing.source_url,
platform: existing.platform,
contentFormat: existing.content_format,
};
}
}
const source = await readFeishuSource(input.feishuUrl, bindings);
if (
input.contentFormat === "video" &&
source.rows.some((row) => row.videos.length === 0)
) {
throw new Error("视频任务中存在未识别到视频的内容行,请检查飞书“视频”列");
}
const inserted = await insertTaskFromSource(source, input);
return {
created: true,
@@ -230,6 +264,8 @@ export async function createDistributionTask(
sheetId: source.sheetId,
sheetName: source.sheetName,
sourceUrl: source.url,
platform: input.platform,
contentFormat: input.contentFormat,
};
}
@@ -287,8 +323,8 @@ export async function createScreenshotTask(
db
.prepare(
`INSERT INTO contents
(id, task_id, title, body, image_assets, status, source, source_row)
VALUES (?, ?, ?, ?, ?, 'available', '截图回收任务', ?)`,
(id, task_id, title, body, image_assets, video_assets, status, source, source_row)
VALUES (?, ?, ?, ?, ?, '[]', 'available', '截图回收任务', ?)`,
)
.bind(
uid("content"),

45
lib/video-file.ts Normal file
View File

@@ -0,0 +1,45 @@
const MP4_BRANDS = new Set([
"avc1",
"dash",
"isom",
"M4A ",
"M4B ",
"M4P ",
"M4V ",
"mp41",
"mp42",
"MSNV",
]);
function fourCharacters(bytes: Uint8Array, offset: number) {
return String.fromCharCode(...bytes.slice(offset, offset + 4));
}
function isMp4Brand(brand: string) {
return (
MP4_BRANDS.has(brand) ||
/^iso[2-9]$/.test(brand) ||
/^3g[2p]$/.test(brand.slice(0, 3))
);
}
/** Rejects HTML/JSON/error payloads and non-MP4 containers before download. */
export function hasMp4FileSignature(input: ArrayBuffer | Uint8Array) {
const bytes = input instanceof Uint8Array ? input : new Uint8Array(input);
if (bytes.byteLength < 12 || fourCharacters(bytes, 4) !== "ftyp") {
return false;
}
const declaredSize = new DataView(
bytes.buffer,
bytes.byteOffset,
bytes.byteLength,
).getUint32(0);
const boxEnd = Math.min(
bytes.byteLength,
declaredSize >= 12 ? declaredSize : bytes.byteLength,
);
for (let offset = 8; offset + 4 <= boxEnd; offset += 4) {
if (isMp4Brand(fourCharacters(bytes, offset))) return true;
}
return false;
}

460
lib/wecom-client.ts Normal file
View File

@@ -0,0 +1,460 @@
const WECOM_API_ORIGIN = "https://qyapi.weixin.qq.com";
const DEFAULT_DUE_DAYS = 3;
export type WecomBindings = {
WECOM_CORP_ID?: string;
WECOM_AGENT_ID?: string;
WECOM_SECRET?: string;
WECOM_ROBOT_WEBHOOK?: string;
WECOM_NOTIFY_DUE_DAYS?: string;
};
export type WecomConfig = {
corpId: string;
agentId: string;
secret: string;
robotWebhook: string;
dueDays: number;
};
type FetchLike = typeof fetch;
type WecomEnvelope = {
errcode?: number;
errmsg?: string;
access_token?: string;
expires_in?: number;
invaliduser?: string;
follow_user?: Array<{ userid: string }>;
external_userid?: string[];
external_contact?: {
external_userid?: string;
name?: string;
avatar?: string;
corp_fullname?: string;
};
next_cursor?: string;
group_chat_list?: Array<{ chat_id?: string; status?: number }>;
group_chat?: {
name?: string;
owner?: string;
member_count?: number;
};
fail_list?: string[];
msgid?: string;
};
export type WecomExternalContact = {
externalUserId: string;
name: string;
avatar: string;
corpName: string;
ownerUserId: string;
};
type CachedAccessToken = {
corpId: string;
secret: string;
token: string;
expiresAt: number;
};
let cachedAccessToken: CachedAccessToken | null = null;
export class WecomClientError extends Error {
status: number;
constructor(message: string, status = 502) {
super(message);
this.name = "WecomClientError";
this.status = status;
}
}
function bindingValue(value: unknown) {
return String(value ?? "").trim();
}
function safeMessage(value: unknown) {
return String(value ?? "").trim().slice(0, 240);
}
export function resolveWecomConfig(bindings: WecomBindings): WecomConfig {
return {
corpId: bindingValue(bindings.WECOM_CORP_ID),
agentId: bindingValue(bindings.WECOM_AGENT_ID),
secret: bindingValue(bindings.WECOM_SECRET),
robotWebhook: bindingValue(bindings.WECOM_ROBOT_WEBHOOK),
dueDays: parseDueDays(bindings.WECOM_NOTIFY_DUE_DAYS),
};
}
function parseDueDays(value: string | undefined) {
const parsed = Number(bindingValue(value));
if (!Number.isFinite(parsed) || parsed < 1) return DEFAULT_DUE_DAYS;
return Math.min(30, Math.floor(parsed));
}
function hasAppCredentials(config: WecomConfig) {
return Boolean(config.corpId && config.agentId && config.secret);
}
async function readEnvelope(
response: Response,
fallbackMessage: string,
): Promise<WecomEnvelope> {
const text = await response.text();
try {
return JSON.parse(text) as WecomEnvelope;
} catch {
throw new WecomClientError(
`${fallbackMessage}(企业微信返回了非 JSON 响应)`,
502,
);
}
}
function ensureOk(
payload: WecomEnvelope,
fallbackMessage: string,
) {
const code = Number(payload.errcode ?? 0);
if (code === 0) return;
const message = safeMessage(payload.errmsg) || fallbackMessage;
if (code === 40014 || code === 42001) {
throw new WecomClientError(`企业微信 access_token 无效:${message}`, 401);
}
throw new WecomClientError(`${fallbackMessage}${message}`, 502);
}
async function fetchAccessToken(
config: WecomConfig,
fetchImpl: FetchLike,
) {
if (
cachedAccessToken?.corpId === config.corpId &&
cachedAccessToken?.secret === config.secret &&
cachedAccessToken.expiresAt > Date.now() + 60_000
) {
return cachedAccessToken.token;
}
const url = new URL(`${WECOM_API_ORIGIN}/cgi-bin/gettoken`);
url.searchParams.set("corpid", config.corpId);
url.searchParams.set("corpsecret", config.secret);
const response = await fetchImpl(url.toString(), {
method: "GET",
signal: AbortSignal.timeout(12_000),
});
const payload = await readEnvelope(response, "获取企业微信 access_token 失败");
if (!response.ok) {
throw new WecomClientError(
`获取企业微信 access_token 失败HTTP ${response.status}`,
502,
);
}
ensureOk(payload, "获取企业微信 access_token 失败");
const token = bindingValue(payload.access_token);
if (!token) {
throw new WecomClientError("企业微信未返回有效 access_token", 502);
}
cachedAccessToken = {
corpId: config.corpId,
secret: config.secret,
token,
expiresAt:
Date.now() + Math.max(300, Number(payload.expires_in) || 7_200) * 1_000,
};
return token;
}
export async function sendWecomRobotMessage(
content: string,
config: WecomConfig,
fetchImpl: FetchLike = fetch,
): Promise<void> {
if (!config.robotWebhook) {
console.warn("[KOC LOOP] wecom robot webhook not configured, skipping");
return;
}
const response = await fetchImpl(config.robotWebhook, {
method: "POST",
headers: { "Content-Type": "application/json; charset=utf-8" },
body: JSON.stringify({
msgtype: "text",
text: { content },
}),
signal: AbortSignal.timeout(10_000),
});
const payload = await readEnvelope(response, "企业微信群机器人推送失败");
if (!response.ok) {
throw new WecomClientError(
`企业微信群机器人推送失败HTTP ${response.status}`,
502,
);
}
ensureOk(payload, "企业微信群机器人推送失败");
}
export async function sendWecomAppMessage(
externalUserIds: string[],
content: string,
config: WecomConfig,
fetchImpl: FetchLike = fetch,
): Promise<{ sent: number; failed: number; skipped: boolean }> {
const normalized = externalUserIds
.map((id) => bindingValue(id))
.filter((id) => id.length > 0);
if (normalized.length === 0) {
return { sent: 0, failed: 0, skipped: true };
}
if (!hasAppCredentials(config)) {
return { sent: 0, failed: 0, skipped: true };
}
const token = await fetchAccessToken(config, fetchImpl);
const url = `${WECOM_API_ORIGIN}/cgi-bin/message/send?access_token=${encodeURIComponent(token)}`;
const response = await fetchImpl(url, {
method: "POST",
headers: { "Content-Type": "application/json; charset=utf-8" },
body: JSON.stringify({
touser: normalized.join("|"),
msgtype: "text",
agentid: Number(config.agentId),
text: { content },
}),
signal: AbortSignal.timeout(12_000),
});
const payload = await readEnvelope(response, "企业微信应用消息推送失败");
if (!response.ok) {
throw new WecomClientError(
`企业微信应用消息推送失败HTTP ${response.status}`,
502,
);
}
ensureOk(payload, "企业微信应用消息推送失败");
const invalid = bindingValue(payload.invaliduser).split("|").filter(Boolean);
return {
sent: Math.max(0, normalized.length - invalid.length),
failed: invalid.length,
skipped: false,
};
}
export function clearWecomAccessTokenCacheForTests() {
cachedAccessToken = null;
}
export type WecomGroupChat = {
chatId: string;
name: string;
ownerUserId: string;
memberCount: number;
status: number;
};
export async function listCustomerGroupChats(
config: WecomConfig,
fetchImpl: FetchLike = fetch,
): Promise<WecomGroupChat[]> {
if (!hasAppCredentials(config)) return [];
const token = await fetchAccessToken(config, fetchImpl);
const chatIds: Array<{ chatId: string; status: number }> = [];
let cursor = "";
do {
const listUrl = `${WECOM_API_ORIGIN}/cgi-bin/externalcontact/groupchat/list?access_token=${encodeURIComponent(token)}`;
const listResp = await fetchImpl(listUrl, {
method: "POST",
headers: { "Content-Type": "application/json; charset=utf-8" },
body: JSON.stringify({ limit: 100, cursor }),
signal: AbortSignal.timeout(12_000),
});
const listPayload = await readEnvelope(listResp, "获取企业微信客户群列表失败");
if (!listResp.ok) {
throw new WecomClientError(
`获取企业微信客户群列表失败HTTP ${listResp.status}`,
502,
);
}
ensureOk(listPayload, "获取企业微信客户群列表失败");
for (const chat of listPayload.group_chat_list ?? []) {
const chatId = bindingValue(chat.chat_id);
if (chatId) chatIds.push({ chatId, status: Number(chat.status ?? 0) });
}
cursor = bindingValue(listPayload.next_cursor);
} while (cursor);
const groups: WecomGroupChat[] = [];
for (const { chatId, status } of chatIds) {
if (status !== 0) continue;
const getUrl = `${WECOM_API_ORIGIN}/cgi-bin/externalcontact/groupchat/get?access_token=${encodeURIComponent(token)}`;
const getResp = await fetchImpl(getUrl, {
method: "POST",
headers: { "Content-Type": "application/json; charset=utf-8" },
body: JSON.stringify({ chat_id: chatId }),
signal: AbortSignal.timeout(12_000),
});
const getPayload = await readEnvelope(getResp, "获取企业微信客户群详情失败");
if (!getResp.ok) continue;
if (Number(getPayload.errcode ?? 0) !== 0) continue;
const detail = getPayload.group_chat;
groups.push({
chatId,
name: bindingValue(detail?.name),
ownerUserId: bindingValue(detail?.owner),
memberCount: Number(detail?.member_count ?? 0),
status,
});
}
return groups;
}
export type WecomGroupMsgLink = {
title: string;
desc?: string;
url: string;
picurl?: string;
};
export async function createGroupMsgTemplate(
config: WecomConfig,
params: {
sender: string;
chatIdList: string[];
text?: string;
link?: WecomGroupMsgLink;
},
fetchImpl: FetchLike = fetch,
): Promise<{ msgid: string; failList: string[] }> {
if (!hasAppCredentials(config)) {
throw new WecomClientError(
"企业微信自建应用未配置WECOM_CORP_ID / WECOM_AGENT_ID / WECOM_SECRET",
400,
);
}
const chatIdList = params.chatIdList.map(bindingValue).filter(Boolean);
if (chatIdList.length === 0) {
throw new WecomClientError("客户群列表为空", 400);
}
const sender = bindingValue(params.sender);
if (!sender) {
throw new WecomClientError("发送成员(群主 userid不能为空", 400);
}
const text = String(params.text ?? "").trim();
const link = params.link;
if (!text && !link) {
throw new WecomClientError("文本与图文附件不能同时为空", 400);
}
const token = await fetchAccessToken(config, fetchImpl);
const url = `${WECOM_API_ORIGIN}/cgi-bin/externalcontact/add_msg_template?access_token=${encodeURIComponent(token)}`;
const response = await fetchImpl(url, {
method: "POST",
headers: { "Content-Type": "application/json; charset=utf-8" },
body: JSON.stringify({
chat_type: "group",
chat_id_list: chatIdList,
sender,
...(text ? { text: { content: text } } : {}),
...(link
? {
attachments: [
{
msgtype: "link",
link: {
title: link.title,
desc: link.desc ?? "",
url: link.url,
picurl: link.picurl ?? "",
},
},
],
}
: {}),
}),
signal: AbortSignal.timeout(12_000),
});
const payload = await readEnvelope(response, "创建企业群发失败");
if (!response.ok) {
throw new WecomClientError(
`创建企业群发失败HTTP ${response.status}`,
502,
);
}
ensureOk(payload, "创建企业群发失败");
return {
msgid: bindingValue(payload.msgid),
failList: (payload.fail_list ?? []).filter(Boolean),
};
}
export async function listExternalContacts(
config: WecomConfig,
fetchImpl: FetchLike = fetch,
): Promise<WecomExternalContact[]> {
if (!hasAppCredentials(config)) return [];
const token = await fetchAccessToken(config, fetchImpl);
const followUrl = `${WECOM_API_ORIGIN}/cgi-bin/externalcontact/get_follow_user_list?access_token=${encodeURIComponent(token)}`;
const followResp = await fetchImpl(followUrl, {
method: "GET",
signal: AbortSignal.timeout(12_000),
});
const followPayload = await readEnvelope(
followResp,
"获取企业微信跟进人列表失败",
);
if (!followResp.ok) {
throw new WecomClientError(
`获取企业微信跟进人列表失败HTTP ${followResp.status}`,
502,
);
}
ensureOk(followPayload, "获取企业微信跟进人列表失败");
const ownerUserIds = (followPayload.follow_user ?? [])
.map((user) => bindingValue(user.userid))
.filter(Boolean);
const ownerToExternalIds: Array<{ externalUserId: string; ownerUserId: string }> = [];
for (const ownerUserId of ownerUserIds) {
const listUrl = `${WECOM_API_ORIGIN}/cgi-bin/externalcontact/list?access_token=${encodeURIComponent(token)}&userid=${encodeURIComponent(ownerUserId)}`;
const listResp = await fetchImpl(listUrl, {
method: "GET",
signal: AbortSignal.timeout(12_000),
});
const listPayload = await readEnvelope(
listResp,
"获取企业微信外部联系人列表失败",
);
if (!listResp.ok) continue;
if (Number(listPayload.errcode ?? 0) !== 0) continue;
const externalIds = (listPayload.external_userid ?? []).filter(Boolean);
for (const externalUserId of externalIds) {
ownerToExternalIds.push({ externalUserId, ownerUserId });
}
}
const contacts: WecomExternalContact[] = [];
for (const { externalUserId, ownerUserId } of ownerToExternalIds) {
const getUrl = `${WECOM_API_ORIGIN}/cgi-bin/externalcontact/get?access_token=${encodeURIComponent(token)}&external_userid=${encodeURIComponent(externalUserId)}`;
const getResp = await fetchImpl(getUrl, {
method: "GET",
signal: AbortSignal.timeout(12_000),
});
const getPayload = await readEnvelope(
getResp,
"获取企业微信外部联系人详情失败",
);
if (!getResp.ok) continue;
if (Number(getPayload.errcode ?? 0) !== 0) continue;
const info = getPayload.external_contact;
if (!info) continue;
contacts.push({
externalUserId,
name: bindingValue(info.name),
avatar: bindingValue(info.avatar),
corpName: bindingValue(info.corp_fullname) || bindingValue(info.name),
ownerUserId,
});
}
return contacts;
}

View File

@@ -0,0 +1,326 @@
import type { DatabaseClient } from "./database";
import { getRuntimeEnv } from "./runtime-env";
import {
createGroupMsgTemplate,
listCustomerGroupChats,
WecomClientError,
type WecomConfig,
} from "./wecom-client";
type FetchLike = typeof fetch;
export const WECOM_GROUP_BATCH_LIMIT = 2000;
const TEXT_MAX_BYTES = 4000;
const LINK_TITLE_MAX_BYTES = 128;
const LINK_DESC_MAX_BYTES = 512;
const LINK_URL_MAX_BYTES = 2048;
export type TaskPushRow = {
id: string;
name: string;
brand: string;
quantity: number;
due_at: string;
task_type: string;
platform: string;
content_format: string;
share_token: string | null;
};
export type TaskPushMessage = {
text: string;
link: { title: string; desc: string; url: string } | null;
};
export type GroupChatRow = {
chat_id: string;
name: string;
owner_user_id: string;
member_count: number;
status: number;
};
export type GroupPushResult = {
sender: string;
msgid: string;
chatCount: number;
failList: string[];
};
export type GroupPushRecord = {
id: string;
taskId: string;
msgid: string;
sender: string;
chatIds: string[];
failList: string[];
createdAt: string;
};
function bindingValue(value: unknown) {
return String(value ?? "").trim();
}
export function trimToBytes(value: string, maxBytes: number) {
let output = "";
let used = 0;
for (const char of Array.from(value)) {
const size = Buffer.byteLength(char, "utf8");
if (used + size > maxBytes) break;
output += char;
used += size;
}
return output;
}
export function buildTaskGroupPushMessage(
task: TaskPushRow,
portalUrl: string,
): TaskPushMessage {
const portal = portalUrl.trim().replace(/\/$/, "");
const shareToken = bindingValue(task.share_token);
const claimUrl = portal && shareToken ? `${portal}/?task=${shareToken}` : "";
const formatLabel =
task.task_type === "screenshot_collect"
? "截图回收"
: task.content_format === "video"
? "视频"
: "图文";
const lines = [
`【新任务】${task.name}`,
`品牌:${task.brand}|数量:${task.quantity} 份|截止:${task.due_at}`,
`平台:${task.platform}|形式:${formatLabel}`,
claimUrl ? `领取链接:${claimUrl}` : "领取链接生成失败,请联系管理员",
];
return {
text: trimToBytes(lines.join("\n"), TEXT_MAX_BYTES),
link: claimUrl
? {
title: trimToBytes(task.name, LINK_TITLE_MAX_BYTES),
desc: trimToBytes(
`品牌 ${task.brand} · ${task.quantity} 份 · 截止 ${task.due_at}`,
LINK_DESC_MAX_BYTES,
),
url: claimUrl.slice(0, LINK_URL_MAX_BYTES),
}
: null,
};
}
export function groupChatsByOwner(chats: GroupChatRow[]) {
const grouped = new Map<string, string[]>();
for (const chat of chats) {
const owner = bindingValue(chat.owner_user_id);
if (!owner) continue;
grouped.set(owner, [...(grouped.get(owner) ?? []), chat.chat_id]);
}
return [...grouped.entries()].map(([ownerUserId, chatIds]) => ({
ownerUserId,
chatIds,
}));
}
export async function syncGroupChats(
db: DatabaseClient,
config: WecomConfig,
fetchImpl: FetchLike = fetch,
): Promise<GroupChatRow[]> {
if (!(config.corpId && config.agentId && config.secret)) {
throw new WecomClientError(
"企业微信自建应用未配置WECOM_CORP_ID / WECOM_AGENT_ID / WECOM_SECRET",
400,
);
}
const groups = await listCustomerGroupChats(config, fetchImpl);
const syncedAt = new Date().toISOString();
await db.prepare("DELETE FROM wecom_group_chats").run();
const inserts = groups.map((group) =>
db
.prepare(
`INSERT INTO wecom_group_chats
(chat_id, name, owner_user_id, member_count, status, synced_at)
VALUES (?, ?, ?, ?, ?, ?)`,
)
.bind(
group.chatId,
group.name,
group.ownerUserId,
group.memberCount,
group.status,
syncedAt,
),
);
if (inserts.length > 0) await db.batch(inserts);
return groups.map((group) => ({
chat_id: group.chatId,
name: group.name,
owner_user_id: group.ownerUserId,
member_count: group.memberCount,
status: group.status,
}));
}
export async function listGroupChatRows(db: DatabaseClient) {
const result = await db
.prepare(
`SELECT chat_id, name, owner_user_id, member_count, status
FROM wecom_group_chats
ORDER BY member_count DESC`,
)
.all<GroupChatRow>();
return result.results;
}
export async function listGroupPushes(
db: DatabaseClient,
taskId?: string,
): Promise<GroupPushRecord[]> {
const filter = taskId ? "WHERE task_id = ?" : "";
const statement = db
.prepare(
`SELECT id, task_id, msgid, sender, chat_ids, fail_list, created_at
FROM wecom_group_pushes
${filter}
ORDER BY created_at DESC
LIMIT 50`,
)
.bind(...(taskId ? [taskId] : []));
const result = await statement.all<{
id: string;
task_id: string;
msgid: string;
sender: string;
chat_ids: string;
fail_list: string;
created_at: string;
}>();
return result.results.map((row) => ({
id: row.id,
taskId: row.task_id,
msgid: row.msgid,
sender: row.sender,
chatIds: safeParseArray(row.chat_ids),
failList: safeParseArray(row.fail_list),
createdAt: row.created_at,
}));
}
export async function pushTaskToGroupChats(
db: DatabaseClient,
params: { taskId: string; chatIds: unknown; text?: unknown },
config: WecomConfig,
fetchImpl: FetchLike = fetch,
): Promise<{ results: GroupPushResult[]; linkUrl: string }> {
const taskId = bindingValue(params.taskId);
if (!taskId) throw new WecomClientError("缺少 taskId", 400);
const requested = [
...new Set(
(Array.isArray(params.chatIds) ? params.chatIds : [])
.map((id) => bindingValue(id))
.filter(Boolean),
),
];
if (requested.length === 0) {
throw new WecomClientError("请选择要发送的客户群", 400);
}
const task = await db
.prepare(
`SELECT id, name, brand, quantity, due_at, task_type, platform,
content_format, share_token
FROM tasks WHERE id = ?`,
)
.bind(taskId)
.first<TaskPushRow>();
if (!task) throw new WecomClientError("任务不存在", 404);
const chatRows: GroupChatRow[] = [];
for (let index = 0; index < requested.length; index += 100) {
const chunk = requested.slice(index, index + 100);
const placeholders = chunk.map(() => "?").join(", ");
const result = await db
.prepare(
`SELECT chat_id, name, owner_user_id, member_count, status
FROM wecom_group_chats WHERE chat_id IN (${placeholders})`,
)
.bind(...chunk)
.all<GroupChatRow>();
chatRows.push(...result.results);
}
const byId = new Map(chatRows.map((row) => [row.chat_id, row]));
const missing = requested.filter((id) => !byId.has(id));
if (missing.length > 0) {
throw new WecomClientError(
`以下客户群未同步到平台,请先同步群列表:${missing.join("、")}`,
400,
);
}
const noOwner = requested
.map((id) => byId.get(id))
.filter((row) => row && !bindingValue(row.owner_user_id));
if (noOwner.length > 0) {
throw new WecomClientError(
`群「${noOwner.map((row) => row?.name || row?.chat_id).join("、")}」缺少群主信息,请重新同步群列表`,
400,
);
}
const portalUrl = bindingValue(getRuntimeEnv().KOC_PORTAL_URL);
const fallback = buildTaskGroupPushMessage(task, portalUrl);
const text = bindingValue(params.text) || fallback.text;
const link = fallback.link;
if (!text && !link) {
throw new WecomClientError("文本与图文附件不能同时为空", 400);
}
const results: GroupPushResult[] = [];
const groups = requested.map((id) => byId.get(id)!);
for (const { ownerUserId, chatIds } of groupChatsByOwner(groups)) {
for (
let index = 0;
index < chatIds.length;
index += WECOM_GROUP_BATCH_LIMIT
) {
const batch = chatIds.slice(index, index + WECOM_GROUP_BATCH_LIMIT);
const sent = await createGroupMsgTemplate(
config,
{ sender: ownerUserId, chatIdList: batch, text, link: link ?? undefined },
fetchImpl,
);
await db
.prepare(
`INSERT INTO wecom_group_pushes
(id, task_id, msgid, sender, chat_ids, text_content, link_url, fail_list)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
)
.bind(
`wgp-${crypto.randomUUID().slice(0, 12)}`,
taskId,
sent.msgid,
ownerUserId,
JSON.stringify(batch),
text,
link?.url ?? "",
JSON.stringify(sent.failList),
)
.run();
results.push({
sender: ownerUserId,
msgid: sent.msgid,
chatCount: batch.length,
failList: sent.failList,
});
}
}
return { results, linkUrl: link?.url ?? "" };
}
function safeParseArray(value: string) {
try {
const parsed = JSON.parse(value);
return Array.isArray(parsed) ? parsed.map((item) => String(item)) : [];
} catch {
return [];
}
}

View File

@@ -0,0 +1,171 @@
import type { DatabaseClient } from "./database";
import { shanghaiDateFromTimestamp } from "./collection-service";
import {
sendWecomAppMessage,
sendWecomRobotMessage,
type WecomConfig,
} from "./wecom-client";
import { getRuntimeEnv } from "./runtime-env";
type FetchLike = typeof fetch;
export type WecomNotifySummary = {
dueSoonAttempted: number;
dueSoonSent: number;
dueSoonFailed: number;
dueSoonSkipped: number;
digestSent: boolean;
};
type DueSoonRow = {
distribution_id: string;
partner_id: string;
partner_name: string;
wecom_external_user_id: string | null;
task_name: string;
due_at: string;
content_title: string;
};
export function computeDueCutoff(today: string, dueDays: number) {
const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec(today);
if (!match) return today;
const [, y, m, d] = match;
const date = new Date(
Date.UTC(Number(y), Number(m) - 1, Number(d)) + dueDays * 24 * 60 * 60 * 1_000,
);
return date.toISOString().slice(0, 10);
}
export async function runDueSoonWecomNotifications(
db: DatabaseClient,
config: WecomConfig,
fetchImpl: FetchLike = fetch,
now: number = Date.now(),
): Promise<WecomNotifySummary> {
const today = shanghaiDateFromTimestamp(now);
const cutoff = computeDueCutoff(today, config.dueDays);
const portalUrl = bindingValue(getRuntimeEnv().KOC_PORTAL_URL);
const result = await db
.prepare(
`SELECT
d.id AS distribution_id,
d.partner_id,
p.name AS partner_name,
p.wecom_external_user_id,
t.name AS task_name,
t.due_at,
c.title AS content_title
FROM distributions d
JOIN tasks t ON t.id = d.task_id
JOIN partners p ON p.id = d.partner_id
JOIN contents c ON c.id = d.content_id
WHERE (d.publish_url IS NULL OR d.publish_url = '')
AND t.due_at IS NOT NULL AND t.due_at != ''
AND t.due_at <= ?
ORDER BY t.due_at ASC, p.name ASC`,
)
.bind(cutoff)
.all<DueSoonRow>();
const grouped = new Map<
string,
{
partnerName: string;
externalUserId: string | null;
rows: DueSoonRow[];
}
>();
for (const row of result.results) {
const entry = grouped.get(row.partner_id) ?? {
partnerName: row.partner_name,
externalUserId: row.wecom_external_user_id,
rows: [],
};
entry.rows.push(row);
if (!entry.externalUserId && row.wecom_external_user_id) {
entry.externalUserId = row.wecom_external_user_id;
}
grouped.set(row.partner_id, entry);
}
let dueSoonAttempted = 0;
let dueSoonSent = 0;
let dueSoonFailed = 0;
let dueSoonSkipped = 0;
const digestTasks: string[] = [];
for (const [, entry] of grouped) {
dueSoonAttempted += 1;
const external = entry.externalUserId
? [entry.externalUserId]
: [];
const lines = entry.rows.slice(0, 5).map((row) => {
return `· 《${truncate(row.task_name, 24)}》— ${truncate(row.content_title, 24)}(截止 ${row.due_at}`;
});
const overflow =
entry.rows.length > 5 ? `\n…还有 ${entry.rows.length - 5}` : "";
const link = portalUrl ? `\n领取链接${portalUrl}` : "";
const content =
`${entry.partnerName},你有 ${entry.rows.length} 条内容待发布:\n${lines.join("\n")}${overflow}${link}`;
const appResult = await sendWecomAppMessage(
external,
content,
config,
fetchImpl,
).catch((error: unknown) => {
console.warn(
"[KOC LOOP] wecom app message failed",
{ partner: entry.partnerName, error: safeError(error) },
);
return null;
});
if (appResult === null) {
dueSoonFailed += 1;
} else if (appResult.skipped) {
dueSoonSkipped += 1;
} else {
dueSoonSent += 1;
}
const earliestDue = entry.rows[0]?.due_at ?? "";
digestTasks.push(
`· ${entry.partnerName}${entry.rows.length} 条,最近截止 ${earliestDue}`,
);
}
let digestSent = false;
if (config.robotWebhook && digestTasks.length > 0) {
const digest =
`今日待发布催办(${today},截止 ≤ ${cutoff}\n${digestTasks.join("\n")}`;
try {
await sendWecomRobotMessage(digest, config, fetchImpl);
digestSent = true;
} catch (error) {
console.warn(
"[KOC LOOP] wecom robot digest failed",
{ error: safeError(error) },
);
}
}
return {
dueSoonAttempted,
dueSoonSent,
dueSoonFailed,
dueSoonSkipped,
digestSent,
};
}
function bindingValue(value: unknown) {
return String(value ?? "").trim();
}
function truncate(value: string, max: number) {
return value.length > max ? `${value.slice(0, max)}` : value;
}
function safeError(error: unknown) {
return error instanceof Error ? error.message : String(error);
}

83
lib/workbook-image.ts Normal file
View File

@@ -0,0 +1,83 @@
import sharp from "sharp";
export type WorkbookSourceImage = {
bytes: Uint8Array;
contentType: string;
width?: number | null;
height?: number | null;
description: string;
};
export type WorkbookImageNormalizationOptions = {
maxDimension?: number;
outputFormat?: "png" | "jpeg";
jpegQuality?: number;
};
const NORMALIZABLE_IMAGE = /^image\/(?:jpe?g|png|webp|gif|tiff?|avif|heic|heif)$/i;
function hasImageSignature(bytes: Uint8Array) {
if (bytes.length < 4) return false;
if (bytes[0] === 0xff && bytes[1] === 0xd8) return true;
if (bytes[0] === 0x89 && bytes[1] === 0x50 && bytes[2] === 0x4e && bytes[3] === 0x47) {
return true;
}
if (bytes[0] === 0x47 && bytes[1] === 0x49 && bytes[2] === 0x46) return true;
if (
bytes.length >= 12 &&
String.fromCharCode(...bytes.slice(0, 4)) === "RIFF" &&
String.fromCharCode(...bytes.slice(8, 12)) === "WEBP"
) {
return true;
}
return (
(bytes[0] === 0x49 && bytes[1] === 0x49 && bytes[2] === 0x2a && bytes[3] === 0x00) ||
(bytes[0] === 0x4d && bytes[1] === 0x4d && bytes[2] === 0x00 && bytes[3] === 0x2a)
);
}
/**
* Excel viewers disagree on whether JPEG EXIF orientation should be applied.
* Bake that orientation into the pixels before the image enters the workbook.
* Callers may also resize and encode large source images as JPEG to keep the
* generated workbook within the upload limit.
*/
export async function normalizeWorkbookImage(
image: WorkbookSourceImage,
options: WorkbookImageNormalizationOptions = {},
): Promise<WorkbookSourceImage> {
if (!NORMALIZABLE_IMAGE.test(image.contentType) && !hasImageSignature(image.bytes)) {
return image;
}
try {
let normalized = sharp(image.bytes, { animated: false }).rotate();
if (options.maxDimension && options.maxDimension > 0) {
normalized = normalized.resize({
width: Math.floor(options.maxDimension),
height: Math.floor(options.maxDimension),
fit: "inside",
withoutEnlargement: true,
});
}
normalized =
options.outputFormat === "jpeg"
? normalized
.flatten({ background: "#ffffff" })
.jpeg({
quality: Math.min(95, Math.max(50, options.jpegQuality ?? 82)),
mozjpeg: true,
})
: normalized.png({ compressionLevel: 6 });
const { data, info } = await normalized.toBuffer({ resolveWithObject: true });
return {
...image,
bytes: new Uint8Array(data),
contentType:
options.outputFormat === "jpeg" ? "image/jpeg" : "image/png",
width: info.width,
height: info.height,
};
} catch {
return image;
}
}

View File

@@ -50,6 +50,9 @@ CREATE TABLE IF NOT EXISTS accounts (
profile_url TEXT NOT NULL DEFAULT (''),
ip_location VARCHAR(255) NOT NULL DEFAULT '待识别',
followers INT NOT NULL DEFAULT 0,
gender VARCHAR(16) NOT NULL DEFAULT '',
bio TEXT NOT NULL DEFAULT (''),
tags VARCHAR(500) NOT NULL DEFAULT '',
post_count INT NOT NULL DEFAULT 0,
avg_views INT NOT NULL DEFAULT 0,
first_seen_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

View File

@@ -0,0 +1,14 @@
ALTER TABLE tasks
ADD COLUMN platform VARCHAR(32) NOT NULL DEFAULT '小红书' AFTER task_type;
-- statement-breakpoint
ALTER TABLE tasks
ADD COLUMN content_format VARCHAR(32) NOT NULL DEFAULT 'image_text' AFTER platform;
-- statement-breakpoint
ALTER TABLE contents
ADD COLUMN video_assets LONGTEXT NOT NULL DEFAULT ('[]') AFTER image_assets;
-- statement-breakpoint
ALTER TABLE distributions
ADD COLUMN latest_shares INT NULL AFTER latest_collects;
-- statement-breakpoint
ALTER TABLE collection_runs
ADD COLUMN shares INT NULL AFTER collects;

View File

@@ -0,0 +1,8 @@
ALTER TABLE accounts
ADD COLUMN gender VARCHAR(16) NOT NULL DEFAULT '' AFTER followers;
-- statement-breakpoint
ALTER TABLE accounts
ADD COLUMN bio TEXT NOT NULL DEFAULT ('') AFTER gender;
-- statement-breakpoint
ALTER TABLE accounts
ADD COLUMN tags VARCHAR(500) NOT NULL DEFAULT '' AFTER bio;

View File

@@ -0,0 +1,2 @@
ALTER TABLE accounts
ADD COLUMN current_contact VARCHAR(255) NOT NULL DEFAULT '' AFTER cooperation_source;

View File

@@ -0,0 +1,2 @@
ALTER TABLE partners
ADD COLUMN wecom_external_user_id VARCHAR(128) AFTER completed_total;

View File

@@ -0,0 +1,23 @@
CREATE TABLE IF NOT EXISTS `wecom_group_chats` (
`chat_id` VARCHAR(128) NOT NULL,
`name` VARCHAR(512) NOT NULL DEFAULT '',
`owner_user_id` VARCHAR(128) NOT NULL,
`member_count` INT NOT NULL DEFAULT 0,
`status` INT NOT NULL DEFAULT 0,
`synced_at` DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
PRIMARY KEY (`chat_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
-- statement-breakpoint
CREATE TABLE IF NOT EXISTS `wecom_group_pushes` (
`id` VARCHAR(64) NOT NULL,
`task_id` VARCHAR(64) NOT NULL,
`msgid` VARCHAR(128) NOT NULL DEFAULT '',
`sender` VARCHAR(128) NOT NULL,
`chat_ids` TEXT NOT NULL,
`text_content` TEXT NOT NULL,
`link_url` TEXT NOT NULL,
`fail_list` TEXT NOT NULL,
`created_at` DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`),
KEY `wecom_group_pushes_task_idx` (`task_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;

View File

@@ -2,7 +2,8 @@ import type { NextConfig } from "next";
const nextConfig: NextConfig = {
output: "standalone",
serverExternalPackages: ["mysql2"],
serverExternalPackages: ["mysql2", "sharp"],
allowedDevOrigins: ["192.168.30.90", "localhost", "127.0.0.1"],
};
export default nextConfig;

5
package-lock.json generated
View File

@@ -16,6 +16,7 @@
"node-cron": "^4.2.1",
"react": "19.2.6",
"react-dom": "19.2.6",
"sharp": "^0.35.3",
"zod": "^4.4.3"
},
"devDependencies": {
@@ -1420,7 +1421,6 @@
"resolved": "https://registry.npmjs.org/@img/colour/-/colour-1.1.0.tgz",
"integrity": "sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==",
"license": "MIT",
"optional": true,
"engines": {
"node": ">=18"
}
@@ -3836,7 +3836,6 @@
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
"integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
"devOptional": true,
"license": "Apache-2.0",
"engines": {
"node": ">=8"
@@ -7492,7 +7491,6 @@
"resolved": "https://registry.npmjs.org/sharp/-/sharp-0.35.3.tgz",
"integrity": "sha512-ej0zVHuZGHCiABXcNxeYhpRnPNPAcvbG8RMdBAhDAxLKkCRVSpK3Iyu7qbqw3JMzoj0REeM6f3tJLtVwl0023Q==",
"license": "Apache-2.0",
"optional": true,
"dependencies": {
"@img/colour": "^1.1.0",
"detect-libc": "^2.1.2",
@@ -7542,7 +7540,6 @@
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",
"integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
"license": "ISC",
"optional": true,
"bin": {
"semver": "bin/semver.js"
},

View File

@@ -7,7 +7,7 @@
},
"scripts": {
"dev": "next dev",
"build": "next build",
"build": "next build --webpack",
"start": "next start",
"test": "npm run build && node --import tsx --test tests/*.test.mjs",
"lint": "eslint . --ignore-pattern dist --ignore-pattern .next --ignore-pattern koc-portal/out",
@@ -26,6 +26,7 @@
"node-cron": "^4.2.1",
"react": "19.2.6",
"react-dom": "19.2.6",
"sharp": "^0.35.3",
"zod": "^4.4.3"
},
"devDependencies": {

Binary file not shown.

7
scripts/setup.sh Executable file
View File

@@ -0,0 +1,7 @@
#!/bin/sh
set -eu
APP_DIR=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
cd "$APP_DIR"
npm ci
npm run build

8
scripts/start.sh Executable file
View File

@@ -0,0 +1,8 @@
#!/bin/sh
set -eu
APP_DIR=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
cd "$APP_DIR"
export HOSTNAME=0.0.0.0
export PORT="${PORT:-9000}"
exec node .next/standalone/server.js

View File

@@ -120,6 +120,55 @@ test("resolves a wiki sheet and imports title, body, tags, and all images", asyn
assert.equal(valuesCall.url.searchParams.get("ranges"), "sheet-one!A1:J20");
});
test("imports video attachments from a Feishu video task sheet", async () => {
clearFeishuAccessTokenCacheForTests();
const { fetchImpl } = fakeFeishu();
const videoFetch = async (input, init = {}) => {
const url = new URL(String(input));
if (url.pathname.endsWith("/values_batch_get")) {
return apiResponse({
valueRanges: [
{
values: [
["标题", "内容(标题+正文+tag", "视频"],
[
"一条测试视频",
"一条测试视频\n视频正文 #测试",
{
type: "attachment",
fileToken: "video-token-one",
text: "demo.mp4",
mimeType: "video/mp4",
size: 1024,
},
],
],
},
],
});
}
return fetchImpl(input, init);
};
const source = await readFeishuSource(
"https://tenant.feishu.cn/wiki/wiki-test",
bindings,
videoFetch,
);
assert.equal(source.rows.length, 1);
assert.equal(source.rows[0].title, "一条测试视频");
assert.equal(source.rows[0].body, "一条测试视频\n视频正文 #测试");
assert.deepEqual(source.rows[0].videos, [
{
index: 1,
fileToken: "video-token-one",
name: "demo.mp4",
mimeType: "video/mp4",
size: 1024,
},
]);
});
test("requires an exact sheet link when a workbook has multiple visible sheets", async () => {
clearFeishuAccessTokenCacheForTests();
const { fetchImpl } = fakeFeishu({

View File

@@ -1,7 +1,9 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
collectMetricsFromMcp,
collectXhsMetricsFromMcp,
resolveAccountProfileFromMcp,
resolveCollectionMcpConfig,
resolveXhsPublicAccountDetails,
resolveXhsPublicAccountId,
@@ -40,10 +42,17 @@ function toolEnvelope(payload, isError = false) {
};
}
function createFakeMcp(toolResults) {
function createFakeMcp(toolResults, redirects = {}) {
let toolIndex = 0;
const calls = [];
const fetchImpl = async (url, init) => {
if (!init?.body) {
const requestUrl = String(url);
calls.push({ url: requestUrl, body: null, headers: new Headers(init?.headers) });
const location = redirects[requestUrl];
if (!location) throw new Error(`Unexpected public request: ${requestUrl}`);
return new Response("", { status: 302, headers: { location } });
}
const body = JSON.parse(init.body);
calls.push({ url: String(url), body, headers: new Headers(init.headers) });
if (body.method === "initialize") {
@@ -103,6 +112,7 @@ test("collects likes, comments and favorites from the verified MCP shape", async
likes: 483,
comments: 41,
collects: 519,
shares: 0,
});
assert.equal(calls.length, 3);
assert.equal(calls[2].body.params.name, "fetch_content_detail");
@@ -147,7 +157,7 @@ test("collects through a stateless MCP server without a session header", async (
fetchImpl,
);
assert.deepEqual(result, { likes: 12, comments: 3, collects: 6 });
assert.deepEqual(result, { likes: 12, comments: 3, collects: 6, shares: 0 });
assert.deepEqual(
calls.map((call) => call.body.method),
["initialize", "tools/call"],
@@ -217,19 +227,24 @@ test("resolves the real XHS account profile from a submitted note link", async (
redId: "94329495984",
ipLocation: "重庆",
followers: 734,
gender: "",
bio: "",
recentNoteTitles: [],
providerTags: [],
});
assert.equal(calls.length, 4);
const mcpCalls = calls.filter((call) => call.body?.method);
assert.equal(mcpCalls.length, 4);
assert.equal(
calls[2].body.params.name,
"collect_xhs_wen_note_detail",
mcpCalls[2].body.params.name,
"fetch_content_detail",
);
assert.equal(
calls[2].body.params.arguments.request.note_id,
"6a671108000000000f004bef",
mcpCalls[2].body.params.arguments.request.link,
"https://www.xiaohongshu.com/discovery/item/6a671108000000000f004bef",
);
assert.equal(calls[3].body.params.name, "parse_xhs_user_summary");
assert.equal(mcpCalls[3].body.params.name, "parse_xhs_user_summary");
assert.equal(
calls[3].body.params.arguments.request.url,
mcpCalls[3].body.params.arguments.request.url,
"https://www.xiaohongshu.com/user/profile/6905cbca0000000037009f49",
);
assert.equal(
@@ -253,7 +268,17 @@ test("resolves followers directly from the supported XHS user summary tool", asy
ipLocation: "福建",
nickname: "555 五",
userId: "1020668113",
gender: "女",
desc: "分享城市周末与美食",
tags: ["本地生活"],
},
notes: [
{
note_id: "note-1",
title: "长沙湘菜探店",
url: "https://www.xiaohongshu.com/explore/note-1",
},
],
},
},
}),
@@ -273,6 +298,10 @@ test("resolves followers directly from the supported XHS user summary tool", asy
followers: 6,
redId: "1020668113",
ipLocation: "福建",
gender: "女",
bio: "分享城市周末与美食",
recentNoteTitles: ["长沙湘菜探店"],
providerTags: ["本地生活"],
});
assert.equal(calls[2].body.params.name, "parse_xhs_user_summary");
});
@@ -338,8 +367,8 @@ test("resolves an xhslink short URL before requesting the author profile", async
"https://www.xiaohongshu.com/user/profile/6905cbca0000000037009f49",
);
assert.equal(
fakeMcp.calls[2].body.params.arguments.request.note_id,
"6a572da40000000021018bd2",
fakeMcp.calls[2].body.params.arguments.request.link,
"http://xhslink.cn/o/AJFyP5dnj7O",
);
});
@@ -400,10 +429,14 @@ test("uses the public note author when MCP profile lookup fails", async () => {
redId: "1020668113",
ipLocation: "待识别",
followers: 6,
gender: "",
bio: "",
recentNoteTitles: [],
providerTags: [],
});
assert.equal(
fakeMcp.calls[2].body.params.name,
"collect_xhs_wen_note_detail",
"fetch_content_detail",
);
});
@@ -431,27 +464,16 @@ test("reads the user-visible Xiaohongshu number from a public profile", async ()
});
});
test("falls back to parse_xhs_note when the primary tool fails", async () => {
test("treats empty interaction counters from the current detail tool as zero", async () => {
const { calls, fetchImpl } = createFakeMcp([
toolEnvelope(
{
response: {
code: 400,
success: false,
msg: "获取内容详情失败",
data: null,
},
},
true,
),
toolEnvelope({
response: {
code: 200,
success: true,
data: {
likes: "1.2万",
comments: 32,
collects: "2,345",
likes: "2",
comments: "",
collects: "",
},
},
}),
@@ -466,11 +488,42 @@ test("falls back to parse_xhs_note when the primary tool fails", async () => {
);
assert.deepEqual(result, {
likes: 12_000,
comments: 32,
collects: 2_345,
likes: 2,
comments: 0,
collects: 0,
shares: 0,
});
assert.equal(calls[3].body.params.name, "parse_xhs_note");
assert.equal(calls.length, 3);
assert.equal(calls[2].body.params.name, "fetch_content_detail");
});
test("surfaces current detail tool failures without calling removed tools", async () => {
const { calls, fetchImpl } = createFakeMcp([
toolEnvelope(
{
response: {
code: 400,
success: false,
msg: "获取内容详情失败",
data: null,
},
},
true,
),
]);
await assert.rejects(
collectXhsMetricsFromMcp(
"https://www.xiaohongshu.com/explore/test",
{
endpoint: "https://collector.example/mcp?key=test-key",
},
fetchImpl,
),
/获取内容详情失败/,
);
assert.equal(calls.length, 3);
assert.equal(calls[2].body.params.name, "fetch_content_detail");
});
test("requires the MCP key without sending a network request", async () => {
@@ -551,6 +604,237 @@ test("rebuilds the MCP session after a gateway session miss", async () => {
fetchImpl,
);
assert.deepEqual(result, { likes: 8, comments: 2, collects: 5 });
assert.deepEqual(result, { likes: 8, comments: 2, collects: 5, shares: 0 });
assert.equal(initializeCount, 2);
});
test("collects Douyin likes, favorites, shares and comments", async () => {
const { calls, fetchImpl } = createFakeMcp([
toolEnvelope({
response: {
code: 200,
success: true,
data: {
digg_count: "120",
collect_count: "30",
share_count: "8",
comment_count: "12",
},
},
}),
]);
const result = await collectMetricsFromMcp(
"https://www.douyin.com/video/7520000000000000000",
"抖音",
{ endpoint: "https://collector.example/mcp", key: "test-key" },
fetchImpl,
);
assert.deepEqual(result, {
likes: 120,
collects: 30,
shares: 8,
comments: 12,
});
assert.equal(calls[2].body.params.name, "fetch_content_detail");
assert.equal(calls[2].body.params.arguments.request.plant, "dy");
});
test("maps the current Douyin MCP metric field names", async () => {
const { fetchImpl } = createFakeMcp([
toolEnvelope({
response: {
code: 200,
success: true,
data: {
liked_count: "5682",
collected_count: "565",
share_count: "6878",
comment_count: "332",
},
},
}),
]);
const result = await collectMetricsFromMcp(
"https://v.douyin.com/5O5VpgomO2U/",
"抖音",
{ endpoint: "https://collector.example/mcp", key: "test-key" },
fetchImpl,
);
assert.deepEqual(result, {
likes: 5682,
collects: 565,
shares: 6878,
comments: 332,
});
});
test("resolves a Douyin account from a submitted work link", async () => {
const secUid = "MS4wLjABAAAA-test-profile-123456";
const { calls, fetchImpl } = createFakeMcp([
toolEnvelope({
response: {
code: 200,
success: true,
data: {
author: {
nickname: "抖音作者",
sec_uid: secUid,
unique_id: "douyin-123",
},
},
},
}),
toolEnvelope({
response: {
code: 200,
success: true,
data: {
user: {
nickname: "抖音作者",
unique_id: "douyin-123",
sec_uid: secUid,
follower_count: "1.5万",
ip_location: "上海",
},
},
},
}),
]);
const profile = await resolveAccountProfileFromMcp(
"https://www.douyin.com/video/7520000000000000000",
"抖音",
"回填昵称",
{ endpoint: "https://collector.example/mcp", key: "test-key" },
fetchImpl,
);
assert.deepEqual(profile, {
platformUid: secUid,
nickname: "抖音作者",
profileUrl: `https://www.douyin.com/user/${secUid}`,
redId: "douyin-123",
ipLocation: "上海",
followers: 15_000,
gender: "",
bio: "",
recentNoteTitles: [],
providerTags: [],
});
assert.equal(calls[2].body.params.arguments.request.plant, "dy");
assert.equal(calls[3].body.params.name, "parse_dy_user_summary");
});
test("does not treat a Douyin short-link device id as the author sec_uid", async () => {
const shortLink = "https://v.douyin.com/5O5VpgomO2U/";
const { calls, fetchImpl } = createFakeMcp(
[
toolEnvelope({
response: {
code: 200,
success: true,
data: {
nickname: "搞怪噜噜😜",
user_id: "4065311277723529",
},
},
}),
],
{
[shortLink]: "https://www.iesdouyin.com/share/video/7671270545631842038/?did=MS4wLjABAAAA-device-token&with_sec_did=1",
},
);
const profile = await resolveAccountProfileFromMcp(
shortLink,
"抖音",
"回填昵称",
{ endpoint: "https://collector.example/mcp", key: "test-key" },
fetchImpl,
);
assert.deepEqual(profile, {
platformUid: "4065311277723529",
nickname: "搞怪噜噜😜",
profileUrl: "",
redId: "",
ipLocation: "待识别",
followers: null,
gender: "",
bio: "",
recentNoteTitles: [],
providerTags: [],
});
assert.equal(calls.find((call) => call.body === null)?.url, shortLink);
assert.equal(
calls.find((call) => call.body?.params?.name === "parse_dy_user_summary"),
undefined,
);
});
test("resolves a Douyin profile only when the public redirect exposes sec_uid", async () => {
const shortLink = "https://v.douyin.com/author-sec-uid/";
const secUid = "MS4wLjABAAAAOqL4Jdu8htr7EWCDAyIr5z_7uvCAxhj-GOzCWg5zn8bDiKOp3WPw7lWkTyHvZpMY";
const { calls, fetchImpl } = createFakeMcp(
[
toolEnvelope({
response: {
code: 200,
success: true,
data: {
nickname: "抖音作者",
user_id: "4065311277723529",
},
},
}),
toolEnvelope({
response: {
code: 200,
success: true,
data: {
user: {
nickname: "抖音作者",
unique_id: "douyin-987",
sec_uid: secUid,
follower_count: "2.3万",
ip_location: "广东",
},
},
},
}),
],
{
[shortLink]: `https://www.iesdouyin.com/share/video/7671270545631842038/?sec_uid=${secUid}`,
},
);
const profile = await resolveAccountProfileFromMcp(
shortLink,
"抖音",
"回填昵称",
{ endpoint: "https://collector.example/mcp", key: "test-key" },
fetchImpl,
);
assert.deepEqual(profile, {
platformUid: secUid,
nickname: "抖音作者",
profileUrl: `https://www.douyin.com/user/${secUid}`,
redId: "douyin-987",
ipLocation: "广东",
followers: 23_000,
gender: "",
bio: "",
recentNoteTitles: [],
providerTags: [],
});
assert.equal(
calls.find((call) => call.body?.params?.name === "parse_dy_user_summary")
?.body.params.arguments.request.url,
`https://www.douyin.com/user/${secUid}`,
);
});

View File

@@ -0,0 +1,57 @@
import assert from "node:assert/strict";
import test from "node:test";
import { strToU8, unzipSync, zipSync } from "fflate";
import { compactPartnerBatchWorkbookForUpload } from "../koc-portal/app/batch-workbook-upload.ts";
const imageBytes = (marker, size) => {
const bytes = new Uint8Array(size);
bytes.set([0x89, 0x50, 0x4e, 0x47, marker]);
for (let index = 5; index < bytes.length; index += 1) bytes[index] = marker;
return bytes;
};
test("slims oversized WPS workbooks without removing backfill screenshots", () => {
const worksheet = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
<sheetData>
<row r="1">
<c r="A1" t="inlineStr"><is><t>序号(不能改)</t></is></c>
<c r="D1" t="inlineStr"><is><t>图片1</t></is></c>
<c r="F1" t="inlineStr"><is><t>笔记截图</t></is></c>
<c r="G1" t="inlineStr"><is><t>数据分析截图(单篇笔记数据分析截图)</t></is></c>
</row>
<row r="2">
<c r="D2" t="str"><f>_xlfn.DISPIMG(&quot;SOURCE&quot;,1)</f><v>=DISPIMG(&quot;SOURCE&quot;,1)</v></c>
<c r="F2" t="str"><f>_xlfn.DISPIMG(&quot;PUBLISH&quot;,1)</f><v>=DISPIMG(&quot;PUBLISH&quot;,1)</v></c>
</row>
</sheetData>
</worksheet>`;
const cellImages = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<etc:cellImages xmlns:etc="http://www.wps.cn/officeDocument/2017/etCustomData" xmlns:xdr="http://schemas.openxmlformats.org/drawingml/2006/spreadsheetDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships">
<etc:cellImage><xdr:pic><xdr:nvPicPr><xdr:cNvPr id="1" name="SOURCE"/></xdr:nvPicPr><xdr:blipFill><a:blip r:embed="rId1"/></xdr:blipFill></xdr:pic></etc:cellImage>
<etc:cellImage><xdr:pic><xdr:nvPicPr><xdr:cNvPr id="2" name="PUBLISH"/></xdr:nvPicPr><xdr:blipFill><a:blip r:embed="rId2"/></xdr:blipFill></xdr:pic></etc:cellImage>
</etc:cellImages>`;
const relationships = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/image" Target="media/source.png"/>
<Relationship Id="rId2" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/image" Target="media/publish.png"/>
</Relationships>`;
const workbook = zipSync(
{
"xl/worksheets/sheet1.xml": strToU8(worksheet),
"xl/cellimages.xml": strToU8(cellImages),
"xl/_rels/cellimages.xml.rels": strToU8(relationships),
"xl/media/source.png": [imageBytes(1, 2_000_000), { level: 0 }],
"xl/media/publish.png": [imageBytes(2, 2_000), { level: 0 }],
},
{ level: 0 },
);
const compacted = compactPartnerBatchWorkbookForUpload(workbook);
const entries = unzipSync(compacted.bytes);
assert.equal(entries["xl/media/source.png"], undefined);
assert.deepEqual(entries["xl/media/publish.png"], imageBytes(2, 2_000));
assert.equal(compacted.removedMediaCount, 1);
assert.equal(compacted.preservedScreenshotCount, 1);
assert.ok(compacted.bytes.byteLength < workbook.byteLength / 10);
});

View File

@@ -0,0 +1,327 @@
import assert from "node:assert/strict";
import test from "node:test";
import { strFromU8, strToU8, unzipSync, zipSync } from "fflate";
import { buildRecoveryWorkbook } from "../lib/recovery-workbook.ts";
import {
PARTNER_BATCH_HEADERS,
buildPartnerBatchWorkbookColumns,
parsePartnerBatchWorkbook,
resolvePartnerWorkbookOrigin,
} from "../lib/partner-batch-workbook.ts";
import { hasMp4FileSignature } from "../lib/video-file.ts";
const png = Uint8Array.from([
0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a,
0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,
0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01,
]);
test("round-trips hidden assignment IDs and embedded backfill screenshots", () => {
const workbook = buildRecoveryWorkbook({
sheetName: "批量回填",
headers: [...PARTNER_BATCH_HEADERS],
columnWidths: [10, 20, 40, 30, 40, 24, 28, 20, 20, 20],
hiddenColumns: [7, 8, 9],
rows: [
{
cells: [
1,
"测试笔记",
"正文 #话题",
"",
"https://www.xiaohongshu.com/explore/1234567890abcdef",
"",
"",
"distribution-1",
"",
"",
],
images: [
{
column: 3,
image: { bytes: png, contentType: "image/png", description: "原图" },
},
{
column: 5,
image: { bytes: png, contentType: "image/png", description: "笔记截图" },
},
{
column: 6,
image: { bytes: png, contentType: "image/png", description: "数据截图" },
},
],
},
],
});
const rows = parsePartnerBatchWorkbook(workbook);
assert.equal(rows.length, 1);
assert.equal(rows[0].distributionId, "distribution-1");
assert.equal(rows[0].title, "测试笔记");
assert.match(rows[0].publishUrl, /xiaohongshu\.com/);
assert.equal(rows[0].publishScreenshot?.contentType, "image/png");
assert.equal(rows[0].creatorScreenshot?.contentType, "image/png");
const entries = unzipSync(workbook);
const worksheet = strFromU8(entries["xl/worksheets/sheet1.xml"]);
assert.match(worksheet, /序号(不能改)/);
assert.doesNotMatch(worksheet, /张原图(见图)|已回填(见图)|请插入笔记截图/);
assert.match(worksheet, /<drawing r:id="rId1"\/>/);
assert.doesNotMatch(worksheet, /#VALUE!/);
const drawing = strFromU8(entries["xl/drawings/drawing1.xml"]);
assert.equal((drawing.match(/<xdr:twoCellAnchor editAs="twoCell">/g) ?? []).length, 3);
assert.match(drawing, /<xdr:col>3<\/xdr:col>/);
assert.match(drawing, /<xdr:col>5<\/xdr:col>/);
assert.match(drawing, /<xdr:col>6<\/xdr:col>/);
assert.match(worksheet, /min="8" max="8"[^>]*hidden="1"/);
});
test("accepts the legacy sequence header for previously exported workbooks", () => {
const legacyHeaders = [...PARTNER_BATCH_HEADERS];
legacyHeaders[0] = "序号";
const workbook = buildRecoveryWorkbook({
sheetName: "批量回填",
headers: legacyHeaders,
columnWidths: [10, 20, 40, 30, 40, 24, 28, 20, 20, 20],
hiddenColumns: [7, 8, 9],
rows: [
{
cells: [
1,
"旧模板笔记",
"正文",
"",
"",
"",
"",
"distribution-legacy",
"",
"",
],
images: [],
},
],
});
const rows = parsePartnerBatchWorkbook(workbook);
assert.equal(rows[0].sequence, "1");
assert.equal(rows[0].distributionId, "distribution-legacy");
});
test("imports dynamic source image columns without confusing screenshot columns", () => {
const headers = [
"序号(不能改)",
"标题",
"笔记内容(正文+话题)",
"图片1",
"图片2",
"发布链接",
"笔记截图",
"数据分析截图(单篇笔记数据分析截图)",
"_系统笔记ID",
"_原笔记截图",
"_原数据分析截图",
];
const workbook = buildRecoveryWorkbook({
sheetName: "批量回填",
headers,
columnWidths: headers.map(() => 20),
hiddenColumns: [8, 9, 10],
rows: [
{
cells: [
1,
"多图笔记",
"正文",
"",
"",
"https://www.xiaohongshu.com/explore/dynamic",
"",
"",
"distribution-dynamic",
"",
"",
],
images: [
{
column: 3,
image: { bytes: png, contentType: "image/png", description: "原图1" },
},
{
column: 4,
image: { bytes: png, contentType: "image/png", description: "原图2" },
},
{
column: 6,
image: { bytes: png, contentType: "image/png", description: "笔记截图" },
},
{
column: 7,
image: { bytes: png, contentType: "image/png", description: "数据截图" },
},
],
},
],
});
const rows = parsePartnerBatchWorkbook(workbook);
assert.equal(rows[0].distributionId, "distribution-dynamic");
assert.equal(rows[0].publishScreenshot?.contentType, "image/png");
assert.equal(rows[0].creatorScreenshot?.contentType, "image/png");
});
test("imports screenshots saved by WPS as DISPIMG cell images", () => {
const sourceImage = Uint8Array.from([...png, 1]);
const publishScreenshot = Uint8Array.from([...png, 2]);
const creatorScreenshot = Uint8Array.from([...png, 3]);
const worksheet = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
<sheetData>
<row r="1">
<c r="A1" t="inlineStr"><is><t>序号(不能改)</t></is></c>
<c r="B1" t="inlineStr"><is><t>标题</t></is></c>
<c r="C1" t="inlineStr"><is><t>笔记内容(正文+话题)</t></is></c>
<c r="D1" t="inlineStr"><is><t>图片1</t></is></c>
<c r="E1" t="inlineStr"><is><t>发布链接</t></is></c>
<c r="F1" t="inlineStr"><is><t>笔记截图</t></is></c>
<c r="G1" t="inlineStr"><is><t>数据分析截图(单篇笔记数据分析截图)</t></is></c>
<c r="H1" t="inlineStr"><is><t>_系统笔记ID</t></is></c>
<c r="I1" t="inlineStr"><is><t>_原笔记截图</t></is></c>
<c r="J1" t="inlineStr"><is><t>_原数据分析截图</t></is></c>
</row>
<row r="2">
<c r="A2"><v>1</v></c>
<c r="B2" t="inlineStr"><is><t>WPS 笔记</t></is></c>
<c r="D2" t="str"><f>_xlfn.DISPIMG(&quot;SOURCE&quot;,1)</f><v>=DISPIMG(&quot;SOURCE&quot;,1)</v></c>
<c r="E2" t="inlineStr"><is><t>https://www.xiaohongshu.com/explore/wps</t></is></c>
<c r="F2" t="str"><f>_xlfn.DISPIMG(&quot;PUBLISH&quot;,1)</f><v>=DISPIMG(&quot;PUBLISH&quot;,1)</v></c>
<c r="G2" t="str"><f>_xlfn.DISPIMG(&quot;CREATOR&quot;,1)</f><v>=DISPIMG(&quot;CREATOR&quot;,1)</v></c>
<c r="H2" t="inlineStr"><is><t>distribution-wps</t></is></c>
</row>
</sheetData>
</worksheet>`;
const cellImages = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<etc:cellImages xmlns:etc="http://www.wps.cn/officeDocument/2017/etCustomData" xmlns:xdr="http://schemas.openxmlformats.org/drawingml/2006/spreadsheetDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships">
<etc:cellImage><xdr:pic><xdr:nvPicPr><xdr:cNvPr id="1" name="SOURCE"/></xdr:nvPicPr><xdr:blipFill><a:blip r:embed="rId1"/></xdr:blipFill></xdr:pic></etc:cellImage>
<etc:cellImage><xdr:pic><xdr:nvPicPr><xdr:cNvPr id="2" name="PUBLISH"/></xdr:nvPicPr><xdr:blipFill><a:blip r:embed="rId2"/></xdr:blipFill></xdr:pic></etc:cellImage>
<etc:cellImage><xdr:pic><xdr:nvPicPr><xdr:cNvPr id="3" name="CREATOR"/></xdr:nvPicPr><xdr:blipFill><a:blip r:embed="rId3"/></xdr:blipFill></xdr:pic></etc:cellImage>
</etc:cellImages>`;
const relationships = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/image" Target="media/source.png"/>
<Relationship Id="rId2" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/image" Target="media/publish.png"/>
<Relationship Id="rId3" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/image" Target="media/creator.png"/>
</Relationships>`;
const workbook = zipSync({
"xl/worksheets/sheet1.xml": strToU8(worksheet),
"xl/cellimages.xml": strToU8(cellImages),
"xl/_rels/cellimages.xml.rels": strToU8(relationships),
"xl/media/source.png": sourceImage,
"xl/media/publish.png": publishScreenshot,
"xl/media/creator.png": creatorScreenshot,
});
const rows = parsePartnerBatchWorkbook(workbook);
assert.equal(rows.length, 1);
assert.equal(rows[0].distributionId, "distribution-wps");
assert.deepEqual(rows[0].publishScreenshot?.bytes, publishScreenshot);
assert.deepEqual(rows[0].creatorScreenshot?.bytes, creatorScreenshot);
});
test("builds video-task workbooks with video columns and no source image columns", () => {
const columns = buildPartnerBatchWorkbookColumns({
contentFormat: "video",
maxSourceImages: 3,
maxSourceVideos: 2,
});
assert.deepEqual(columns.headers.slice(0, 8), [
"序号(不能改)",
"标题",
"笔记内容(正文+话题)",
"视频1",
"视频2",
"发布链接",
"笔记截图",
"数据分析截图(单篇笔记数据分析截图)",
]);
assert.equal(columns.headers.some((header) => /^图片\d+$/.test(header)), false);
const workbook = buildRecoveryWorkbook({
sheetName: "视频批量回填",
headers: columns.headers,
columnWidths: columns.columnWidths,
hiddenColumns: [
columns.systemColumn,
columns.systemColumn + 1,
columns.systemColumn + 2,
],
rows: [
{
cells: [
1,
"视频笔记",
"视频正文 #测试",
"下载视频1",
"下载视频2",
"",
"",
"",
"distribution-video",
"",
"",
],
images: [],
hyperlinks: [
{
column: columns.sourceVideoStartColumn,
url: "https://koc.example.com/api/partner-image?kind=video&download=1",
},
],
},
],
});
const entries = unzipSync(workbook);
const worksheet = strFromU8(entries["xl/worksheets/sheet1.xml"]);
const relationships = strFromU8(
entries["xl/worksheets/_rels/sheet1.xml.rels"],
);
assert.match(worksheet, /视频1/);
assert.doesNotMatch(worksheet, /图片1/);
assert.match(relationships, /https:\/\/koc\.example\.com\/api\/partner-image/);
assert.match(relationships, /download=1/);
});
test("uses the configured public origin before proxy or container addresses", () => {
const request = new Request("http://app:3000/api/partner-batch-workbook", {
headers: {
host: "app:3000",
"x-forwarded-host": "internal-proxy:8080",
"x-forwarded-proto": "https",
},
});
assert.equal(
resolvePartnerWorkbookOrigin(request, "https://koc.example.com/koc/"),
"https://koc.example.com",
);
});
test("preserves a forwarded non-standard port when no origin is configured", () => {
const request = new Request("http://app:3000/api/partner-batch-workbook", {
headers: {
host: "app:3000",
"x-forwarded-host": "localhost:8080",
"x-forwarded-proto": "http",
},
});
assert.equal(resolvePartnerWorkbookOrigin(request), "http://localhost:8080");
});
test("recognizes MP4 bytes instead of trusting a response content type", () => {
const mp4Header = Uint8Array.from([
0x00, 0x00, 0x00, 0x18,
0x66, 0x74, 0x79, 0x70,
0x69, 0x73, 0x6f, 0x6d,
0x00, 0x00, 0x02, 0x00,
0x69, 0x73, 0x6f, 0x6d,
0x6d, 0x70, 0x34, 0x32,
]);
assert.equal(hasMp4FileSignature(mp4Header), true);
assert.equal(hasMp4FileSignature(new TextEncoder().encode("not a video")), false);
});

View File

@@ -16,7 +16,7 @@ test("creates an xlsx archive whose pictures are embedded in worksheet cells", (
columnWidths: [8, 24, 24, 24],
rows: [
{
cells: [1, "测试笔记", "见图", "见图"],
cells: [1, "测试笔记", "", ""],
images: [
{
column: 2,
@@ -43,12 +43,24 @@ test("creates an xlsx archive whose pictures are embedded in worksheet cells", (
],
});
const archive = unzipSync(workbook);
const worksheet = strFromU8(archive["xl/worksheets/sheet1.xml"]);
assert.ok(archive["xl/media/image1.png"]);
assert.ok(archive["xl/media/image2.png"]);
assert.match(strFromU8(archive["xl/worksheets/sheet1.xml"]), /<drawing r:id="rId1"\/>/);
assert.match(strFromU8(archive["xl/drawings/drawing1.xml"]), /oneCellAnchor/);
assert.match(strFromU8(archive["xl/drawings/_rels/drawing1.xml.rels"]), /image2\.png/);
assert.match(worksheet, /<drawing r:id="rId1"\/>/);
assert.doesNotMatch(worksheet, /#VALUE!/);
const drawing = strFromU8(archive["xl/drawings/drawing1.xml"]);
assert.equal((drawing.match(/<xdr:twoCellAnchor editAs="twoCell">/g) ?? []).length, 2);
assert.match(drawing, /<xdr:col>2<\/xdr:col>/);
assert.match(drawing, /<xdr:col>3<\/xdr:col>/);
assert.match(
strFromU8(archive["xl/drawings/_rels/drawing1.xml.rels"]),
/image2\.png/,
);
assert.doesNotMatch(
worksheet,
/见图/,
);
});
test("creates clickable external hyperlinks for resource exports", () => {

View File

@@ -23,6 +23,30 @@ test("builds the KOC LOOP product shell", async () => {
await access(new URL("../.next/static", import.meta.url));
});
test("keeps distribution filters as always-visible fuzzy search fields", async () => {
const [adminApp, globalCss] = await Promise.all([
readFile(new URL("../app/admin-app.tsx", import.meta.url), "utf8"),
readFile(new URL("../app/globals.css", import.meta.url), "utf8"),
]);
assert.match(globalCss, /\.distribution-task-search\s*\{[^}]*flex-direction:\s*row/s);
assert.match(globalCss, /\.distribution-task-search\s*\{[^}]*margin:\s*0/s);
assert.match(globalCss, /\.distribution-task-search\s*>\s*span\s*\{[^}]*flex:\s*0\s+0\s+18px/s);
assert.match(globalCss, /\.distribution-task-filter-combobox\.brand\s*\{[^}]*flex-basis:\s*176px/s);
assert.match(globalCss, /\.distribution-task-filter-menu\s*\{[^}]*position:\s*absolute[^}]*z-index:\s*60/s);
assert.match(globalCss, /\.distribution-task-filter-input\s*\{[^}]*display:\s*flex[^}]*margin:\s*0/s);
assert.match(adminApp, /aria-expanded=\{openTaskFilter === filter\.key\}/);
assert.match(adminApp, /role="combobox"/);
assert.match(adminApp, /role="listbox"/);
assert.match(adminApp, /placeholder=\{`搜索\$\{filter\.label\}`\}/);
assert.match(adminApp, /setTaskFilterValue\(filter\.key, event\.target\.value\)/);
assert.doesNotMatch(adminApp, /distribution-task-filter-trigger/);
assert.doesNotMatch(adminApp, /distribution-task-filter-menu-search/);
assert.doesNotMatch(adminApp, /taskFilterQuery/);
assert.doesNotMatch(adminApp, /distribution-task-option-panel/);
assert.doesNotMatch(adminApp, /<select value=\{contentTypeFilter\}/);
});
test("stacks user management and securely removes departed accounts", async () => {
const [usersPage, usersRoute, globalCss] = await Promise.all([
readFile(new URL("../app/users-page.tsx", import.meta.url), "utf8"),
@@ -137,8 +161,8 @@ test("issues external task links and supports one-to-one note submissions", asyn
assert.match(partnerRoute, /claimantIdentifier\.canonical/);
assert.match(partnerRoute, /legacyPartnerId/);
assert.match(partnerRoute, /微信号或手机号/);
assert.match(partnerRoute, /extractXhsPublishUrl/);
assert.match(partnerRoute, /小红书长链或短链/);
assert.match(partnerRoute, /extractPublishUrl/);
assert.match(partnerRoute, /task\.platform/);
assert.match(partnerRoute, /请填写发布链接/);
assert.doesNotMatch(partnerRoute, /请填写发布账号昵称和发布链接/);
assert.match(partnerRoute, /没有找到领取记录/);
@@ -147,11 +171,16 @@ test("issues external task links and supports one-to-one note submissions", asyn
assert.match(partnerRoute, /enrichDistributionAccount/);
assert.match(partnerRoute, /runInBackground\(enrichment/);
assert.match(accountEnrichment, /profile_url = excluded\.profile_url/);
assert.match(accountEnrichment, /resolveXhsAccountProfileFromMcp/);
assert.match(accountEnrichment, /resolveXhsProfileDetailsFromMcp/);
assert.match(accountEnrichment, /resolveAccountProfileFromMcp/);
assert.match(accountEnrichment, /resolveProfileDetailsFromMcp/);
assert.match(accountEnrichment, /isVerifiedXhsProfileUrl/);
assert.match(accountEnrichment, /endsWith\("\.xiaohongshu\.com"\)/);
assert.match(accountEnrichment, /DELETE FROM accounts/);
assert.match(accountEnrichment, /WHERE platform = \? AND platform_uid = \?/);
assert.match(accountEnrichment, /existingAccount\?\.id \|\| canonicalAccountId/);
assert.match(accountEnrichment, /cl\.claimant_name AS claimant_contact/);
assert.match(accountEnrichment, /current_contact = CASE/);
assert.match(accountEnrichment, /!row\.resolved_account_id/);
assert.match(accountEnrichment, /a\.followers/);
assert.match(accountEnrichment, /followers = CASE/);
assert.match(accountEnrichment, /Number\(row\.followers \?\? 0\) === 0/);
@@ -171,6 +200,11 @@ test("issues external task links and supports one-to-one note submissions", asyn
assert.match(imageRoute, /creator-center\//);
assert.match(imageRoute, /imageKind === "publish"/);
assert.match(imageRoute, /imageKind === "creator"/);
assert.match(imageRoute, /isMutableEvidence/);
assert.match(imageRoute, /"private, no-store"/);
assert.match(imageRoute, /Content-Type", "video\/mp4"/);
assert.match(imageRoute, /video-\$\{imageIndex\}\.mp4/);
assert.match(imageRoute, /downloadRequested \? "attachment" : "inline"/);
assert.match(imageRoute, /cl\.claim_token/);
assert.match(imageUploadRoute, /isAdminRequest/);
assert.match(cors, /KOC_PORTAL_URL/);
@@ -178,6 +212,17 @@ test("issues external task links and supports one-to-one note submissions", asyn
assert.match(cors, /X-KOC-Upload-Kind/);
assert.match(cors, /Access-Control-Allow-Origin/);
assert.match(adminApp, /hasCreatorMetrics/);
assert.match(adminApp, /function PlatformBadge/);
assert.match(adminApp, /function resourceProfileLink/);
assert.match(adminApp, /搜索主页/);
assert.match(adminApp, /latest_publish_url/);
assert.match(adminApp, /通过作品查看主页/);
assert.match(adminApp, /platform-logo/);
assert.match(adminApp, /按任务名称模糊搜索/);
assert.match(adminApp, /全部品牌\/项目/);
assert.match(adminApp, /全部内容类型/);
assert.match(adminApp, /全部平台/);
assert.match(adminApp, /task-scope-subline/);
assert.match(adminApp, /待KOC填写数据/);
assert.match(adminApp, /AdminImageLightbox/);
assert.match(adminApp, /CreatorScreenshotPreview/);
@@ -195,6 +240,43 @@ test("issues external task links and supports one-to-one note submissions", asyn
assert.match(migration, /claim_token/);
});
test("exports and imports claim-bound Excel backfill workbooks", async () => {
const [route, parser, workbook, imageNormalizer, nginx] = await Promise.all([
readFile(new URL("../app/api/partner-batch-workbook/route.ts", import.meta.url), "utf8"),
readFile(new URL("../lib/partner-batch-workbook.ts", import.meta.url), "utf8"),
readFile(new URL("../lib/recovery-workbook.ts", import.meta.url), "utf8"),
readFile(new URL("../lib/workbook-image.ts", import.meta.url), "utf8"),
readFile(new URL("../deploy/nginx/koc-loop.conf", import.meta.url), "utf8"),
]);
assert.match(route, /批量回填/);
assert.match(route, /publish-evidence/);
assert.match(route, /creator-center/);
assert.match(route, /extractPublishUrl/);
assert.match(route, /findAccess/);
assert.match(route, /isDifferentFromStoredImage/);
assert.match(route, /rowIndex \+ 1/);
assert.match(route, /buildPartnerBatchWorkbookColumns/);
assert.match(route, /resolvePartnerWorkbookOrigin/);
assert.match(route, /download: "1"/);
assert.match(route, /columns\.sourceImageStartColumn \+ index/);
assert.match(parser, /_系统笔记ID/);
assert.match(parser, /笔记截图/);
assert.match(parser, /数据分析截图(单篇笔记数据分析截图)/);
assert.match(parser, /parseImages/);
assert.match(workbook, /hiddenColumns/);
assert.match(workbook, /offsetX/);
assert.doesNotMatch(workbook, /value \|\| "见图"/);
assert.match(imageNormalizer, /\.rotate\(\)/);
assert.match(imageNormalizer, /\.png\(/);
assert.match(nginx, /client_max_body_size 85m/);
assert.equal((nginx.match(/proxy_set_header Host \$http_host;/g) ?? []).length, 2);
assert.equal(
(nginx.match(/proxy_set_header X-Forwarded-Host \$http_host;/g) ?? []).length,
2,
);
});
test("supports task collection schedules and latest public metrics", async () => {
const [
adminApp,
@@ -206,6 +288,7 @@ test("supports task collection schedules and latest public metrics", async () =>
migration,
accountMigration,
compose,
mcpClient,
] = await Promise.all([
readFile(new URL("../app/admin-app.tsx", import.meta.url), "utf8"),
readFile(new URL("../app/api/action/route.ts", import.meta.url), "utf8"),
@@ -216,6 +299,7 @@ test("supports task collection schedules and latest public metrics", async () =>
readFile(new URL("../mysql/0001_init.sql", import.meta.url), "utf8"),
readFile(new URL("../mysql/0001_init.sql", import.meta.url), "utf8"),
readFile(new URL("../docker-compose.self-hosted.yml", import.meta.url), "utf8"),
readFile(new URL("../lib/mcp-collection-client.ts", import.meta.url), "utf8"),
]);
for (const label of [
@@ -238,12 +322,19 @@ test("supports task collection schedules and latest public metrics", async () =>
assert.match(adminApp, /sortWithNullsLast/);
assert.match(adminApp, /内容 \/ 发布账号/);
assert.match(adminApp, /recovery-title-link/);
assert.match(adminApp, /打开小红书笔记/);
assert.match(adminApp, /打开\$\{selectedTask\.platform\}作品/);
assert.match(adminApp, /target="_blank"/);
assert.match(adminApp, /noopener noreferrer/);
assert.match(adminApp, /const noteUrl = xhsPublishUrl\(item\.publish_url\)/);
assert.match(adminApp, /const noteUrl = publicPublishUrl\(item\.publish_url\)/);
assert.match(adminApp, /noteUrl \? \(/);
assert.match(adminApp, /updateDistributionPublishUrl/);
assert.match(adminApp, /填写链接/);
assert.match(adminApp, /更新链接/);
assert.match(adminApp, /hostname === "xhslink\.cn"/);
assert.match(actionRoute, /update_distribution_publish_url/);
assert.match(actionRoute, /extractPublishUrl/);
assert.match(actionRoute, /DELETE FROM collection_runs WHERE distribution_id = \?/);
assert.match(actionRoute, /enrichDistributionAccount/);
assert.match(actionRoute, /save_collection_schedule/);
assert.match(actionRoute, /collect_now/);
assert.match(actionRoute, /backfill_account_profiles/);
@@ -253,7 +344,11 @@ test("supports task collection schedules and latest public metrics", async () =>
assert.match(actionRoute, /createCollectionRunTasks/);
assert.match(bootstrapRoute, /runDueScheduledCollections/);
assert.match(collectionService, /INSERT OR IGNORE INTO collection_runs/);
assert.match(collectionService, /collectXhsMetricsFromMcp/);
assert.match(
collectionService,
/run\.status === "success" && source !== "manual"/,
);
assert.match(collectionService, /collectMetricsFromMcp/);
assert.doesNotMatch(collectionService, /hashText/);
assert.match(collectionService, /runScheduledCollections/);
assert.match(collectionService, /runDueScheduledCollections/);
@@ -272,6 +367,10 @@ test("supports task collection schedules and latest public metrics", async () =>
assert.match(migration, /collection_runs_distribution_date_idx/);
assert.match(accountMigration, /public_account_id/);
assert.match(compose, /ENABLE_SCHEDULER/);
assert.match(mcpClient, /"fetch_content_detail"/);
assert.match(mcpClient, /"parse_xhs_user_summary"/);
assert.doesNotMatch(mcpClient, /"parse_xhs_note"/);
assert.doesNotMatch(mcpClient, /"collect_xhs_wen_note_detail"/);
});
test("supports fixed screenshot collection tasks without publish metrics", async () => {
@@ -314,7 +413,7 @@ test("exports complete task recovery data to Excel with embedded images", async
assert.match(adminApp, /导出全部数据/);
assert.match(adminApp, /\/api\/recovery-export/);
assert.match(adminApp, /图片和截图已嵌入表格/);
assert.match(exportRoute, /小红书昵称/);
assert.match(exportRoute, /`\$\{task\.platform\}昵称`/);
assert.match(exportRoute, /曝光量-实际第7天/);
assert.match(exportRoute, /阅读量-实际第7天/);
assert.match(exportRoute, /publish_screenshot_key/);
@@ -323,8 +422,9 @@ test("exports complete task recovery data to Excel with embedded images", async
assert.match(exportRoute, /isAdminRequest/);
assert.match(exportRoute, /consumeMcpExportToken/);
assert.match(workbook, /xl\/drawings\/drawing1\.xml/);
assert.match(workbook, /twoCellAnchor editAs="twoCell"/);
assert.match(workbook, /xl\/media\/image/);
assert.match(workbook, /oneCellAnchor/);
assert.match(workbook, /relationships\/image/);
});
test("provides simple username-password login and three server-enforced roles", async () => {
@@ -393,6 +493,7 @@ test("filters and exports the current KOC resource result set", async () => {
]);
assert.match(adminApp, /搜索账号名称 \/ 账号ID/);
assert.match(adminApp, /当前联系人/);
assert.match(adminApp, /搜索IP地区/);
assert.match(adminApp, /搜索合作来源/);
assert.match(adminApp, /ipLocation\.includes\(ipKeyword\)/);
@@ -401,6 +502,7 @@ test("filters and exports the current KOC resource result set", async () => {
assert.match(adminApp, /filteredAccounts\.map\(\(item\) => item\.account\.id\)/);
assert.match(exportRoute, /小红书号\/抖音号/);
assert.match(exportRoute, /历史合作来源/);
assert.match(exportRoute, /当前联系人/);
assert.match(exportRoute, /合作社资源 · 不可直联/);
assert.match(exportRoute, /isManagerRequest/);
assert.match(exportRoute, /consumeMcpExportToken/);
@@ -408,21 +510,61 @@ test("filters and exports the current KOC resource result set", async () => {
});
test("imports existing KOC resources through a validated spreadsheet preview", async () => {
const [adminApp, importRoute, resourceParser, accountMigration] = await Promise.all([
const [adminApp, globalCss, importRoute, resourceParser, resourceWrite, accountMigration, profileMigration, contactMigration] = await Promise.all([
readFile(new URL("../app/admin-app.tsx", import.meta.url), "utf8"),
readFile(new URL("../app/globals.css", import.meta.url), "utf8"),
readFile(new URL("../app/api/resources-import/route.ts", import.meta.url), "utf8"),
readFile(new URL("../lib/resource-import.ts", import.meta.url), "utf8"),
readFile(new URL("../lib/resource-write.ts", import.meta.url), "utf8"),
readFile(new URL("../mysql/0002_resource_import.sql", import.meta.url), "utf8"),
readFile(new URL("../mysql/0006_account_profile_tags.sql", import.meta.url), "utf8"),
readFile(new URL("../mysql/0007_account_current_contact.sql", import.meta.url), "utf8"),
]);
assert.match(adminApp, /下载导入模板/);
assert.match(adminApp, /校验并预览/);
assert.match(adminApp, /确认导入/);
assert.match(importRoute, /isManagerRequest/);
assert.match(importRoute, /mode !== "commit"/);
assert.match(resourceParser, /RESOURCE_IMPORT_MAX_ROWS = 100/);
assert.match(resourceParser, /当前自动解析仅支持小红书账号主页/);
assert.match(importRoute, /resolveXhsProfileDetailsFromMcp/);
assert.match(resourceParser, /RESOURCE_IMPORT_MAX_ROWS = 10_000/);
assert.match(resourceParser, /RESOURCE_IMPORT_MAX_BYTES = 20 \* 1024 \* 1024/);
assert.match(adminApp, /单次最多 10,000 个账号,文件不超过 20MB/);
assert.match(resourceWrite, /RESOURCE_IMPORT_DB_BATCH_SIZE = 100/);
assert.match(importRoute, /bulk resource profile enrichment/);
assert.match(adminApp, /异常数据将自动跳过,不会导入/);
assert.match(
adminApp,
/summary\.create \+ importPreview\.summary\.update === 0 \|\| importWorking/,
);
assert.doesNotMatch(
adminApp,
/disabled=\{importPreview\.summary\.error > 0 \|\| importWorking\}/,
);
assert.match(importRoute, /const importableRows = analyzed\.filter/);
assert.match(importRoute, /跳过 \$\{summary\.error\} 条异常数据/);
assert.match(importRoute, /previewAnalyzedRows\(analyzed\)/);
assert.match(resourceParser, /当前自动解析仅支持小红书或抖音账号主页/);
assert.match(resourceWrite, /resolveProfileDetailsFromMcp/);
assert.match(accountMigration, /cooperation_source/);
assert.match(profileMigration, /ADD COLUMN gender/);
assert.match(profileMigration, /ADD COLUMN bio/);
assert.match(profileMigration, /ADD COLUMN tags/);
assert.match(contactMigration, /ADD COLUMN current_contact/);
assert.match(adminApp, /gender-icon male/);
assert.match(adminApp, /gender-icon female/);
assert.match(adminApp, /resource-profile-avatar/);
assert.match(adminApp, /resource-account-number/);
assert.match(adminApp, /resource-platform-line/);
assert.match(adminApp, /resource-latest/);
assert.match(adminApp, /合作来源/);
assert.match(adminApp, /待打标/);
assert.doesNotMatch(adminApp, /className="verified-dot"/);
assert.match(globalCss, /\.resource-tags\s*\{[^}]*margin-bottom:\s*auto/s);
assert.match(globalCss, /\.resource-card-foot\s*\{[^}]*margin-top:\s*10px/s);
assert.match(adminApp, /resource-tags/);
assert.match(adminApp, /最多 5 个/);
assert.match(resourceParser, /gender: \["性别"\]/);
assert.match(resourceParser, /bio: \["简介"/);
assert.match(resourceParser, /tags: \["标签"/);
});
test("supports anonymous partner delegation without creating a second data flow", async () => {

View File

@@ -1,7 +1,9 @@
import assert from "node:assert/strict";
import test from "node:test";
import { strFromU8, strToU8, unzipSync, zipSync } from "fflate";
import { buildRecoveryWorkbook } from "../lib/recovery-workbook.ts";
import {
RESOURCE_IMPORT_MAX_ROWS,
mergeCooperationSources,
normalizeProfileUrl,
parseResourceFollowers,
@@ -10,6 +12,72 @@ import {
resourcePlatformUid,
} from "../lib/resource-import.ts";
test("accepts several thousand accounts in one import file", () => {
const csv = [
"账号主页,账号昵称,账号ID,IP属地,粉丝数,合作来源",
...Array.from({ length: 3_000 }, (_, index) => {
const id = String(index + 1).padStart(6, "0");
return `https://www.xiaohongshu.com/user/profile/bulk${id},账号${id},${id},上海,100,批量资源`;
}),
].join("\n");
const rows = parseResourceImportFile(
"bulk-resources.csv",
new TextEncoder().encode(csv),
);
assert.equal(RESOURCE_IMPORT_MAX_ROWS, 10_000);
assert.equal(rows.length, 3_000);
assert.equal(rows[0].rowNumber, 2);
assert.equal(rows.at(-1)?.rowNumber, 3_001);
assert.equal(rows.every((row) => row.errors.length === 0), true);
});
test("parses several thousand accounts from an XLSX workbook", () => {
const workbook = buildRecoveryWorkbook({
sheetName: "KOC资源导入",
headers: ["账号主页", "账号昵称", "账号ID", "IP属地", "粉丝数", "合作来源"],
columnWidths: [48, 24, 20, 16, 14, 24],
rows: Array.from({ length: 3_000 }, (_, index) => {
const id = String(index + 1).padStart(6, "0");
return {
cells: [
`https://www.xiaohongshu.com/user/profile/xlsx${id}`,
`账号${id}`,
id,
"北京",
200,
"Excel批量资源",
],
images: [],
};
}),
});
const rows = parseResourceImportFile("bulk-resources.xlsx", workbook);
assert.equal(rows.length, 3_000);
assert.equal(rows[0].profileUrl.endsWith("xlsx000001"), true);
assert.equal(rows.at(-1)?.profileUrl.endsWith("xlsx003000"), true);
});
test("keeps a bounded 10,000-row safety limit", () => {
const csv = [
"账号主页",
...Array.from(
{ length: RESOURCE_IMPORT_MAX_ROWS + 1 },
(_, index) => `https://www.douyin.com/user/bulk-account-${index + 1}`,
),
].join("\n");
assert.throws(
() =>
parseResourceImportFile(
"too-many-resources.csv",
new TextEncoder().encode(csv),
),
/单次最多导入 10000 个账号/,
);
});
test("parses CSV resources and normalizes public profile data", () => {
const csv = [
"账号主页,合作来源",
@@ -26,6 +94,9 @@ test("parses CSV resources and normalizes public profile data", () => {
ipLocation: "",
followers: 0,
followersResolved: false,
gender: "",
bio: "",
tags: [],
cooperationSource: "林林KOC社群",
errors: [],
});
@@ -34,8 +105,8 @@ test("parses CSV resources and normalizes public profile data", () => {
test("uses optional account fields directly and only requires the profile URL", () => {
const csv = [
"账号链接,账号昵称,账号ID,IP属地,粉丝数,合作来源",
"https://www.xiaohongshu.com/user/profile/abc123,番茄不炒蛋,4171542126,江西,10+,历史资源",
"账号链接,账号昵称,账号ID,IP属地,粉丝数,性别,简介,标签,合作来源",
"https://www.xiaohongshu.com/user/profile/abc123,番茄不炒蛋,4171542126,江西,10+,女,分享江西本地生活,本地生活、美食探店,历史资源",
].join("\n");
const [row] = parseResourceImportFile(
"resources.csv",
@@ -46,9 +117,36 @@ test("uses optional account fields directly and only requires the profile URL",
assert.equal(row.ipLocation, "江西");
assert.equal(row.followers, 10);
assert.equal(row.followersResolved, true);
assert.equal(row.gender, "女");
assert.equal(row.bio, "分享江西本地生活");
assert.deepEqual(row.tags, ["本地生活", "美食探店"]);
assert.deepEqual(resourceImportMissingFields(row), []);
});
test("validates optional gender", () => {
const csv = [
"账号链接,性别,标签",
"https://www.xiaohongshu.com/user/profile/abc123,其他,美食探店",
].join("\n");
const [row] = parseResourceImportFile(
"resources.csv",
new TextEncoder().encode(csv),
);
assert.match(row.errors.join(""), /性别格式不正确/);
});
test("rejects more than five tags in one optional tag cell", () => {
const csv = [
"账号链接,标签",
'https://www.xiaohongshu.com/user/profile/abc123,"美食探店,旅游出行,数码产品,本地生活,婚嫁备婚,美妆护肤"',
].join("\n");
const [row] = parseResourceImportFile(
"resources.csv",
new TextEncoder().encode(csv),
);
assert.match(row.errors.join(""), /最多填写 5 个标签/);
});
test("normalizes common follower formats and identifies missing enrichment fields", () => {
assert.deepEqual(parseResourceFollowers("1.3万"), {
value: 13_000,
@@ -86,11 +184,49 @@ test("parses the first matching worksheet from an XLSX workbook", () => {
assert.equal(rows[0].followersResolved, false);
});
test("keeps values in their columns after a self-closing blank XLSX cell", () => {
const workbook = buildRecoveryWorkbook({
sheetName: "KOC资源导入",
headers: ["账号主页", "账号昵称", "账号ID", "IP属地", "粉丝数", "合作来源"],
columnWidths: [48, 24, 20, 16, 14, 24],
rows: [
{
cells: [
"https://www.xiaohongshu.com/user/profile/blank-ip-cell",
"空白IP账号",
"123456789",
"",
10,
"",
],
images: [],
},
],
});
const entries = unzipSync(workbook);
const sheetPath = "xl/worksheets/sheet1.xml";
const sheetXml = strFromU8(entries[sheetPath]);
entries[sheetPath] = strToU8(
sheetXml.replace('<c r="E2"', '<c r="D2"/><c r="E2"'),
);
const [row] = parseResourceImportFile(
"self-closing-blank.xlsx",
zipSync(entries),
);
assert.equal(row.ipLocation, "");
assert.equal(row.followers, 10);
assert.equal(row.followersResolved, true);
assert.deepEqual(row.errors, []);
});
test("reports invalid required fields without hiding valid rows", () => {
const csv = "账号主页,合作来源\n,历史资源\nhttps://www.douyin.com/user/demo,社群";
const csv = "账号主页,合作来源\n,历史资源\nhttps://www.douyin.com/user/demo,社群\nhttps://example.com/user/demo,其他";
const rows = parseResourceImportFile("resources.csv", new TextEncoder().encode(csv));
assert.match(rows[0].errors.join(""), /账号主页不能为空/);
assert.match(rows[1].errors.join(""), /仅支持小红书账号主页/);
assert.equal(rows[1].platform, "抖音");
assert.equal(rows[1].errors.length, 0);
assert.match(rows[2].errors.join(""), /仅支持小红书或抖音账号主页/);
});
test("rejects invalid optional follower values without requiring other optional fields", () => {
@@ -105,6 +241,18 @@ test("rejects invalid optional follower values without requiring other optional
assert.match(row.errors.join(""), /粉丝数格式不正确/);
});
test("rejects a numeric value entered as an IP location", () => {
const csv = [
"账号链接,IP属地,粉丝数",
"https://www.xiaohongshu.com/user/profile/abc123,10,100",
].join("\n");
const [row] = parseResourceImportFile(
"resources.csv",
new TextEncoder().encode(csv),
);
assert.match(row.errors.join(""), /IP属地格式不正确/);
});
test("normalizes profile URLs and merges cooperation sources", () => {
assert.equal(
normalizeProfileUrl("https://www.xiaohongshu.com/user/profile/abc/?foo=1#top"),

607
tests/wecom-client.test.mjs Normal file
View File

@@ -0,0 +1,607 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
clearWecomAccessTokenCacheForTests,
createGroupMsgTemplate,
listCustomerGroupChats,
listExternalContacts,
resolveWecomConfig,
sendWecomAppMessage,
sendWecomRobotMessage,
WecomClientError,
} from "../lib/wecom-client.ts";
const baseBindings = {
WECOM_CORP_ID: "corp-test",
WECOM_AGENT_ID: "1000001",
WECOM_SECRET: "secret-test",
WECOM_ROBOT_WEBHOOK:
"https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=robot-test",
WECOM_NOTIFY_DUE_DAYS: "3",
};
function fakeWecom(options = {}) {
const calls = [];
const fetchImpl = async (input, init = {}) => {
const url = new URL(String(input));
calls.push({ url, init });
if (url.pathname.endsWith("/cgi-bin/gettoken")) {
return Response.json({
errcode: 0,
errmsg: "ok",
access_token: "token-test",
expires_in: 7200,
});
}
if (url.pathname.endsWith("/cgi-bin/webhook/send")) {
return Response.json({
errcode: options.robotErrcode ?? 0,
errmsg: options.robotErrmsg ?? "ok",
});
}
if (url.pathname.endsWith("/cgi-bin/message/send")) {
const body = JSON.parse(String(init.body ?? "{}"));
const invalid = options.invalidUser
? body.touser.split("|").filter((u) => u === options.invalidUser)
: [];
return Response.json({
errcode: 0,
errmsg: "ok",
invaliduser: invalid.join("|"),
});
}
return new Response("not found", { status: 404 });
};
return { calls, fetchImpl };
}
test.beforeEach(() => {
clearWecomAccessTokenCacheForTests();
});
test("resolveWecomConfig applies dueDays and trims values", () => {
const config = resolveWecomConfig({
WECOM_CORP_ID: " corp ",
WECOM_AGENT_ID: " 10 ",
WECOM_SECRET: " s ",
WECOM_ROBOT_WEBHOOK: " https://hook ",
WECOM_NOTIFY_DUE_DAYS: "5",
});
assert.equal(config.corpId, "corp");
assert.equal(config.agentId, "10");
assert.equal(config.secret, "s");
assert.equal(config.robotWebhook, "https://hook");
assert.equal(config.dueDays, 5);
});
test("resolveWecomConfig falls back to default dueDays for bad input", () => {
assert.equal(resolveWecomConfig({ WECOM_NOTIFY_DUE_DAYS: "0" }).dueDays, 3);
assert.equal(resolveWecomConfig({ WECOM_NOTIFY_DUE_DAYS: "abc" }).dueDays, 3);
assert.equal(
resolveWecomConfig({ WECOM_NOTIFY_DUE_DAYS: "100" }).dueDays,
30,
);
});
test("sendWecomRobotMessage posts text payload and returns void", async () => {
const { calls, fetchImpl } = fakeWecom();
await sendWecomRobotMessage(
"hello",
resolveWecomConfig(baseBindings),
fetchImpl,
);
const hookCall = calls.find((c) =>
c.url.pathname.endsWith("/cgi-bin/webhook/send"),
);
assert.ok(hookCall, "robot webhook was called");
const body = JSON.parse(String(hookCall.init.body));
assert.equal(body.msgtype, "text");
assert.equal(body.text.content, "hello");
});
test("sendWecomRobotMessage skips silently when webhook is empty", async () => {
const { calls, fetchImpl } = fakeWecom();
const config = resolveWecomConfig({ ...baseBindings, WECOM_ROBOT_WEBHOOK: "" });
await sendWecomRobotMessage("hello", config, fetchImpl);
assert.equal(
calls.filter((c) =>
c.url.pathname.endsWith("/cgi-bin/webhook/send"),
).length,
0,
);
});
test("sendWecomRobotMessage throws WecomClientError on provider error", async () => {
const { fetchImpl } = fakeWecom({
robotErrcode: 93000,
robotErrmsg: "invalid webhook url",
});
await assert.rejects(
() =>
sendWecomRobotMessage(
"hello",
resolveWecomConfig(baseBindings),
fetchImpl,
),
(error) => {
assert.ok(error instanceof WecomClientError);
assert.equal(error.status, 502);
return true;
},
);
});
test("sendWecomAppMessage fetches access_token then sends to message/send", async () => {
const { calls, fetchImpl } = fakeWecom();
const result = await sendWecomAppMessage(
["user-a", "user-b"],
"催办内容",
resolveWecomConfig(baseBindings),
fetchImpl,
);
assert.equal(result.sent, 2);
assert.equal(result.failed, 0);
assert.equal(result.skipped, false);
const tokenCall = calls.find((c) =>
c.url.pathname.endsWith("/cgi-bin/gettoken"),
);
assert.ok(tokenCall, "gettoken was called");
const sendCall = calls.find((c) =>
c.url.pathname.endsWith("/cgi-bin/message/send"),
);
assert.ok(sendCall, "message/send was called");
assert.equal(sendCall.url.searchParams.get("access_token"), "token-test");
const body = JSON.parse(String(sendCall.init.body));
assert.equal(body.touser, "user-a|user-b");
assert.equal(body.agentid, 1000001);
assert.equal(body.text.content, "催办内容");
});
test("sendWecomAppMessage skips when no external user ids", async () => {
const { calls, fetchImpl } = fakeWecom();
const result = await sendWecomAppMessage(
[],
"催办",
resolveWecomConfig(baseBindings),
fetchImpl,
);
assert.equal(result.skipped, true);
assert.equal(
calls.filter((c) =>
c.url.pathname.endsWith("/cgi-bin/message/send"),
).length,
0,
);
});
test("sendWecomAppMessage skips when corp credentials are missing", async () => {
const { calls, fetchImpl } = fakeWecom();
const result = await sendWecomAppMessage(
["user-a"],
"催办",
resolveWecomConfig({
...baseBindings,
WECOM_CORP_ID: "",
WECOM_AGENT_ID: "",
WECOM_SECRET: "",
}),
fetchImpl,
);
assert.equal(result.skipped, true);
assert.equal(
calls.filter((c) =>
c.url.pathname.endsWith("/cgi-bin/gettoken"),
).length,
0,
);
});
test("sendWecomAppMessage reports failed when provider marks invaliduser", async () => {
const { fetchImpl } = fakeWecom({ invalidUser: "user-a" });
const result = await sendWecomAppMessage(
["user-a", "user-b"],
"催办",
resolveWecomConfig(baseBindings),
fetchImpl,
);
assert.equal(result.sent, 1);
assert.equal(result.failed, 1);
});
test("access_token is cached across calls", async () => {
const { calls, fetchImpl } = fakeWecom();
const config = resolveWecomConfig(baseBindings);
await sendWecomAppMessage(["u1"], "a", config, fetchImpl);
await sendWecomAppMessage(["u2"], "b", config, fetchImpl);
const tokenCalls = calls.filter((c) =>
c.url.pathname.endsWith("/cgi-bin/gettoken"),
);
assert.equal(tokenCalls.length, 1, "access_token cached for second call");
});
function fakeExternalContacts() {
const calls = [];
const fetchImpl = async (input, init = {}) => {
const url = new URL(String(input));
calls.push({ url, init });
if (url.pathname.endsWith("/cgi-bin/gettoken")) {
return Response.json({
errcode: 0,
errmsg: "ok",
access_token: "token-test",
expires_in: 7200,
});
}
if (url.pathname.endsWith("/cgi-bin/externalcontact/get_follow_user_list")) {
return Response.json({
errcode: 0,
errmsg: "ok",
follow_user: [{ userid: "owner-a" }, { userid: "owner-b" }],
});
}
if (url.pathname.endsWith("/cgi-bin/externalcontact/list")) {
const userid = url.searchParams.get("userid");
if (userid === "owner-a") {
return Response.json({
errcode: 0,
errmsg: "ok",
external_userid: ["ext-1", "ext-2"],
});
}
if (userid === "owner-b") {
return Response.json({
errcode: 0,
errmsg: "ok",
external_userid: ["ext-3"],
});
}
return Response.json({ errcode: 60020, errmsg: "user not found" });
}
if (url.pathname.endsWith("/cgi-bin/externalcontact/get")) {
const eid = url.searchParams.get("external_userid");
if (eid === "ext-1") {
return Response.json({
errcode: 0,
errmsg: "ok",
external_contact: {
external_userid: "ext-1",
name: "张三",
avatar: "https://example.com/a.png",
corp_fullname: "ACME 公司",
},
});
}
if (eid === "ext-2") {
return Response.json({
errcode: 0,
errmsg: "ok",
external_contact: {
external_userid: "ext-2",
name: "李四",
avatar: "",
corp_fullname: "",
},
});
}
if (eid === "ext-3") {
return Response.json({
errcode: 0,
errmsg: "ok",
external_contact: {
external_userid: "ext-3",
name: "王五",
avatar: "https://example.com/c.png",
corp_fullname: "Other 公司",
},
});
}
return Response.json({ errcode: 60111, errmsg: "not found" });
}
return new Response("not found", { status: 404 });
};
return { calls, fetchImpl };
}
test("listExternalContacts flattens follow_user → list → get into a single array", async () => {
const { fetchImpl } = fakeExternalContacts();
const contacts = await listExternalContacts(
resolveWecomConfig(baseBindings),
fetchImpl,
);
assert.equal(contacts.length, 3);
const first = contacts.find((c) => c.externalUserId === "ext-1");
assert.ok(first);
assert.equal(first.name, "张三");
assert.equal(first.avatar, "https://example.com/a.png");
assert.equal(first.corpName, "ACME 公司");
assert.equal(first.ownerUserId, "owner-a");
const second = contacts.find((c) => c.externalUserId === "ext-2");
assert.ok(second);
assert.equal(second.corpName, "李四");
const third = contacts.find((c) => c.externalUserId === "ext-3");
assert.ok(third);
assert.equal(third.ownerUserId, "owner-b");
});
test("listExternalContacts skips members with non-zero errcode", async () => {
const { fetchImpl } = fakeExternalContacts();
const contacts = await listExternalContacts(
resolveWecomConfig(baseBindings),
fetchImpl,
);
const ownerBIds = contacts
.filter((c) => c.ownerUserId === "owner-b")
.map((c) => c.externalUserId);
assert.deepEqual(ownerBIds, ["ext-3"]);
});
test("listExternalContacts returns empty array when app credentials missing", async () => {
const { calls, fetchImpl } = fakeExternalContacts();
const contacts = await listExternalContacts(
resolveWecomConfig({
...baseBindings,
WECOM_CORP_ID: "",
WECOM_AGENT_ID: "",
WECOM_SECRET: "",
}),
fetchImpl,
);
assert.deepEqual(contacts, []);
assert.equal(calls.length, 0, "no API calls without credentials");
});
function fakeGroupPush(options = {}) {
const calls = [];
const pages =
options.pages ??
[
{
next_cursor: "page-2",
group_chat_list: [
{ chat_id: "wr-normal-1", status: 0 },
{ chat_id: "wr-dissolved", status: 1 },
],
},
{ next_cursor: "", group_chat_list: [{ chat_id: "wr-normal-2", status: 0 }] },
];
const details =
options.details ??
{
"wr-normal-1": { name: "KOC 一群", owner: "zhangsan", member_count: 42 },
"wr-normal-2": { name: "KOC 二群", owner: "lisi", member_count: 7 },
};
const fetchImpl = async (input, init = {}) => {
const url = new URL(String(input));
calls.push({ url, init });
if (url.pathname.endsWith("/cgi-bin/gettoken")) {
return Response.json({
errcode: 0,
errmsg: "ok",
access_token: "token-test",
expires_in: 7200,
});
}
if (url.pathname.endsWith("/cgi-bin/externalcontact/groupchat/list")) {
const body = JSON.parse(String(init.body ?? "{}"));
const index = body.cursor ? Number(body.cursor.replace("page-", "")) - 1 : 0;
const page = pages[index] ?? { next_cursor: "", group_chat_list: [] };
return Response.json({
errcode: 0,
errmsg: "ok",
next_cursor: page.next_cursor,
group_chat_list: page.group_chat_list,
});
}
if (url.pathname.endsWith("/cgi-bin/externalcontact/groupchat/get")) {
const body = JSON.parse(String(init.body ?? "{}"));
const detail = details[body.chat_id];
if (!detail) {
return Response.json({ errcode: 60111, errmsg: "not found" });
}
return Response.json({ errcode: 0, errmsg: "ok", group_chat: detail });
}
if (url.pathname.endsWith("/cgi-bin/externalcontact/add_msg_template")) {
return Response.json({
errcode: options.templateErrcode ?? 0,
errmsg: options.templateErrmsg ?? "ok",
msgid: options.msgid ?? "msgGTEST",
fail_list: options.failList ?? [],
});
}
return new Response("not found", { status: 404 });
};
return { calls, fetchImpl };
}
test("listCustomerGroupChats paginates groupchat/list and keeps only normal groups", async () => {
const { calls, fetchImpl } = fakeGroupPush();
const groups = await listCustomerGroupChats(
resolveWecomConfig(baseBindings),
fetchImpl,
);
assert.equal(groups.length, 2);
const first = groups.find((g) => g.chatId === "wr-normal-1");
assert.ok(first);
assert.equal(first.name, "KOC 一群");
assert.equal(first.ownerUserId, "zhangsan");
assert.equal(first.memberCount, 42);
const second = groups.find((g) => g.chatId === "wr-normal-2");
assert.ok(second);
assert.equal(second.ownerUserId, "lisi");
assert.equal(
groups.some((g) => g.chatId === "wr-dissolved"),
false,
"non-normal status groups are filtered",
);
const listCalls = calls.filter((c) =>
c.url.pathname.endsWith("/cgi-bin/externalcontact/groupchat/list"),
);
assert.equal(listCalls.length, 2, "cursor pagination followed");
const secondBody = JSON.parse(String(listCalls[1].init.body));
assert.equal(secondBody.cursor, "page-2");
});
test("listCustomerGroupChats skips groups whose detail lookup fails", async () => {
const { fetchImpl } = fakeGroupPush({
pages: [
{
next_cursor: "",
group_chat_list: [
{ chat_id: "wr-normal-1", status: 0 },
{ chat_id: "wr-gone", status: 0 },
],
},
],
details: {
"wr-normal-1": { name: "KOC 一群", owner: "zhangsan", member_count: 42 },
},
});
const groups = await listCustomerGroupChats(
resolveWecomConfig(baseBindings),
fetchImpl,
);
assert.deepEqual(
groups.map((g) => g.chatId),
["wr-normal-1"],
);
});
test("createGroupMsgTemplate posts group payload with text and link attachment", async () => {
const { calls, fetchImpl } = fakeGroupPush();
const result = await createGroupMsgTemplate(
resolveWecomConfig(baseBindings),
{
sender: "zhangsan",
chatIdList: ["wr-normal-1", "wr-normal-2"],
text: "【新任务】测试任务",
link: {
title: "测试任务",
desc: "品牌 A · 3 份",
url: "https://portal.example.com/?task=tok-1",
},
},
fetchImpl,
);
assert.equal(result.msgid, "msgGTEST");
assert.deepEqual(result.failList, []);
const sendCall = calls.find((c) =>
c.url.pathname.endsWith("/cgi-bin/externalcontact/add_msg_template"),
);
assert.ok(sendCall, "add_msg_template was called");
assert.equal(sendCall.url.searchParams.get("access_token"), "token-test");
const body = JSON.parse(String(sendCall.init.body));
assert.equal(body.chat_type, "group");
assert.equal(body.sender, "zhangsan");
assert.deepEqual(body.chat_id_list, ["wr-normal-1", "wr-normal-2"]);
assert.equal(body.text.content, "【新任务】测试任务");
assert.equal(body.attachments.length, 1);
assert.equal(body.attachments[0].msgtype, "link");
assert.equal(body.attachments[0].link.url, "https://portal.example.com/?task=tok-1");
});
test("createGroupMsgTemplate omits text and attachments keys when absent", async () => {
const { calls, fetchImpl } = fakeGroupPush();
await createGroupMsgTemplate(
resolveWecomConfig(baseBindings),
{
sender: "zhangsan",
chatIdList: ["wr-normal-1"],
link: { title: "只有图文", url: "https://portal.example.com/?task=tok-1" },
},
fetchImpl,
);
const sendCall = calls.find((c) =>
c.url.pathname.endsWith("/cgi-bin/externalcontact/add_msg_template"),
);
assert.ok(sendCall, "add_msg_template was called");
const body = JSON.parse(String(sendCall.init.body));
assert.equal("text" in body, false);
assert.equal(body.attachments[0].link.title, "只有图文");
assert.equal(body.attachments[0].link.desc, "");
});
test("createGroupMsgTemplate passes through fail_list from provider", async () => {
const { fetchImpl } = fakeGroupPush({
msgid: "msgFAIL",
failList: ["wr-normal-2"],
});
const result = await createGroupMsgTemplate(
resolveWecomConfig(baseBindings),
{
sender: "zhangsan",
chatIdList: ["wr-normal-1", "wr-normal-2"],
text: "内容",
},
fetchImpl,
);
assert.equal(result.msgid, "msgFAIL");
assert.deepEqual(result.failList, ["wr-normal-2"]);
});
test("createGroupMsgTemplate throws WecomClientError on provider error", async () => {
const { fetchImpl } = fakeGroupPush({
templateErrcode: 81053,
templateErrmsg: "user not in visible scope",
});
await assert.rejects(
() =>
createGroupMsgTemplate(
resolveWecomConfig(baseBindings),
{ sender: "zhangsan", chatIdList: ["wr-normal-1"], text: "内容" },
fetchImpl,
),
(error) => {
assert.ok(error instanceof WecomClientError);
assert.equal(error.status, 502);
assert.match(error.message, /user not in visible scope/);
return true;
},
);
});
test("createGroupMsgTemplate rejects empty text and link before calling API", async () => {
const { calls, fetchImpl } = fakeGroupPush();
await assert.rejects(
() =>
createGroupMsgTemplate(
resolveWecomConfig(baseBindings),
{ sender: "zhangsan", chatIdList: ["wr-normal-1"] },
fetchImpl,
),
(error) => {
assert.ok(error instanceof WecomClientError);
assert.equal(error.status, 400);
return true;
},
);
assert.equal(
calls.filter((c) =>
c.url.pathname.endsWith("/cgi-bin/externalcontact/add_msg_template"),
).length,
0,
"no API call for invalid payload",
);
});
test("createGroupMsgTemplate rejects missing sender or empty chat list", async () => {
const { fetchImpl } = fakeGroupPush();
const config = resolveWecomConfig(baseBindings);
await assert.rejects(
() =>
createGroupMsgTemplate(config, {
sender: "",
chatIdList: ["wr-normal-1"],
text: "内容",
}, fetchImpl),
/群主/,
);
await assert.rejects(
() =>
createGroupMsgTemplate(config, {
sender: "zhangsan",
chatIdList: [],
text: "内容",
}, fetchImpl),
/客户群列表为空/,
);
});

View File

@@ -0,0 +1,262 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
buildTaskGroupPushMessage,
groupChatsByOwner,
pushTaskToGroupChats,
trimToBytes,
} from "../lib/wecom-group-push-service.ts";
import {
clearWecomAccessTokenCacheForTests,
resolveWecomConfig,
WecomClientError,
} from "../lib/wecom-client.ts";
const baseBindings = {
WECOM_CORP_ID: "corp-test",
WECOM_AGENT_ID: "1000001",
WECOM_SECRET: "secret-test",
};
const baseTask = {
id: "task-1",
name: "测试任务",
brand: "品牌A",
quantity: 3,
due_at: "2026-09-01",
task_type: "content_publish",
platform: "小红书",
content_format: "image_text",
share_token: "tok-123",
};
test.beforeEach(() => {
clearWecomAccessTokenCacheForTests();
});
test("trimToBytes cuts on UTF-8 boundaries", () => {
assert.equal(trimToBytes("abc", 10), "abc");
assert.equal(trimToBytes("abcdef", 3), "abc");
// 「汉」占 3 字节
assert.equal(trimToBytes("a汉b", 4), "a汉");
assert.equal(trimToBytes("汉汉汉", 7), "汉汉");
});
test("buildTaskGroupPushMessage composes text and link with claim url", () => {
const message = buildTaskGroupPushMessage(baseTask, "https://portal.example.com/");
assert.match(message.text, /【新任务】测试任务/);
assert.match(message.text, /品牌品牌A数量3 份截止2026-09-01/);
assert.match(message.text, /平台:小红书|形式:图文/);
assert.match(message.text, /领取链接https:\/\/portal\.example\.com\/\?task=tok-123/);
assert.ok(message.link);
assert.equal(message.link.title, "测试任务");
assert.equal(message.link.url, "https://portal.example.com/?task=tok-123");
assert.match(message.link.desc, /品牌A · 3 份 · 截止 2026-09-01/);
});
test("buildTaskGroupPushMessage marks video and screenshot tasks", () => {
const video = buildTaskGroupPushMessage(
{ ...baseTask, content_format: "video" },
"https://portal.example.com",
);
assert.match(video.text, /形式:视频/);
const screenshot = buildTaskGroupPushMessage(
{ ...baseTask, task_type: "screenshot_collect" },
"https://portal.example.com",
);
assert.match(screenshot.text, /形式:截图回收/);
});
test("buildTaskGroupPushMessage drops link when share_token missing", () => {
const message = buildTaskGroupPushMessage(
{ ...baseTask, share_token: null },
"https://portal.example.com",
);
assert.equal(message.link, null);
assert.match(message.text, /领取链接生成失败/);
});
test("buildTaskGroupPushMessage trims long content to byte limits", () => {
const longName = "长".repeat(200);
const longBrand = "牌".repeat(400);
const message = buildTaskGroupPushMessage(
{ ...baseTask, name: longName, brand: longBrand },
"https://portal.example.com",
);
assert.ok(Buffer.byteLength(message.text, "utf8") <= 4000);
assert.ok(Buffer.byteLength(message.link.title, "utf8") <= 128);
assert.ok(Buffer.byteLength(message.link.desc, "utf8") <= 512);
});
test("groupChatsByOwner groups chat ids by owner and skips empty owners", () => {
const grouped = groupChatsByOwner([
{ chat_id: "wr-1", name: "一", owner_user_id: "zhangsan", member_count: 1, status: 0 },
{ chat_id: "wr-2", name: "二", owner_user_id: "lisi", member_count: 2, status: 0 },
{ chat_id: "wr-3", name: "三", owner_user_id: "zhangsan", member_count: 3, status: 0 },
{ chat_id: "wr-4", name: "四", owner_user_id: " ", member_count: 4, status: 0 },
]);
assert.deepEqual(grouped, [
{ ownerUserId: "zhangsan", chatIds: ["wr-1", "wr-3"] },
{ ownerUserId: "lisi", chatIds: ["wr-2"] },
]);
});
function fakeGroupPushDb(taskRow) {
const executed = [];
const groupRows = [
{ chat_id: "wr-1", name: "一群", owner_user_id: "zhangsan", member_count: 10, status: 0 },
{ chat_id: "wr-2", name: "二群", owner_user_id: "lisi", member_count: 5, status: 0 },
];
const makeStatement = (sql, params = []) => ({
sql,
params,
bind(...args) {
return makeStatement(sql, args);
},
async run() {
executed.push({ sql, params });
return {};
},
async first() {
executed.push({ sql, params });
if (/FROM tasks WHERE id = \?/.test(sql)) {
return params[0] === taskRow.id ? taskRow : null;
}
return null;
},
async all() {
executed.push({ sql, params });
if (/FROM wecom_group_chats/.test(sql)) {
return {
results: sql.includes(" IN (")
? groupRows.filter((row) => params.includes(row.chat_id))
: groupRows,
};
}
return { results: [] };
},
});
return {
executed,
prepare(sql) {
return makeStatement(sql);
},
async batch(statements) {
for (const statement of statements) await statement.run();
},
};
}
function fakeTemplateFetch(calls) {
return async (input, init = {}) => {
const url = new URL(String(input));
calls.push({ url, init });
if (url.pathname.endsWith("/cgi-bin/gettoken")) {
return Response.json({
errcode: 0,
errmsg: "ok",
access_token: "token-test",
expires_in: 7200,
});
}
if (url.pathname.endsWith("/cgi-bin/externalcontact/add_msg_template")) {
return Response.json({ errcode: 0, errmsg: "ok", msgid: "msg-1", fail_list: [] });
}
return new Response("not found", { status: 404 });
};
}
test("pushTaskToGroupChats batches per owner and records each send", async () => {
process.env.KOC_PORTAL_URL = "https://portal.example.com";
try {
const db = fakeGroupPushDb(baseTask);
const calls = [];
const summary = await pushTaskToGroupChats(
db,
{ taskId: "task-1", chatIds: ["wr-2", "wr-1"] },
resolveWecomConfig(baseBindings),
fakeTemplateFetch(calls),
);
assert.equal(summary.results.length, 2);
const senders = summary.results.map((item) => item.sender).sort();
assert.deepEqual(senders, ["lisi", "zhangsan"]);
const templateCalls = calls.filter((c) =>
c.url.pathname.endsWith("/cgi-bin/externalcontact/add_msg_template"),
);
assert.equal(templateCalls.length, 2, "one call per owner");
const bodies = templateCalls.map((c) => JSON.parse(String(c.init.body)));
const bySender = new Map(bodies.map((body) => [body.sender, body]));
assert.deepEqual(bySender.get("zhangsan").chat_id_list, ["wr-1"]);
assert.deepEqual(bySender.get("lisi").chat_id_list, ["wr-2"]);
assert.match(bySender.get("zhangsan").text.content, /【新任务】测试任务/);
assert.equal(
bySender.get("zhangsan").attachments[0].link.url,
"https://portal.example.com/?task=tok-123",
);
const inserts = db.executed.filter((item) =>
item.sql.includes("INSERT INTO wecom_group_pushes"),
);
assert.equal(inserts.length, 2);
} finally {
delete process.env.KOC_PORTAL_URL;
}
});
test("pushTaskToGroupChats rejects unknown chat ids", async () => {
const db = fakeGroupPushDb(baseTask);
const calls = [];
await assert.rejects(
() =>
pushTaskToGroupChats(
db,
{ taskId: "task-1", chatIds: ["wr-404"] },
resolveWecomConfig(baseBindings),
fakeTemplateFetch(calls),
),
(error) => {
assert.ok(error instanceof WecomClientError);
assert.equal(error.status, 400);
assert.match(error.message, /未同步/);
return true;
},
);
});
test("pushTaskToGroupChats rejects missing task", async () => {
const db = fakeGroupPushDb(baseTask);
await assert.rejects(
() =>
pushTaskToGroupChats(
db,
{ taskId: "task-404", chatIds: ["wr-1"] },
resolveWecomConfig(baseBindings),
fakeTemplateFetch([]),
),
(error) => {
assert.ok(error instanceof WecomClientError);
assert.equal(error.status, 404);
return true;
},
);
});
test("pushTaskToGroupChats prefers explicit text over template", async () => {
process.env.KOC_PORTAL_URL = "https://portal.example.com";
try {
const db = fakeGroupPushDb(baseTask);
const calls = [];
await pushTaskToGroupChats(
db,
{ taskId: "task-1", chatIds: ["wr-1"], text: "运营手动编辑的文案" },
resolveWecomConfig(baseBindings),
fakeTemplateFetch(calls),
);
const templateCall = calls.find((c) =>
c.url.pathname.endsWith("/cgi-bin/externalcontact/add_msg_template"),
);
const body = JSON.parse(String(templateCall.init.body));
assert.equal(body.text.content, "运营手动编辑的文案");
} finally {
delete process.env.KOC_PORTAL_URL;
}
});

View File

@@ -0,0 +1,15 @@
import assert from "node:assert/strict";
import test from "node:test";
import { computeDueCutoff } from "../lib/wecom-notifier-service.ts";
test("computeDueCutoff advances by dueDays and crosses month/year", () => {
assert.equal(computeDueCutoff("2026-08-18", 3), "2026-08-21");
assert.equal(computeDueCutoff("2026-08-30", 3), "2026-09-02");
assert.equal(computeDueCutoff("2026-12-30", 3), "2027-01-02");
assert.equal(computeDueCutoff("2026-08-18", 0), "2026-08-18");
assert.equal(computeDueCutoff("2026-08-18", 10), "2026-08-28");
});
test("computeDueCutoff returns input unchanged when malformed", () => {
assert.equal(computeDueCutoff("not-a-date", 3), "not-a-date");
});

View File

@@ -0,0 +1,98 @@
import assert from "node:assert/strict";
import test from "node:test";
import sharp from "sharp";
import { normalizeWorkbookImage } from "../lib/workbook-image.ts";
test("bakes EXIF orientation into exported workbook image pixels", async () => {
const source = await sharp({
create: {
width: 8,
height: 4,
channels: 3,
background: "#e95420",
},
})
.jpeg()
.withMetadata({ orientation: 6 })
.toBuffer();
const normalized = await normalizeWorkbookImage({
bytes: new Uint8Array(source),
contentType: "image/jpeg",
width: 8,
height: 4,
description: "手机照片",
});
const metadata = await sharp(normalized.bytes).metadata();
assert.equal(normalized.contentType, "image/png");
assert.equal(normalized.width, 4);
assert.equal(normalized.height, 8);
assert.equal(metadata.width, 4);
assert.equal(metadata.height, 8);
assert.equal(metadata.orientation, undefined);
});
test("keeps unsupported image bytes unchanged", async () => {
const bytes = Uint8Array.from([1, 2, 3]);
const normalized = await normalizeWorkbookImage({
bytes,
contentType: "application/octet-stream",
description: "未知文件",
});
assert.equal(normalized.bytes, bytes);
assert.equal(normalized.contentType, "application/octet-stream");
});
test("normalizes recognizable images even when storage metadata has no image MIME type", async () => {
const source = await sharp({
create: {
width: 8,
height: 4,
channels: 3,
background: "#22c55e",
},
})
.jpeg()
.withMetadata({ orientation: 6 })
.toBuffer();
const normalized = await normalizeWorkbookImage({
bytes: new Uint8Array(source),
contentType: "application/octet-stream",
description: "方向元数据缺失测试",
});
assert.equal(normalized.contentType, "image/png");
assert.equal(normalized.width, 4);
assert.equal(normalized.height, 8);
});
test("can downsize full-resolution source images for compact workbook exports", async () => {
const source = await sharp({
create: {
width: 4000,
height: 3000,
channels: 3,
background: "#d4a72c",
},
})
.png({ compressionLevel: 0 })
.toBuffer();
const normalized = await normalizeWorkbookImage(
{
bytes: new Uint8Array(source),
contentType: "image/png",
width: 4000,
height: 3000,
description: "批量回填原图",
},
{ maxDimension: 1600, outputFormat: "jpeg", jpegQuality: 82 },
);
const metadata = await sharp(normalized.bytes).metadata();
assert.equal(normalized.contentType, "image/jpeg");
assert.equal(normalized.width, 1600);
assert.equal(normalized.height, 1200);
assert.equal(metadata.format, "jpeg");
assert.ok(normalized.bytes.byteLength < source.byteLength / 20);
});