Kimi vs Claude for long documents and research
Our hands-on test of these two tools is under way. The take below is based on the official plans and documentation; scores from the test will be added here when it is finished.
Early take
Both are built for long context. Test them on the same document set; Chinese-heavy material and budget favour Kimi, careful English writing and codebase work favour Claude.
Side by side
| Kimi | Claude | |
|---|---|---|
| Made by | Moonshot AI | Anthropic |
| Free tier | Yes | Yes |
| Paid from | $19/mo | $20/mo |
| Open weights | Yes | No |
| API | Yes | Yes |
| Open weights | Yes (K-series) | No |
| Coding agent | Measured in our hands-on test | Claude Code |
| Same-document test | Measured in our hands-on test | Measured in our hands-on test |
How we test this
We are running these tasks now. Scores for each tool will appear here as soon as testing is finished.
Setup
- Kimi
- Kimi Plus ($19/month), default model; Kimi Code for the coding-agent task
- Claude
- Claude Pro ($20/month), default model; Claude Code for the coding-agent task
Documents are uploaded as the same PDF files to both tools in a fresh chat. Web search stays off except for the web-research task. Question-answer tasks are scored against an answer key checked against the source text before testing; each set includes one question the document does not answer, to catch invented answers.
Both plans cost about the same, so the test is about which one handles long, mixed-language material more reliably within a normal working day.
Tasks and prompts
01144-page English regulation
Using only the attached EU AI Act (Regulation (EU) 2024/1689), answer the following and give the article for each answer:
1. What is the maximum fine for prohibited AI practices?
2. From what date do the prohibitions apply?
3. From what date do the obligations for general-purpose AI models apply?
4. Which annex lists the high-risk use cases?
5. Which article defines an "AI system"?
6. From what date does the Regulation generally apply?
7. What does the Regulation say about the price of GPU hardware?只根据附件中的《欧盟人工智能法案》（Regulation (EU) 2024/1689）回答以下问题，并为每个答案注明条款：
1. 违反禁止性 AI 实践的最高罚款是多少？
2. 禁止性条款从哪天开始适用？
3. 通用目的 AI 模型的义务从哪天开始适用？
4. 哪个附件列出了高风险用途？
5. 哪一条定义了“AI 系统”？
6. 法案整体从哪天开始适用？
7. 法案对 GPU 硬件价格有什么规定？What we check
- Six answers match the key with the right article
- Question 7 is answered as not covered by the document
02Long Chinese policy report
Using only the attached 2025 Chinese Government Work Report, answer: 1. The GDP growth target. 2. The target for new urban jobs. 3. The CPI target. 4. The deficit-to-GDP ratio. 5. The amount of ultra-long special treasury bonds. 6. The amount of local government special-purpose bonds. 7. What the report says about the exchange rate between the yuan and the euro. Quote the sentence each answer comes from.只根据附件中的 2025 年《政府工作报告》回答：1. 国内生产总值增长预期目标；2. 城镇新增就业目标；3. 居民消费价格涨幅目标；4. 赤字率；5. 超长期特别国债规模；6. 地方政府专项债券规模；7. 报告对人民币兑欧元汇率有何具体表述。每个答案请引用原文句子。What we check
- Six answers match the key
- Each answer quotes the source sentence
- Question 7 is answered as not covered
03Table from four pricing pages
The four attached PDFs are the pricing pages of Runway, Kling, Hailuo and Kimi, saved today. Build one table with: tool, plan name, monthly price when billed monthly (USD), and what the plan includes in credits or usage. Mark any value you could not find as "not stated" rather than guessing.附件四份 PDF 是今天保存的 Runway、可灵、海螺、Kimi 价格页。请整理成一张表：工具、套餐名、按月付费的月价（美元）、套餐包含的积分或用量。找不到的数值请写“未注明”，不要猜。What we check
- Every plan on the four pages is present
- Prices match the pages
- Missing values marked "not stated" instead of invented
04Bilingual web research
Search the web. What is the cheapest paid monthly plan for Kling and for Hailuo AI right now, at list price, and how many credits does each include? Cite the official page for each figure.请联网搜索：可灵和海螺 AI 目前最便宜的按月付费套餐原价分别是多少？各包含多少积分？每个数字都请附官方页面链接。What we check
- Prices and credits match the official pages on test day
- List price separated from first-month offers
- Each link is the official page and supports the figure
05Cut 20% without losing facts
Edit the attached draft down to about 480 words (it is about 600 now). Keep every fact, figure and name. Keep the author's tone. Return only the edited text.把附件中的英文稿件删减到约 480 词（现在约 600 词）。保留所有事实、数字和名称，保持作者语气，只返回修改后的正文。What we check
- All 12 tracked facts still present and unchanged
- Length 456–504 words
- Tone and point of view unchanged
06Coding agent in a small repo
In this repository, implement parse_duration in duration.py so that all tests in test_duration.py pass. Do not edit the tests. Run the tests before you finish.在这个代码库里实现 duration.py 中的 parse_duration，让 test_duration.py 中的测试全部通过。不要修改测试文件。结束前请运行测试。What we check
- Share of the 14 tests that pass
- Test file left untouched
- Number of turns we had to intervene
Scoring (1–5 per criterion)
| Criterion | Weight | What the scores mean |
|---|---|---|
| Accuracy | 40% | 1: wrong or invented facts · 3: mostly right with a material slip · 5: fully correct |
| Grounded in the source | 25% | 1: no locations or sources · 3: some answers located · 5: every answer points to the article, page or link |
| Follows instructions | 20% | 1: ignores constraints · 3: misses one constraint · 5: meets every constraint |
| Language quality | 15% | 1: awkward · 3: correct but stiff · 5: reads like a skilled native writer |
Also measured
- Objective tasks: share of answer-key items correct, mapped to 1–5 (1 + 4 × share)
- Admits when the document has no answer (yes / no)
- Facts kept in the editing task, out of 12
- Hit a usage limit during the test (yes / no)
Kimi: Good fit for
- Reading long PDFs and many files at once
- Research with web search in Chinese and English
- Developers evaluating open-weight agentic models
Claude: Good fit for
- Long documents and multi-round editing
- Codebase work with Claude Code
- Teams using Projects as shared knowledge bases