choizhang
commited on
Commit
·
b58e0b2
1
Parent(s):
81e036c
docs: Add Token Statistics Function Description in README
Browse files- README-zh.md +48 -0
- README.md +49 -0
README-zh.md
CHANGED
@@ -410,6 +410,54 @@ if __name__ == "__main__":
|
|
410 |
|
411 |
</details>
|
412 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
413 |
### 对话历史
|
414 |
|
415 |
LightRAG现在通过对话历史功能支持多轮对话。以下是使用方法:
|
|
|
410 |
|
411 |
</details>
|
412 |
|
413 |
+
### Token统计功能
|
414 |
+
<details>
|
415 |
+
<summary> <b>概述和使用</b> </summary>
|
416 |
+
|
417 |
+
LightRAG提供了TokenTracker工具来跟踪和管理大模型的token消耗。这个功能对于控制API成本和优化性能特别有用。
|
418 |
+
|
419 |
+
#### 使用方法
|
420 |
+
|
421 |
+
```python
|
422 |
+
from lightrag.utils import TokenTracker
|
423 |
+
|
424 |
+
# 创建TokenTracker实例
|
425 |
+
token_tracker = TokenTracker()
|
426 |
+
|
427 |
+
# 方法1:使用上下文管理器(推荐)
|
428 |
+
# 适用于需要自动跟踪token使用的场景
|
429 |
+
with token_tracker:
|
430 |
+
result1 = await llm_model_func("你的问题1")
|
431 |
+
result2 = await llm_model_func("你的问题2")
|
432 |
+
|
433 |
+
# 方法2:手动添加token使用记录
|
434 |
+
# 适用于需要更精细控制token统计的场景
|
435 |
+
token_tracker.reset()
|
436 |
+
|
437 |
+
rag.insert()
|
438 |
+
|
439 |
+
rag.query("你的问题1", param=QueryParam(mode="naive"))
|
440 |
+
rag.query("你的问题2", param=QueryParam(mode="mix"))
|
441 |
+
|
442 |
+
# 显示总token使用量(包含插入和查询操作)
|
443 |
+
print("Token usage:", token_tracker.get_usage())
|
444 |
+
```
|
445 |
+
|
446 |
+
#### 使用建议
|
447 |
+
- 在长会话或批量操作中使用上下文管理器,可以自动跟踪所有token消耗
|
448 |
+
- 对于需要分段统计的场景,使用手动模式并适时调用reset()
|
449 |
+
- 定期检查token使用情况,有助于及时发现异常消耗
|
450 |
+
- 在开发测试阶段积极使用此功能,以便优化生产环境的成本
|
451 |
+
|
452 |
+
#### 实际应用示例
|
453 |
+
您可以参考以下示例来实现token统计:
|
454 |
+
- `examples/lightrag_gemini_track_token_demo.py`:使用Google Gemini模型的token统计示例
|
455 |
+
- `examples/lightrag_siliconcloud_track_token_demo.py`:使用SiliconCloud模型的token统计示例
|
456 |
+
|
457 |
+
这些示例展示了如何在不同模型和场景下有效地使用TokenTracker功能。
|
458 |
+
|
459 |
+
</details>
|
460 |
+
|
461 |
### 对话历史
|
462 |
|
463 |
LightRAG现在通过对话历史功能支持多轮对话。以下是使用方法:
|
README.md
CHANGED
@@ -443,6 +443,55 @@ if __name__ == "__main__":
|
|
443 |
|
444 |
</details>
|
445 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
446 |
### Conversation History Support
|
447 |
|
448 |
|
|
|
443 |
|
444 |
</details>
|
445 |
|
446 |
+
### Token Usage Tracking
|
447 |
+
|
448 |
+
<details>
|
449 |
+
<summary> <b>Overview and Usage</b> </summary>
|
450 |
+
|
451 |
+
LightRAG provides a TokenTracker tool to monitor and manage token consumption by large language models. This feature is particularly useful for controlling API costs and optimizing performance.
|
452 |
+
|
453 |
+
#### Usage
|
454 |
+
|
455 |
+
```python
|
456 |
+
from lightrag.utils import TokenTracker
|
457 |
+
|
458 |
+
# Create TokenTracker instance
|
459 |
+
token_tracker = TokenTracker()
|
460 |
+
|
461 |
+
# Method 1: Using context manager (Recommended)
|
462 |
+
# Suitable for scenarios requiring automatic token usage tracking
|
463 |
+
with token_tracker:
|
464 |
+
result1 = await llm_model_func("your question 1")
|
465 |
+
result2 = await llm_model_func("your question 2")
|
466 |
+
|
467 |
+
# Method 2: Manually adding token usage records
|
468 |
+
# Suitable for scenarios requiring more granular control over token statistics
|
469 |
+
token_tracker.reset()
|
470 |
+
|
471 |
+
rag.insert()
|
472 |
+
|
473 |
+
rag.query("your question 1", param=QueryParam(mode="naive"))
|
474 |
+
rag.query("your question 2", param=QueryParam(mode="mix"))
|
475 |
+
|
476 |
+
# Display total token usage (including insert and query operations)
|
477 |
+
print("Token usage:", token_tracker.get_usage())
|
478 |
+
```
|
479 |
+
|
480 |
+
#### Usage Tips
|
481 |
+
- Use context managers for long sessions or batch operations to automatically track all token consumption
|
482 |
+
- For scenarios requiring segmented statistics, use manual mode and call reset() when appropriate
|
483 |
+
- Regular checking of token usage helps detect abnormal consumption early
|
484 |
+
- Actively use this feature during development and testing to optimize production costs
|
485 |
+
|
486 |
+
#### Practical Examples
|
487 |
+
You can refer to these examples for implementing token tracking:
|
488 |
+
- `examples/lightrag_gemini_track_token_demo.py`: Token tracking example using Google Gemini model
|
489 |
+
- `examples/lightrag_siliconcloud_track_token_demo.py`: Token tracking example using SiliconCloud model
|
490 |
+
|
491 |
+
These examples demonstrate how to effectively use the TokenTracker feature with different models and scenarios.
|
492 |
+
|
493 |
+
</details>
|
494 |
+
|
495 |
### Conversation History Support
|
496 |
|
497 |
|