feat: add token-budget truncation for file mentions

[roomote]

This PR attempts to address Issue #10824. Feedback and guidance are welcome.

Summary

Implements token-budget aware truncation for file mentions to prevent context exhaustion when large files are referenced in the initial prompt.

Problem

When users mention large files (e.g., 2.7MB JSON files) using @/path/to/file syntax in their prompts, the entire file content was being included in the context, which could immediately exhaust the context window.

Solution

Added token-budget based truncation for file mentions, similar to how the read_file tool works:

1. New parameter maxFileTokenBudget - Added to parseMentions(), processUserContentMentions(), and getFileOrFolderContent() functions

2. Token-budget calculation - Each file mention is limited to 10% of the model's context window. For example, with a 200k context window, each mentioned file is limited to ~20k tokens.

3. Uses readFileWithTokenBudget() - Leverages the existing incremental token counting implementation to read files up to the budget and then stop.

4. Clear truncation message - When a file is truncated, users see:

   [File truncated: read X lines (Y tokens) within token budget of Z. Use the read_file tool to examine specific sections.]
   

Testing

• Added new tests for token-budget truncation in src/core/mentions/__tests__/index.spec.ts
• Updated existing tests in processUserContentMentions.spec.ts to account for the new parameter
• All 42+ tests pass

Related Issue

Closes #10824

---

Important

Introduces token-budget truncation for file mentions to prevent context exhaustion, limiting each file to 10% of the model's context window.

• Behavior:
  • Implements token-budget truncation for file mentions in parseMentions() and getFileOrFolderContent().
  • Limits each file mention to 10% of the model's context window.
  • Uses readFileWithTokenBudget() for reading files within the budget.
  • Displays a truncation message when a file is truncated.
• Functions:
  • Adds maxFileTokenBudget parameter to parseMentions(), processUserContentMentions(), and getFileOrFolderContent().
• Testing:
  • Adds tests for token-budget truncation in index.spec.ts.
  • Updates tests in processUserContentMentions.spec.ts for new parameter.
• Misc:
  • Calculates token budget in Task.ts based on 10% of context window.

^{This description was created by for 46f8582. You can customize this summary. It will automatically update as commits are pushed.}

[roomote]

Rooviewer   See task on Roo Cloud

Review complete. No issues found.

This PR correctly implements token-budget aware truncation for file mentions. The implementation uses 10% of the context window per file mention and leverages the existing readFileWithTokenBudget() infrastructure. All 25 tests pass.

_{Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues.}