Skip to content

flamehaven01/Flamehaven-Filesearch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

FLAMEHAVEN FileSearch

FLAMEHAVEN FileSearch

Self-hosted RAG search engine. Production-ready in 3 minutes.

License: MIT Version Python Docker

Quick Start β€’ Features β€’ Documentation β€’ API Reference β€’ Contributing


🎯 Why FLAMEHAVEN?

Stop sending your sensitive documents to third-party services. Get enterprise-grade semantic search running locally in minutes, not days.

# One command. Three minutes. Done.
docker run -d -p 8000:8000 -e GEMINI_API_KEY="your_key" flamehaven-filesearch:1.4.1

πŸš€ Fast

Production deployment in 3 minutes
Vector generation in <1ms
Zero ML dependencies

πŸ”’ Private

100% self-hosted
Your data never leaves your infrastructure
Enterprise-grade security

πŸ’° Cost-Effective

Free tier: 1,500 queries/month
No infrastructure costs
Open source & MIT licensed


Features ✨

Core Capabilities

  • πŸ” Smart Search Modes - Keyword, semantic, and hybrid search with automatic typo correction
  • πŸ“„ Multi-Format Support - PDF, DOCX, TXT, MD, and common image formats
  • ⚑ Ultra-Fast Vectors - DSP v2.0 algorithm generates embeddings in <1ms without ML frameworks
  • 🎯 Source Attribution - Every answer includes links back to source documents

What's New in v1.4.1

  • Usage Tracking & Quotas - Per-API-key request/token tracking with daily/monthly limits
  • Admin Usage APIs - Detailed usage stats, quota management, and alert monitoring
  • pgvector Maintenance - HNSW reindexing, VACUUM ANALYZE, and index statistics
  • pgvector Tuning Guide - Comprehensive production tuning and optimization documentation
  • Circuit Breaker - Automatic failure recovery for database connections
  • Performance Monitoring - Complete observability with health checks and metrics

Production Features (v1.4.0+)

  • Multimodal Search - Text + image search endpoint (optional)
  • HNSW Vector Index - High-performance similarity search with pgvector
  • OAuth2/OIDC Support - JWT validation alongside API keys
  • PostgreSQL Backend - Enterprise-grade persistence and vector store
  • Vision Processing - Image metadata extraction with size limits and timeouts

Enterprise Features (v1.2.2+)

  • πŸ” API Key Authentication - Fine-grained permission system
  • ⚑ Rate Limiting - Configurable per-user quotas
  • πŸ“Š Audit Logging - Complete request history
  • πŸ“¦ Batch Processing - Process 1-100 queries per request
  • πŸ“ˆ Admin Dashboard - Real-time metrics and management

Quick Start πŸš€

Option 1: Docker (Recommended)

The fastest path to production:

docker run -d \
  -p 8000:8000 \
  -e GEMINI_API_KEY="your_gemini_api_key" \
  -e FLAMEHAVEN_ADMIN_KEY="secure_admin_password" \
  -v $(pwd)/data:/app/data \
  flamehaven-filesearch:1.4.1

βœ… Server running at http://localhost:8000

Option 2: Python SDK

Perfect for integrating into existing applications:

from flamehaven_filesearch import FlamehavenFileSearch, FileSearchConfig

# Initialize
config = FileSearchConfig(google_api_key="your_gemini_key")
fs = FlamehavenFileSearch(config)

# Upload and search
fs.upload_file("company_handbook.pdf", store="docs")
result = fs.search("What is our remote work policy?", store="docs")

print(result['answer'])
# Output: "Employees can work remotely up to 3 days per week..."

Option 3: REST API

For language-agnostic integration:

# 1. Generate API key
curl -X POST http://localhost:8000/api/admin/keys \
  -H "X-Admin-Key: your_admin_key" \
  -d '{"name":"production","permissions":["upload","search"]}'

# 2. Upload document
curl -X POST http://localhost:8000/api/upload/single \
  -H "Authorization: Bearer sk_live_abc123..." \
  -F "[email protected]" \
  -F "store=my_docs"

# 3. Search
curl -X POST http://localhost:8000/api/search \
  -H "Authorization: Bearer sk_live_abc123..." \
  -H "Content-Type: application/json" \
  -d 
  '{ 
    "query": "What are the main findings?",
    "store": "my_docs",
    "search_mode": "hybrid"
  }'

πŸ“¦ Installation

# Core package
pip install flamehaven-filesearch

# With API server
pip install flamehaven-filesearch[api]

# With HNSW vector index
pip install flamehaven-filesearch[vector]

# With PostgreSQL backend (metadata + vector store)
pip install flamehaven-filesearch[postgres]

# With vision delegate support
pip install flamehaven-filesearch[vision]

# Development setup
pip install flamehaven-filesearch[all]

# Build from source
git clone https://github.com/flamehaven01/Flamehaven-Filesearch.git
cd Flamehaven-Filesearch
docker build -t flamehaven-filesearch:1.4.1 .

Configuration βš™οΈ

Required Environment Variables

export GEMINI_API_KEY="your_google_gemini_api_key"
export FLAMEHAVEN_ADMIN_KEY="your_secure_admin_password"

Optional Configuration

export HOST="0.0.0.0"              # Bind address
export PORT="8000"                  # Server port
export REDIS_HOST="localhost"       # Distributed caching
export REDIS_PORT="6379"            # Redis port

Advanced Configuration

Create a config.yaml for fine-tuned control:

vector_store:
  quantization: int8
  compression: gravitas_pack
  
search:
  default_mode: hybrid
  typo_correction: true
  max_results: 10
  
security:
  rate_limit: 100  # requests per minute
  max_file_size: 52428800  # 50MB

πŸ“Š Performance

Metric Value Notes
Vector Generation <1ms DSP v2.0, zero ML dependencies
Memory Footprint 75% reduced Int8 quantization vs float32
Metadata Size 90% smaller Gravitas-Pack compression
Test Suite 0.33s 19/19 tests passing
Cold Start 3 seconds Docker container ready

Real-World Benchmarks

Environment: Docker on Apple M1 Mac, 16GB RAM
Document Set: 500 PDFs, ~2GB total

Health Check:           8ms
Search (cache hit):     9ms
Search (cache miss):    1,250ms  (includes Gemini API call)
Batch Search (10):      2,500ms  (parallel processing)
Upload (50MB file):     3,200ms  (with indexing)

Architecture πŸ—οΈ

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Your Documents β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚
         β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                  REST API Layer                      β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚  β”‚   Upload     β”‚  β”‚    Search    β”‚  β”‚   Admin   β”‚ β”‚
β”‚  β”‚   Endpoint   β”‚  β”‚   Endpoint   β”‚  β”‚ Dashboard β”‚ β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”˜
          β”‚                  β”‚                 β”‚
          β–Ό                  β–Ό                 β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  File Parser     β”‚  β”‚ Semantic Search  β”‚  β”‚  Metrics β”‚
β”‚  (PDF/DOCX/TXT)  β”‚  β”‚  DSP v2.0       β”‚  β”‚  Logger  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚                     β”‚
         β–Ό                     β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Store Manager   β”‚  β”‚  Gemini API      β”‚
β”‚  (SQLite + Vec)  β”‚  β”‚  (Reasoning)     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚
         β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Redis Cache     β”‚
β”‚  (Optional)      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Security πŸ”’

FLAMEHAVEN takes security seriously:

  • βœ… API Key Hashing - SHA256 with salt
  • βœ… Rate Limiting - Per-key quotas (default: 100/min)
  • βœ… Permission System - Granular access control
  • βœ… Audit Logging - Complete request history
  • βœ… OWASP Headers - Security headers enabled by default
  • βœ… Input Validation - Strict file type and size checks

Security Best Practices

# Use strong admin keys
export FLAMEHAVEN_ADMIN_KEY=$(openssl rand -base64 32)

# Enable HTTPS in production
# (use nginx/traefik as reverse proxy)

# Rotate API keys regularly
curl -X DELETE http://localhost:8000/api/admin/keys/old_key_id \
  -H "X-Admin-Key: $FLAMEHAVEN_ADMIN_KEY"

Roadmap πŸ—ΊοΈ

Full roadmap lives in ROADMAP.md. Summary below:

v1.4.x (Q1 2026)

  • Multimodal search (image + text)
  • HNSW vector indexing for faster search
  • OAuth2/OIDC integration
  • PostgreSQL backend option (metadata + vector store)
  • Usage-budget controls and reporting
  • pgvector tuning and reliability hardening

v2.0.0 (Q2 2026)

  • Multi-language support (15+ languages)
  • XLSX, PPTX, RTF format support
  • WebSocket streaming for real-time results
  • Kubernetes Helm charts

Community Requests

See ROADMAP.md for backlog curation and request intake.


Troubleshooting πŸ›

❌ 401 Unauthorized Error

Problem: API returns 401 when making requests.

Solutions:

  1. Verify FLAMEHAVEN_ADMIN_KEY environment variable is set
  2. Check Authorization: Bearer sk_live_... header format
  3. Ensure API key hasn't expired (check admin dashboard)
# Debug: Check if admin key is set
echo $FLAMEHAVEN_ADMIN_KEY

# Regenerate API key
curl -X POST http://localhost:8000/api/admin/keys \
  -H "X-Admin-Key: $FLAMEHAVEN_ADMIN_KEY" \
  -d '{"name":"debug","permissions":["search"]}'
🐌 Slow Search Performance

Problem: Searches taking >5 seconds.

Solutions:

  1. Check cache hit rate: FLAMEHAVEN_METRICS_ENABLED=1 curl http://localhost:8000/metrics
  2. Enable Redis for distributed caching
  3. Verify Gemini API latency (should be <1.5s)
# Enable Redis caching
docker run -d --name redis redis:7-alpine
export REDIS_HOST=localhost
πŸ’Ύ High Memory Usage

Problem: Container using >2GB RAM.

Solutions:

  1. Enable Redis with LRU eviction policy
  2. Reduce max file size in config
  3. Monitor with Prometheus endpoint
# Configure Redis memory limit
docker run -d \
  -p 6379:6379 \
  redis:7-alpine \
  --maxmemory 512mb \
  --maxmemory-policy allkeys-lru

More solutions in our Wiki Troubleshooting Guide.


Documentation πŸ“š

Documentation Hub

Use the links below to jump to the most relevant guide.

Topic Description
Troubleshooting Step-by-step debugging playbook
Configuration Reference Full list of environment variables and config fields
Production Deployment Docker, systemd, reverse proxy, scaling tips
API Reference REST endpoints, payloads, rate limits
Architecture How the FastAPI, cache, metrics, and validation layers fit together
Benchmarks Performance measurements and methodology

These Markdown files live inside the repository so they stay versioned alongside the code. Feel free to contribute improvements via pull requests.

Additional Resources


Contributing 🀝

We love contributions! FLAMEHAVEN is better because of developers like you.

Good First Issues

  • 🟒 [Easy] Add dark mode to admin dashboard (1-2 hours)
  • 🟑 [Medium] Implement XLSX file support (2-3 hours)
  • πŸ”΄ [Advanced] Add HNSW vector indexing (4-6 hours)

See CONTRIBUTING.md for development setup and guidelines.

Contributors


Community & Support πŸ’¬


License πŸ“„

Distributed under the MIT License. See LICENSE for more information.


πŸ™ Acknowledgments

Built with amazing open source tools:

  • FastAPI - Modern Python web framework
  • Google Gemini - Semantic understanding and reasoning
  • SQLite - Lightweight, embedded database
  • Redis - In-memory caching (optional)

⭐ Star us on GitHub β€’ πŸ“– Read the Docs β€’ πŸš€ Deploy Now

Built with πŸ”₯ by the Flamehaven Core Team

Last updated: December 28, 2025 β€’ Version 1.4.1