> ## Documentation Index
> Fetch the complete documentation index at: https://docs.booleinference.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Boole AI: Local-First High-Throughput LLM Inference

> Boole AI delivers high-throughput LLM inference via a local binary or cloud API — OpenAI-compatible, zero telemetry, and optimized nightly.

Boole AI is a high-throughput LLM inference platform built for teams that need speed, privacy, and simplicity. You can run models in two ways: download a single self-contained binary and serve models directly on your own NVIDIA GPU, or call the Boole cloud API — a fully OpenAI-compatible endpoint that gets you from signup to first token in minutes. Both modes share the same interface, so switching between them requires nothing more than a one-line config change.

Every night the Boole compiler re-ingests every supported model, fuses kernels, quantizes weights, and pages the KV cache — running 2,057 optimization iterations and keeping the 217 best-performing variants. The result is a continuously improving runtime that you consume through the same familiar OpenAI-style API you already know.

## Two Ways to Run Models

<CardGroup cols={2}>
  <Card title="Local Deployment" icon="microchip" href="/quickstart-local">
    Download a single executable with weights embedded. No Python, no dependencies, cold start under 400 ms. Runs fully offline after download.
  </Card>

  <Card title="Cloud API" icon="cloud" href="/quickstart-api">
    Call `https://api.boole.dev/v1` with your existing OpenAI client. Sign up in seconds — no card required, \$20 in free credits included.
  </Card>
</CardGroup>

## Key Capabilities

<CardGroup cols={2}>
  <Card title="312 Tokens / Second" icon="bolt">
    Serve Llama 3.3 70B at 312 tokens/sec on a single NVIDIA A10 GPU — outpacing most multi-GPU setups.
  </Card>

  <Card title="Nightly Compiler Optimization" icon="gear">
    Kernel fusion, weight quantization, and KV-cache paging rebuilt every night. Your throughput improves automatically, no action needed.
  </Card>

  <Card title="Zero Data Retention" icon="shield-halved">
    Prompts are never stored. Zero telemetry, fully offline-capable, and SOC 2 Type II compliant — your data stays yours.
  </Card>

  <Card title="OpenAI-Compatible Interface" icon="plug">
    Drop Boole into any app that already uses the OpenAI SDK. Change one line — `base_url` — and you're done.
  </Card>
</CardGroup>

## Next Steps

Whether you want to run models on your own hardware or start calling the cloud API right now, the pages below will get you there. Explore the model catalogue to pick the right model for your use case, review pricing before you scale, or jump straight into the API reference.

<CardGroup cols={2}>
  <Card title="Models" icon="layer-group" href="/concepts/models">
    Browse all 42 available models, including Llama, Qwen, Mixtral, DeepSeek, and Whisper.
  </Card>

  <Card title="Pricing" icon="circle-dollar-to-slot" href="/concepts/pricing">
    Per-token and per-minute rates for every model. Prices decrease as the compiler improves.
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/overview">
    Full reference for every endpoint, parameter, and response field.
  </Card>

  <Card title="FAQ" icon="circle-question" href="/support/faq">
    Answers to common questions about setup, billing, rate limits, and compliance.
  </Card>
</CardGroup>