> ## Documentation Index
> Fetch the complete documentation index at: https://docs.booleinference.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Boole AI FAQ: Pricing, Privacy, and Technical Questions

> Answers to common questions about Boole AI's inference engine, pricing, data privacy, model availability, and OpenAI compatibility.

Answers to the most common questions about Boole AI — covering local deployment, the cloud API, pricing, data privacy, and model availability. If you don't find what you're looking for here, reach out to [support@boole.ai](mailto:support@boole.ai).

## General

<AccordionGroup>
  <Accordion title="What is Boole AI?">
    Boole AI is a high-throughput LLM inference platform. You can run models locally using the Boole binary on your own GPU, or call the cloud API hosted at `https://api.boole.dev/v1`. Both surfaces expose the same OpenAI-compatible interface, so switching between local and cloud requires only a `base_url` change.

    Boole's nightly compiler optimizes every supported model for maximum throughput — delivering, for example, **312 tokens/sec** on Llama 3.3 70B on a single NVIDIA A10 GPU.
  </Accordion>

  <Accordion title="How does Boole achieve 312 tokens/sec?">
    Boole runs a nightly compiler pipeline called Crucible that performs three core optimizations automatically:

    * **Kernel fusion** — combines adjacent GPU operations to eliminate memory round-trips between kernels.
    * **Weight quantization** — reduces weight precision where quality impact is negligible, shrinking memory bandwidth requirements.
    * **KV cache paging** — manages the attention cache in fixed-size pages to prevent memory fragmentation and enable larger effective batch sizes.

    To date the compiler has run **2,057 iterations**, retaining **217** of the best-performing optimization passes. The result compounds: throughput improves continuously without any changes on your end.
  </Accordion>

  <Accordion title="Is Boole OpenAI-compatible?">
    Yes. Boole exposes the same API surface as OpenAI — the same endpoints, request shapes, and response formats. To migrate an existing OpenAI integration, change two values:

    ```python theme={null}
    from openai import OpenAI

    client = OpenAI(
        base_url="https://api.boole.dev/v1",  # was: https://api.openai.com/v1
        api_key="YOUR_BOOLE_API_KEY",          # was: your OpenAI key
    )
    ```

    No other code changes are required. Streaming, tool calls, and function calling are all supported.
  </Accordion>

  <Accordion title="How many models are available?">
    Boole currently supports **42 models**, all optimized by the nightly compiler. The catalogue includes Llama 3.3 70B, Qwen 2.5 72B, Mixtral 8x22B, DeepSeek V3, Whisper Large v3, and many others.

    See the full list and per-model specifications in [Models](/concepts/models).
  </Accordion>
</AccordionGroup>

***

## Pricing & Billing

<AccordionGroup>
  <Accordion title="How much does it cost?">
    Local deployment is **free** — there are no per-token fees when you run Boole on your own GPU.

    Cloud API pricing starts at **\$0.08 per million input tokens** (DeepSeek V3). Example rates:

    | Model            | Input       | Output      |
    | ---------------- | ----------- | ----------- |
    | DeepSeek V3      | \$0.08/MTok | \$0.12/MTok |
    | Llama 3.3 70B    | \$0.09/MTok | \$0.14/MTok |
    | Qwen 2.5 72B     | \$0.11/MTok | \$0.16/MTok |
    | Mixtral 8x22B    | \$0.13/MTok | \$0.19/MTok |
    | Whisper Large v3 | \$0.02/min  | —           |

    See the full pricing table at [Pricing](/concepts/pricing).
  </Accordion>

  <Accordion title="Do prices ever go up?">
    No — Boole's prices only decrease over time. As the nightly compiler finds more efficient optimization passes, the cost to serve each token falls, and those savings are passed to you.

    In the event of any price increase (which has not happened), Boole will provide a minimum of **30 days' advance notice**.
  </Accordion>

  <Accordion title="What are the free credits?">
    Every new account receives **\$20 in credits** on signup — equivalent to approximately **1 million tokens** on Llama 3.3 70B. No credit card is required to claim them.

    Credits apply automatically to cloud API usage and expire after 12 months if unused.
  </Accordion>

  <Accordion title="Does local deployment cost anything?">
    No. When you run the Boole binary on your own hardware, there are no per-token fees, no subscription fees, and no license fees. You pay only for the GPU hardware and electricity — Boole charges nothing.
  </Accordion>
</AccordionGroup>

***

## Privacy & Security

<AccordionGroup>
  <Accordion title="Are my prompts stored?">
    No. Boole operates a **zero-retention policy**: your prompts and completions are processed in memory to generate a response and are not written to disk or logged after the response is returned.

    This applies to both the cloud API and the local binary.
  </Accordion>

  <Accordion title="Is Boole SOC 2 compliant?">
    Yes. Boole AI is **SOC 2 Type II** certified, covering security, availability, and confidentiality. All cloud API traffic is encrypted with **TLS 1.3**.

    Contact [security@boole.ai](mailto:security@boole.ai) to request a copy of the audit report or a security review questionnaire.
  </Accordion>

  <Accordion title="Can I run Boole completely offline?">
    Yes. The local binary is a single self-contained executable with weights embedded. Once downloaded, it requires no internet connection at inference time — no telemetry calls, no license checks, no external requests of any kind.

    This makes Boole suitable for air-gapped environments and deployments with strict network egress policies.
  </Accordion>

  <Accordion title="Does the local binary phone home?">
    No. The local binary has **zero telemetry** by default. It does not send usage data, crash reports, or any other information to Boole's servers. All inference happens entirely on your machine.
  </Accordion>
</AccordionGroup>

***

## Technical

<AccordionGroup>
  <Accordion title="What GPU do I need for local deployment?">
    Any NVIDIA GPU with **CUDA 12.0 or later** is supported. Recommended hardware by model size:

    | Model size | Minimum VRAM | Recommended GPU   |
    | ---------- | ------------ | ----------------- |
    | 7B         | 8 GB         | RTX 3080 / A10    |
    | 13B        | 16 GB        | RTX 4090 / A10    |
    | 70B        | 40 GB        | A100 / A10 (INT4) |

    The NVIDIA A10 (24 GB) can run 70B models with INT4 quantization enabled via `--quantize int4`. For full-precision 70B inference, use an A100 (40 GB or 80 GB).
  </Accordion>

  <Accordion title="What's the rate limit?">
    The default rate limit is **60 requests per second** per API key on the cloud API. There is no rate limit on the local binary.

    If your workload requires a higher cloud rate limit, contact [support@boole.ai](mailto:support@boole.ai) with your account email and expected request volume.
  </Accordion>

  <Accordion title="Can I use streaming?">
    Yes. All text generation models support **Server-Sent Events (SSE) streaming**. Enable it by setting `stream=True` in your request:

    ```python theme={null}
    stream = client.chat.completions.create(
        model="llama-3.3-70b-instruct",
        messages=[{"role": "user", "content": "Hello!"}],
        stream=True,
    )

    for chunk in stream:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    ```

    Streaming is supported on both the cloud API and the local binary.
  </Accordion>

  <Accordion title="Does Boole support tool calls / function calling?">
    Yes. Tool calls and function calling are **fully supported** on all chat models. The request and response format matches the OpenAI specification exactly, so any existing function-calling code works without modification.
  </Accordion>
</AccordionGroup>