Skip to main content
Boole AI delivers high-throughput LLM inference two ways: download a single binary and run models on your own GPU at zero per-token cost, or call our cloud API at prices well below market alternatives. Either way, you get the same OpenAI-compatible interface — no code changes needed.

Local Quickstart

Deploy a model binary on your GPU and serve 312 tokens/sec in under 4 minutes.

API Quickstart

Get an API key and make your first inference call in minutes — 1M free tokens included.

Models

Browse 42 supported models including Llama 3.3 70B, Qwen 2.5 72B, and DeepSeek V3.

API Reference

Explore the full OpenAI-compatible REST API with request and response schemas.

​
Get Started in Minutes

1

Create an account

Sign up at booleinference.com/signup — no credit card required. You receive $20 in free credits (1M tokens) automatically.
2

Choose your deployment mode

Run locally for zero per-token cost and full data privacy, or use the cloud API for instant access with no hardware required.
3

Point your existing code at Boole

Set base_url to https://api.boole.dev/v1 (cloud) or http://localhost:8000/v1 (local). No other code changes needed — Boole is fully OpenAI-compatible.
4

Start inferring

Call chat/completions, stream responses, use tool calls and structured outputs — everything works out of the box.

​
Why Boole AI

Fastest Local Throughput

312 tokens/sec on Llama 3.3 70B on a single A10 GPU — faster than vLLM, TGI, and TensorRT-LLM.

Nightly Compiler Optimization

An automated agent rewrites the model runtime every night, fusing kernels and quantizing weights — only keeping changes that pass a frozen quality gate.

Zero Data Retention

Prompts and completions are never stored after the response is returned. Full SOC 2 Type II compliance.

OpenAI-Compatible

Drop-in replacement for the OpenAI API. Works with every existing SDK and integration unmodified.