Python Integration Guide for Boole AI — OpenAI SDK

Install the openai Python package and make chat, streaming, async, and tool-call requests against Boole AI’s high-throughput inference API.

Boole AI works with the official OpenAI Python SDK. Install it once, point it at Boole, and every pattern you already use — synchronous calls, async coroutines, streaming, tool calls — works without any further changes. There is no Boole-specific library to learn.

Installation

pip install openai

Store your API key in an environment variable — never hardcode it in source files. Set BOOLE_API_KEY in your shell profile or a .env file and load it with os.environ or a library like python-dotenv.

Basic Chat Completion

Create a client pointing at the Boole Cloud API and call chat.completions.create exactly as you would with OpenAI.

example.py

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.boole.dev/v1",
    api_key=os.environ["BOOLE_API_KEY"],
)

response = client.chat.completions.create(
    model="llama-3.3-70b-instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"},
    ],
)

print(response.choices[0].message.content)

Streaming

Set stream=True to receive tokens as they are generated. Use the context-manager form to ensure the connection closes cleanly when you’re done reading.

streaming.py

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.boole.dev/v1",
    api_key=os.environ["BOOLE_API_KEY"],
)

with client.chat.completions.create(
    model="llama-3.3-70b-instruct",
    messages=[{"role": "user", "content": "Write a short poem about the ocean."}],
    stream=True,
) as stream:
    for chunk in stream:
        print(chunk.choices[0].delta.content or "", end="", flush=True)

Async Usage

Use AsyncOpenAI in async applications — FastAPI handlers, async pipelines, or any asyncio-based code. The interface is identical to the sync client; just await the call.

async_example.py

import os
import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="https://api.boole.dev/v1",
    api_key=os.environ["BOOLE_API_KEY"],
)

async def main():
    response = await client.chat.completions.create(
        model="llama-3.3-70b-instruct",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    print(response.choices[0].message.content)

asyncio.run(main())

Local Deployment

When you’re running the Boole local binary, change base_url to point at localhost and pass any non-empty string as the API key — the local server does not perform key validation.

local.py

import os
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="local",                  # any non-empty string
)

response = client.chat.completions.create(
    model="llama-3.3-70b-instruct",
    messages=[{"role": "user", "content": "Hello from my local GPU!"}],
)

print(response.choices[0].message.content)

The local binary starts in under 400 ms and exposes the same /v1 interface as the cloud API. You can switch between local and cloud by changing one line — everything else in your code stays the same.