A hybrid edge-cloud AI engine for mobile devices & wearables.
┌─────────────────┐
│ Cactus Engine │ ←── OpenAI-compatible APIs for text, speech, and vision.
└─────────────────┘
│
┌─────────────────┐
│ Cactus Graph │ ←── Zero-copy computation graph
└─────────────────┘
│
┌─────────────────┐
│ Cactus Kernels │ ←── CPU/GPU kernels for (Apple, Samsung, Pixel, etc.)
└─────────────────┘
│
┌─────────────────┐
│ Cactus Quants │ ←── Custom rotation-based quantization technique
└─────────────────┘
- Step 1:
brew install cactus-compute/cactus/cactus - Step 2:
cactus run
#include "cactus_engine.h"
cactus_model_t model = cactus_init(
"path/to/weight/folder",
"path to txt or dir of txts for auto-rag",
false
);
const char* messages = R"([
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "My name is Henry Ndubuaku"}
])";
const char* options = R"({
"max_tokens": 50,
"stop_sequences": ["<|im_end|>"]
})";
char response[4096];
int result = cactus_complete(
model, // model handle
messages, // JSON chat messages
response, // response buffer
sizeof(response), // buffer size
options, // generation options
nullptr, // tools JSON
nullptr, // streaming callback
nullptr, // user data
nullptr, // pcm audio buffer
0 // pcm buffer size
);Example response from Gemma4-E2B
{
"success": true, // generation succeeded
"error": null, // error details if failed
"cloud_handoff": false, // true if cloud model used
"response": "Hi there!",
"function_calls": [], // parsed tool calls
"segments": [], // transcription segments (empty for chat)
"confidence": 0.8193, // model confidence
"confidence_threshold": 0.7, // resolved handoff threshold (model-dependent)