Built-in AI

jakobhoeg/built-in-ai is a community provider that serves as the base AI SDK provider for client side in-browser AI models. It currently provides a model provider for Chrome & Edge's native browser AI models via the JavaScript Prompt API, as well as a model provider for using open-source in-browser models with WebLLM.

We are also working hard to include other browser-based AI frameworks such as transformers.js.

Setup

Installation

The @built-in-ai/core package is the AI SDK provider for Chrome and Edge browser's built-in AI models. You can install it with:

pnpm add @built-in-ai/core

The @built-in-ai/web-llm package is the AI SDK provider for popular open-source models using the WebLLM inference engine. You can install it with:

pnpm add @built-in-ai/web-llm

Browser-specific setup (@built-in-ai/core)

1. You need Chrome (v128 or higher) or Edge Dev/Canary (v138.0.3309.2 or higher)

2. Enable these experimental flags:

   • If you're using Chrome:
     1. Go to chrome://flags/#prompt-api-for-gemini-nano and set it to Enabled
     2. Go to chrome://flags/#optimization-guide-on-device-model and set it to Enabled BypassPrefRequirement
     3. Go to chrome://components and click Check for Update on Optimization Guide On Device Model
   • If you're using Edge:
     1. Go to edge://flags/#prompt-api-for-phi-mini and set it to Enabled

For more information, check out this guide

Provider Instances

@built-in-ai/core

You can import the default provider instance builtInAI from @built-in-ai/core:

import { builtInAI } from '@built-in-ai/core';

const model = builtInAI();

You can use the following optional settings to customize the model:

• temperature number

  Controls randomness in the model's responses. For most models, 0 means almost deterministic results, and higher values mean more randomness.

• topK number

  Control the diversity and coherence of generated text by limiting the selection of the next token.

@built-in-ai/web-llm

You can import the default provider instance webLLM from @built-in-ai/web-llm:

import { webLLM } from '@built-in-ai/web-llm';

const model = webLLM();

Language Models

@built-in-ai/core

The provider will automatically work in all browsers that support the Prompt API since the browser handles model orchestration. For instance, if your client uses Edge, it will use Phi4-mini, and for Chrome it will use Gemini Nano.

@built-in-ai/web-llm

The provider allows using a ton of popular open-source models such as Llama3 and Qwen3. To see a complete list, please refer to the official WebLLM documentation

Example usage

@built-in-ai/core

import { streamText } from 'ai';
import { builtInAI } from '@built-in-ai/core';

const result = streamText({
  model: builtInAI(), // will default to the specific browser model
  prompt: 'Hello, how are you',
});

for await (const chunk of result.textStream) {
  console.log(chunk);
}

@built-in-ai/web-llm

import { streamText } from 'ai';
import { webLLM } from '@built-in-ai/web-llm';

const result = streamText({
  model: webLLM('Qwen3-0.6B-q0f16-MLC'),
  prompt: 'Hello, how are you',
});

for await (const chunk of result.textStream) {
  console.log(chunk);
}

For more examples and API reference, check out the documentation