Integrations
AI SDK
A chat endpoint on the Vercel AI SDK: the model comes from config, and the reply streams straight onto ctx.response.
On this page
Start from the template. src/app.service.ts and src/app.controller.ts stay as they are; everything below is added.
Install#
bun add ai @ai-sdk/openai @rhythmjs/config zodConfig picks the model#
A config namespace holds the model id, so swapping models is an environment change: AI_MODEL=gpt-4o. An empty value stops the app from booting: the config loads at startup, when await configModule.forRoot(...) runs, and a bad value rejects with a ConfigError before any request.
// src/config/ai.config.ts
import { registerAs } from "@rhythmjs/config";
import { z } from "zod";
export const aiConfig = registerAs("ai", z.object({ model: z.string().min(1).default("gpt-4o-mini") }), () => ({
model: process.env.AI_MODEL,
}));
export const load = [aiConfig] as const;A service creates the model#
modelService is a plain service. The chat module puts it on its startup context, and tests stub modelService.create with the SDK's mock model.
// src/chat/models.ts
import { openai } from "@ai-sdk/openai";
import type { LanguageModel } from "ai";
export interface ChatModel {
model: LanguageModel;
modelInfo: { id: string };
}
export const modelService = {
create(id: string): ChatModel {
return { model: openai(id), modelInfo: { id } };
},
};The controller streams the reply#
This is the chat controller, a RhythmRouter with full /api/... paths. A handler fills ctx.response, and its body takes a byte stream, so result.textStream piped through a TextEncoderStream is the whole response, with no headers to set. abortSignal stops the model call when the client disconnects. Clients read the stream with TextStreamChatTransport; for tool calls or reasoning in the chat UI, use the SDK's UI message stream instead, which needs its own headers.
// src/chat/chat.controller.ts
import { RhythmRouter } from "@rhythmjs/router";
import type { RhythmHttpContext } from "@rhythmjs/router/context";
import { convertToModelMessages, generateText, streamText, type UIMessage } from "ai";
import type { ChatModel } from "./models";
import { error, json } from "@rhythmjs/router/response";
export type ChatContext = RhythmHttpContext & ChatModel;
export const chatController = new RhythmRouter<ChatContext>()
.get("/api/model", (ctx) => {
json(ctx, ctx.modelInfo);
})
.post("/api/chat", async (ctx) => {
const { messages } = (await ctx.request.json()) as { messages?: UIMessage[] };
if (!Array.isArray(messages) || messages.length === 0) {
error(ctx, 400, "messages are required");
return;
}
const result = streamText({
model: ctx.model,
system: "You are a concise assistant.",
messages: await convertToModelMessages(messages),
abortSignal: ctx.request.signal,
});
ctx.response.body = result.textStream.pipeThrough(new TextEncoderStream());
})
.post("/api/summarize", async (ctx) => {
const { text } = (await ctx.request.json()) as { text?: string };
if (!text) {
error(ctx, 400, "text is required");
return;
}
const result = await generateText({
model: ctx.model,
prompt: `Summarize in one sentence: ${text}`,
abortSignal: ctx.request.signal,
});
json(ctx, { summary: result.text });
});The chat module owns the feature#
The module decorates its startup context with the service, includes the awaited config module, derives the model from the configured id, and mounts the controller.
// src/chat/chat.module.ts
import { configModule } from "@rhythmjs/config";
import { Rhythm, decorate, derive, include, mount } from "@rhythmjs/rhythm";
import { load } from "../config/ai.config";
import { chatController } from "./chat.controller";
import { modelService } from "./models";
export const chatModule = new Rhythm({ name: "chat" })
.register(decorate(() => ({ modelService })))
.register(include(await configModule.forRoot(...load), ({ configService }) => ({ configService })))
.use(derive((ctx) => ctx.modelService.create(ctx.configService.get("ai.model"))))
.use(mount(chatController));Mount it in the app#
The app module mounts the chat module next to the template's own controller. Because configModule.forRoot is awaited when the chat module is imported, a bad config fails at startup and not at the first request; main.ts needs no setup call.
// src/app.module.ts
import { Rhythm, decorate, mount } from "@rhythmjs/rhythm";
import { appController } from "./app.controller";
import { appService } from "./app.service";
import { chatModule } from "./chat/chat.module";
export const appModule = new Rhythm({ name: "app" })
.register(decorate(() => ({ appService })))
.use(mount(appController))
.use(mount(chatModule));// src/main.ts
import { toFetchHandler } from "@rhythmjs/router/fetch";
import { appModule } from "./app.module";
const port = Number(process.env.PORT ?? 3008);
const server = Bun.serve({ port, fetch: toFetchHandler(appModule) });
console.log(`listening on ${server.url}`);Things to check#
- Every request costs money: add
bodyLimit, an authentication guard and rate limiting before exposing the endpoint. - Tests stub
modelService.create, so they need no API key or network.
The full app, with tests, is examples/ai-sdk, built from the template.