Hono is now AI ready

Hono is now AI ready

The web technologies needed to create AI, especially ChatGPT applications, are OpenAPI and Streaming.

OpenAPI is not OpenAI, but you can create OpenAI applications with OpenAPI. You write in natural language what you want your application to do with OpenAPI, and ChatGPT will interpret it in a nice way.

Streaming is suited for applications like ChatGPT that return text bit by bit. While it takes 30 seconds to return a response all at once, returning it little by little is less stressful for the user, and it seems more like “talking” to them that way.

Now, Hono supports both. You can use the Hono extension called Zod OpenAPI and you can stream text with the new c.streamText().

Zod OpenAPI Hono

ZodOpen API Hono is an extension of Hono that allows you to define OpenAPI in your code base and create its logic as a Hono application.

The beauty is that Zod defines the schema.

Let’s create a super simple blog. Expect and then validate input from users as follows:

const schema = z.object({
  title: z.string().min(1),
  content: z.string().min(1),
})

You can use .openapi() in this Zod OpenAPI (This function is based on Zod to OpenAPI, thanks a lot!).

const schema = z.object({
  title: z.string().min(1).openapi({ example: 'About today' }),
  content: z.string().min(1).openapi({ example: 'Today is a good day...' }),
})

Use this and define the path, content type, and response information as a route:

export const routeAddPost = createRoute({
  method: 'post',
  path: '/posts',
  request: {
    body: {
      content: {
        'application/json': {
          schema: schema,
        },
      },
    },
  },
  responses: {
    '200': {
      description: 'OK',
      content: {
        'application/json': {
          schema: z.object({
            ok: z.boolean(),
          }),
        },
      },
    },
  },
})

That’s everything ”OpenAPI-like”. Now it’s time to implement the logic as usual. Here we are using Cloudflare D1 as our database.

app.openapi(routeAddPost, async (c) => {
  const { title, content } = c.req.valid('json')
  const id = crypto.randomUUID()
  await c.env.DB.prepare('INSERT INTO posts(id, title, content) values (?, ?, ?)')
    .bind(id, title, content)
    .run()
  return c.jsonT({
    ok: true,
  })
})

The great thing about this Zod OpenAPI is that it is terribly Type-Safe. If the request’s Content-Type is application/json, it will suggest the 1st arg of c.req.valid() as json and expect the response to be ok:boolean as defined in the response schema.

The API spec is generated as JSON and you can view the documentation in the Swagger UI.

ChatGPT plugin

Let’s make this a plugin for ChatGPT. It’s easy. Just write ”This plugin is for Blogging” in the manifest file.

export const aiPluginJson = {
  schema_version: 'v1',
  name_for_model: 'Blog',
  description_for_model: 'Plug-ins for blogging',
  api: {
    type: 'openapi',
    url: 'http://localhost:8787/openapi.json',
  },
  //...
}

Now it’s finished. If you give it a hint, ChatGPT will come up with the content for your blog and put it in the database. And this can be deployed to the edge, such as Cloudflare Workers.

This is one of two stories about AI.

c.streamText()

You can use Hono’s new functions c.stream() and c.streamText() to make streaming of contents easier. It’s easy to use. Just look at the code.

app.get('/', (c) => {
  return c.streamText(async (stream) => {
    stream.writeln('Hello!')
    await stream.sleep(1000)
    stream.writeln('Hono!')
  })
})

ChatGPT Gateway

The ChatGPT API supports ”stream”. Therefore, you can create a ChatGPT Gateway with an elegant code.

app.post('/api', async (c) => {
  const body = await c.req.json<{ message: string }>()

  const openai = new OpenAI({ apiKey: c.env.OPENAI_API_KEY })
  const chatStream = await openai.chat.completions.create({
    messages: PROMPT(body.message),
    model: 'gpt-3.5-turbo',
    stream: true
  })

  return c.streamText(async (stream) => {
    for await (const message of chatStream) {
      await stream.write(message.choices[0]?.delta.content ?? '')
    }
  })
})

This is just a Gateway, but if you give it the right prompts and save the history for each user, you will have an excellent AI!

AI is fun

Creating these AI applications is a lot of fun! In particular, Hono’s Zod OpenAPI, c.streamText(), and Cloudflare Workers are a perfect match for AI.

By the way, this article was written by a human.