Beyond Text-only AI

Fatih Kadir Akın, "Beyond Text-Only AI: On-Demand UI Generation for Better Conversational Experience":

What if AI systems could generate precisely the right interface components when needed, without requiring developers to build every possible UI variation in advance? This post explores my work on a new prototype that enables LLMs to create dynamic, interactive UI components on demand, transforming how users interact with AI systems and MCP servers.

This is where LLMs are very interesting to me, where applications with them are most able to escape hype into practical building. A thing they're good at is taking some of the vague stuff we hand-wave at like "you know, that thing with the days in squares on it" and jumping to "you mean a calendar?" But then, skip straight from that to presenting a calendar for input as part of the workflow.

I'm finding LLMs most handy when they work in smaller universes. Give it a relatively small inventory of UI components, properly described with context, and they can be great at picking which one to use and when. Hard to hallucinate when there's not a wide range for wandering, easy to mitigate what hallucinations might occur when you can automatically kick it back into bounds. Meanwhile, the workflow can feel more smooth and demand less fiddly navigation from a user. That seems like a really promising glimmer to me.

blog comments powered by Disqus
I like (reading about) making video games  Previous Foggy Blogging Next