Creating fluent Frontend Experiences in an age of AI

Frontend Nation 2025 / 28:16

Watch this talk on YouTube

Transcript

26 paragraphs

This is an automatic transcript of the recording above. It is published in full and unedited, apart from correcting names the recogniser reliably mishears. It will contain mistakes.

00:09I'm Tejas Kumar and I've been building on the web for over 20 years at places like Verscell, Spotify, Zeta and more. Today I'm a developer relations engineer at data stacks for Genai. Um I get to work with study and research AI and then teach AI and today that's what we're here to do. A big question in the front-end world is about AI and how can we create truly great front-end experiences with AI. Um, particularly how do we deal with streaming content that comes in from a model that may not be finished. Um, previously uh with when working with AI APIs, we would receive we would sometimes just block uh when when we ask a question of say Chad GPT and this isn't the best. So in our time together, this is front-end nation. And so this is a front-end talk about how we can build better front-end that is web UI experiences um with AI. Uh we may not need to we'll look at that maybe if there's time at the end. Uh we'll look at something like MCP and how front end is maybe changing. Um but for now, let's just get started. So uh I have here my old school app. It's just a search. And um what what we can do is start querying stuff. So we can make maybe make it say um hi. And do we get a reply? We don't.

01:27Um so let's take a look at the code. Well, we've got index.tsx here. And this is the code. Um and you can see that handle search does nothing. You can see in terms of markup, we've got h1 search, a form with an input and a button. And this is all live. So if I, you know, I have live reload here. Um and so our search isn't doing anything. Okay. Well, what should it do? Well, we want to um talk to some type of API, some type of AI API. We want to get back an AI payload. So, this will be implemented on the server side. So, let's go to our server. So, we'll go to um server. Nope. Go to server.ts. And we have a server and it's kind of empty. Um let's fix it. So, slash ask. What do we want? Well, we want query from the query params. And what do we want to do with this query? Well, we want to get an answer. So we'll say con answer is await open AI and chat completions create just like that. Um and we'll instantiate open AAI as well.

02:23So we'll say um const open AAI is new open AAI just like that. So now we're doing this and it looks kind of good. Uh we respond with a JSON payload of answer and we have this is our basic server and now we just have to go fetch this on the client side. So if we do this, we kind of have it. But let's maybe set the answer in state and then we'll go and um like render the answer here. Okay, this is a nice full stack solution that works. Let's go to our terminal and run the server. So we'll npx uh vit node and we'll do I think it's main.ts. And so now sure it wants to install vit node. Um there we go. So the server is running on port 3000. So now if we go back to our browser and reload, this should work. So we'll say um how are you today? And we should Okay, cool. I'm just a computer program. So but you saw there was some type of delay there. In fact, the the delay scales linearly with the response and since this is generative, we'll have problems. Like if we go to the network tab, watch this. I'll ask write an essay about front end. Um, and what we can see is the essay is being written, but the user is kind of just left hang. This is not good front-end experience, right?

03:40Like it takes quite a long time. Um, let's see. Wow. See, it takes a really long time. Imagine this is user experience. Not okay, so there 15 seconds. 15 seconds delve spotted was this really sucks. And so, what's the next best thing for front end here? The next best thing is streaming. Uh so let's look at how we can stream text for better UX instead of um instead of doing uh just like the the classic um waiting. So to do that let's go here and instead of all this we'll just add stream is true. And now instead of we have no answer choices. So when you say stream is true the the the response changes. In fact this is not an answer it's a response. So let's do now for await chunk of response. We get back a chunk response. So for each chunk, write them into the response and then end when there's nothing left. So we'll save our new server implementation and we'll go back and restart our dev server. Okay.

04:39And then we'll update our client. So resz here the response is a it has a body which is a stream. It's a readable stream. So let's read the stream. Uh what we're doing is we're getting the reader. Um we create a text decoder because it's a byte stream aka a stream of bytes. Okay. And so it's a byte stream. So then what we do is not this this is not what we do. We'll create a function called read. Um and this function does that it it reads from the reader incrementally and then if we're done it just exits. If we're not done we decode the bytes and then set it into state. However we need to recursively call this. So we call read inside read and then we call read here to kick it off. It's a recursive process until the stream is empty and then it just does nothing. It returns. Okay. So now if we save this, let's try that again. Write an an essay on the history of front end.

05:34And uh wow, look at that. Incredible. So much better. The the time to something happening is faster. This is great. But we don't work with text mostly. Yeah, we work with JSON. Like if if you go on literally anything like Amazon, you get a here. Let's just Amazon.de. What you have is you have a list of products. this this like list plus then detail is a common UX pattern. list. See, this is a list and then you click on it, you get a detail page, right? This is how we browse the web. Lists are usually JSON. So, how do you deal with streaming JSON for great front end? Well, let's take a look at that. So, let's go back to our um localhost example 5173 and let's say give me 20 kinds of tripods, right? This is not cool. We want JSON.

06:24So let's go update our server implementation and let's add a system prompt. So we'll do this and instead of whatever slop we got, we'll say you respond only in JSON. Let's try that. Um and let's go restart our server. And uh we'll just keep our terminal right here because we're going to need to do this quite often. So we'll restart our server and come back. Reload. 20 kinds of cameras. Okay. So, it's nice, but we get a JSON snippet. So, we'll have to update the system prompt. Um, no back ticks. This is not a snippet. It's real JSON. All right, let's save this and go back to our terminal. Restart the server and we'll try again. 20 kinds of chocolate. Okay, that's nicer. Um, the problem is it's a JSON array, right? where chocolates is the key and then um it's another array with name name. Uh it's okay but this cannot be incrementally parsed because if we get a like a chunk up to here for example um and we try to JSON look JSON JSON.parse and we'll just like an incremental chunk it fails. This sucks. So what do we do?

07:46Do we try catch and then brute force until it doesn't throw? There's a better way to do this. Um, and maybe instead of a JSON array, we use a format called NDJSON or new line delimited JSON to delimit by new line. And then when we get a valid line of JSON, then we can easily parse line by line. So let's try that. So let's go to um you actually let's we have it here. ND-JSON. Uh, it's a real NDJSON. You use this schema with um name string just like that. Okay. Uh, and so now we we're asked for NDJSON. Let's try this. So, we'll go back here and uh server is running. Let's try 20 kinds of dessert. What do we got? Perfect. So, we're getting back this like JSON line by line. That's awesome. Uh let's go back to the front end and now console log each chunk we get. So we can console.log and we can say gotchunk and then the text. So 20 kinds of phones. But you notice the chunks we get they're not like cohesive JSON. We need one final adjustment to work with incremental JSON and that is to um flush single objects over the network response and not just like random chunks. So to do that we'll update our server again and we'll say um before we start iterating we'll say let obj is just an empty string and then instead of just writing to the stream uh what we can do is we can say I don't know what AI wants to do we're not going to do that we'll first start by assigning this to something so we'll say um const partial

09:32is this okay and this is how things work but we can say ob we can increment this and if it ends with a new line because it's new line delimited JSON then we write it to the response and reset the object and then we don't do this does this make sense and so now we always only send a valid object at a time so let's go restart our server okay come back here great and let's try it now 20 kinds of islands uh look at that you see that beautiful we get back an actual chunk that's an actual so now that we get back a streamed JSON chunk. Instead of just setting the answer as text, we can say items. Set items as an array of strings with name. And then we set items here. And we go back and instead of a div, we can do a ul and showcase a list. We can fulfill that pattern. So now check it out. We could do uh 20 kinds of frameworks and we get back beautiful.

10:35You see that crashed at some point because we are trying to parse an an u maybe a wrong character. So we can actually check if there's even a response. We can say if there is no partial do nothing. I think that was missing. Let's finish this. So now let's try this again and say 20 kinds of JavaScript. I have no idea anyway what's going to happen. But here it totally works. We get back streamed incre. This is like UX improve. started waiting for like a whole thing and now we just get it. Incredible. Okay, but the we're we're not we're only halfway through. What we need to understand is when we use tools like React, View, Angular, spelt, they make use of the DOM. What is the DOM? The document object model. What is document? It's just HTML. But the object model is the document modeled as objects as JavaScript objects. And so if the document object model is objects and this is how we stream an array of objects then can we stream DOM? Indeed we can because you're just stream and now we know how to stream object and so uh the Verscell AI SDK does exactly this to stream like actual components which is nothing more than JSON uh to create some really great user experience. In fact we can just see this on chatgbt.com here. Let's go to Let's go to chatgbt.com.

12:02Uh chatgbt.com. And what we'll do is we'll say um you know uh what's the weather in Berlin. And um what we'll see is oh this may this may take a long time because I chose 03. 03 loves to think. But let's see what happens. Uh okay. You know what? We're not going to do that. I'm going to choose a faster model. Uh and we'll try this again. What's the weather in Berlin? Come on. for Berlion. Um, so it's going to perform rag. It's going to search the web and then look, it rendered some UI here. That was just JSON. You know how to do that. Now, uh, to illustrate this, I built an app. I built an app called Movies++. Some of you may have seen this where if you try to search like a normal human being on a tool like Netflix, you maybe will have a bad time. And that's because um, that's because Netflix optimizes for keyword search, but not like human search. Like when I want to watch a movie, I want to watch like a movie that's about superheroes or a movie with a strong female lead or a movie with large monsters, you know? I don't want to watch like specific names of movies.

13:06Often I don't know, you know. And so let's try that. If we go on Netflix right now, um, and we can honestly just stop the talk here and watch Netflix if uh that's what we want. But okay, so we're on Netflix and we'll do movies with a strong female lead. Look at that. It's just like try different keyword. That's the best we can do. So, I built an app that uses AI and streams in content that does slightly better. And I call it movies++. Let's take a look. So, if we go to localhost 3001, um, this is what it looks like. Let's try that same prompt. Movies with a strong female lead. Um, this is what we get. So, it starts to fetch movies. That nextgs issue is not not my application code. Uh, starts to get movies.

13:51And at some point, um, there we go. So we have these all of them do have strong female leads but you may have noticed that wasn't streamed and this is not UI um it's this is a demo chill but like now we can do some we can say like what might this look like as UI you know and now um look at that it it's this is just JSON and so you might be wondering like how is this possible um it's J we just looked at how this is possible so we just render these as elements after receiving JSON and the cool thing is because of that we can bind event listeners after we rendered them and so on. Uh but it gets even better like show me the trailer for Madame Web, right? We can do anything because it's just J the DOM is just JSON. And so look at it. Um it just like totally playing where can I watch it? Um and it will even like open maps and show me the nearest cinema where maybe I can go watch it. So it's so cool because it's just JSON. We looked at how we could stream JSON. Now, we can do even better because I don't know about you, but there's been times where I've wanted like customer support and uh I find myself going to I don't know like some intercom widget and asking for help and I often get back instructions. But what if we could do better? Like the case where you forgot your password and you get back like click here, go there, do that. You get instructions. What if we just did this?

15:10What if I forgot my password, right? And immediately you have the password reset form like right here, you know? um can do this uh and it's incredible. But while we can do this, I I suspect and there's some talks even here that suggest um like for example yesterday with Kent that suggest front end is changing. It's just front end if he's different. Okay, what is front end? Well, let's redefine front. It's it's the end of the application that users interact with. It's your UX layer. It's front end front you know and so we can't ignore the rise of AI we can't ignore the fact that we have chat interfaces like chat GPT we can't ignore that in many cases it's just better chat interfaces are just better um why are they better well they're accessible like if somebody is blind then you can use voice and you can hear if somebody is deaf then you can use your eyes and read. If somebody is blind and deaf, then all it takes is one team at OpenAI or Anthropic or one of these chat app vendors to really really zero in on their accessibility and a huge amount of the population benefits. This is a significant step up to the way things are in the world today. Today, each front end, each website is determined by a team of developers, designers, and product people, some experts who are fallible human beings like me who are likely to make mistakes or ignore certain groups of people, and things are not

16:52accessible. Sometimes you have interfaces that are tabs or carousels, but they're not screen reader key. It's just a huge problem. And and with the chat interface, the the honestly the experience is better. Like if I could today um talk to something like chat GPT and say, "Hey, do I have any emails coming up that I need to pay attention to? Hey, what are my meetings for tomorrow? Um, hey, book me a flight to go to Macedonia in September and attend what the stack and I want to spend 300 bucks. Zero connection." Like like I can if I could just issue commands. If I could just speak things into existence, I would I would 100% prefer this to the experience of going and clicking on some website vendor's form and then finding out it doesn't respond to the enter key and like all of this mess. Okay, thanks to model context protocol and AI, um, front end is changing. Front end is different.

17:45And I'd like to show you that uh in in in the last few minutes we have because I want us to I want to invite us to reconsider front end in the age of AI as we get more fluent as the talk title says in frontend in the age of AI. Okay. And so what I want to do is I want to show you a tool called Langlow. Um it's open source. We don't sell it. We don't make any money from it. But I it's the easiest way to reason about the evolution of front end in an AI error at least for me. So um it's it's it's sort of like runnable locally since it's open source and you can host it wherever you want. And so um here how can we host it? Well, it's Docker, so do that. But we can go to local I'm hosting it locally.

18:19Local host 7860 is where Langlow lives. And of course, that's not running, so I need to start it. Um, so we'll Docker run uh 7860 is the port 7860. And then 7860 Langflow AI/LFlow. That's with with this command. Of course, Docker is not running. Um, okay. Sure. Yes, of course. You would think uh tis would prepare for this talk anyway. So with this docker command um you can run lang flow and when you run lang flow you can I we can explore what new front end might look like. Okay. So docker is running. Let's run lang flow here which I should have. Thank you docker desktop. Um and it's going to come up in just a second. While it does I'm going to make sure that the rest of my stuff doesn't need to update um and cause problems. So, this is Claude Desktop. Some of you may be familiar with Claude Desktop. It's just like chat GBT. Honestly, it's just an app. Um, and I'm just going to clear out all my caches here. Yes. Yes. Go away. Okay.

19:21And this is just like chat GPT. Hi, how are you chat GPT? You know, um, and see if it answers. This would be an interesting answer. Pondering. Okay. Uh, you can keep pondering. Are we good with Langlow? Cool. Langlow open. So let's now go visit Langflow running on local host. And here's what we have. It's brand new. There's nothing here. I'm going to make an agent that invites us to rethink front end. So here is I'm just going to do a blank flow. And I'm going to get an input, which is a chat input, an output, which guess what? It's a chat output. I'm making a chat interface because we just talked about how they're better. Um, and in between I will have an agent just like this. Um, and I'm going to just, you know, input goes here, output goes there. And I'm just going to test that this works. I need an API key. So, we'll get my OpenAI API key right here.

20:14And let's test that this works. Hi. Um, okay, cool. It totally works. Now, what I want to do, again, rethinking front end, is create an assistant that can read my emails and maybe manage my calendar. So, to do that, I will grab the calendar component here. Uh, it's it's from Composio, which is a great tool. And I'm going to get my API key. I'm going to paste it right there. Um, and I'll do the same for Gmail. Um, and it needs to authenticate with Compose here. So, I'm just going to zoom back out and I'll paste those two. And right now, authentication's kind of in flight. Um, it's starting to authenticate. Let's go see what Claude said. I'm doing well. As for comparisons, I see we have different Oh, what anyway. So, sorry, I got distracted. So, I'm going to give my calendar as a tool um to my agent. So, I turn on tool mode. Boom. Just like that.

21:08And here, I'll do a similar thing. I'll turn on tool mode. And just like that, go here. And I can select what I want the agent to do with my calendar. So, I'll say, you know, um I want you to be able to create events, to find events, find free slots, get my calendar, quick add events. Sure. So, you can now the agent can do all of these things. Um, and I promise this is going somewhere. So now my agent exists. Let's go take a look at my calendar. So this is my calendar, uh, right here. Today is the 2nd of June, um, 3rd, fourth, 5ifth, whatever. And so, um, let's go to the frontend nation website, Frontend Nation, and find out what the schedule is. So, we'll go to the schedule and we have a bunch of things. Is the sk how do how do we get the schedule? Is it like over the network? Let's check workshops.

21:55Oh my goodness. SVG is being served over fetch uh in Oh. Oh, here we go. Um, this is the schedule it looks like. Let's check if Kent C dots. No. Um, okay. I guess these are like actually workshops. These are Okay, these are workshops. I'm looking for the schedule like talks because I would love to reconsider the front end here. Is this like server rendered? It's definitely not server rendered. That's the markup. So, we are indeed getting the schedule from somewhere here. Um, let's see. I'm going to refresh one more time real quick. So, we get back a bunch of work. I want to know where this data is coming from. Anyway, so let's make let's just do the workshop, you know. Let's do this. Do they have here schedule?

22:41There's times 2025 March 31st. Um, cool. Let's actually do this. Let's create an assistant a different front end for this workshop UI because my thesis is anything that can become anything that has an API can become can be expressed with a different front end. That's what I'm trying to say. So let's take this one June 18th, 2024. There's a workshop. I'm just going to copy this URL. Copy um as URL right here. I'm going to go to Langflow and add API request. Uh and we'll add this right there. And we'll copy this URL conference.view school. Really? Is that just a JSON? Okay, cool. Wow, they have a public JSON endpoint. Nice. And um I'm going to parse that data into a string. Data parse data just like this. So we'll take this and we will plug the data from there into this. Now we get back text and we can construct a prompt prompt and we'll the prompt will be um it will have a variable here are some workshops and we'll create a variable workshops answer the question and we'll do input.

23:55So now the workshops go to workshops and the input goes to chat input and then the prompt itself becomes the new input to the agent because it contains workshops. So, we can ask questions about workshops. Okay, let's just validate that this works real quick because honestly, I did a bunch of stuff. Let's see. Tell me about June workshops 2025. And so now it should pull that data and answer my question because we just built that. So, uh, let's take a quick look. Uh-oh, we have some issues. List of data objects is not supported. Well, we want to stringify it. That's okay. Um, let's try again. So, tell me about June 2025 workshops. Um, and so it's it's getting some there. The the workshops are for 2024, not 2025. Here are some workshops from 2024. Fine, we have this data. We'll work with it. This looks good. Um, how might we work with this? Well, Langflow supports MCP. So, I'm going to publish this as an MCP server, and I'm going to quickly edit this and give it a name and stuff. So, I'd be like, um, get workshop info.

25:02Get work. use this tool to learn about workshops and manage a Google calendar, right? And so now it exposes a model context protocol server which I can just copy like this and paste in an app like claude. And again, Claude desktop is really like chat GPD. It's just a regular AI app that's also an MCP client. And I'm going to quickly add this here. So I'll come here and just paste my Langflow server. I'll close this. Close this. Close this. And now if I restart the app. Um, and again keep in mind the user interface here is not Langflow. It's not something I wrote. This is like chat GPT. It's a foreign thing and it's still a front end. It's just on my front end. You can see that I have my workshop info. So now I'd be like, tell me which workshop to attend in 2024 for databases.

25:53In fact, just add it to my calendar. So this is front end, but it's different, right? because like I'm not browsing the web and I'm not doing any of these things. Um I'm just talking right front end UI UX is changing and it's very interesting. Okay, so um here we go. I'll help you find workshops and add it to your calendar. So yes, we'll allow the use of this tool that I just added over MCP and it's searching for workshops. Let's see what it comes up with. I found some great workshops for 2024. Here are your options. Supercharge your full stack. This is June 18th, 2024. And it gave me a bunch of options here. Since we're already past the first workshop may have already Let me add the second one to your calendar. So, it's already going July 9th, 2024, which let's go to my calendar right now, but back to July 9th, 2024. So, let's do year 2024. July 9th.

27:06Perfect. I've added it. And let's go check here. Tracing front-end issues with backend solutions workshop. This thing just operated my calendar. What? uh the front there's a front end that I would have used on the web but this front end is radically different. It's all front end but front end is changing. I'll leave the talk here. I realize I'm maybe a minute or so over but I want to invite you to think. We already looked at how we can create fluent front end for AI today with streaming JSON and and creating great UX. But we also need to consider how UX is changing itself and how maybe the website is going away. Maybe we won't click on buttons and navigate pages with questionable accessibility, but we just talk. We just talk and front end is not even our front end. We just make the APIs and we operate highly accessible AIdriven conversational frontends. I'll leave that here. I'm happy to take questions and continue the conversation. Thank you so much for your time and I can't wait to continue talking to you.

More talks

All 19 transcripts

Elsewhere

There is every talk I have given, all 69 of them, ConTejas Code, the podcast, and Fluent React, the O'Reilly book on how React works inside.