Sean Rugge: ...came across a video on LinkedIn of somebody using an agentic browser to take a course, complete a course. And that to me raised alarm bells. I'm like, "Oh man, I really don't like that." AI won't do your job for you. And if you try and have it do your job for you and put it on autopilot, it will do something with extreme confidence. And that something is probably going to be a hot garbage pile.


Carolyn Ford: Sean Rugge, who is senior educational technologist at Marine Corps University, learned this firsthand when he took a 200-page PDF and turned it into an interactive online training course with the help of AI. The AI structured modules, generated quizzes, built interactive content frameworks, and helped him organize everything. But it was all under his direction.


And here's the catch: Sean is an expert in his field. He's been doing instructional design for years, so he knows what great looks like. He also spent a lot of time iterating and fact-checking. He even reviewed the content with a Marine expert to make sure that it was right. And he caught moments where the AI hallucinated. It left out entire chapters and it presented wildly confident, but wrong metrics.


The AI was a fast, tireless collaborator, but it took human brainpower to keep it on track and make sure that everything was right. Today, Sean is going to show us what he built, how he built it, talk about the tools he used, and where AI saved him time and where he had to step in to keep it from making a mess.


Sean, I want to skip all the "AI is changing everything" talk, and I want to go straight to the good stuff. I saw you demo a training course that you built using AI, Gen AI, agentic AI—which I want you to get into the difference when we get there. And I think our listeners will really benefit from seeing what you did. So, in just a minute, I'm going to ask you to screen share and take us through what you did. But first, just give us an overview of what you built, what your objectives were, and how you did it.


Sean Rugge: Sure. Um, so I work here at Marine Corps University, and I should probably start with a little bit of a disclaimer that the opinions that I express are my own, and not that of the Marine Corps, Marine Corps University, the Department of Defense, or the National Football League, or any of its affiliates.


All right, I think I've got everybody covered. What I was looking for really is a proof of concept, and to prove that it could be done with the tools that the Marine Corps has available to them. And if I couldn't, then I would venture out into tools that, "Wow, it would be really nice if the Marine Corps had this available to them." So that was kind of my approach and my goal.


The audience that I was looking to help—and, you know, I'm a problem solver, right? So I want to help people solve problems, and that's pretty much 99% of my job. But I work with a lot of Marines who rotate constantly from one unit to another, right? And through their rotations, they may show up at a school where they're tasked with building a course or modernizing a course, putting a course online that wasn't online before. And they're not ISDs. They're, you know, considered subject matter experts in their MOS by the time they reach that school.


And so they have familiarity with the material, but it might be something that's just like PowerPoint presentations or PDFs. And you know that the Marine Corps wants that content modernized. And so that can be a tall ask for, you know, someone who's just shown up at a school, and their first tasker is, "Hey, build a course online, make it professional, modernize it, improve on it." It's a lot, and I wanted to demonstrate that AI could help them—kind of give them that sense of hope and "Hey, I can do this. I don't just need to hang the PowerPoint slides and hang the PDF and call it a day. I can actually do what we want to do and make the education better."


Carolyn Ford: Okay. What was your methodology for deciding what the AI should do, what the human should do, and where you should absolutely not trust, especially the agentic AI—like what you should not let it have access to?


Sean Rugge: Yeah. Um, so that's one of the things that kind of turned me onto this project to begin with. During the furlough period, I had 40 days to kind of sit around and ponder. I came across a video on LinkedIn of somebody using an agentic browser to take a course, complete a course. And that to me raised alarm bells, and I'm like, "Oh man, I really don't like that."


Carolyn Ford: So wait, they just let the agent be them and do the course for them? Is that what you said?


Sean Rugge: Yeah, they logged in. They logged in and demonstrated that, hey, the quiz can be done, the discussion board can be completed. And, you know, I'm sitting there going, "Wow, that's a lot of online education these days." Um, I don't really like that. But there might be some positive uses as well. And one of them could be: it takes a long time to build a course and click all the buttons and know where all the buttons are. And I'd like to see if the agent browser can do that.


So that was I pitched the idea to my boss, and he said, "Yeah, go with it." And let's—


Carolyn Ford: So when did you do this? I mean, genai.mil was put in place like recently, last December.


Sean Rugge: Yeah. Yeah, so that's when genai.mil came online, and I knew that it—sorry.


Carolyn Ford: Yes.


Sean Rugge: Yeah, it didn't have agentic capability. Um, so I knew that I'd have to use something else to meet that need of the actual building of the course for me, and I just give it direction.


Right, but I started in January when I learned about the Marine Corps AI Workshop and that they were looking for presenters. And I thought, "Hey, if I could do this in a month and turn it around and present in March, that would be a pretty cool proof of concept that you can have a really quick turnaround time with the AI tools that are out there."


Carolyn Ford: Okay. So before we screen share and you can walk us through this—we'll do a little hands-on today. So listeners, highly encourage you to go to YouTube so you can watch this today. Um, the course that you built is a course on how to build a course using AI?


Sean Rugge: No, this was using something that a staff sergeant might have when they show up at a school, where you're provided with a PDF. And that's what I started with—a PDF document that was publicly available. And that was part of the parameters I set for the project and that I pitched: that we'll use publicly available source materials so there's no concern over CUI or sensitive things like that.


And so I pulled up a PDF for Small Unit Leader Mountain Warfare, a publicly available PDF. It's between 200 and 300 pages of light reading material.


Carolyn Ford: Good lord.


Sean Rugge: And, you know, that makes up the information that you have to pass on to the students.


Carolyn Ford: Right, right.


Sean Rugge: And it's intended really to be a field manual. That's the intent, is that the soldier has it with them and that they can use it as a reference when they need it. But there's a lot of knowledge in there that they need, and it can be difficult to find the right information when you need it.


Carolyn Ford: That's right. Well, and you and I were chatting before I hit record, and in another lifetime I was a teacher, so I understand how hard it is to do instructional design, to build a training course in something that's actually usable. So why don't we share your screen, okay? And show us what you did and how you did it.


Sean Rugge: So I started with a PDF document that basically made up the entirety of the content here, and I put that into the genai.mil platform. And that's really—I tried as much as I could to use that platform because I wanted to demonstrate that it was a capable platform.


And so my first ask was, "Hey, I need to build a course. It needs to be online, it's instructor-led." I was putting out the parameters within the prompt of how I envisioned this content being transformed into an online course. So I had some kind of concept for what I wanted to do. I didn't know how many modules, but I asked GenAI.mil to kind of break out the content into modules so that it could be in an online format. And when I do stuff like this, people say, "Well, how do you know where to even start?" And I say, "I ask it."


Carolyn Ford: Right.


Sean Rugge: I babble. I tell it what I'm trying to do, and I ask it to make a recommendation, and then I just keep going from there.


Carolyn Ford: Yeah, and if you don't like the recommendation, you tell it to revise the recommendation and it will.


Sean Rugge: That's right. It's a great cognitive partner, and that's the way I approach any AI session that I start. And it will not do the work for you. I want to emphasize that it requires a lot of brainpower on my part to develop anything like this. Like you said, I'm sure—


Carolyn Ford: Well, I'm going to let you keep going, but I imagine that you had to iterate a lot, and it's hours and hours that you spent on this course.


Sean Rugge: Yes, there is definitely an iterative process to this. And so this is essentially what it came up with within a day. It had the module framework, and it associated different chapters with each one of the modules. And I looked over, and the chapter assignment to each module made sense. I did notice that it was missing a chapter completely; it was missing the logistics chapter.


Carolyn Ford: Right. I find that too, and I'm like, "Why would you leave this out?" And then it comes back and says, "Oh, my bad. You're right. I'm glad you caught that."


Sean Rugge: Yes. Gosh. Yeah, and so it started to kind of put it in one particular place, and I said, "You know, really logistics has to do with all of these modules. Can we weave it throughout?" And it agreed that we could. And so it took elements of logistics and wove them throughout the different modules. So that was kind of the first human-in-the-loop moment where I needed to help it revise so that we could get a better product.


Carolyn Ford: So did you look at it side by side? You went through the PDF, you're comparing the modules, and that's how you said, "Oh, we're missing a whole chapter here."


Sean Rugge: Yeah, and I even had it map out the alignment of the chapters to the modules so I could see how it was drawing the material, right?


Carolyn Ford: Yeah. Did you, when you first put it in—did you, to use my highly technical term, just babble and say, "Here's what I'm trying to do," or did you have it write a prompt for what you were trying to do?


Sean Rugge: You know, I think I started with, "Well, you are a staff sergeant at a new mountain warfare training..." I tried to give it that persona. But what I found in the response, I didn't like. I didn't like that it was acting, right? I just wanted to have a dialogue with somebody who was on the same page as me. We're collaboratively working on a project together.


So I stopped—I got away from giving it that persona because I think the first response it came back with was like, "All right, recruits..." You know, thought it was a drill sergeant. And I didn't need it to be.


Carolyn Ford: Yeah.


Sean Rugge: So yeah, we transitioned away from that and just focused on, "Hey, let's just design a course here and pretend we're doing that."


Carolyn Ford: Okay.


Sean Rugge: Um, but I started off by just some simple things. So I'll go into a couple of the modules here. You know, the learning objectives—I asked it in one of the initial prompts to have learning objectives associated with each one of the modules that it came up with. And I definitely wanted to have a quiz or a knowledge check piece to it. So that was another thing in mind.


And I do want to emphasize, like, it took me knowing the ins and outs of the software that I'm building for, which is the Moodle application.


Carolyn Ford: Is Moodle a military thing? I don't know that application.


Sean Rugge: It's not. It's an open-source learning management system. We do use it in the Marine Corps. There are a couple other services I think that use it as well—obviously a secured version. And this here is a commercial—I have it on a personal server where I can build and demonstrate proof of concept rapidly.


Carolyn Ford: Right.


Sean Rugge: But yeah, I had it identify the learning objectives, extract those from the text. And I was working with a subject matter expert, a recently retired Marine who actually did this stuff for his MOS school. So it was good to kind of bounce things off of him and say, "Hey, does this look right? Does it make sense?" And having that partner available as another check was good.


The modernization piece—like, I wanted it to produce H5P content, and this is the interactive stuff. So this is an interactive book where they can go through, and they've got different questions they can answer. And you can embed things like images or video into stuff like this. And this is an image that I had NotebookLM build, right?


Carolyn Ford: It's a great image, too! Did it get it—I'm surprised how well the text came out, because sometimes AI struggles with text, right?


Sean Rugge: It does, it does. And NotebookLM has gotten better. I think there was like one typo in here I was agonizing over.


Carolyn Ford: And you find that one typo, and you tell the AI to fix it, and then it destroys everything, right?


Sean Rugge: Or it just doesn't fix it. It's like, "Yep, I fixed it," and it makes everything worse.


Carolyn Ford: Yeah, exactly. But this looks really good. So did you find that NotebookLM is one of the best ones if you're trying to create a graphic married to text? This is really good.


Sean Rugge: Yeah, for the infographics. I had it do a video as well, because it does the video and audio podcasts.


Carolyn Ford: Yes, yes. I've heard many of those.


Sean Rugge: Yeah, it's gotten to the point where you can tell like, "Oh, this is done in LM," right?


Carolyn Ford: Yeah, yeah. But this is a great—I mean, this is a great graphic. But yeah, for a Marine to come in and not have the graphic skills to be able to create this, right? You know, being able to push that button and, with a few words, have NotebookLM do it. And this is something that is becoming available in genai.mil, I think later this year. It's scheduled to be released.


Carolyn Ford: What was your prompt? Sorry, I'm so impressed with this graphic right now. Listeners, you've got to go look at this graphic, because if you've tried to make graphics with text in AI, you know mine and Sean's pain. So what was your prompt to get it to generate this?


Sean Rugge: So, well, I took that PDF, and that's what I provided as the source material to NotebookLM. But I broke it into the chapters, so there was some manual work on my part to break that PDF into the individual chapters. And then when I created the graphic, I just selected the chapters that were responsible for this module as the source material, and then asked for some different graphics that were associated with the source material essentially, that I could use in an interactive book in a course.


Carolyn Ford: Okay. And we're experiencing—what you're sharing on your screen right now is the LMS, the Moodle LMS. This is the way you would deliver the course to the Marines, right?


Sean Rugge: Correct.


Carolyn Ford: Okay.


Sean Rugge: And so, you know, that's one of the areas where I was really looking to get some modernization in, because for me that's better than the PowerPoint presentation that the student is just told to download and look at, or the PDF document that they're told to download and read, right? And I've run into so many cases of military training where that's the case. Like, go get the PowerPoint. All right, you've accessed the PowerPoint, now you're trained. And maybe you had an instructor who ran you through the PowerPoint in a class, right? And to me, that doesn't capture nearly enough.


Carolyn Ford: Nope. That's not training, that's not learning.


Sean Rugge: Yeah. And so one of the more ambitious things that I had it do, I think it was in Module 4—yes, this was the branching scenario. And that's where I had to use a few different tools. And I was looking for some different source material. I used DVIDS, which is a multimedia web environment that people can get access to and download video and things like that.


Let me see if I can—there we go, yeah. And so this is one of the videos that was generated for that branching scenario.


Carolyn Ford: And you generated it in DVIDS, that's the tool?


Sean Rugge: DVIDS is where I got some of the source image content that I was using with a prompt to generate the video and other graphic content in genai.mil. Not in genai.mil—this was one of the shortfalls that I identified, where we need media capabilities. Since then though, we have Nano Banana as part of the genai.mil product, and they are incorporating the NotebookLM stuff. And I know they have video content generation coming later this year as well.


So, you know, I'm hopeful that—I'm sure they understood that we needed these capabilities, but I'm hopeful that, in some small way, this helped to raise awareness and get more people on board with, "Yeah, this is why people need it."


Carolyn Ford: Okay. Can we play a little bit of the video?


Sean Rugge: Sure.


Carolyn Ford: I'm not hearing sound, but does it have sound as well?


Sean Rugge: It does, it has a little bit of sound. And it's not narration; it's just sound effects that are associated with the video content.


Carolyn Ford: Yeah, that's super.


Sean Rugge: But to get that created, I used a tool called FlexClip. And that was something I had to pay for a subscription for briefly to be able to create that content.


Carolyn Ford: So these layers that you're building in all the other tools, eventually you pulled them into Moodle, into your LMS, to make one complete training course.


Sean Rugge: Yes, yeah.


Carolyn Ford: Okay, okay.


Sean Rugge: And so I had it build out different image content and different video content at certain decision points. But I was able to have—and this was kind of the nice thing about the agentic browser. I was using Comet, and I could give it the prompt of, "I need the skeleton of the branching scenario built out, and here are the decision points. Here's the entire branching scenario written out in text, and build in the decision points for me." And so I could fill in the media later, because it couldn't do files, but it could do text-based actions.


So that was a huge time—I mean, the creative energy it takes, and then just the testing and the iterating of, "Okay, well, is the decision working right, but is it also the right decision versus the set of wrong decisions that we also want there?" That all takes time to creatively build, right? So it was nice to have that agent that could take on a lot of those manual button-pushing actions and get me to a point of where I've got a prototype that I can review and then start putting in the bells and whistles of the multimedia.


Carolyn Ford: So you couldn't do that just in genai.mil or NotebookLM? I don't use the agentic side of things yet, so I don't totally understand why you needed to use the agentic browser and you couldn't just do it in...


Sean Rugge: Yeah, so I could just go in and push the buttons in the LMS, but it's extremely time-consuming, right?genai.mil can't produce H5P products.


Carolyn Ford: Oh, actually, okay, let's back up a little bit. What is an H5P product? Define that.


Sean Rugge: Thank you, yeah. It's an HTML5 package. So it's kind of what has taken over when Flash went away—animated elements, interactive elements. So really, it's helpful for producing microlearning types of activities students can engage with.


Carolyn Ford: Okay, so the agentic browser was able to interact with the LMS tool with Moodle to build this out? Am I getting that right?


Sean Rugge: Correct. Yes, you're exactly right. So I logged in as an administrator, then I went over to the prompt window. I can pull up a couple of those—I think I've got a... most of them have gone since my subscription expired, but I've got a couple here.


Carolyn Ford: How did you hook it into Moodle? It just has an API or...


Sean Rugge: In the browser, I just log in. And so you've got an assistant here, and you can give it a prompt: "Build a quiz for me." And if I'm on this page and I ask it to build a quiz, I might have to give it more specific instructions like, "Turn edit mode on, add a quiz activity using the activity chooser." I have to use terms that are specific to Moodle or building within this software, and that's kind of helpful for it. But if I'm really vague in the instructions, then I notice that it's not as responsive in the result, right? But yeah, this is an example.


Carolyn Ford: The more specific you can be, the more information you can give it, the better your results are going to be, right?


Sean Rugge: Exactly, yeah. And it's kind of like that—I don't know if you did it in elementary school, but I remember an elementary school project where teachers had us write out instructions for a peanut butter and jelly sandwich, and then we kind of followed different sets of instructions to... yes, chaos ensued. You definitely get some of that chaos if you don't give really good instruction. So the prompt is important, for sure.


Here's a couple of examples of where I asked it to build out in Module 3: "Create an interactive book." But yeah, go ahead.


Carolyn Ford: Well, this is Perplexity that you're in right now, but is it...


Sean Rugge: Yeah, it's not the agentic browser. This is Comet, this is Perplexity's Comet browser.


Carolyn Ford: That's what I'm telling you, Sean, I haven't touched the agentic stuff! So, okay.


Sean Rugge: Yeah, and this was the one that I actually saw like taking a quiz in a course and completing a discussion forum. So that's what drew me to it, and I was able to get an edu subscription for a couple of months and was able to use that and play with it while I was working on this project.


Carolyn Ford: So I see your prompt. It says, "Please continue creating the H5P interactive book in the content bank. Book should have instructional pages that cover the Module 3 learning objectives. The interactive book should have knowledge check questions that allow students to earn points as they go through the book and complete it for a grade." Which I want to just point out here that because you are a subject matter expert in instructional design, you know what to tell it to do. So from this prompt, show me what it produced for you.


Sean Rugge: Yes, so this is Module 3 here. So this was the output, this is what Comet produced for you.


Carolyn Ford: Okay, I see. I understand now when you say all the buttons that you need to push; I'm with you.


Sean Rugge: Yes, and to do this—I mean, as an instructor or instructional designer, you've got to turn on edit mode, you've got to go to the front page of the course, you've got to add the activity, you've got to type in all the text, you've got to know how to use the tool to do all of that.


Carolyn Ford: Definitely. And it just did it for you. Did you tell it, "I'm going to be using Moodle, so build it for Moodle," or you just told it H5P?


Sean Rugge: Um, I think in one of the earlier chat sessions, we established that it was Moodle. I think the first time I asked it to build an H5P item, it wasn't quite sure where, so I had to specify in the course content bank, "Create it there."So there was some back and forth in that regard, just getting my prompt right. And then realizing that, as the subject matter expert, I was kind of training the assistant. The assistant was kind of the staff sergeant; they didn't know how to use the software, they had the textual knowledge, right?


Carolyn Ford: Right.


Sean Rugge: They had the PDF, but they needed the instructions on, "Okay, well, what buttons do I push to build it?"And so I had to help it and gently nudge it along. But once we built one together, it was pretty easy to kind of say, "Hey, remember that one? Let's do it."


Carolyn Ford: That's right, that's right. Once you get it right for one. Okay, we're going to take a quick break, we'll be right back.


(Sponsor Announcement)


This episode is sponsored by Owl Cyber Defense, a pure-play cybersecurity company delivering made-in-the-USA data diode and cross-domain solutions trusted to protect some of the most sensitive government and commercial networks worldwide. Owl enables secure, near-instant collaboration across network boundaries, helping military, federal, and critical infrastructure organizations make faster, safer decisions. To learn more, visit owlcyberdefense.com.


Carolyn Ford: And we're back. I'm Carolyn Ford, this is Tech Transforms, and I'm here with Sean Rugge, senior educational technologist at Marine Corps University.


How did you decide that building this course, modernizing this 200-page PDF using AI was worth the effort, or if you could have just done this faster on your own?


Sean Rugge: Sure. There were probably times where I was going back and forth with the AI a little bit and wasn't quite getting the result that I wanted, and I thought, "You know, I could probably just click the buttons myself."


Carolyn Ford: Well, and I start to get confused because, man, the AI—it sounds like it knows what it's talking about, and it sounds really good, and it's so confident. But then you're like, "What? What the hell does that sentence even mean? Like, where did you even get this?"


Sean Rugge: Right, yeah. It is supremely confident in its hallucinations, for sure. So I've been kind of on this quest to quantify what learning is happening, what time is saved, what's the return on investment. And so the way I approached that was within the chat sessions that I was in, I asked that question of the AI tool that I was using, right? And then through that, I came up with—what we came up with together, my collaborative partner came up with—a standardized way to approach it for different scenarios.


And what I quickly realized is, okay, it's pretty good at just looking at raw text and starting to structure it, right? Taking unstructured data and structuring it and providing some analysis to it. My initial attempts at, "Hey, give me some numbers for how much time was saved," were met with wild hallucinations.


Carolyn Ford: Like, I've experienced the same thing! Like, "You saved a million dollars by using AI."


Sean Rugge: Yeah, "99% of all problems were solved."


Carolyn Ford: Sure.


Sean Rugge: "But the ones that we didn't solve, I didn't even present to you because, you know..." So I could tell it was hallucinating, and then I realized, "Okay, well, I've got to adjust my prompt. The question isn't—if you're giving me hallucinations, then I've got to adjust." And that's where the teacher in me started: "Let's show your work. You've got to have an equation behind that number, right? Show me that equation."


And so it became a little more forthcoming with the equations that were behind the analysis. And then I could see places where it was like "AI assessment" or basically the opinion of the AI—in the AI's opinion, this is the value. Okay, well, that's not quite good enough, let me make that better. So there was an iterative process to get that ROI value.


Carolyn Ford: Yeah, and again, because you are a subject matter expert in what you do, you were able to recognize the hallucination. And I like what you said at the very beginning, that you also collaborated on the training itself, the material, with a Marine who's been in these situations. So you brought in yet another—so you're the subject matter expert on instructional design, you brought in a subject matter expert on the course material. You were not relying on AI carte blanche, right?


Sean Rugge: Right, and nor should you. It can definitely help, but you've got to approach it as a member of the team, right?


Carolyn Ford: That's right, that's right. It's a really fast assistant—sometimes a really dumb one, but really confident.


Sean Rugge: Yes, right! And people are like that too sometimes, you know?


Carolyn Ford: So, same-same.


Sean Rugge: Right, and I keep coming back to that because I tend to do the opposite of anthropomorphize with the AI. Every sort of article that I read that's like, "Look what AI just did! How horrible is this?", I'm like, "Well, you know, people tend to do that, too."


Carolyn Ford: Well, and also, guess where AI got that? From you!


Sean Rugge: Yes, yes! Right, right, exactly.


Carolyn Ford: All right, well, let's talk about the learning intelligence dashboard that you created. What does it measure? What signals mattered most? And how does it help answer whether learning is actually happening instead of just checking the compliance box? And can you share the dashboard? Can you show it?


Sean Rugge: Yes, yeah. I think it would be helpful if I do.


Carolyn Ford: So you created—and tell me how you created the dashboard.


Sean Rugge: So this came from as an offshoot from this project, right? When I started to evaluate: okay, how much content did we produce together, how much time did we save by having the agentic browser perform the actions in the LMS? When I was trying to quantify that, I quickly realized: okay, it's looking at text iterations, me correcting the prompt and going back and forth and saying, "Oh, well, this isn't quite right. Let's fix it." And it was putting some metrics behind that. Here's an example of one of the dashboards that I applied to.


Carolyn Ford: And you created this after the course was done? You went into which tool to say, "Create this dashboard"?


Sean Rugge: So I used Claude primarily to create the dashboard.


Carolyn Ford: And did you give it your whole course, the whole course material, and say, "Create me a dashboard"?


Sean Rugge: Well, this was more focused. I'll give you—this is the one that I started with, and this is a dashboard of just: hey, let's evaluate the chat sessions that I'm having with AI and put some metrics behind that. And part of this was just me wanting to be able to demonstrate that you can use AI very effectively, and you can demonstrate some of the skill sets that are being shown within a chat session and start to quantify that.


So these are kind of some metrics that we put together behind that effort of just interacting with an AI chatbot, and building, prototyping, and revising. There's a cognitive depth score, a strategic thinking metric, a metacognition self-awareness of your own learning. There's a competency radar map, and these are derived competencies. So it was just looking at the context of the chat session and deriving competencies that were demonstrated within that session. There's a Bloom's taxonomy, a Bloom's progression.


And this is for one where—oh, so this is all aggregated. If I dig into one individually, you can see—and I wanted to see some variance, right? I wanted—there were some sessions where I knew I produced a lot more with the AI tool than others. But as an example, I pretended to be a student who was asking for help on math homework, and I couldn't really articulate the topic very well, and I kind of knew a little bit about what I wanted it to do, but wasn't really able to solve any problems together. And you can see the output is much different; it's much lower.


Carolyn Ford: This learning dashboard—you could use this not just with your training module, but with any AI chat session with your training module, is that right?


Sean Rugge: Correct, yeah. And this is really designed to look at students interacting with AI, and what are they getting and what are they producing out of that session?


Carolyn Ford: How do you plug it into what the student's doing? How do you integrate it?


Sean Rugge: So what I did was devise a prompt with Claude. We put together a prompt that does session analysis, and so it basically puts together a rubric and a set of rules for how to evaluate the session.


Carolyn Ford: Oh, and so you plug the prompt into the session that you have been in in whatever AI application you're using, and then it produces this?


Sean Rugge: It will give you a JSON output, so it gives you like the data that you would need to plug into this. So you could upload that file that it kicks out into this dashboard, which is an HTML page, and then it adds it.


Carolyn Ford: Wow, this is so cool. I'm going to put you on the spot, and we can edit this if we need to: would you be willing to share the prompt that you use to build this dashboard?


Sean Rugge: Sure! I've got it on my GitHub, I can share that with you.


Carolyn Ford: Thank you.


Sean Rugge: And then this one is one that I applied to where I saw text discussion in courses, and we use this frequently. Because I could get the dashboard from me interacting with a chatbot, I'm like, "Well, students are interacting with each other and an instructor, and can we evaluate that?" So this is something that came out of that: just looking at student discussion, and then performing essentially the same analysis on student discussion to look at the cognitive performance of either an individual student or as an aggregate class for a discussion topic.


Carolyn Ford: But it's discussion that's happening online, so you can... yeah, okay.


Sean Rugge: Yeah. And so what happens there is it aggregates the discussion, sends it off to a large language model to do the analysis, so that's kind of automated in the background. And then the large language model sends back the JSON that's needed for the dashboard to render.


Carolyn Ford: That is so cool. Well, what are some of the measurable outcomes or changes that you've seen since implementing AI-supported course design workflows? Time saved, better structure? How well—I guess I should ask, how much have you implemented?


Sean Rugge: So it varies, right? And we're still looking at different tools that can be used to help that staff sergeant, because that staff sergeant at the school—that's my primary audience there. I want to help them to be able to produce materials that are better than what they started with.


Carolyn Ford: Well, and just sharing your prompts with them and the methodology, now they can take these 200-page PDFs and you've given them a formula to turn them into this training.


Sean Rugge: Right, and we have others out there that are also producing similar prompts, which is really encouraging to see early adopters out there doing that and solving that problem. But in terms of the time saved, or the metric that I'm looking for—is not one that I really have, one that I can present on a dashboard for you, but it's giving that Marine that sense of hope that they can accomplish it, that they can move from PDF and PowerPoint to a modernized online course that's put together with some thought and some of the design methodology behind it.


Carolyn Ford: It is very empowering. You're going to make them as confident as these AI chatbots!


So here's a question for you: what do you do about students that use the AI to game the system? I mean, they use agentic AI to go take the course for them.


Sean Rugge: Yeah, I'm glad you asked that. You've got to design your course in a way that encourages use of AI. And one of the things that I was looking for in terms of the Bloom's progression is: what products are we creating, what are we revising, how quickly are we getting to building something? And I think assignments—the asks of students—need to shift from, "Hey, memorize these questions and be able to respond to this quiz," to build and task AIs with building, and come up with a solution to a problem, right? I think that's where learning really has to shift, right?


Carolyn Ford: So, and that's critical thinking. That's creative thinking on the part of the human being, right? What to ask AI to do.


Sean Rugge: And not just that, it's the management of it. Because I've got one slide here that I did want to share—let me share this real quick. And this is kind of one of the cornerstones of the presentation, was that I had a whole development squad that I was managing, right? And so as the ISD or the project manager essentially, I had all these agents at my disposal that were helping me complete the project.


Carolyn Ford: Yeah, these are all your—this is your staff! You've got genai.mil, M365 Copilot, FlexClip, NotebookLM, Perplexity, Comet. So five different staff members essentially, virtual staff members, helping you build this, right?


Sean Rugge: Right, and I task them each with different things to accomplish, and I was kind of the orchestrator at that point and the project manager. And we need to put more students in this position so that they can learn how to use AI effectively.


Carolyn Ford: I love that. When I first started teaching—it was a long time ago, but I think this is still a thing—where, you know, worksheets... to learn spelling, you'd have to write the same word over 10 times or whatever, fill-in-the-blank worksheets. I hated them, and as a teacher, I refused to have my kids do them. Instead, what I had them do was—they could read, and the only rules around reading were no pornography and nothing like ultra-gory or violent. And I had them write. I had them write, and I didn't care what they wrote about—no pornography, nothing ultra-violent, you know, basic rules like that—but I had them write their story, I had them write about their day. That's how I had them learn. And just what you said about these Marines learning this critical thinking, it reminded me of that: you have to—they want to learn these creative, critical thinking skills.


Sean Rugge: Yeah, and there's an emphasis on having the thinking Marine, right?


Carolyn Ford: Yeah. So what's interesting about what you just said about the thinking Marine... I have been told that in the military, it has in the past been encouraged to—your brains belong in a foot locker, and your leaders will tell you what to do. Like, have you heard that? Put your brains in the foot locker.


Sean Rugge: Yeah, "If I want your opinion, I'll give it to you."


Carolyn Ford: So have we moved away from that a little bit, or just in your area?


Sean Rugge: It's a balance, right? It's a balance, because I think when you're in a situation where you have the fog-of-war element, you need to be able to have that creative thinking that's instantaneous, that you know how to solve a problem instinctively. But you're going to be faced with some tough problems to solve, and being able to have that element of critical thinking is critically important.


Carolyn Ford: Yeah, and I hear you where you say it's a balance; you also need to be able to follow a command.


Sean Rugge: Sure.


Carolyn Ford: That's part—I think it's part of the critical thinking, right? Knowing when to, and when you've got to step aside and say, "Let's think about this."


Yeah, I like that. Okay, we're going to go to our Tech Talk questions. This is rapid-fire questions, just from the gut. So, first question: in course development, which task should never be handed fully to AI?


Sean Rugge: I would say the deployment before review. I mean, and I know that's going to happen—I hope that it's not going to happen here, but I feel like it's going to happen out there in the world. But yeah, you have to review it. You have to stay in the loop and catch that Chapter 11 that was missed, right?


Carolyn Ford: Right. All right, would you rather have a perfect prompt library or a really good human instructional designer?


Sean Rugge: So I'm going to say yes to that question.


Carolyn Ford: I agree, I concur.


Sean Rugge: I'd like both of them. I want my cake and I want to eat it, too.


Carolyn Ford: I'm with you, Sean. I don't know how I would—I suppose if I were forced to choose, I'm going to go with the human, I'm going to go with you. So, all right.


What is something that you are reading—newsletter, blog, Substack, whatever it is—that you actually read instead of just letting it pile up in your inbox?


Sean Rugge: So I'm going to say a lot of the stuff that Nvidia is putting out lately. And it's not so much reading material; it's like they have a lot of really good webinars that have been really thought-provoking.


Carolyn Ford: Okay, I have not followed any of their stuff. So see, these questions—again, this is all for me! So, last question, also for me: what's the last book, article, or podcast that made you immediately want to try something new in your workflow?


Sean Rugge: I thought about this one. I'm going to go back to the Nvidia webinars, but as far as a book, I'm going to throw one book out there, and it's a book I read as a child, but a book I also taught as an adult: The Phantom Tollbooth. And I don't know if you're familiar with the book.


Carolyn Ford: I'm not. I'm an English teacher!


Sean Rugge: Oh my gosh! Okay, well then you've got homework; I'm giving you homework. Read the book, it's incredible. I don't want to spoil it for you, but it's about learning, right?


Carolyn Ford: The Phantom Tollbooth.


Sean Rugge: The Phantom Tollbooth by Norton Juster.


Carolyn Ford: Okay, I am so excited! I love new reading material. Sean, thank you so much for joining us today. This has been so much fun for me, and I really appreciate your time.


Sean Rugge: Thank you, I really appreciate the opportunity. Thank you, Carolyn.


Carolyn Ford: Where can our listeners follow you?


Sean Rugge: You can find me on LinkedIn. Hit me up on there.


Carolyn Ford: All right, we'll drop your LinkedIn handle in the show notes. So, thank you.


Thanks for tuning in. If you found this episode valuable, be sure to share it, leave a review, and smash that like button to help us reach more people who could benefit from the conversation.


I'm Carolyn Ford. Tech Transforms is produced by Show and Tell and sponsored by Owl Cyber Defense. Until next time, stay curious and keep imagining the future.