Pushkin. There's a good chance you've heard of con Academy about how Salcon started out more than a decade ago tutoring his young cousins and then started posting simple tutoring videos on YouTube, like how to solve a quadratic equation, that kind of thing. He founded con Academy in two thousand and eight, and it grew into this big thing beloved by Bill Gates and the Ted talk set. Today, con Academy has tens of millions of monthly users and provides not just those YouTube videos, but thousands of practice questions across math and science and the humanities, along with software that monitors students progress. All this has been in service of a big idea using technology to give everyone the huge benefit that Salcon's cousins got from one on one tutoring. But in spite of Salza, the con Academy has not been able to fully deliver on that idea. The technology just hasn't been there to match what a human can do. But now sal thinks that may be about to change. Earlier this year, con Academy launched something they call Conmego, an AI tutor built on top of GPT four. I'm Jacob Goldstein, and this is what's your problem. My guest today is Sal Con. He's the founder and CEO of con Academy. Sal's problem is this, how do you use AI to bring students the benefit of working with a human tutor. Sal says his real work with AI started last summer when he heard from the co founders of the company that built.

Pushkin. There's a good chance you've heard of con Academy about how Salcon started out more than a decade ago tutoring his young cousins and then started posting simple tutoring videos on YouTube, like how to solve a quadratic equation, that kind of thing. He founded con Academy in two thousand and eight, and it grew into this big thing beloved by Bill Gates and the Ted talk set. Today, con Academy has tens of millions of monthly users and provides not just those YouTube videos, but thousands of practice questions across math and science and the humanities, along with software that monitors students progress. All this has been in service of a big idea using technology to give everyone the huge benefit that Salcon's cousins got from one on one tutoring. But in spite of Salza, the con Academy has not been able to fully deliver on that idea. The technology just hasn't been there to match what a human can do. But now sal thinks that may be about to change. Earlier this year, con Academy launched something they call Conmego, an AI tutor built on top of GPT four. I'm Jacob Goldstein, and this is what's your problem. My guest today is Sal Con. He's the founder and CEO of con Academy. Sal's problem is this, how do you use AI to bring students the benefit of working with a human tutor. Sal says his real work with AI started last summer when he heard from the co founders of the company that built.

GPT About a year ago. Summer of twenty twenty two, I get an email from Sam Altman and Greg Brockman, the founders of open AI, who I knew, you know, I bumped into them at different events and things like that, and they said, hey, you know, we're working on our next model. If we would love to talk to you all about a few, you know, things that we might be able to collaborate on. I was skeptical that it would really have any implications for con Academy, but as a nerd, I said, Oh, this must be GPT four that they must be working on. And I have a lot of respect for Greg and Sam what they had already accomplished, so I was like, yeah, just I'm just curious. Yeah, let's meet is Actually at that point hadn't finished training what would eventually be GPT four, but they were about two weeks away from it, but they said, look, we think this is going to be the model, the generative AI that really opens people's minds to what's possible here. And because of that, it's going to be exciting and a little bit scary, and so we want to launch with some social positive use cases. The first one that came to our mind is education, and the first organization that came to our mind is Kin Academy, because y'all.

GPT About a year ago. Summer of twenty twenty two, I get an email from Sam Altman and Greg Brockman, the founders of open AI, who I knew, you know, I bumped into them at different events and things like that, and they said, hey, you know, we're working on our next model. If we would love to talk to you all about a few, you know, things that we might be able to collaborate on. I was skeptical that it would really have any implications for con Academy, but as a nerd, I said, Oh, this must be GPT four that they must be working on. And I have a lot of respect for Greg and Sam what they had already accomplished, so I was like, yeah, just I'm just curious. Yeah, let's meet is Actually at that point hadn't finished training what would eventually be GPT four, but they were about two weeks away from it, but they said, look, we think this is going to be the model, the generative AI that really opens people's minds to what's possible here. And because of that, it's going to be exciting and a little bit scary, and so we want to launch with some social positive use cases. The first one that came to our mind is education, and the first organization that came to our mind is Kin Academy, because y'all.

Want they want you to be the warm, fuzzy, happy face of AI.

Want they want you to be the warm, fuzzy, happy face of AI.

Yeah, to see if we'd be interested to see and they want you know, they don't want to just be optics. I think they generally want AI to have warm, fuzzy applications. I believe, of course, the dark and sordid applications that.

Yeah, to see if we'd be interested to see and they want you know, they don't want to just be optics. I think they generally want AI to have warm, fuzzy applications. I believe, of course, the dark and sordid applications that.

They clearly do, right, They clearly recognized the downside.

They clearly do, right, They clearly recognized the downside.

Yeah, exactly. But the other reason they actually reached out to us and I connected the dots later, they said, you know, we really want it to be good at academic things like That's where GPT three it really had no solid handle on knowledge, and we think GPT four will and y'all have a lot of items we were just talking about how deep of an item bank we have it academy, across subjects and grades. And they were particularly interested in ap bio, which I later learned from Bill Gates. It was because when he saw GPT three he said he essentially told him, look, this is cute, guys, but I'll be impressed if this could pass an ap biology exam. So I think Greg and Sam literally said, Okay, you know, Bill Gates, that's a pretty good benchmark. Can this pass the ap bile exam? So I think that's the other thing. They said, hey, can we use some of your items to test to evaluate the model.

Yeah, exactly. But the other reason they actually reached out to us and I connected the dots later, they said, you know, we really want it to be good at academic things like That's where GPT three it really had no solid handle on knowledge, and we think GPT four will and y'all have a lot of items we were just talking about how deep of an item bank we have it academy, across subjects and grades. And they were particularly interested in ap bio, which I later learned from Bill Gates. It was because when he saw GPT three he said he essentially told him, look, this is cute, guys, but I'll be impressed if this could pass an ap biology exam. So I think Greg and Sam literally said, Okay, you know, Bill Gates, that's a pretty good benchmark. Can this pass the ap bile exam? So I think that's the other thing. They said, hey, can we use some of your items to test to evaluate the model.

So they wanted you have like whatever, thousands of questions about all these different subjects kind of test questions, test like questions about ap biology and calculus and lots of other subjects, and they wanted to train the model on your questions because you have the questions and the answers and exactly what you would want if you were training an AI model on sort of high school college level knowledge.

So they wanted you have like whatever, thousands of questions about all these different subjects kind of test questions, test like questions about ap biology and calculus and lots of other subjects, and they wanted to train the model on your questions because you have the questions and the answers and exactly what you would want if you were training an AI model on sort of high school college level knowledge.

Well, it's interesting out of the gate, they were actually more interested in just on the evaluation side. So once you produce a model, you want to see how good it is so you literally make it take questions and see how many gets right. So I was like, yeah, you know, maybe you know, I wasn't sure if this was really going to be something we could use, but I still wanted to see. I still wanted to see the demo when I was available. So two weeks later they said, oh, we finished training, we'd love to show you a demo. And I remember that meeting they put up and you know, this isn't that mind blowing the folks now because they've gotten used to this. But in summer of twenty twenty two, we were on a video conference and they on Greg's screen. He showed an ap bio question and he said, sah, what do you think the answer is? And I like read the question was about osmosis. I'm like, oh c. And then they asked the AI. I said yeah, see, I'm like, okay, that's kind of cool. And then I said, well, you know, ask it why is? The answer explained it very well, and that's when I was like, wow, that is interesting. I said, well, why aren't the other choices? The answer explained it very well. I'm like okay, yeah, this is getting.

Well, it's interesting out of the gate, they were actually more interested in just on the evaluation side. So once you produce a model, you want to see how good it is so you literally make it take questions and see how many gets right. So I was like, yeah, you know, maybe you know, I wasn't sure if this was really going to be something we could use, but I still wanted to see. I still wanted to see the demo when I was available. So two weeks later they said, oh, we finished training, we'd love to show you a demo. And I remember that meeting they put up and you know, this isn't that mind blowing the folks now because they've gotten used to this. But in summer of twenty twenty two, we were on a video conference and they on Greg's screen. He showed an ap bio question and he said, sah, what do you think the answer is? And I like read the question was about osmosis. I'm like, oh c. And then they asked the AI. I said yeah, see, I'm like, okay, that's kind of cool. And then I said, well, you know, ask it why is? The answer explained it very well, and that's when I was like, wow, that is interesting. I said, well, why aren't the other choices? The answer explained it very well. I'm like okay, yeah, this is getting.

Kind of wow. Right, that's still like I know, it's not really cognition. Maybe it doesn't understand, but something is happening there, right, something something well.

Kind of wow. Right, that's still like I know, it's not really cognition. Maybe it doesn't understand, but something is happening there, right, something something well.

I think it's starting to make all of us start to parse words, like we start to realize that like sentience and cognition and intelligence are not necessarily the in fact, they aren't same thing, and we can separate these things. Something can be intelligent. And I think it's going to challenge our words, our semantics for some of these things. But then I said, hey, can it can write another question like this? And it did it, and I'm like that's a pretty good question. And then I said write ten more questions like this, and it did it. And you know, at the moment.

I think it's starting to make all of us start to parse words, like we start to realize that like sentience and cognition and intelligence are not necessarily the in fact, they aren't same thing, and we can separate these things. Something can be intelligent. And I think it's going to challenge our words, our semantics for some of these things. But then I said, hey, can it can write another question like this? And it did it, and I'm like that's a pretty good question. And then I said write ten more questions like this, and it did it. And you know, at the moment.

At your moment where you were like, oh, no, AI is coming from my job, I kind.

At your moment where you were like, oh, no, AI is coming from my job, I kind.

Of thought that a little bit. But then what really opened my mid They're like, oh, well, would you want access to it and to at least try it out see how you might be able to use it. And I was like, yeah, I want access. I want to try out this thing, and so myself and actually eventually our whole organization got under a non disclosure agreement with open ai back back in August, and we started testing it in that first weekend, and I remember it was myself in.

Of thought that a little bit. But then what really opened my mid They're like, oh, well, would you want access to it and to at least try it out see how you might be able to use it. And I was like, yeah, I want access. I want to try out this thing, and so myself and actually eventually our whole organization got under a non disclosure agreement with open ai back back in August, and we started testing it in that first weekend, and I remember it was myself in.

The August twenty twenty two.

The August twenty twenty two.

Yes, myself, our Chief Learning Officer Christen, and our CTO Paul were the first people to have access to and I think we even got access to it before many folks that open Ai access to it, and we were just playing around with it, and we were also slacking with the open AI team. We're like, hey, you have any ideas. We're trying to get it to do this or we got it closed, but it's still and they were giving us some really good tips and by the end of that weekend we had it being able to take on personas and modeling you know, pretty good tutor behavior. And that's when it really when I started saying, Okay, this is a this is a game changer. This is science fiction has talked about artificially intelligent tutors forever, most famously Diamond Age Young Ladies. Illustrated primer Neil Stevenson wrote about this in the nineteen nineties about you know, an intelligent tutor being able to educate essentially. I mean, this book was set in the not too far future and it educates all these young girls in China living on barges were orphans and then they take over the world because this tutor was able to empower them so much so.

Yes, myself, our Chief Learning Officer Christen, and our CTO Paul were the first people to have access to and I think we even got access to it before many folks that open Ai access to it, and we were just playing around with it, and we were also slacking with the open AI team. We're like, hey, you have any ideas. We're trying to get it to do this or we got it closed, but it's still and they were giving us some really good tips and by the end of that weekend we had it being able to take on personas and modeling you know, pretty good tutor behavior. And that's when it really when I started saying, Okay, this is a this is a game changer. This is science fiction has talked about artificially intelligent tutors forever, most famously Diamond Age Young Ladies. Illustrated primer Neil Stevenson wrote about this in the nineteen nineties about you know, an intelligent tutor being able to educate essentially. I mean, this book was set in the not too far future and it educates all these young girls in China living on barges were orphans and then they take over the world because this tutor was able to empower them so much so.

Sidebar Neil Stevenson amazingly prescient, right like he's the like president guy for this, he's the president guy for for crypto, right like it's weird, right.

Sidebar Neil Stevenson amazingly prescient, right like he's the like president guy for this, he's the president guy for for crypto, right like it's weird, right.

Oh, he's pretty good. And he wrote, you know, diamond Age was nineteen ninety four, so yeah, you know, almost as an aside, it was a tablet app so he was prescient for tablet and mobile. But but that's when when I when I saw that it could take on personas and act as a tutor and not superficially but actually do some things that are I would view like actually quite thoughtful as a tutor, I said, Wow, we I think we're at the cusp of something here. And I didn't know if it was going to happen in like three months or three years, but I'm like, we've got we've got to work on this.

Oh, he's pretty good. And he wrote, you know, diamond Age was nineteen ninety four, so yeah, you know, almost as an aside, it was a tablet app so he was prescient for tablet and mobile. But but that's when when I when I saw that it could take on personas and act as a tutor and not superficially but actually do some things that are I would view like actually quite thoughtful as a tutor, I said, Wow, we I think we're at the cusp of something here. And I didn't know if it was going to happen in like three months or three years, but I'm like, we've got we've got to work on this.

Just to pause there and kind of go broader, Like the big idea behind con Academy the whole time, right since its inception whatever what more than a decade ago now is the power of a tutor right, that like a tutor is profoundly valuable, and that's sort of empirically clear, right.

Just to pause there and kind of go broader, Like the big idea behind con Academy the whole time, right since its inception whatever what more than a decade ago now is the power of a tutor right, that like a tutor is profoundly valuable, and that's sort of empirically clear, right.

That's exactly right. It all got started back in two thousand and four with me tutoring cousins just on the side, one cousin Nada needed help, and then I started tutoring brothers. Wordspreads in my family, free tutoring is going on, and I saw it with my own cousins just on an anecdotal level, that everyone that I was tutoring, I was able to put in, you know, thirty forty minutes a day with all of them collectively, it was it was dramatically accelerating them. And so to a large degree, when I started making exercises and software for my cousins and then eventually making videos for my cousins that obviously many many more folks ended up using, I was always in the mindset of how can I help scale my tutoring, How can I start making that tutoring a little bit more self service so that my cousins and eventually other people could help themselves. So that's been the journey of coin Academy for the last I mean it's been almost it's been nineteen years. Since I started tutoring my cousins, and so this technology held the potential to take it that much further.

That's exactly right. It all got started back in two thousand and four with me tutoring cousins just on the side, one cousin Nada needed help, and then I started tutoring brothers. Wordspreads in my family, free tutoring is going on, and I saw it with my own cousins just on an anecdotal level, that everyone that I was tutoring, I was able to put in, you know, thirty forty minutes a day with all of them collectively, it was it was dramatically accelerating them. And so to a large degree, when I started making exercises and software for my cousins and then eventually making videos for my cousins that obviously many many more folks ended up using, I was always in the mindset of how can I help scale my tutoring, How can I start making that tutoring a little bit more self service so that my cousins and eventually other people could help themselves. So that's been the journey of coin Academy for the last I mean it's been almost it's been nineteen years. Since I started tutoring my cousins, and so this technology held the potential to take it that much further.

Yeah, so you're playing with this thing. It's very early. The world doesn't even know about GPT for yet. What what are you doing? What are you working on? Within con Academy.

Yeah, so you're playing with this thing. It's very early. The world doesn't even know about GPT for yet. What what are you doing? What are you working on? Within con Academy.

At this point we started to see, Wow, this could just could really work, This could be really powerful, and we started figuring out ways to minimize some of the rough spots of Jenai, like hallucinations where it can make up things like it's making math errors, which is obviously a problem if we want it to be a math tutor. So we started going working through that. At the same time, we started having some pretty intense debates inside of our organization. Roughly speaking, half the organization was like, this is the most important technological advancement ever or at least in our lifetimes. We've got to go all in on this, like it's our duty. And then the other half of the team, not that they disagreed, but they said, look, kind academy stands for a lot it and we're there to help students, and if we go out there with something that's either not well baked or it leads to something that's suspicious or shady or just scary for folks. Because JENAI could be scary, it's going to be bad for us. So we started having those debates. But within a couple of months, as we were just kept prototyping, well, two things happened. One is I think a consensus. I don't know if it was a consensus, but I started to drive alignment around the idea like, these these fears and these risks that people are articulating are real. We should not ignore them. But there's ways to turn them into features that actually not only mitigate risks, but they actually can be enhancements. So, for example, we said, well, what if on what eventually would be konmego we didn't call it. Then, then all the sessions of a student who's under eighteen are recorded and accessible by parents or teachers. What if we have a second AI that moderates the conversations, and if the conversations go anywhere shady can actively notify the parent or teacher. What if our AI doesn't, It doesn't just give you the answer. Even before you know this was before chat GPT came out, we knew that this could be used for cheating that the raw technology could. We said, we're not going to do that, but it can, it can support you. So we started thinking through all of these what you know, we aren't going to make We aren't going to use information to train the AI, at least in this stage where people aren't sure how it might go. We weren't going to let any personally identifiable information go between the student and the artificially intelligent model. And so in a lot of ways, it's like we're gonna put more safeguards on jen AI than frankly there exist on the Internet, and when kids are just randomly on the Internet.

At this point we started to see, Wow, this could just could really work, This could be really powerful, and we started figuring out ways to minimize some of the rough spots of Jenai, like hallucinations where it can make up things like it's making math errors, which is obviously a problem if we want it to be a math tutor. So we started going working through that. At the same time, we started having some pretty intense debates inside of our organization. Roughly speaking, half the organization was like, this is the most important technological advancement ever or at least in our lifetimes. We've got to go all in on this, like it's our duty. And then the other half of the team, not that they disagreed, but they said, look, kind academy stands for a lot it and we're there to help students, and if we go out there with something that's either not well baked or it leads to something that's suspicious or shady or just scary for folks. Because JENAI could be scary, it's going to be bad for us. So we started having those debates. But within a couple of months, as we were just kept prototyping, well, two things happened. One is I think a consensus. I don't know if it was a consensus, but I started to drive alignment around the idea like, these these fears and these risks that people are articulating are real. We should not ignore them. But there's ways to turn them into features that actually not only mitigate risks, but they actually can be enhancements. So, for example, we said, well, what if on what eventually would be konmego we didn't call it. Then, then all the sessions of a student who's under eighteen are recorded and accessible by parents or teachers. What if we have a second AI that moderates the conversations, and if the conversations go anywhere shady can actively notify the parent or teacher. What if our AI doesn't, It doesn't just give you the answer. Even before you know this was before chat GPT came out, we knew that this could be used for cheating that the raw technology could. We said, we're not going to do that, but it can, it can support you. So we started thinking through all of these what you know, we aren't going to make We aren't going to use information to train the AI, at least in this stage where people aren't sure how it might go. We weren't going to let any personally identifiable information go between the student and the artificially intelligent model. And so in a lot of ways, it's like we're gonna put more safeguards on jen AI than frankly there exist on the Internet, and when kids are just randomly on the Internet.

That's a pretty low bar. You definitely want to be higher than random, let's be honest.

That's a pretty low bar. You definitely want to be higher than random, let's be honest.

Yeah, we made a much higher because we knew this was gonna kind of you know, people were going to have mixed feelings about it.

Yeah, we made a much higher because we knew this was gonna kind of you know, people were going to have mixed feelings about it.

A lot of ways it could go wrong. There's a lot of ways. That's right.

A lot of ways it could go wrong. There's a lot of ways. That's right.

That's right. Then, I would say the other big thing that happened end of November, chat GPT comes out, and that now that captured everyone's imagination and we all remember those first few weeks and months where people were you know, everyone was getting on chat GPT and taking screenshots and putting it on social media of what it was doing. And it was doing some amazing things. It was doing some very imperfect things too, the hallucinations, the math, theres, et cetera. I was worried initially because the narrative and education immediately became this thing as error prone, this thing can be used to cheat. It's the end of term papers, homework, et cetera. School district started banning chat GPT. I was like, oh no, we're working so much on this, and GPT four is so much better, and what was eventually going to become conmego was so much better, Like I hope the baby doesn't get thrown out with the bathwater. In hindsight, that was a good thing because it made us even internally say, look, the genie's out of the bottle. Now it's how we use the genie, Like we're going to be a force for good. Hopefully we have to work feverishly to show that it can be used well. And then I think by the time GPT four came out and Conmego came out with it in March, the education system and frankly society had a chance to process it and they had come around like, well, the technology isn't bad. We just need reasonable guardrails and we just need tools that are built for the actual use cases. And then we were able to show up with like you mean like this, you mean like con migo, and then most people and yes, exactly like that. And now we're seeing, I mean honestly, more schools and districts and parents want it for their students and their children than we can currently handle.

That's right. Then, I would say the other big thing that happened end of November, chat GPT comes out, and that now that captured everyone's imagination and we all remember those first few weeks and months where people were you know, everyone was getting on chat GPT and taking screenshots and putting it on social media of what it was doing. And it was doing some amazing things. It was doing some very imperfect things too, the hallucinations, the math, theres, et cetera. I was worried initially because the narrative and education immediately became this thing as error prone, this thing can be used to cheat. It's the end of term papers, homework, et cetera. School district started banning chat GPT. I was like, oh no, we're working so much on this, and GPT four is so much better, and what was eventually going to become conmego was so much better, Like I hope the baby doesn't get thrown out with the bathwater. In hindsight, that was a good thing because it made us even internally say, look, the genie's out of the bottle. Now it's how we use the genie, Like we're going to be a force for good. Hopefully we have to work feverishly to show that it can be used well. And then I think by the time GPT four came out and Conmego came out with it in March, the education system and frankly society had a chance to process it and they had come around like, well, the technology isn't bad. We just need reasonable guardrails and we just need tools that are built for the actual use cases. And then we were able to show up with like you mean like this, you mean like con migo, and then most people and yes, exactly like that. And now we're seeing, I mean honestly, more schools and districts and parents want it for their students and their children than we can currently handle.

So you mentioned some of the guardrails you built into it, you know, not accepting personal information, having a second AI monitoring for anything shady, not just giving answers. There's another piece of the process of building con Migo that I've heard you discuss elsewhere that is really interesting to me. And that's that's about having the AI sort of think about its answers, think about its responses. Right, I probably haven't articulated that well, but you can so tell me about that piece of it.

So you mentioned some of the guardrails you built into it, you know, not accepting personal information, having a second AI monitoring for anything shady, not just giving answers. There's another piece of the process of building con Migo that I've heard you discuss elsewhere that is really interesting to me. And that's that's about having the AI sort of think about its answers, think about its responses. Right, I probably haven't articulated that well, but you can so tell me about that piece of it.

As you can imagine, one of the hardest things to resolve was that even GPT four, which is dramatically better than GPT three point five at math, it was making an uncomfortable number of math errors, especially when it came to being in the tutoring use case where let's say there's an algebra problem and I the student take a step and maybe I distribute it, distribut use the distributive property and correctly will it recognize it? Will it not? How will it provide that feedback to the student? And out the box, it wasn't doing it that well. It was making mistakes a lot, unacceptably often. Then we had a it's.

As you can imagine, one of the hardest things to resolve was that even GPT four, which is dramatically better than GPT three point five at math, it was making an uncomfortable number of math errors, especially when it came to being in the tutoring use case where let's say there's an algebra problem and I the student take a step and maybe I distribute it, distribut use the distributive property and correctly will it recognize it? Will it not? How will it provide that feedback to the student? And out the box, it wasn't doing it that well. It was making mistakes a lot, unacceptably often. Then we had a it's.

Really bad if the tutor is getting the problem wrong.

Really bad if the tutor is getting the problem wrong.

No, tutor, Yeah, that's not acceptable. And then an open AI researcher gave us an idea, which is, instead of just having the AI respond immediately to the student, instead, what if you were to have the AI essentially on its own, not show this part to the student, but generate what it thinks could have been reasonable responses for the student HU and then use that plus the conversation with the student to then respond to the student.

No, tutor, Yeah, that's not acceptable. And then an open AI researcher gave us an idea, which is, instead of just having the AI respond immediately to the student, instead, what if you were to have the AI essentially on its own, not show this part to the student, but generate what it thinks could have been reasonable responses for the student HU and then use that plus the conversation with the student to then respond to the student.

Yeah, and we did this thingly next level, right, like a teacher would always be doing that, but there's no reason to think that just a raw large language model would be right.

Yeah, and we did this thingly next level, right, like a teacher would always be doing that, but there's no reason to think that just a raw large language model would be right.

That's right, But you know what's interesting about this. We did it and it dramatically improved the math. And then we started tweaking it more and more, and we got it better and better. Now we do a bunch of fancy stuff along along those lines to get it a lot better. But then we realize, to your point, that's exactly what a teacher would do. So that's what con Migo does. It works on it on its own and its own scratch paper, so to speak. Compares the student's response if they got something different. Con Migo doesn't assume the student is wrong. It says, because con Migo can sometimes be wrong even on its own work, It says, hey, I got something different, can you explain your reasoning? And then when the student explains the reasoning, that's really good for large language models. It's actually able to understand that. And what's interesting, not only is that very pedagogically good, we've been getting a strong a lot of feedback from students that they really appreciate that type of an interaction. It eerily feels human that this thing you were so used to computers being so perfect, and it's like you're wrong, you're right here, you know, that's what a robot would do, but no, this is what a tutor would do. It's like, hey, I didn't get the same thing. Let's work through it together. Let's see who made the mistake.

That's right, But you know what's interesting about this. We did it and it dramatically improved the math. And then we started tweaking it more and more, and we got it better and better. Now we do a bunch of fancy stuff along along those lines to get it a lot better. But then we realize, to your point, that's exactly what a teacher would do. So that's what con Migo does. It works on it on its own and its own scratch paper, so to speak. Compares the student's response if they got something different. Con Migo doesn't assume the student is wrong. It says, because con Migo can sometimes be wrong even on its own work, It says, hey, I got something different, can you explain your reasoning? And then when the student explains the reasoning, that's really good for large language models. It's actually able to understand that. And what's interesting, not only is that very pedagogically good, we've been getting a strong a lot of feedback from students that they really appreciate that type of an interaction. It eerily feels human that this thing you were so used to computers being so perfect, and it's like you're wrong, you're right here, you know, that's what a robot would do, but no, this is what a tutor would do. It's like, hey, I didn't get the same thing. Let's work through it together. Let's see who made the mistake.

In a minute, how conmego compares to GPT today and what it might be like a few years from now. When I was getting ready for this interview, I played around with con Migo and chat GPT. I put them side by side in different tabs in my browser, and I asked each one a basic calculus question, how do you take a derivative? This is a thing that I knew how to do a long time ago, but I forgot also a long time ago, so it seemed like a good test, and it was striking how different the response is were GVT gave me something like the Wikipedia entry about derivatives, a bunch of text with some rules, some equations, and then I tried it on conmego, and first it asked me what I knew, and I said, I knew algebra. And then it explained one rule, a sort of first rule, the power rule for finding derivatives, and it gave me a problem to try, and I got that problem wrong, and it sort of asked me what I was thinking, and I tried it again and I got it right. And then Konnigo asked if I wanted to try another problem, and I said yes, and it gave me another practice problem. So clearly this is a very different experience than the plain vanilla chat GPT. But it was still a little bit awkward and a little bit hard to follow. And when I closed the tab and came back, it didn't remember our previous conversation. So overall, very good, but not quite there yet. I asked sal if that seemed about right to him.

In a minute, how conmego compares to GPT today and what it might be like a few years from now. When I was getting ready for this interview, I played around with con Migo and chat GPT. I put them side by side in different tabs in my browser, and I asked each one a basic calculus question, how do you take a derivative? This is a thing that I knew how to do a long time ago, but I forgot also a long time ago, so it seemed like a good test, and it was striking how different the response is were GVT gave me something like the Wikipedia entry about derivatives, a bunch of text with some rules, some equations, and then I tried it on conmego, and first it asked me what I knew, and I said, I knew algebra. And then it explained one rule, a sort of first rule, the power rule for finding derivatives, and it gave me a problem to try, and I got that problem wrong, and it sort of asked me what I was thinking, and I tried it again and I got it right. And then Konnigo asked if I wanted to try another problem, and I said yes, and it gave me another practice problem. So clearly this is a very different experience than the plain vanilla chat GPT. But it was still a little bit awkward and a little bit hard to follow. And when I closed the tab and came back, it didn't remember our previous conversation. So overall, very good, but not quite there yet. I asked sal if that seemed about right to him.

Yeah, I think that's pretty accurate. I think where it is today, it's like if you really wanted to learn how to take a derivative, or if you're learning calculus for the first time, I would say, go to the calculus course on kon Academy, start watching some of those videos, do those practice And what con migo is going to be really good at is if those videos, or the practice problems, or the articles we have or the hints we provide, there's still some itchy conceptual thing you're not getting. Con Migo's really good at trying to unlock that one conceptual thing.

Yeah, I think that's pretty accurate. I think where it is today, it's like if you really wanted to learn how to take a derivative, or if you're learning calculus for the first time, I would say, go to the calculus course on kon Academy, start watching some of those videos, do those practice And what con migo is going to be really good at is if those videos, or the practice problems, or the articles we have or the hints we provide, there's still some itchy conceptual thing you're not getting. Con Migo's really good at trying to unlock that one conceptual thing.

Like what I think really kind of a narrow problem, like you're almost there, but you just need like a little one more explanation or a different kind of explanation or something.

Like what I think really kind of a narrow problem, like you're almost there, but you just need like a little one more explanation or a different kind of explanation or something.

Exactly, or there's just some conceptual dimension that maybe the video didn't address that you're curious about or you want to connect it. You're learning about entropy and chemistry, and you're like, I've heard this word in computer science, how are they similar? Con Migo is great for that. I don't think conmego by itself is a place where you would just say, start being my calculus tutor for the next year year and you're gonna work through calculus with me.

Exactly, or there's just some conceptual dimension that maybe the video didn't address that you're curious about or you want to connect it. You're learning about entropy and chemistry, and you're like, I've heard this word in computer science, how are they similar? Con Migo is great for that. I don't think conmego by itself is a place where you would just say, start being my calculus tutor for the next year year and you're gonna work through calculus with me.

So that's that's where you are today. Where do you think you're going to be in a year or five years?

So that's that's where you are today. Where do you think you're going to be in a year or five years?

Yeah, one year, I'm quite confident we're going to have I mean it might be six months memory. So kind of migo is going to be able to know about previous conversations. And memory isn't just even about that, it's also about being able to report back to teachers, so the teachers can say, Hey, con Migo, what have you been working on with my students? Have you noticed any general patterns amongst my students, any conceptual gaps? In fact that functionality we're already prototyped that one and we're going to launch that in the coming months, but it can also develop insights about the students, like hey, you know Mary is really into anime. Whatever I give an anime example, she lights up, or you know, Billy really likes money and so and so we're developing were already have a prototype of that where it develops these insights, but we're making it transparent to the user, like these are the insights that it's collected about you, so the.

Yeah, one year, I'm quite confident we're going to have I mean it might be six months memory. So kind of migo is going to be able to know about previous conversations. And memory isn't just even about that, it's also about being able to report back to teachers, so the teachers can say, Hey, con Migo, what have you been working on with my students? Have you noticed any general patterns amongst my students, any conceptual gaps? In fact that functionality we're already prototyped that one and we're going to launch that in the coming months, but it can also develop insights about the students, like hey, you know Mary is really into anime. Whatever I give an anime example, she lights up, or you know, Billy really likes money and so and so we're developing were already have a prototype of that where it develops these insights, but we're making it transparent to the user, like these are the insights that it's collected about you, so the.

User, so people don't freak out like why does this machine know that I like anime?

User, so people don't freak out like why does this machine know that I like anime?

Or so doesn't climb the ladder of inference, which unfortunately a lots of humans do about other people, where the student can say no, actually, I just I used to be into anime. I'm not into it anymore. Or no, I know you think I really like that, but that's not you just you just went a little bit too far with that. So you're gonna have that, I think within a year. I don't know if this if we're going to have this out to hundreds of thousands or millions of people yet, but you're gonna have the ability to talk to conmigo much as you would talk to you know, your your your your Amazon or Apple devices. But it's going to be far more intelligent than those devices. It's going to know about all your context on con Academy, even though even if you're not talking to it.

Or so doesn't climb the ladder of inference, which unfortunately a lots of humans do about other people, where the student can say no, actually, I just I used to be into anime. I'm not into it anymore. Or no, I know you think I really like that, but that's not you just you just went a little bit too far with that. So you're gonna have that, I think within a year. I don't know if this if we're going to have this out to hundreds of thousands or millions of people yet, but you're gonna have the ability to talk to conmigo much as you would talk to you know, your your your your Amazon or Apple devices. But it's going to be far more intelligent than those devices. It's going to know about all your context on con Academy, even though even if you're not talking to it.

So you basically made a voice interface instead of a typing interface. Just to be clear, when you say talk, you mean speak yes.

So you basically made a voice interface instead of a typing interface. Just to be clear, when you say talk, you mean speak yes.

And I think the other thing that will surprise folks is how human like the voice will be in a year. In a year, I think a year from now, that whole loop of teachers develop assignments, creating rubrics, assigning it to students, students doing the assignment with an AI, the AI reporting back to the teacher that won. Yes, the student really did the work with me, like we worked together. It didn't just get copy and pasted from chat GBT. So that solves the hopefully addresses the cheating issue, gives students more support, and then also the AI gives a preliminary grade to the teacher. I think that whole workflow you're going to see in about a year, and you're going to see it beyond con Academy. We made an announcement a couple of weeks ago with Instructure, the people who make the you know, the biggest learning management system for k twelve and higher ed you might start seeing conmego there and other places they.

And I think the other thing that will surprise folks is how human like the voice will be in a year. In a year, I think a year from now, that whole loop of teachers develop assignments, creating rubrics, assigning it to students, students doing the assignment with an AI, the AI reporting back to the teacher that won. Yes, the student really did the work with me, like we worked together. It didn't just get copy and pasted from chat GBT. So that solves the hopefully addresses the cheating issue, gives students more support, and then also the AI gives a preliminary grade to the teacher. I think that whole workflow you're going to see in about a year, and you're going to see it beyond con Academy. We made an announcement a couple of weeks ago with Instructure, the people who make the you know, the biggest learning management system for k twelve and higher ed you might start seeing conmego there and other places they.

Are, meaning in schools that use this particular platform.

Are, meaning in schools that use this particular platform.

Yeah, right now, con Migo's only on con Academy's website. I think in a year you're going to see con Migo sit on other websites as well, not just on con Academy's website.

Yeah, right now, con Migo's only on con Academy's website. I think in a year you're going to see con Migo sit on other websites as well, not just on con Academy's website.

That's a year which those all seem like sort of within this new universe we've learned to live in of where generative AI is. Now, what's the five year? Where's it going to be in five years?

That's a year which those all seem like sort of within this new universe we've learned to live in of where generative AI is. Now, what's the five year? Where's it going to be in five years?

Yeah? The five years kind of wild?

Yeah? The five years kind of wild?

Yeah, the five years are so wild, right, like it we might not need you anymore in five years, right, That's part of what I was thinking about, like truly, like I don't know what is the five year.

Yeah, the five years are so wild, right, like it we might not need you anymore in five years, right, That's part of what I was thinking about, like truly, like I don't know what is the five year.

I think in five years you're going to be able to have an interview with a gen AI version of sal that will look like this interview you're doing with me right now. And for those listening, like you can see me right now. We're on a video conference, so you can actually see me. I actually think.

I think in five years you're going to be able to have an interview with a gen AI version of sal that will look like this interview you're doing with me right now. And for those listening, like you can see me right now. We're on a video conference, so you can actually see me. I actually think.

Point that AI could teach me whatever derivatives as well as you could teach me derivatives at some point in the future. Do you have a sense of how far off that?

Point that AI could teach me whatever derivatives as well as you could teach me derivatives at some point in the future. Do you have a sense of how far off that?

I think. I think in the five year timeframe, it's an engineering problem more than a science problem at this point, where you just have to make the stuff fit together so it feels seamless and it feels really natural and magical. And that's what we're spending a lot of time doing so that the memory feels natural, so that how it holds you accountable feels natural.

I think. I think in the five year timeframe, it's an engineering problem more than a science problem at this point, where you just have to make the stuff fit together so it feels seamless and it feels really natural and magical. And that's what we're spending a lot of time doing so that the memory feels natural, so that how it holds you accountable feels natural.

I mean, understand a little bit. It needs to understand better than understands now based on my very limited experience. No, or communicate its understanding or something.

I mean, understand a little bit. It needs to understand better than understands now based on my very limited experience. No, or communicate its understanding or something.

I think it's more. I think it's more of a even with GPT four, it's more about the prompting and the communication and what you're passing to what okay fair. I mean, in five years you're probably gonna have GPT five or six, and you're gonna have these other large language models, because you know you have at least five or six major groups are throwing billions at it.

I think it's more. I think it's more of a even with GPT four, it's more about the prompting and the communication and what you're passing to what okay fair. I mean, in five years you're probably gonna have GPT five or six, and you're gonna have these other large language models, because you know you have at least five or six major groups are throwing billions at it.

Will conmigo be basically redundant to GPT X whatever, GPT six and then like where do teachers fit in? Right? Those seem like two logical questions there.

Will conmigo be basically redundant to GPT X whatever, GPT six and then like where do teachers fit in? Right? Those seem like two logical questions there.

I've been thinking a lot about this and speaking a lot about this. I think you're going to see job disruption in a lot of places, but I do not think it's going to happen in teaching. I think you're going to I think if I told every teacher on the planet that Hey, all of a sudden, we discovered all this money, and we're going to hire three teaching assistants for every one of you to go into your classroom and they're going to help you with lesson planning. They're going to help you grade papers, are going to help you write progress supports, and while you know, while you're in the class session, they're going to circulate and answer any questions that the students have and then report back to you. I mean, I think every teacher like, finally.

I've been thinking a lot about this and speaking a lot about this. I think you're going to see job disruption in a lot of places, but I do not think it's going to happen in teaching. I think you're going to I think if I told every teacher on the planet that Hey, all of a sudden, we discovered all this money, and we're going to hire three teaching assistants for every one of you to go into your classroom and they're going to help you with lesson planning. They're going to help you grade papers, are going to help you write progress supports, and while you know, while you're in the class session, they're going to circulate and answer any questions that the students have and then report back to you. I mean, I think every teacher like, finally.

So that's the teacher side. What about the con academy side? Like, could AI render con Academy obsolete? Yeah?

So that's the teacher side. What about the con academy side? Like, could AI render con Academy obsolete? Yeah?

I think in five years, honestly, Jenna, I might be able to make real time videos very similar to Conacademy videos, but they feel like real time explanations. It will be that much better at potentially creating exercises and things like that. But I think it's still going to be better when it's anchored on a framework, on a scope and sees just as a teacher, a real teacher, right, A real teacher can do all of these things, but they are better when they have a curriculum, when they have textbooks, when they have con academy, when they have all of these other tools around it. I think the same thing is going to be true of Jenai for a very long time.

I think in five years, honestly, Jenna, I might be able to make real time videos very similar to Conacademy videos, but they feel like real time explanations. It will be that much better at potentially creating exercises and things like that. But I think it's still going to be better when it's anchored on a framework, on a scope and sees just as a teacher, a real teacher, right, A real teacher can do all of these things, but they are better when they have a curriculum, when they have textbooks, when they have con academy, when they have all of these other tools around it. I think the same thing is going to be true of Jenai for a very long time.

We'll be back in a minute with the lightning round, in which I try to get Salt to reach into his days as the frontman of a heavy metal band. Let's do a lightning round. What's the first Bollywood movie I should watch? If I've never watched a Bollywood movie.

We'll be back in a minute with the lightning round, in which I try to get Salt to reach into his days as the frontman of a heavy metal band. Let's do a lightning round. What's the first Bollywood movie I should watch? If I've never watched a Bollywood movie.

You know, I mean, my standard disclaimer with all Bollywood movies are there will be moments in almost every Bollywood movies that will make you cringe. Cringe. There's gonna be a little bit overacting. People are going to be dancing at inappropriate times. But I'm your Khan, who's one of the more famous Indian actors and he also produces movies. He has this movie called Three Idiots that one I recommend, and it has some not you know to I think western sensibilities, some cringe worthy moments, but it's it's it's about education, you know, not to be too self aggrandizing, but the whole, the main story is about this guy. It's actually based loosely on a real person who he was very unsatisfied with the education system.

You know, I mean, my standard disclaimer with all Bollywood movies are there will be moments in almost every Bollywood movies that will make you cringe. Cringe. There's gonna be a little bit overacting. People are going to be dancing at inappropriate times. But I'm your Khan, who's one of the more famous Indian actors and he also produces movies. He has this movie called Three Idiots that one I recommend, and it has some not you know to I think western sensibilities, some cringe worthy moments, but it's it's it's about education, you know, not to be too self aggrandizing, but the whole, the main story is about this guy. It's actually based loosely on a real person who he was very unsatisfied with the education system.

Is it based on you? Yes? Or no?

Is it based on you? Yes? Or no?

No, it's not based on me. It's not based on me. But he uh uh. He ends up eventually starting his own school. But he also ends up falling in love with the doctor and marrying her. And I'm like, that's my life. That's that's what.

No, it's not based on me. It's not based on me. But he uh uh. He ends up eventually starting his own school. But he also ends up falling in love with the doctor and marrying her. And I'm like, that's my life. That's that's what.

Uh? What was the what was the name of the heavy metal band you were in in your youth?

Uh? What was the what was the name of the heavy metal band you were in in your youth?

The name was Malignancy until we realized someone else.

The name was Malignancy until we realized someone else.

Had that had that good name. It sounds like a heavy metal band. What can you give me a few bars of one of your songs?

Had that had that good name. It sounds like a heavy metal band. What can you give me a few bars of one of your songs?

No, but come on, I have a certain brand now. I can't growling into a into a microphone anymore. A whole series of lyrics just went through my head, and my my podcast filter vetoed all of them.

No, but come on, I have a certain brand now. I can't growling into a into a microphone anymore. A whole series of lyrics just went through my head, and my my podcast filter vetoed all of them.

Just one phrase, one phrase. You don't even have to sing it, just say it.

Just one phrase, one phrase. You don't even have to sing it, just say it.

No, it was I was angry.

No, it was I was angry.

Yeah, yeah, what's the last non work thing? You use chat GPT for?

Yeah, yeah, what's the last non work thing? You use chat GPT for?

Oh, last non work thing? I partially used it to planification several vacations actually, and it worked.

Oh, last non work thing? I partially used it to planification several vacations actually, and it worked.

It was helpful. Yeah. So, you know, one of the things that's interesting to me looking at the arc or the ramp of con Academy from the outside is it's been clearly, like profoundly successful and kind of a darling of this sort of Bill Gates ted talk Silicon Valley universe and reasonably so rightly so. But I'm curious, like, what on the inside of building the organization was was hard? Harder than it looked.

It was helpful. Yeah. So, you know, one of the things that's interesting to me looking at the arc or the ramp of con Academy from the outside is it's been clearly, like profoundly successful and kind of a darling of this sort of Bill Gates ted talk Silicon Valley universe and reasonably so rightly so. But I'm curious, like, what on the inside of building the organization was was hard? Harder than it looked.

I think when you scale an organization, you know, I was a one person shop for many years, and then you start scaling. And I definitely think when we got to between between twenty people and one hundred and fifty, we had a lot of growing pains. A lot of organizations do I now see. I mean, when you're in it, you're like, is it just to us that we can't seem to But I realized that there's a when you get My old management philosophy was just get the smartest, most passionate people in the room. And then we'll figure it out. And I've now realized I'll get the smartest, passionate, most aligned folks in the room and work constantly to align around a true north. So yeah, yes, I think like many startups, when we were in a startup mode, we probably zigged and zagged more than we necessarily had to. But maybe it's part of just natural growing pains. But that's that's probably the biggest source of tension over the years of like, our mission is so big, should we just do one part of it? What about international? What about domestic? What about you know, English language arts? What about math? And so we've been pulled in so many different directions, and we violated a lot of basic business strategy. Basic business strategy would be like focus on just one thing, and I'd be like, I'm impatient, Like I'm I mean, I used to be the to your point. I used to be like the young guy on the scene, you know, with this A. I started out in my early thirties, and now I'm approaching fifty. I'm about to turn forty seven years old, and I'm like, wow, I only have like probably, if I'm lucky, I have another good twenty twenty five years. If I'm lucky and I started tooting now the nineteen years ago, like, we got to get on this. We got to do all the subjects, all the great all the countries as soon as possible. I want kind of kind of me to be around for hundreds of years, thousands of years. I read a lot of science fiction books, and so you know, even if I even if I wanted around one hundred years, two hundred years and serve this mission free world class education, how does it do that when I'm not there or when people who didn't know me are no longer there. And I don't want it to just exist. I mean, there is a success scenario where in a hundred years it's it's successful, but then it becomes it's the incumbent. It's just like every other large publisher. And that's a lame scenario either. I you know, if I was reincarnated at that time, I'd want to disrupt that incumbent. So how does it stay innovative and doesn't take itself too seriously? But it really is and hopefully at that point serving most of humanity?

I think when you scale an organization, you know, I was a one person shop for many years, and then you start scaling. And I definitely think when we got to between between twenty people and one hundred and fifty, we had a lot of growing pains. A lot of organizations do I now see. I mean, when you're in it, you're like, is it just to us that we can't seem to But I realized that there's a when you get My old management philosophy was just get the smartest, most passionate people in the room. And then we'll figure it out. And I've now realized I'll get the smartest, passionate, most aligned folks in the room and work constantly to align around a true north. So yeah, yes, I think like many startups, when we were in a startup mode, we probably zigged and zagged more than we necessarily had to. But maybe it's part of just natural growing pains. But that's that's probably the biggest source of tension over the years of like, our mission is so big, should we just do one part of it? What about international? What about domestic? What about you know, English language arts? What about math? And so we've been pulled in so many different directions, and we violated a lot of basic business strategy. Basic business strategy would be like focus on just one thing, and I'd be like, I'm impatient, Like I'm I mean, I used to be the to your point. I used to be like the young guy on the scene, you know, with this A. I started out in my early thirties, and now I'm approaching fifty. I'm about to turn forty seven years old, and I'm like, wow, I only have like probably, if I'm lucky, I have another good twenty twenty five years. If I'm lucky and I started tooting now the nineteen years ago, like, we got to get on this. We got to do all the subjects, all the great all the countries as soon as possible. I want kind of kind of me to be around for hundreds of years, thousands of years. I read a lot of science fiction books, and so you know, even if I even if I wanted around one hundred years, two hundred years and serve this mission free world class education, how does it do that when I'm not there or when people who didn't know me are no longer there. And I don't want it to just exist. I mean, there is a success scenario where in a hundred years it's it's successful, but then it becomes it's the incumbent. It's just like every other large publisher. And that's a lame scenario either. I you know, if I was reincarnated at that time, I'd want to disrupt that incumbent. So how does it stay innovative and doesn't take itself too seriously? But it really is and hopefully at that point serving most of humanity?

That is wildly ambitious.

That is wildly ambitious.

It is.

It is.

Salcon is the founder and CEO of con Academy. Today's show was produced by Gabriel Hunter Chang and Edith Russlo, and it was edited by Sarah Nix and engineered by Amanda k Wong. I'm Jacob Goldstein, and we'll be back next week with another episode of What's Your Problem.

Salcon is the founder and CEO of con Academy. Today's show was produced by Gabriel Hunter Chang and Edith Russlo, and it was edited by Sarah Nix and engineered by Amanda k Wong. I'm Jacob Goldstein, and we'll be back next week with another episode of What's Your Problem.