You're listening to a CNA podcast.

You're listening to a CNA podcast.

So hey everyone, welcome to another episode of Deep Dive with Oelli and myself Steven. today we're talking about

So hey everyone, welcome to another episode of Deep Dive with Oelli and myself Steven. today we're talking about

AI or rather the use of AI when it comes to doing your homework. Are you an AI fan? Do you use it for work, not homework, but work?

AI or rather the use of AI when it comes to doing your homework. Are you an AI fan? Do you use it for work, not homework, but work?

OK, who doesn't use it nowadays unless you are like living under a rock, but everyone uses, you know, large language models to a certain extent, right, to improve writing. I mean, in our line of work, we use it to refine the original sentences that I came up with. I

OK, who doesn't use it nowadays unless you are like living under a rock, but everyone uses, you know, large language models to a certain extent, right, to improve writing. I mean, in our line of work, we use it to refine the original sentences that I came up with. I

like I basically to learn and to work.

like I basically to learn and to work.

Efficiently. It helps me do what I do.

Efficiently. It helps me do what I do.

Yeah. So we can imagine a lot of students from the Higher Institutes of learning, they're also doing that and as a matter of fact, recently, right, there was this issue that blew up in the news. It's when NTU accused 3 students from the School of Social Sciences of academic fraud. It's a very, very harsh term to think about it, saying that they basically use generative AI tools in their assignments and were given zero marks for that.

Yeah. So we can imagine a lot of students from the Higher Institutes of learning, they're also doing that and as a matter of fact, recently, right, there was this issue that blew up in the news. It's when NTU accused 3 students from the School of Social Sciences of academic fraud. It's a very, very harsh term to think about it, saying that they basically use generative AI tools in their assignments and were given zero marks for that.

Yeah, 0. That's a big blow. Of course, the students are upset with that and they're fighting back. They're asking the university to justify why they're doing that, uh, you know, and trying to understand just, well, where do you draw the line? When is it OK and when is it not OK, especially when it comes to students who are still in school. And today we have some people who will help us understand that issue better. So let's welcome Associate Professor Ben Leong. He's the director of AI Center for Educational Technologies at NUS.

Yeah, 0. That's a big blow. Of course, the students are upset with that and they're fighting back. They're asking the university to justify why they're doing that, uh, you know, and trying to understand just, well, where do you draw the line? When is it OK and when is it not OK, especially when it comes to students who are still in school. And today we have some people who will help us understand that issue better. So let's welcome Associate Professor Ben Leong. He's the director of AI Center for Educational Technologies at NUS.

Hello, uh, it's a pleasure to be here and I'm happy to, to, you know, discuss this issue, which is quite close to heart. Jeremy Su, he's a co-founder of Next AI and graduated from Neon Pay just last year.

Hello, uh, it's a pleasure to be here and I'm happy to, to, you know, discuss this issue, which is quite close to heart. Jeremy Su, he's a co-founder of Next AI and graduated from Neon Pay just last year.

Yes, happy to be here. Thanks for having me. I think this is gonna be really interesting.

Yes, happy to be here. Thanks for having me. I think this is gonna be really interesting.

So OK, let's just start off with a very big picture question, right? Help us understand how is AI being used both by teachers and schools. So Jeremy from a student's point of view because you just graduated, right? So when it comes to the students, how they actually use it, they use it for quite a wide range of things, right? There is the obvious writing my essays, doing my homework, that's a very, very standard use case. We've had it since years ago, even before Chad GPT because we have

So OK, let's just start off with a very big picture question, right? Help us understand how is AI being used both by teachers and schools. So Jeremy from a student's point of view because you just graduated, right? So when it comes to the students, how they actually use it, they use it for quite a wide range of things, right? There is the obvious writing my essays, doing my homework, that's a very, very standard use case. We've had it since years ago, even before Chad GPT because we have

For example, GBT 3 original models 2, which were all pretty good autocomplete models, these were used by different students to, for example, write their essays or kind of refine certain sentences. And then right now people are using it for more creative things, so things like research, they're able to kind of trigger an AI, have it go online, search for a bunch of different information, uh, pull it together, synthesize it, and then eventually create some sort of report and so they don't need to do like.

For example, GBT 3 original models 2, which were all pretty good autocomplete models, these were used by different students to, for example, write their essays or kind of refine certain sentences. And then right now people are using it for more creative things, so things like research, they're able to kind of trigger an AI, have it go online, search for a bunch of different information, uh, pull it together, synthesize it, and then eventually create some sort of report and so they don't need to do like.

work anymore, right? That's a big part. I think specifically for me, what I've seen is because more from the creative side. So in, for example, media school, people are using it in art, people are using it to actually edit. So people are using it in many different ways which are not conventionally, I think what most people think of when they think of AI in the academic sense. It just reports and

work anymore, right? That's a big part. I think specifically for me, what I've seen is because more from the creative side. So in, for example, media school, people are using it in art, people are using it to actually edit. So people are using it in many different ways which are not conventionally, I think what most people think of when they think of AI in the academic sense. It just reports and

essays.

essays.

Ben, I'll ask you, so at the university again, you know, you are facing this, many students will also be using AI and AI is of course a lot better now. It can literally write the essay for you.

Ben, I'll ask you, so at the university again, you know, you are facing this, many students will also be using AI and AI is of course a lot better now. It can literally write the essay for you.

So at which point do you say, OK, that's all right, at which point do you say no. OK, so I think it's it's a complex question, right, really depends on the discipline that you know that we're dealing with. So I teach computer science and mostly programming. So in programming, you know, you write code and I think my colleagues who have students write essays, they have a different kind of requirement. It may be harder for us to detect plagiarism for coding.

So at which point do you say, OK, that's all right, at which point do you say no. OK, so I think it's it's a complex question, right, really depends on the discipline that you know that we're dealing with. So I teach computer science and mostly programming. So in programming, you know, you write code and I think my colleagues who have students write essays, they have a different kind of requirement. It may be harder for us to detect plagiarism for coding.

Because the fact is that AI is really good at generating good code and the code looks kind of like standard to be fair, right? Pages is not a new thing.

Because the fact is that AI is really good at generating good code and the code looks kind of like standard to be fair, right? Pages is not a new thing.

So universities

So universities

that use these AI detectors, right, like turn it in and all that, does that detect like the code? So,

that use these AI detectors, right, like turn it in and all that, does that detect like the code? So,

so the truth is that we have had very good AI code detectors in the in the past before AI, right, and.

so the truth is that we have had very good AI code detectors in the in the past before AI, right, and.

The way I was going to explain, ever since I start teaching, we have caught students cheating by copying their friends' code, and generally what happens is we catch them because they write bad code, not because they write good code. I mean write good code, you have two clever people writing good code and look the same. It's actually there's some standard ways of writing code. So the general way to catch people writing copying code is that they have a friend who writes bad code and wrong code, and you have make the same mistake, right? And the best part is.

The way I was going to explain, ever since I start teaching, we have caught students cheating by copying their friends' code, and generally what happens is we catch them because they write bad code, not because they write good code. I mean write good code, you have two clever people writing good code and look the same. It's actually there's some standard ways of writing code. So the general way to catch people writing copying code is that they have a friend who writes bad code and wrong code, and you have make the same mistake, right? And the best part is.

It came from somewhere that you don't know where it came from, right? So, so there's no way to kind of prove that, in fact, no easy way to prove because code is 2 + 2 is 4. So when you write good code, good code is good code, as in it should all look pretty much the same, right? Correct. I mean, so there are many ways to do the same thing, but there is, you're right, there's a certain style that good code, right? So so my question then becomes, what's wrong with that?

It came from somewhere that you don't know where it came from, right? So, so there's no way to kind of prove that, in fact, no easy way to prove because code is 2 + 2 is 4. So when you write good code, good code is good code, as in it should all look pretty much the same, right? Correct. I mean, so there are many ways to do the same thing, but there is, you're right, there's a certain style that good code, right? So so my question then becomes, what's wrong with that?

Because they're getting you the the result that they need and in the future when they work, why would they bother going through it themselves when they can use the air to write it? That's an excellent question, right? So, so then you need to understand what we're doing in school here, right? There are two things that we are trying to do.

Because they're getting you the the result that they need and in the future when they work, why would they bother going through it themselves when they can use the air to write it? That's an excellent question, right? So, so then you need to understand what we're doing in school here, right? There are two things that we are trying to do.

Number one is that we're trying to help the students learn something, OK, and let me blunt, I mean, homework is there for a reason, right? I mean, if I give you homework, frankly it's, it's going to cost us work, right, because someone has to grade it. Of course you can outsource to AI, but frankly, you know, if we don't give issue homework, then we do less. But the homework is there to the homework. No, no, no, no, there are these things called formative assessments and there are these things called summative assessments. So formative assessments is to help you learn something. So you're going through the struggle to learn the stuff, right?

Number one is that we're trying to help the students learn something, OK, and let me blunt, I mean, homework is there for a reason, right? I mean, if I give you homework, frankly it's, it's going to cost us work, right, because someone has to grade it. Of course you can outsource to AI, but frankly, you know, if we don't give issue homework, then we do less. But the homework is there to the homework. No, no, no, no, there are these things called formative assessments and there are these things called summative assessments. So formative assessments is to help you learn something. So you're going through the struggle to learn the stuff, right?

Now, so the problem with AI in this regard is that it allows the students to kind of like avoid this this struggle, right? And they avoid learning, right? I mean, so, so frankly, if I give you a question, they get the result they get the results and so they're not, OK, at some level they are short change themselves because their parents paid good money for them to come to. OK, so that's number one. Number 2, there's also a very important uh aspect of this assessment thing which is that there's this thing called a submitive assessment, but universities have a role to play in the economy, right?

Now, so the problem with AI in this regard is that it allows the students to kind of like avoid this this struggle, right? And they avoid learning, right? I mean, so, so frankly, if I give you a question, they get the result they get the results and so they're not, OK, at some level they are short change themselves because their parents paid good money for them to come to. OK, so that's number one. Number 2, there's also a very important uh aspect of this assessment thing which is that there's this thing called a submitive assessment, but universities have a role to play in the economy, right?

Because ultimately these kids graduate and they get jobs. OK, so let me ask a question, right? If there's no reason for us to give the first class honors, and second class and so forth, we actually have a role in society, right, to sort of like grade the students, right, so that employers have an easier time with the hiring.

Because ultimately these kids graduate and they get jobs. OK, so let me ask a question, right? If there's no reason for us to give the first class honors, and second class and so forth, we actually have a role in society, right, to sort of like grade the students, right, so that employers have an easier time with the hiring.

Right, so if everybody gets an A, then it's debatable whether or not this is still a kind of function of the university, but I'm certain that if we give every student A's, I think that would be and there's somehow you see, OK, the whole process as you mentioned is for the student to go through the learning journey so that when they face other situations in the real world that AI maybe hasn't encountered yet.

Right, so if everybody gets an A, then it's debatable whether or not this is still a kind of function of the university, but I'm certain that if we give every student A's, I think that would be and there's somehow you see, OK, the whole process as you mentioned is for the student to go through the learning journey so that when they face other situations in the real world that AI maybe hasn't encountered yet.

Then they can help solve it. So you're teaching them how to think, basically, right? But now with the AI doing everything for them, they miss out on this.

Then they can help solve it. So you're teaching them how to think, basically, right? But now with the AI doing everything for them, they miss out on this.

I've got a

I've got a

question as well that a lot of people are turning to your large language models

question as well that a lot of people are turning to your large language models

right

right

to learn stuff. Let's say you go into like GPT or Gemini and whatnot.

to learn stuff. Let's say you go into like GPT or Gemini and whatnot.

As you are learning stuff, right, it picks up your learning style, it becomes individualized, so to speak. So then does that mean that the role of a teacher becomes so teachers be left behind, I'm trying to say because the, yeah, exactly, they already used to learning from GBT. So when the teacher is trying to teach them, it becomes increasingly difficult for teachers to teach

As you are learning stuff, right, it picks up your learning style, it becomes individualized, so to speak. So then does that mean that the role of a teacher becomes so teachers be left behind, I'm trying to say because the, yeah, exactly, they already used to learning from GBT. So when the teacher is trying to teach them, it becomes increasingly difficult for teachers to teach

students.

students.

I just have my private tutor

I just have my private tutor

at home

at home

when it comes to this, I think there are a few points.

when it comes to this, I think there are a few points.

So the personalized learning pathway is always going to be better than the generalized one because it's simply it's going to be a better fit. What we've seen is that motivated individuals who are trying to pick up some new skills or they're trying to accomplish a task or if they're even trying to do less work in school, they will turn to these AI models and for you mentioned large language models and all sorts of different models, they will turn to these tools to try and accomplish their task. Now, the thing is, I think many people think that just because you use AI, you know, people become dumber, people don't think about it anymore.

So the personalized learning pathway is always going to be better than the generalized one because it's simply it's going to be a better fit. What we've seen is that motivated individuals who are trying to pick up some new skills or they're trying to accomplish a task or if they're even trying to do less work in school, they will turn to these AI models and for you mentioned large language models and all sorts of different models, they will turn to these tools to try and accomplish their task. Now, the thing is, I think many people think that just because you use AI, you know, people become dumber, people don't think about it anymore.

And, you know, I'm seeing Reddit students are saying that it's rare nowadays to see people think on their own because they're sending their assignments to GPT before even thinking the devil is in the details, how are they sending it in, right? So the degree of specificity that you would need to include in a prompt in an instruction requires that you identify your own thinking and label and actually describe step by step exactly how you would go about attacking this task. So if I were to just

And, you know, I'm seeing Reddit students are saying that it's rare nowadays to see people think on their own because they're sending their assignments to GPT before even thinking the devil is in the details, how are they sending it in, right? So the degree of specificity that you would need to include in a prompt in an instruction requires that you identify your own thinking and label and actually describe step by step exactly how you would go about attacking this task. So if I were to just

GBT write this essay for me. You're not going to get that good of an essay. If you say write and style of, you know, so and so, it does a little better. If I wanted to think really in depth, you know, go through this point, uh, articulate this particular argument, quote an example here, and make sure it makes sense in the grand scheme of things, and then go step by step. You end up with these prompts that are pretty normal. I think in industry to write 34 paragraphs about the right questions are you actually learning anything?

GBT write this essay for me. You're not going to get that good of an essay. If you say write and style of, you know, so and so, it does a little better. If I wanted to think really in depth, you know, go through this point, uh, articulate this particular argument, quote an example here, and make sure it makes sense in the grand scheme of things, and then go step by step. You end up with these prompts that are pretty normal. I think in industry to write 34 paragraphs about the right questions are you actually learning anything?

What is learning then?

What is learning then?

I

I

think you are. So it's like having when we first had Google as a resource, how do you use it and use it effectively, right? So you can use it the wrong way, you can use it the right way. Same thing with the AI. You can use it to make whatever you're doing better and to learn more and understand better, or you can just be lazy and just chuck it the whole problem and let it do everything for you. The question is how are you using the AI and at which point can you say it's OK to use it this way, but it's not OK to use it that way.

think you are. So it's like having when we first had Google as a resource, how do you use it and use it effectively, right? So you can use it the wrong way, you can use it the right way. Same thing with the AI. You can use it to make whatever you're doing better and to learn more and understand better, or you can just be lazy and just chuck it the whole problem and let it do everything for you. The question is how are you using the AI and at which point can you say it's OK to use it this way, but it's not OK to use it that way.

Yeah, how do we differentiate misuse and misjudgment?

Yeah, how do we differentiate misuse and misjudgment?

I think there are the two things I mentioned that matter, right? Number 1 is the learning. Number 2 is the fairness, right, because the trouble now is that some students use AI and they get better grade other students who don't use AI, uh, it's not unfair. I'm like just need to switch on your computer, that's all.

I think there are the two things I mentioned that matter, right? Number 1 is the learning. Number 2 is the fairness, right, because the trouble now is that some students use AI and they get better grade other students who don't use AI, uh, it's not unfair. I'm like just need to switch on your computer, that's all.

Too lazy to

Too lazy to

do that. The problem here is that there's there's a class where the prophet said that you shall not use AI. OK, so you have people who are this, this man looks like he looks like he's a man, right.

do that. The problem here is that there's there's a class where the prophet said that you shall not use AI. OK, so you have people who are this, this man looks like he looks like he's a man, right.

I'm wrong, but suppose it's an honest man, right, and you dishonest man and this honest one actually, you know, gets a better grade, then then there's something wrong, right, because then we have a society which actually, you know, rewards people who are dishonest. Is that something you want in Singapore? I think it's, is that the fault also of the way they're being tested. The professor may be a bit naive to say you cannot use AI, but in this day and age, you know what I mean? OK, let's set it aside, but, but I think if you ask me what what is the issue here, it is the issue of fairness, OK.

I'm wrong, but suppose it's an honest man, right, and you dishonest man and this honest one actually, you know, gets a better grade, then then there's something wrong, right, because then we have a society which actually, you know, rewards people who are dishonest. Is that something you want in Singapore? I think it's, is that the fault also of the way they're being tested. The professor may be a bit naive to say you cannot use AI, but in this day and age, you know what I mean? OK, let's set it aside, but, but I think if you ask me what what is the issue here, it is the issue of fairness, OK.

So to address this issue of fairness, if, if the professor says you can't use AI, people should not use AI, right? Now, in the case where by first process that I don't care you'll use AI, then yeah, go ahead, but the trouble will be a problem, right, which is that then, uh, what's the whole point, right, because you get pretty much the same thing and worse still, we still, you know, there are different models, paid models, unpaid models, and suppose this guy is rich, right? He buys the advantage and he may get advantage we could argue the kids who have tuition versus those who do not also have an unfair advantage, right.

So to address this issue of fairness, if, if the professor says you can't use AI, people should not use AI, right? Now, in the case where by first process that I don't care you'll use AI, then yeah, go ahead, but the trouble will be a problem, right, which is that then, uh, what's the whole point, right, because you get pretty much the same thing and worse still, we still, you know, there are different models, paid models, unpaid models, and suppose this guy is rich, right? He buys the advantage and he may get advantage we could argue the kids who have tuition versus those who do not also have an unfair advantage, right.

Well that's true. That's true, but, but I think that's the way the real world works. That's how, how far you want to bring that. But but ultimately there is the issue of fairness

Well that's true. That's true, but, but I think that's the way the real world works. That's how, how far you want to bring that. But but ultimately there is the issue of fairness

the fairness point in that case, is it even a good proxy of the real world because the real world is not really fair. People have to learn. I think the whole point of having the school and having that environment is.

the fairness point in that case, is it even a good proxy of the real world because the real world is not really fair. People have to learn. I think the whole point of having the school and having that environment is.

To simulated almost a 1 to 1 mimic of the real world and it trains them to operate in the real world. Does it help them learn in that case?

To simulated almost a 1 to 1 mimic of the real world and it trains them to operate in the real world. Does it help them learn in that case?

That's an

That's an

excellent philosophical question. But, but so but to address these two issues. The first issue is this, right? It is that the learning part, I think the learning part is real. There are students that are really not getting, I mean, what I'm seeing, again, this is not scientific. I actually put up some of the grant to learn to study this thing more carefully, but based on what I'm seeing on the ground today, there's a small growth students, OK, who are using AI very effectively.

excellent philosophical question. But, but so but to address these two issues. The first issue is this, right? It is that the learning part, I think the learning part is real. There are students that are really not getting, I mean, what I'm seeing, again, this is not scientific. I actually put up some of the grant to learn to study this thing more carefully, but based on what I'm seeing on the ground today, there's a small growth students, OK, who are using AI very effectively.

They are actually moving ahead faster, better than before. OK, so you compare them today, the same same students to those in the past, these guys will do better, and that's the rest. I really think the rest are actually getting dumber. I don't know for using AI they are actually not OK, so

They are actually moving ahead faster, better than before. OK, so you compare them today, the same same students to those in the past, these guys will do better, and that's the rest. I really think the rest are actually getting dumber. I don't know for using AI they are actually not OK, so

you are

you are

saying that because they have lost their authenticity, their creative juices, they are counting on.

saying that because they have lost their authenticity, their creative juices, they are counting on.

Right,

Right,

so we're just saying, so do I get this project and just chuck the whole thing to AI and say give me an answer, or do I understand the project, understand what are the 5 tools I need, then go to AI and say, help me figure out what are these 5 tools. So there are different ways of using that right? That's not how the human psychology works. That's not how the kids think, right?

so we're just saying, so do I get this project and just chuck the whole thing to AI and say give me an answer, or do I understand the project, understand what are the 5 tools I need, then go to AI and say, help me figure out what are these 5 tools. So there are different ways of using that right? That's not how the human psychology works. That's not how the kids think, right?

So, so what I'm telling you on the ground is that there's a small group, and they are actually moving ahead, right? And, and actually, to be honest, we don't fully understand how the kids are using AI today because it's quite new, like 2 years, OK. We don't watch over them 24/7, right? And, and they are doing some some things that way.

So, so what I'm telling you on the ground is that there's a small group, and they are actually moving ahead, right? And, and actually, to be honest, we don't fully understand how the kids are using AI today because it's quite new, like 2 years, OK. We don't watch over them 24/7, right? And, and they are doing some some things that way.

Really kind of wild, there are some ways but then the question becomes, is it then, are we then testing the students the wrong way? Should we also evolve in the, you know, how we

Really kind of wild, there are some ways but then the question becomes, is it then, are we then testing the students the wrong way? Should we also evolve in the, you know, how we

analyze more face to face um sort of discussion about certain topic or

analyze more face to face um sort of discussion about certain topic or

excellent

excellent

point. There are two kinds of disciplines. There's my kind of disciplines and there's the poor humanities people. The difference is this, what do we do?

point. There are two kinds of disciplines. There's my kind of disciplines and there's the poor humanities people. The difference is this, what do we do?

We lock them in the exam hall. We lock them in a lab, but I, I do have some colleagues who are in humanities whereby the grading is all done by essays. Uh, they are struggling and, and I, I, I feel bad because I don't really have a good solution for them because

We lock them in the exam hall. We lock them in a lab, but I, I do have some colleagues who are in humanities whereby the grading is all done by essays. Uh, they are struggling and, and I, I, I feel bad because I don't really have a good solution for them because

you are in humanities, right? I've been operating in tech for 5 years.

you are in humanities, right? I've been operating in tech for 5 years.

So do you discuss among your friends, OK, what is the best way, you know, to go around this problem, to ask these large language models, how can we sort of get around the system? I mean, I think it's it's interesting you bring that up because I think many people now use the conversational models as almost that partner. So instead of asking friends, they just ask the models will probably answer better. I think on the point of essentially essays being a poor measure, it's it's a horrible measure. Essays are the new MCQs.

So do you discuss among your friends, OK, what is the best way, you know, to go around this problem, to ask these large language models, how can we sort of get around the system? I mean, I think it's it's interesting you bring that up because I think many people now use the conversational models as almost that partner. So instead of asking friends, they just ask the models will probably answer better. I think on the point of essentially essays being a poor measure, it's it's a horrible measure. Essays are the new MCQs.

Because what we learned about language from all the years of research in AI is basically that language given the rules, uh, given the syntax, given the semantics, the meaning of each word, if you just apply a few rules and you keep in mind all the exceptions, you can algorithmically generate anything you want and that's what all these models do. So it's basically standardized testing, it just doesn't look like standardized testing when when you give an essay.

Because what we learned about language from all the years of research in AI is basically that language given the rules, uh, given the syntax, given the semantics, the meaning of each word, if you just apply a few rules and you keep in mind all the exceptions, you can algorithmically generate anything you want and that's what all these models do. So it's basically standardized testing, it just doesn't look like standardized testing when when you give an essay.

100% agree, it's a horrible, it's horrible to test it. I know you're not really in the humanities because that's where it seems like the where it's questionable that a very gray area you see. So therefore, I mean, can we come to the conclusion that something has to change? Because right now if you ask a lot of students, right, it's not clear what exactly is the demarcation, you know, like can I use AI? I can use AI, but to what extent?

100% agree, it's a horrible, it's horrible to test it. I know you're not really in the humanities because that's where it seems like the where it's questionable that a very gray area you see. So therefore, I mean, can we come to the conclusion that something has to change? Because right now if you ask a lot of students, right, it's not clear what exactly is the demarcation, you know, like can I use AI? I can use AI, but to what extent?

You know, how do I know? And now there's a lot of anxiety among the students as well because I don't want to. I know people that don't use AI because they're scared

You know, how do I know? And now there's a lot of anxiety among the students as well because I don't want to. I know people that don't use AI because they're scared

that I think that's wrong because these are the real world resources, they're available to everyone. Why are we saying don't use it?

that I think that's wrong because these are the real world resources, they're available to everyone. Why are we saying don't use it?

But in the real world,

But in the real world,

you want to be labeled risk being labeled academic fraud because I wouldn't

you want to be labeled risk being labeled academic fraud because I wouldn't

want that in a sense has failed to evolve and adapt to what is happening in the real

want that in a sense has failed to evolve and adapt to what is happening in the real

world.

world.

Or maybe they're not catching up fast.

Or maybe they're not catching up fast.

I try to address that point, right? So, so I think I've explained to you the two concerns we have in school, right? But, but the problem is this, right, we've always had this complaint from students that

I try to address that point, right? So, so I think I've explained to you the two concerns we have in school, right? But, but the problem is this, right, we've always had this complaint from students that

Uh, like I said, we lock them up in, in, in the lab, right, to do a coding exam, right? They're like, what the heck? I mean, you know, in real life I get to go to GitHub and then I copy I'm better, right? OK, so that's a very good night now. But you see, the whole point of school actually is that we're trying to train them to to learn some things, OK, and

Uh, like I said, we lock them up in, in, in the lab, right, to do a coding exam, right? They're like, what the heck? I mean, you know, in real life I get to go to GitHub and then I copy I'm better, right? OK, so that's a very good night now. But you see, the whole point of school actually is that we're trying to train them to to learn some things, OK, and

I don't know why people think that coding is easy lah, but I must tell you it is not that easy, OK? And the fact is this, I actually don't use AI because sadly AI doesn't write as well as I can write, and AI does not write code, uh, doesn't write code the way I can write code either, right? So, OK, so for coding, actually sometimes I might use AI because it's just convenient because the truth is that we are trying to teach the students to a certain level, OK.

I don't know why people think that coding is easy lah, but I must tell you it is not that easy, OK? And the fact is this, I actually don't use AI because sadly AI doesn't write as well as I can write, and AI does not write code, uh, doesn't write code the way I can write code either, right? So, OK, so for coding, actually sometimes I might use AI because it's just convenient because the truth is that we are trying to teach the students to a certain level, OK.

And and the AI today, while it may seem like magical people, they are doing things that are relatively competent but not expert level. OK, but the challenge is this, right, if the kids, the students today never even reach that level of minimal competency, they will never cross at the chasm. OK. So you can think of this curve, right? There's this like big part of people that then there's.

And and the AI today, while it may seem like magical people, they are doing things that are relatively competent but not expert level. OK, but the challenge is this, right, if the kids, the students today never even reach that level of minimal competency, they will never cross at the chasm. OK. So you can think of this curve, right? There's this like big part of people that then there's.

AI and there's this experts. I call this AI because of that. So if they're not able to bring the skill level to this level, they will never cross over to become experts, and that's where the money is, that's where the job. So in other words, you're saying they need to have a good foundation of that learning first before so that they can evolve later. But then I also argue like mathematics, we learn about, I know, simultaneous equations and all that. I used to tease my uncle as a math professor. I said what for at the end of the day, all I need to know is if I give $10 at the hawker center, I get back $5 change when I buy my Nasi Lemma.

AI and there's this experts. I call this AI because of that. So if they're not able to bring the skill level to this level, they will never cross over to become experts, and that's where the money is, that's where the job. So in other words, you're saying they need to have a good foundation of that learning first before so that they can evolve later. But then I also argue like mathematics, we learn about, I know, simultaneous equations and all that. I used to tease my uncle as a math professor. I said what for at the end of the day, all I need to know is if I give $10 at the hawker center, I get back $5 change when I buy my Nasi Lemma.

The rest, I mean, and I have a calculator, I have a computer, so I don't need those steps involved, right? So, so in a way.

The rest, I mean, and I have a calculator, I have a computer, so I don't need those steps involved, right? So, so in a way.

That foundation, yes, if it, some of it is lost, but some would argue that some of it is not needed in this day and age, and I used to learn chemistry. I could do all the chemistry equation isn't that what is doing? No, no, no, but you must understand what we're doing in school, right? OK, so I would tell, OK, I'm actually a teacher of teachers, so my students are actually MOE teachers. So I tell them actually you can teach anything you want, right, because the practice is that they are not in in K-12 to learn other equations. They're not there to learn, uh, you know, benzene rings, how to react with each other.

That foundation, yes, if it, some of it is lost, but some would argue that some of it is not needed in this day and age, and I used to learn chemistry. I could do all the chemistry equation isn't that what is doing? No, no, no, but you must understand what we're doing in school, right? OK, so I would tell, OK, I'm actually a teacher of teachers, so my students are actually MOE teachers. So I tell them actually you can teach anything you want, right, because the practice is that they are not in in K-12 to learn other equations. They're not there to learn, uh, you know, benzene rings, how to react with each other.

Right, they are there to learn how to learn. That's the key, and that's what we need in life, right? And let me address this this question which I try to address, which is that why, why don't we let the kids do what they have done in real life? The reason is very simple. It's because we are trying to get them to reach a certain level of mastery so that hope, I must admit, I mean, every math prof hopes that the kids will become math profs, and then the computer scientists.

Right, they are there to learn how to learn. That's the key, and that's what we need in life, right? And let me address this this question which I try to address, which is that why, why don't we let the kids do what they have done in real life? The reason is very simple. It's because we are trying to get them to reach a certain level of mastery so that hope, I must admit, I mean, every math prof hopes that the kids will become math profs, and then the computer scientists.

to be good programmers, right? So we are trying in good faith to help our students acquire skills to become masters in this. Some may want to. I agree with you, we should be doing that, but maybe we need to be doing it a different way now that the tools have changed. No, no, no, there's no difference because the learning is in your head. OK, no, no, no, let me explain to you what the difference is. OK, so first of all.

to be good programmers, right? So we are trying in good faith to help our students acquire skills to become masters in this. Some may want to. I agree with you, we should be doing that, but maybe we need to be doing it a different way now that the tools have changed. No, no, no, there's no difference because the learning is in your head. OK, no, no, no, let me explain to you what the difference is. OK, so first of all.

I want to make a very bold claim. There are no revolutions in education, you know, 1015 years ago that the MOOCs, right, came out and said, oh, we're going to change the world, and I'll be out of a job I last checked, I just had 800 students last semester. It's not gonna happen, and I tell you why, because actually,

I want to make a very bold claim. There are no revolutions in education, you know, 1015 years ago that the MOOCs, right, came out and said, oh, we're going to change the world, and I'll be out of a job I last checked, I just had 800 students last semester. It's not gonna happen, and I tell you why, because actually,

The business of education and teaching is not about communicating knowledge. We are in the business of managing motivation. OK,

The business of education and teaching is not about communicating knowledge. We are in the business of managing motivation. OK,

so can I ask

so can I ask

you, with large language models, you said your job is to motivate the kids, right? In general, with the with with LLM, would you say the motivation has gone up or gone down?

you, with large language models, you said your job is to motivate the kids, right? In general, with the with with LLM, would you say the motivation has gone up or gone down?

OK, so that's the problem. So that's why I was telling my students, my students who are not teachers that actually the trouble is that we may think that we are offer 20 years for good teachers. Yeah, maybe we have certain skills.

OK, so that's the problem. So that's why I was telling my students, my students who are not teachers that actually the trouble is that we may think that we are offer 20 years for good teachers. Yeah, maybe we have certain skills.

The trouble is that the clientele has changed. The kids have evolved, you know, and what has happened over the last 20 years? Mobile phones, social media, COVID actually also impacted the kids a lot. And, and now today we have LMs. So and then the internet also also affected everything, and that will keep changing. So you as a student just graduated, I mean.

The trouble is that the clientele has changed. The kids have evolved, you know, and what has happened over the last 20 years? Mobile phones, social media, COVID actually also impacted the kids a lot. And, and now today we have LMs. So and then the internet also also affected everything, and that will keep changing. So you as a student just graduated, I mean.

Do you agree with what Ben saying?

Do you agree with what Ben saying?

I

I

think for the most part it's

think for the most part it's

probably

probably

correct, but, but, but the implications of that, you know, if, if we really go with the implications of that and and I agree you're you're in the business of motivation, I think it's quite telling because what it means if you if you extrapolate it out is that students who are not motivated.

correct, but, but, but the implications of that, you know, if, if we really go with the implications of that and and I agree you're you're in the business of motivation, I think it's quite telling because what it means if you if you extrapolate it out is that students who are not motivated.

in the first place, they're going to go to school, they're not going to be motivated and it's going to be the same thing. They're gonna you're just keeping the same student, not motivated and then giving them a bad score. So that's the value for those people, which is the majority of the people who we are saying are not at that mastery, they're not at that level where they've passed the curve, and then for the other people that are going in and getting this validation. So the value, I don't see it basically in this case. So Jeremy, if you have to teach.

in the first place, they're going to go to school, they're not going to be motivated and it's going to be the same thing. They're gonna you're just keeping the same student, not motivated and then giving them a bad score. So that's the value for those people, which is the majority of the people who we are saying are not at that mastery, they're not at that level where they've passed the curve, and then for the other people that are going in and getting this validation. So the value, I don't see it basically in this case. So Jeremy, if you have to teach.

Aspiring students who want to ace their exams, how to use AI, right? What would you say to them? I think one of the things is I believe people are quite output focused, especially in today's world. So you can see that people want instant results and then after that they want to know how to replicate those instant results. It's very easy to hook people in if you show them that this is the output you could get and you just kind of need to figure out what's the secret sauce, what are the inputs that leads to this output. So I think actually one of the interesting things is with AI I have started touching on a lot of topics which previously I wouldn't have.

Aspiring students who want to ace their exams, how to use AI, right? What would you say to them? I think one of the things is I believe people are quite output focused, especially in today's world. So you can see that people want instant results and then after that they want to know how to replicate those instant results. It's very easy to hook people in if you show them that this is the output you could get and you just kind of need to figure out what's the secret sauce, what are the inputs that leads to this output. So I think actually one of the interesting things is with AI I have started touching on a lot of topics which previously I wouldn't have.

Because AI, when I ask, for example, Chat GBT currently I'm using Cloud, I'm using Gemini, some other tools as well, it starts telling me at a very high level, this is how it works, this is how it works. It condenses the information down to a 5 year old level, then I say, OK, cool. Now let's elaborate, let's go into this particular topic, let's dive even deeper. And so,

Because AI, when I ask, for example, Chat GBT currently I'm using Cloud, I'm using Gemini, some other tools as well, it starts telling me at a very high level, this is how it works, this is how it works. It condenses the information down to a 5 year old level, then I say, OK, cool. Now let's elaborate, let's go into this particular topic, let's dive even deeper. And so,

At all of these levels, I think my motivation constantly increases a little bit, a little bit, a little bit, and there's this compounding effect rather than I think what's more common in a lot of educational institutions is they start off kind of giving you the A, B's and C's, but you don't see that you're going to write an essay. So you don't see the story, you don't see where the final outcome really.

At all of these levels, I think my motivation constantly increases a little bit, a little bit, a little bit, and there's this compounding effect rather than I think what's more common in a lot of educational institutions is they start off kind of giving you the A, B's and C's, but you don't see that you're going to write an essay. So you don't see the story, you don't see where the final outcome really.

It's going to be and I think all throughout, it's just kind of this hazy black box. Students are not that motivated partly because the methodology is not there. But you also see students like obviously um abusing it, right? Was it the UCLA graduate we saw, you know, it went viral because he posted like he showed he was, you know, using

It's going to be and I think all throughout, it's just kind of this hazy black box. Students are not that motivated partly because the methodology is not there. But you also see students like obviously um abusing it, right? Was it the UCLA graduate we saw, you know, it went viral because he posted like he showed he was, you know, using

Yeah, GPT and he's graduated,

Yeah, GPT and he's graduated,

so should we stop and eliminate and create all these barriers to entry just because a small handful of people abuse.

so should we stop and eliminate and create all these barriers to entry just because a small handful of people abuse.

I think all systems are capable. So is that where again the institutions need to again set clearer defined lines like how to use what and what to use.

I think all systems are capable. So is that where again the institutions need to again set clearer defined lines like how to use what and what to use.

I think

I think

it be fair to universities, right? It's only been 2 years.

it be fair to universities, right? It's only been 2 years.

You know, it's not been like that. OK, it may seem like lifetime like around, uh, but, uh, I mean things evolve move not so quickly, right? And, and to be honest, we, we are also not completely, like I said, I admitted, right, we're not completely sure what the states are doing today, how to use it, and that's part of the learning process. So I, I, I think that the thing about fraud, like I said, the whole this whole about fraud is really about rules. I mean, the prof, OK, whoever is instructor has the right to set the rules, right.

You know, it's not been like that. OK, it may seem like lifetime like around, uh, but, uh, I mean things evolve move not so quickly, right? And, and to be honest, we, we are also not completely, like I said, I admitted, right, we're not completely sure what the states are doing today, how to use it, and that's part of the learning process. So I, I, I think that the thing about fraud, like I said, the whole this whole about fraud is really about rules. I mean, the prof, OK, whoever is instructor has the right to set the rules, right.

And the real question here is not whether you agree the rules or not, right, is that if the rule is broken.

And the real question here is not whether you agree the rules or not, right, is that if the rule is broken.

What, what should be our response, right? I mean, OK, you sound like anarchist. I like, OK, we just

What, what should be our response, right? I mean, OK, you sound like anarchist. I like, OK, we just

training to be slaves?

training to be slaves?

No, no, no, no, fair enough. So you say this is my class. This is what we're gonna learn. You are not allowed to use, for example, you're not allowed to use a calculator for this math equation, right? But you make it clear, right? If you agree to sign up for the course, you play by these rules. I think they did make it clear, right then on that basis that it.

No, no, no, no, fair enough. So you say this is my class. This is what we're gonna learn. You are not allowed to use, for example, you're not allowed to use a calculator for this math equation, right? But you make it clear, right? If you agree to sign up for the course, you play by these rules. I think they did make it clear, right then on that basis that it.

Prosecute and and and you know, and it didn't seem like

Prosecute and and and you know, and it didn't seem like

it's clear

it's clear

because it's still in limbo. I mean it it it's it's pending,

because it's still in limbo. I mean it it it's it's pending,

you

you

see. Why don't you just let NTU and the appeals panel do their work, right? So, so that would be the question just what is clear and what isn't clear. As they go through it, they will then have to further elaborate and then seek clarity. And even if it's not clear, I think it to be fair to the instructors.

see. Why don't you just let NTU and the appeals panel do their work, right? So, so that would be the question just what is clear and what isn't clear. As they go through it, they will then have to further elaborate and then seek clarity. And even if it's not clear, I think it to be fair to the instructors.

This is a brave new world we are entering and, and I, you know, so there are also many of them are also struggling to cope, uh, many of them are, I mean, many of us I think are ill equipped to, to,

This is a brave new world we are entering and, and I, you know, so there are also many of them are also struggling to cope, uh, many of them are, I mean, many of us I think are ill equipped to, to,

to know, but the tuition centers can do it really well because we, we service some other tuition centers. I have friends doing personal tuition. I know other business owners who own tuition centers. The teachers are kind of forcing it down on them and saying, OK, this is our business. We're at stake if we don't adapt, we're gonna die. So I'm gonna give you guys a suite of new tools. Learn how to use and teach your students with it.

to know, but the tuition centers can do it really well because we, we service some other tuition centers. I have friends doing personal tuition. I know other business owners who own tuition centers. The teachers are kind of forcing it down on them and saying, OK, this is our business. We're at stake if we don't adapt, we're gonna die. So I'm gonna give you guys a suite of new tools. Learn how to use and teach your students with it.

They they seem to be adapting fine.

They they seem to be adapting fine.

So, so you're saying that the tuition centers are teaching the students

So, so you're saying that the tuition centers are teaching the students

how to,

how to,

I mean they are using AI and then they understand that the students will use it as well. So the way they come up, that kind of ideas and the methods of verification, whether the work is good and whether they thinking is solid, they're coming up with brand new ways to. It's

I mean they are using AI and then they understand that the students will use it as well. So the way they come up, that kind of ideas and the methods of verification, whether the work is good and whether they thinking is solid, they're coming up with brand new ways to. It's

like here at the company they say we have this new system for operating your leave and all that.

like here at the company they say we have this new system for operating your leave and all that.

Use it or you can't take leave. So I learned how to use it real quick, right, same thing with AI, but this is new to me. I'm not aware but his argument being that we can, we can force this change to be faster and it feels to a certain extent that some of our educational institutions may be struggling to keep up with the rapid pace of change. Would you agree with that? I think we need to go back to what is our role in society, right?

Use it or you can't take leave. So I learned how to use it real quick, right, same thing with AI, but this is new to me. I'm not aware but his argument being that we can, we can force this change to be faster and it feels to a certain extent that some of our educational institutions may be struggling to keep up with the rapid pace of change. Would you agree with that? I think we need to go back to what is our role in society, right?

And I'm a little bit idealistic. I, I think our role is to help our students learn better, OK? And, and this thing about learning stuff, right? So, so the fact is this, you're right, you are complaining about learning.

And I'm a little bit idealistic. I, I think our role is to help our students learn better, OK? And, and this thing about learning stuff, right? So, so the fact is this, you're right, you are complaining about learning.

See these equations. OK. The fact is that many of the things they learn in college actually don't don't need to care about, OK, but it's part of the process. It's good to know more things. But if they're going to be a lawyer, a doctor, you know, you better learn it well, right? They're going to die, right, you know, and as a software engineer, they learn the craft very well. So, so for that, in terms of learning for the things that matter, then I, I think we need to do what is right, OK, and, and let me just tell you what I tell my students and I tell my teachers to tell their students, right?

See these equations. OK. The fact is that many of the things they learn in college actually don't don't need to care about, OK, but it's part of the process. It's good to know more things. But if they're going to be a lawyer, a doctor, you know, you better learn it well, right? They're going to die, right, you know, and as a software engineer, they learn the craft very well. So, so for that, in terms of learning for the things that matter, then I, I think we need to do what is right, OK, and, and let me just tell you what I tell my students and I tell my teachers to tell their students, right?

Which is a very simple argument, which is the following. If it is quite clear that if they use AI to do something they cannot do.

Which is a very simple argument, which is the following. If it is quite clear that if they use AI to do something they cannot do.

They will never be able to do it. OK. They will never be able to do it. On the other hand, if they can do it already, and then, you know, if they use AI to do it, they they can go home and get a cute girl, but if I if I see

They will never be able to do it. OK. They will never be able to do it. On the other hand, if they can do it already, and then, you know, if they use AI to do it, they they can go home and get a cute girl, but if I if I see

my other friends, they're using this chat GPT and they're scoring higher marks than me and then I hear the incentive structures.

my other friends, they're using this chat GPT and they're scoring higher marks than me and then I hear the incentive structures.

But

But

then you're saying good luck to this person because if he gets into the real world, maybe they won't be able to think.

then you're saying good luck to this person because if he gets into the real world, maybe they won't be able to think.

I think what you're going is that along the way we may be losing some of that learning experience. They may not be learning how to think, how to analyze. You now have a device or AI to do it for you.

I think what you're going is that along the way we may be losing some of that learning experience. They may not be learning how to think, how to analyze. You now have a device or AI to do it for you.

So at the end of the day, I I I do think that the people who actually put the effort, right, will have much better chance of surpassing that that barrier with the AI barrier and become experts. So you are learning from the best, learning how he has learned how to use technology and teaching that to the next generation of students, which also means being a bit more open to allowing the technology to come in. OK, I don't think we're going to come to a clear black and white answer today, but I guess my view is that I think we need to evolve a bit more and to be a bit more open.

So at the end of the day, I I I do think that the people who actually put the effort, right, will have much better chance of surpassing that that barrier with the AI barrier and become experts. So you are learning from the best, learning how he has learned how to use technology and teaching that to the next generation of students, which also means being a bit more open to allowing the technology to come in. OK, I don't think we're going to come to a clear black and white answer today, but I guess my view is that I think we need to evolve a bit more and to be a bit more open.

To the use of technology and that we need to change our way of assessment to find what is the best in that person and to see whether they've actually learned how to figure it out. I think that the truth is that the, you know, profs actually know their business, they know what they're trying to do, what they trying to achieve for the students, and there is no one size fits all kind of solution, and I hope you believe that.

To the use of technology and that we need to change our way of assessment to find what is the best in that person and to see whether they've actually learned how to figure it out. I think that the truth is that the, you know, profs actually know their business, they know what they're trying to do, what they trying to achieve for the students, and there is no one size fits all kind of solution, and I hope you believe that.

NTU NUS, all of us are trying to do our best, the right things for students, right? So they don't punish students because it's fun to do because they like the publicity like this, it's not something that they do because I think that's why

NTU NUS, all of us are trying to do our best, the right things for students, right? So they don't punish students because it's fun to do because they like the publicity like this, it's not something that they do because I think that's why

they're taking it seriously.

they're taking it seriously.

They're still deliberating

They're still deliberating

over

over

the other

the other

students, yeah, and then this is this is a learning platform for all of us, right, the students, the institutions, the lecturers, and hopefully from here on, you know, we'd be able to.

students, yeah, and then this is this is a learning platform for all of us, right, the students, the institutions, the lecturers, and hopefully from here on, you know, we'd be able to.

At least the students would have just a clearer idea of like how, how, how do you make sure that you are learning it properly and and not, I won't say not getting caught, but you know you want to use,

At least the students would have just a clearer idea of like how, how, how do you make sure that you are learning it properly and and not, I won't say not getting caught, but you know you want to use,

you made a good point. I think what I what I'm gonna ask you is, so if I'm a student now, should I go and ask my professor A prof. So for this next project, can I use AI? What can I do? What can I not do and get those boundaries set up.

you made a good point. I think what I what I'm gonna ask you is, so if I'm a student now, should I go and ask my professor A prof. So for this next project, can I use AI? What can I do? What can I not do and get those boundaries set up.

As you mentioned, it's different for every subject, every different area. I think I need to tell you what is available, what is allowable, not, not, I come to my prof and I ask you what can I use it for? What are you allowing me to use but I should assignment to you. I will tell you explicitly. That's right, right. And then that's the approach that actually the school is pro proposing, which is that the profs figure it out and communicate this clearly, but like I said.

As you mentioned, it's different for every subject, every different area. I think I need to tell you what is available, what is allowable, not, not, I come to my prof and I ask you what can I use it for? What are you allowing me to use but I should assignment to you. I will tell you explicitly. That's right, right. And then that's the approach that actually the school is pro proposing, which is that the profs figure it out and communicate this clearly, but like I said.

Be be kind to the props. I mean, we have enough students to do with, you know, this is a new thing. It will take some time, I think, for, for, you know, the vast majority to actually, you know, figure this out, and, and, but generally I think the, the guidelines is that the school does, uh, ask that the instructors be explicit what is the AI policy, right? And generally like I said, yeah you can use it, you cannot use it, you can use it normally you need to declare. So at the end of the day, if it's clear.

Be be kind to the props. I mean, we have enough students to do with, you know, this is a new thing. It will take some time, I think, for, for, you know, the vast majority to actually, you know, figure this out, and, and, but generally I think the, the guidelines is that the school does, uh, ask that the instructors be explicit what is the AI policy, right? And generally like I said, yeah you can use it, you cannot use it, you can use it normally you need to declare. So at the end of the day, if it's clear.

And everybody knows what is allowed, what is not allowed, then there's no argument. No no no not so simple.

And everybody knows what is allowed, what is not allowed, then there's no argument. No no no not so simple.

Yeah

Yeah

We

We

have run out of time

have run out of time

that it's clear. Unfortunately, the students these days, they will try to wriggle.

that it's clear. Unfortunately, the students these days, they will try to wriggle.

The way around that. I have no comment on that because we don't know the case well. I can't agree or disagree whether it's clear or not, so we will have to let it play out on its own and we will hear what the final outcome is. I'm sure we'll get there. A, it's here to stay and it's a part of our lives, so there's no running away from it. OK. Alright.

The way around that. I have no comment on that because we don't know the case well. I can't agree or disagree whether it's clear or not, so we will have to let it play out on its own and we will hear what the final outcome is. I'm sure we'll get there. A, it's here to stay and it's a part of our lives, so there's no running away from it. OK. Alright.

And

And

uh that wraps up this week's edition of Deep Dive. Big thanks to Tiffany Ag, Junai Juhari, Joann Chan, Sae, Hoeing.

uh that wraps up this week's edition of Deep Dive. Big thanks to Tiffany Ag, Junai Juhari, Joann Chan, Sae, Hoeing.

Shaza Dalila, Ellison Jenner and video by Marcus Ramos, Faith Ho and Hanida Armin.

Shaza Dalila, Ellison Jenner and video by Marcus Ramos, Faith Ho and Hanida Armin.

Thank you. You know that's one day we're gonna take a video and show you who's in the team because you're probably thinking, who are these people, you know, behind us, of course, with everything we do here is a big team effort. So thanks so much for joining us. Any comments, drop us a note. Love to hear from you. See you next week.

Thank you. You know that's one day we're gonna take a video and show you who's in the team because you're probably thinking, who are these people, you know, behind us, of course, with everything we do here is a big team effort. So thanks so much for joining us. Any comments, drop us a note. Love to hear from you. See you next week.