The Tool is Not Your Course - The Labster Podcast, Episode 13
April [00:00:04] Hey everyone, I'm April, and you're listening to The Labster Podcast. I'm proud to say that at Labster we are guided by our mission to empower the next generation of scientists to change the world and contribute to solving global challenges. If you're an educator listening to this podcast, we know you share that mission. So thank you. With me, as always, is my friend and fellow Labsterite SJ Boulton, an educational designer and former university lecturer who now develops Labster's virtual lab simulations for students and high school, college and university.
April [00:00:43] In today's episode, we'll get to speak with Dr. Karen Vignare, a strategic innovator who is working to improve access, success and flexibility within higher education by leveraging digital technologies. Karen is the Vice President of Digital Transformation for Student Success and Executive Director for the Personalized Learning Consortium at the Association of Public and Land Grant Universities. As part of her role at the APLU, Karen oversees the adaptive courseware grant, providing leadership and support to eight pioneering universities that are scaling adaptive courseware in introductory level courses. She is also published extensively on online learning, analytics and open educational resources. So with that, welcome to the podcast Karen.
Karen [00:01:36] Thanks, April. Nice to be here. Looking forward to a great conversation with you and SJ.
SJ [00:01:41] So I've been diving into a little bit of the literature that you've published of the past a while. And it's been a few real eye openers for me as somebody who's working with an ad tech kind of from the inside out, we're learning more about how people are working to increase educational access and reduce equity gaps, especially in higher education. Is is really it's so common for me to know that somebody has got a good handle on it and that's just been there. So as somebody who's dedicated so much of the time and work to increase in the educational access and reduce equity gaps, I was wondering what some of the ways that you feel institutions can start the process of transitioning away from the so-called data to educational and teaching processes that leave some students unable to succeed, especially in those high priority or foundational gateway type courses? How can they get started with that?
Karen [00:02:38] Yeah, SJ. I think our recommendation is really that institutions and organizations think about departmentally approaches. We recognize that faculty have been innovators for a very, very long time and we thank those faculty for doing that kind of innovation work and making a difference for the students in their own individual classes. But going at this problem one at a time is simply not fast enough. It's not collaborative enough.
Karen [00:03:15] And so in our work, what we really recognize is the place where we need to get students started is really in those introductory courses. And we we know from the data that there is a wide variety of students who come to those classes, students who could have gone to extremely poorly resourced institutions, students who simply didn't have at their high schools certain levels of courses being taught. And then we have incredibly talented students. So our faculty are often faced with teaching to a wide range. And we certainly believe that the experience we've all lived through and Covid has actually increased that wide variety in terms of the learner preparedness that comes for us. On the other hand, we really encourage people to think about the fact that there's a lot of great learning science research that tells us about the capability of humans to learn if learning is structured in a way that engages them and quite honestly motivates them.
Karen [00:04:38] So for that kind of approach, we need some levers and we often look at those levers as assistance from educational technologies. Now, we would say that you can probably use some of your technologies like the learning management system better. But oftentimes there are tools out there, let's say adaptive courseware. And I do believe probably the products that you guys have have the same kind of data. And it is about the data.
Karen [00:05:15] Faculty need to know much sooner how students are doing in the classroom. There's some absolutely phenomenal research by a colleague of mine, Chuck Dziubin, and his partner, Dr. Patsy Moskal. And both of them have looked at courseware, essentially, if we don't address learners within and this is in a quote unquote US semester of 15 weeks, if we don't address those learners very early, probably before three weeks, their trajectory downward cannot be reversed.
SJ [00:05:56] What does addressing the learner mean to you?
Karen [00:05:59] So addressing the learner means knowing a lot more - and I want faculty to hear this very carefully - knowing a lot more about where the understanding of their learning is. And what that means is do they understand the kinds of concepts and can they do the practice and assessment, regular assessment, the formative regular assessment in a way that shows they are progressing through the content. And this is important because I do believe faculty have done a great job hearing, we need to have varied kinds of assessment.
Karen [00:06:46] We need to be make sure that when we create content, it is culturally relevant, it's inviting to students. But I think faculty don't have this information in a data set that helps them visualize how any particular student is progressing.
SJ [00:07:08] So are there specific types of analytics that you feel are appropriate? Do you believe that there should be a more personalized approach for these adaptive technologies?
Karen [00:07:18] Yes, but yes is very complicated. So when it comes to the analytics, so often here in the US, many of our institutions have started down the pathway of using all kinds of analytics. And they sort of came in a construct what we would talk about in terms of predictive analytics, but predictive analytics, by and large, use demographic data.
Karen [00:07:48] What we're really talking about in the learning data is behavioral data. We're looking for data that helps us understand whether a student is working on a problem, looking at content, being able to actually complete an assessment, or if they haven't actually done any of that work. How do we reach out to them and show them that process? Right. And we're hearing anecdotally from those that had been using some of these tools and and not just anecdotally, there's been some qualitative surveys that came out of Every Learner Everywhere, led by Digital Promise that shows faculty that had been using these tools. By and large, if we compare them to faculty using tools that say they use tools, that they understood the importance of data analytics twice and double the rate that what faculty nationally had been thinking about and why is that important?
Karen [00:08:55] The importance is we're not saying that this data is the only thing happening in the class. Everything that our faculty do in the class adds value. But if you don't know how students who are coming to you and I think SJ, you said this very well, with this even wider variety of skills and preparation are actually doing very early on that keeping the student who for whatever reason, whether it is an unfortunate incident of whether their Internet connection is working or whether it is more fundamental about their learning preparedness. If we don't know that, we can't guide them to where they need to be.
Karen [00:09:41] The activity planned for the classroom has to be based on where the students are. And some people will say, well, I have students who have no knowledge and I have other students who are already prepared to to get to sort of material and have the class. Well, for a long time, our faculty have done tremendous work around, you know, setting up groups in order to inspire peer learning. There are opportunities here for faculty to use tools and techniques, even in the online classroom that can actually bring that classroom further along, including everybody.
Karen [00:10:23] So for us, the analytics are 'how are they learning?' So these are the learning analytics.
Karen [00:10:31] And while we would love to say in a traditional class, like maybe just using an LMS, you might be able to see most of that data is very static. And what I mean by that is you're having students check in once, twice, three times a week, but you're not having them actively show you something that they're learning through. And so that static kind of information isn't as helpful as something that is actually taking through a well-designed learning exercise that shows us a student can get through this process. Maybe it's slower, maybe it's faster.
Karen [00:11:15] But we need to know a lot more at the granular level about their learning behavior rather than assuming that their demographic information is going to predict how they do in the classroom.
SJ [00:11:32] No, because it often doesn't.
Karen [00:11:34] Absolutely. I agree,
SJ [00:11:36] And it's far too general to say that a person because of where they've come from, I'm a first generation university student that comes from an area that's historically very low tech. And I'm not sure what demographic data would have accurately predicted my trajectory. Much like many of the people that were like me with my cohort. So I'm very much with you on shifting away from demographic predictive data sets. And instead of looking at something that looks at what's happening right now and what are the attitudes in this moment, it's so refreshing to hear somebody take that approach from my perspective.
Karen [00:12:11] Well, and I think I'd want to add that doing this in a collaborative way. And the reason I started off by saying we really think the department and there is new research and literature, sort of like in the STEM community around departmentally action teams. Right. But thinking about this, I mean, the university is often structured through the provost, then to the deans, then to the department chairs. The activity that we think are critical is that this is a collaborative approach.
Karen [00:12:46] And we already talked about those pioneering faculty who we we've learned so much from. But if we don't think about this work in a way that all faculty teaching that course are collaborating on both the instructional sort of set up and design the instructional design of the learning, but also in that critical period of what we would call instruction or instructional management, then we don't have a student first approach because we are allowing a wider variety of learning across to our faculty.
Karen [00:13:26] And what that teaching, I should say, what that means is you students are at risk, unfortunately, of encountering a faculty who has not had the same kinds of training that another faculty member has had. And we need to bring that together. We don't think of that as customization. We don't think of that as standardization. What we think about is that we as individuals that - we in the learning design community as as individuals and as researchers have a lot of information to help faculty redesign their courses. But doing it in a collaborative way allows faculty to begin sharing much more about how they help students because they do help students. Right. And what we know is, I mean,.
Karen [00:14:27] While our work can be more scientific, it is still very much a human based activity. We don't know how to motivate every single student and faculty need to collaborate on what has worked for them in the past to help motivate these students through learning experiences. And this really takes a collective, collaborative approach. When we think about those courses that are particularly important, those introductory courses, those priority courses, those courses that are often the ones that keep our students from being successful in actually making it, graduating, retaining, learning enough so that they can move through the process.
SJ [00:15:18] So we've heard or we've learned that these early adopters, like faculty, like pioneer teams, are hugely important in kind of setting the tone or I wonder if it's more about connecting the preliminary data to show that something is useful, to show that something can be done. I was wondering if you could talk a little bit about the importance of those pioneer teams, how they get started or where they come from, who kind of initiates that in general, and how they can set the tone for potential future scale and institution. They seem very important.
Karen [00:15:51] Yeah, SJ. I think for the audience perspective, we've had the privilege of writing a guide for implementing adaptive courseware from planning through scaling. And then there has been another publication that I'm also particularly important for this approach. It's around improving critical courses, using digital evidence based pedagogy and those to really focus on this phased approach from pioneer, from innovation to scale. And they focus on the team approach. Right.
Karen [00:16:27] So one of the things that I think has been missing in our faculty pioneers is they are looking for evidence, but they have to do all of this work on their own and they have to sustain it, right. So now if you think about this work where the faculty teaching, so it becomes a collaborative team and that team includes folks like instructional designers when available and includes folks from the institutional researcher, institutional effectiveness. It includes an executive team member, meaning not just the department chair, but probably a dean or a provost or a vice chancellor. And those folks, collectively, they will all have different parts but are working at this problem.
Karen [00:17:17] So it doesn't mean that we won't still pilot and try things, but it means that the pilot is not itself the event. It means that we are going and we are committing to a process. This process is a change management process and it says it's a continuous improvement process.
Karen [00:17:40] There are pilots that don't go well. We do recognize that. But the pilot shouldn't necessarily be abandoned, right? It should be improved. So did we not use the right technologies or did we not have the right implementation?
Karen [00:17:57] One of the things that I do really want to say to the audience, because we're often asked this question, 'please just tell me what is the right tool to use'. And that sort of really minimizes everybody's role in this work. The fact is there may be some tools that work better. But what we're really trying to say is you have to work through a process, find the tools that support your students, and then make sure you do the work of actually aligning those tools and the information those tools provide in to your assessment and your continuous improvement process.
Karen [00:18:43] So, so many people, when they ask that question, it makes us think - and I hope that's not really the case - it makes us think all you want is a tool and everything will be done. Know the tool is absolutely important. It's a lever. It provides you with data. It provides you with a guide on where I might try different things in the rest of my classroom. But it is not your course. It requires you to integrate into your entire course to make sure it's aligned.
SJ [00:19:21] Which I find very refreshing. It's not just we've selected this tool. We've committed to a tendering process. We're going to buy this thing and it's down to, you know, faculty to make it work. That doesn't seem to be the approach it's described here is very much about. Let's try something out. Let's learn from it. Let's pivot and make sure that we address why something didn't work, if it didn't work or if it does work. Let's do more of that for me. I would have loved to have been involved with something like that as a faculty member. There wasn't the same pressure as there is now and a kind of COVID post-COVID landscape.
SJ [00:19:54] I imagine that one of the greatest resistances that you might have experienced or that you've observed is that age old thing of time. And folks do say, just give me the thing that works and tell me what has worked for you. And I will just do that because they feel there isn't the time to necessarily implement test retrospective and learn from that that implementation. They just want to get something and get it done and make it work. Do you feel that time is a big pressure on this? Do you feel that there needs to be a more institutional consensus on, yes, we're going to take this forward and we're going to leverage the right people at the right time to really commit to figuring out what's best?
Karen [00:20:33] Definitely time is one of the factors that are really important when institutions and departments think about this work. And that's why we often talk about aligning faculty incentives. So how are you rewarding faculty? How are you rewarding their participation in this work? How are you changing your meeting structure? So does the department even talk about teaching? Because we have plenty of departments where people say it never comes up. How is teaching going this semester?
Karen [00:21:09] The truth is we have to think about how we use our time and what we spend time on. I don't think the issue is there is a lack of time. It's really how the time is spent. And we need to adjust that because in order to get sustainable change, we're going to need to change how we approach the problem.
Karen [00:21:34] And in a collaborative, project-managed way, we're now putting together multiple people with with cross disciplinary experience, instructional experience, learning, science, experience, technology experience and that all important faculty, subject matter, knowledge, experience. And by having that work collectively, we can actually achieve improved outcomes.
Karen [00:22:02] One of the things I'm extremely proud about is in the large grant that we ran. One of the findings were in 300,000 enrollments, we were able to prevent 8700 students from having to repeat. Of course, now we may think of that and say, well, why not more? But for those students, this was real. They saved $16.5 million. Right. So this is actually really important when we put a student first hat on. And when we think about the way we spend time, we have to recognize our faculty work very hard. We recognize that. But what we ask them to do and how we plan for that activity becomes incredibly important if we're going to do this work.
SJ [00:23:05] Doing the testing and forming the strategy and ensuring that the right people are at the table, but it seems to pay dividends afterwards. Karen time is coming to a close already. And I wanted to ask, is there anything we haven't covered today or anything we haven't talked about that you would have liked to have talked about whether you would like to have Covid.
Karen [00:23:27] Fundamentally, we believe humans are extremely capable of learning. We believe faculty have the best intent and we believe that we're going to have to use technologies and instructional design and learning science in better ways to make that student learning actually materialize and catapult. There is a need, I think, really across the world, to have more educated people and we have to start putting the effort into doing this much more collaboratively for the sake of those students that we've left behind in the past.
SJ [00:24:14] That's amazing. Thank you so much.
Karen [00:24:16] Thank you. Stay and thank you, April.
Karen [00:24:22] Thank you, Dr. Karen Vignare. This has been a fascinating conversation on making a commitment to collaborating on a process that can drive student success by transforming teaching and learning. Dr. Vignare touched on the importance of using behavioral learning data from adaptive courseware in order to personalize learning. We hope this conversation has helped to spark some new ideas for you about how you can get the greatest benefit from the digital learning tools available to you. And as we close out, I want to thank you, our listeners. We'd love to hear your thoughts about this episode. And we invite you to share them with us at Labster dot com slash feedback. Thanks for listening. If you like this episode, we hope you'll share it with a fellow teacher and subscribe to The Labster Podcast until next time. Keep teaching, keep learning and stay safe.
