In our recent story about Google buying BrightBytes, there was a lot of discussion about what kind of data is useful for understanding learning, and the general limits that educational data has on measuring the things that we really care about.
One area where there might be some promise is with predicting whether a student is likely to fail a course or drop out of college. A recent research paper tested whether “clickstream data” from a Learning Management System (LMS) in community colleges could outperform “administrative data” like HS GPA, previous course grades, and demographic information.
The most striking finding is that previous college course grades outperformed any “clickstream data”. This means that for any students who had previous experience in college courses, those course grades were going to be far more informative and useful than information about what the student was clicking on or where in the LMS they were spending their time.
But for students who were arriving at their first semester of college, adding the LMS “clickstream data” to the administrative data that followed the student increased the accuracy of the prediction.
These sorts of prediction models are already being used at many universities around the US, although most of them only use the “administrative data” to identify people who might be struggling or forecast failure rates and dropout rates for the campus as a whole. Introducing data from an LMS
One of the hard parts about educational data sometimes is knowing what outcomes to optimize for, but student failure rates and dropout rates can be a useful place to start and the implication behind this data is that high school performance is a far less accurate predictor of success than college performance.
The paper does not address why HS GPAs are less useful than college GPAs. is could be a question of rigor, and in some ways that would validate the concerns around grade inflation. although that doesn’t mean that standardized tests are better at judging first year student performance. It just forces us to re-evaluate some of the ways that we prepare our students for college.
It should also be noted that the study was only done within the Virginia Community College System (VCCS) because that was the system that the researchers had a high degree of access to. So the point about HS GPAs being a poor predictor could be hiding a finding that the GPA earned within one system is a mediocre predictor of the GPA earned in another system. Since transfer rates from post-secondary institutions into community colleges are low, and often little GPA data accompanies a student who takes a community college course while also being enrolled at a 4-year institution, it is unclear whether a student in their first semester within a VCCS school could experience a performance gap irregardless of whether they had previous college experience or just high school experience.
University of the People shows how accessible education could look in the future
One of the more low-key but compelling talks at SXSW EDU this year was about a new kind of University....
Grading ChatGPT: How to get students to work with AI
Buried in a panel called Learning in the Age of AI and Technology at SXSW EDU, Paul Kim, the Chief...
AI, and especially ChatGPT, was everywhere at SXSW EDU
Nearly every presenter brought up AI or specifically ChatGPT at SXSW EDU over the course of it's four days. Several...
Digital Game Based Learning with Dr. André Denham
Gamification has long been a buzzword in education but studying the ways to make games truly useful is becoming more...
6 Questions to ask people when they try to sell you their EdTech Product
SXSW EDU starts today in Austin, Texas, and with any education convention comes dozens and dozens of people trying to...
5 things to check out on Monday at SXSW EDU
This is the first year we will be covering SXSW Edu and we are looking forward to some fascinating seassions....