via The New York Times
SAN ANTONIO — Several Texas A&M professors know something that generations of teachers could only hope to guess: whether students are reading their textbooks.
They know when students are skipping pages, failing to highlight significant passages, not bothering to take notes — or simply not opening the book at all.
“It’s Big Brother, sort of, but with a good intent,” said Tracy Hurley, the dean of the school of business.
The faculty members here are neither clairvoyant nor peering over shoulders. They, along with colleagues at eight other colleges, are testing technology from a Silicon Valley start-up, CourseSmart, that allows them to track their students’ progress with digital textbooks.
Major publishers in higher education have already been collecting data from millions of students who use their digital materials. But CourseSmart goes further by individually packaging for each professor information on all the students in a class — a bold effort that is already beginning to affect how teachers present material and how students respond to it, even as critics question how well it measures learning. The plan is to introduce the program broadly this fall.
Adrian Guardia, a Texas A&M instructor in management, took notice the other day of a student who was apparently doing well. His quiz grades were solid, and so was what CourseSmart calls his “engagement index.” But Mr. Guardia also saw something else: that the student had opened his textbook only once.
“It was one of those aha moments,” said Mr. Guardia, who is tracking 70 students in three classes. “Are you really learning if you only open the book the night before the test? I knew I had to reach out to him to discuss his studying habits.”
Students do not see their engagement indexes unless a professor shows them, but they know the books are watching them. For a few, merely hearing the number is a shock. Charles Tejeda got a C on the last quiz, but the real revelation that he is struggling was a low CourseSmart index.
“They caught me,” said Mr. Tejeda, 43. He has two jobs and three children, and can study only late at night. “Maybe I need to focus more,” he said.
CourseSmart is owned by Pearson, McGraw-Hill and other major publishers, which see an opportunity to cement their dominance in digital textbooks by offering administrators and faculty a constant stream of data about how students are doing.
In the old days, teachers knew if students understood the course from the expressions on their faces. Now some classes, including one of Mr. Guardia’s, are entirely virtual. Engagement information could give the colleges early warning about which students might flunk out, while more broadly letting teachers know if the whole class is falling behind.
Eventually, the data will flow back to the publishers, to help prepare new editions.
Academic and popular publishers, as well as some authors, have dreamed for years of such feedback to direct sales and editorial efforts more efficiently. Amazon and Barnes & Noble are presumed to be collecting a trove of data from readers, although they decline to say what, if anything, they will do with it.
“It’s Big Brother, sort of, but with a good intent.”
The predigital era, when writers wrote and publishers published without a clue, is seen as an amazingly ignorant time. “Before this, the publisher never knew if Chapter 3 was even looked at,” said Sean Devine, CourseSmart’s chief executive.
More than 3.5 million students and educators use CourseSmart textbooks and are already generating reams of data about Chapter 3. Among the colleges experimenting this semester are Clemson, Central Carolina Technical College and Stony Brook University, as well as Texas A&M-San Antonio, a new offshoot.
Texas A&M has one of the highest four-year graduation rates in the state, but only half the students make it out in that time. “If CourseSmart offers to hook it up to every class, we wouldn’t decline,” said Dr. Hurley, the dean.
At a recent session here of a management training class, Mr. Guardia addressed how to intervene efficiently with underperformers. The students watched a video of a print shop manager chewing out an employee without knowing the circumstances. The moral: The manager needed better data.
Then Mr. Guardia discussed with his students the analytics of their own reading, which he had e-mailed to them. The students suggested that once again better information was needed. Several said their score was being minimized because they took notes on paper.
Others complained there were software bugs, a response Mr. Guardia has heard before. The student who was cramming at the last minute said, for example, that he had opened the textbook several times, not just once. Perhaps these are the digital equivalent of “the dog ate my homework.” CourseSmart said it knew of no problems with its software.
The start-up said its surveys indicated few privacy concerns among students or colleges, and this was borne out by the class. “Big Brother,” said one student, but that was a joke, and everyone snickered. Being watched is a fundamental part of the world they live in.
“Amazon has such a footprint on me,” said Carol Johnson, 51, who works in the tech industry. “It knows more than my mother.”
Chris Dede, a professor of learning technologies at Harvard’s Graduate School of Education, is more apprehensive. He believes analytics are important in the classroom, but they must be based on high-quality data.
The CourseSmart system has other potential problems; students could easily game the highlighting or note-taking functions. Or a student might improve his score by leaving his textbook open and doing something else.
“The possibilities of harm are tremendous if teachers are naïve enough to think these scores mean anything for the vast majority of students,” Professor Dede said.
CourseSmart says the data it collects now is a beginning. “We’ll ultimately show how the student traverses the book,” Mr. Devine said. “There’s a correlation and causality between engagement and success.”
There is also correlation, the students are learning, between perception and success.
Hillary Torres, a senior, is a good student with a low engagement index, probably because she is taking notes into a computer file not being tracked. This could be a problem; she is a member of the Society for Human Resource Management, whose local chapter is advised by Mr. Guardia. “If he looks and sees, ‘Hillary is not really reading as much as I thought,’ does that give him a negative image of me?” she wondered. “His opinion really matters. Maybe I need to change my study habits.”
After two months of using the system, Mr. Guardia is coming to some conclusions of his own. His students generally are scoring well on quizzes and assignments. In the old days, that might have reassured him. But their engagement indexes are low.
“Maybe the course is too easy and I need to challenge them a bit more,” Mr. Guardia said. “Or maybe the textbooks are not as good as I thought.
