Rising tides
It’s not a competition, of course. Because everyone would celebrate and eventually benefit from test scores rising across all categories of students.
It remains notable, however, that girls widened their margin over boys in the recently announced results for the 2018 National Assessment of Educational Progress Technology and Engineering Literacy (TEL) test. In 2014, girls averaged three points higher than boys; in the 2018 results, this gap increased to five points.
Snapshots on STEM
Overall, scores on the test involving 15,400 eighth-graders in 600 schools increased by two points, up to 152 out of a possible 300. Many familiar features of the STEM education landscape cropped up: achievement gaps based on race and ethnicity, persistent divides correlated to income, and the mutual reinforcement of learning inside and outside of school. Between the lines, though, the results also opened up some provocative questions about the nature of technology and engineering coursework available to these test-taking students. More on that below.
TEL is structured around three core content areas and three core practices.
Test mechanics
TEL measures students’ performance in areas of both content and practice. Subject matter areas include:
Technology and society, or effects and ethical dimensions of technology’s impact on the world.
Design and systems, or how people produce and manage technology.
Information and communication technology, or all the tools and procedures people use to express themselves.
The test also asks students to apply ways of thinking and reasoning about technology, or to put “practices” to use:
Understanding technological principles, or using knowledge about technology.
Developing solutions and achieving goals, or solving problems with technology.
Communicating and collaborating, or using technology to communicate for various purposes.
No teaching to this test
TEL is given entirely on computers and involves scenario-based tasks. Students cannot be prepared specifically for the test, so their performance reflects more fully just the reasoning and intellectual capabilities they bring to taking it.
For example, one scenario asks students to select an image suitable for use in an online media campaign promoting a TV show about the Andromeda galaxy, as well as secure permission to use it and appropriately credit the copyright holder. Students make choices among options presented through the course of the scenario that reveal what sense they have made of the instructions and subject matter content embedded within the scenario. It’s actually really fun, grounded in real-world circumstances, and cognitively cross-cutting. You can take the task to see for yourself.
Digging into the results
As noted, 2018 scores on TEL rose to 152, with 46 percent of students scoring at or above proficient. The largest increases in average score appeared in the higher percentiles of results. At the 90th percentile, scores increased from 193 to 197. At the 10th percentile, scores remained the same at 104. At intervening percentiles, scores increased by two or three points.
TEL results showed continuing achievement gaps among different groups of students.
How it breaks down
Achievement gaps also correlated to demographics, though across the board nearly all student groups increased their scores:
African-American students averaged 132; Hispanics, 139; Asians, 169; white students, 163.
Students eligible for free and reduced lunch averaged 138; those not eligible, 164.
Rural students averaged 152; urban students, 147; suburban students, 156.
Test results also bore out the enduring emphasis on STEM education in the K-12 arena, with achievement correlating to learning and exposure inside and outside of school.
The number of students reporting taking classes in areas related to engineering or technology increased from 52 to 57 percent.
Class-taking students registered an average score of 156, compared to 148 for students not taking such classes.
Students who "often" or "sometimes" learned about or even just discussed TEL-related topics outside of school scored better than those who did not, by margins between five and 17 points.
Both formal and informal encounters with technology and engineering topics seem to improve students’ understanding and abilities in these areas.
A paradox appears
Comparing course-taking and test results among boys and girls, though, reveals a puzzling disconnect. More boys than girls reported taking an engineering- or technology-related class, 61 percent compared to 53 percent. Yet girls performed better by an even wider margin in 2018 than in 2014.
Given the innovative, cross-curricular nature of TEL, it is easy to imagine that the courses boys are taking in greater numbers focus more on narrower, technical content than on the synthesizing, multi-disciplinary abilities rewarded on the test. Anecdotal evidence about middle school course options would seem to support this idea, but it would take much more rigorous study to flesh out this idea. At any rate, the results bear out the notion that TEL is not a test that’s easy to teach to.
Broadening the appeal
The Department of Education administers TEL every four years. So far given only to eighth-graders, the framework for testing is meant to extend to fourth- and twelfth-graders, too, but high costs have so far kept this from happening. Which is bad. As a learning vehicle, NAEP TEL has clearly solved some of the problems with inclusiveness and relevance that limit the appeal of STEM education to wider groups of students. For both students and educators, wider exposure to the test could help illuminate solutions to STEM pipeline problems that take root in K-12 and bedevil later STEM study and career environments of all kinds.
Eric Iversen is VP for Learning and Communications at Start Engineering. He has written and spoken widely on engineering education in the K-12 arena. You can write to him about this topic, especially when he gets stuff wrong, at eiversen@start-engineering.com.
You can also follow along on Twitter @StartEnginNow.
Our new Cybersecurity Career Guide shows middle and high schoolers what cybersecurity is all about and how they can find the career in the field that’s right for them. Now with a Student Workbook for classroom or afterschool use!
To showcase STEM career options, pair our cybersecurity books with the recently updated 3rd edition of our Start Engineering Career Guide.
We’ve also got appealing, fun engineering posters and engaging books for PreK-2 and K-5.
Our books cover the entire PreK-12 range. Get the one that’s right for you at our online shop.