Sunday, April 24, 2016

Motivating STEM Engagement in Children, Families, and Communities

By Daniel Hickey

 This extended post is the second in a series of post about my work with non-profit STEM education start up called Iridescent.  In a previous post I describe their Curiosity Machine and an evaluation I carried out of a program designed to encourage families to complete Engineering Design Challenges on that website from home, after attending two-hour weekly evening workshops. In this post I explore the challenges that organizations like Iridescent and science museums face in attempting to motivate inquiry learning in informal contexts. I am also using this post as my weekly contribution to the graduate education class on motivation that I am currently teaching.

In the last year, I have had the pleasure of working with Iridescent Inc. a science education non-profit in LA. Iridescent was founded in 2006 by an aerospace engineer named Tara Chklovski. Tara and her colleagues are bringing three core engineering design practices (curiositycreativity, and persistence) to girls, boys, and families, with a particular focus on underserved minoritie
Over the last two years, Iridescent ran a program that worked with parents and their elementary-age children in two-hour workshops to help them complete online Design Challenges. Each Design Challenge features inspiring videos with practicing engineers who exemplify the engineering design practices while they talk about their work and introduce a handful of more specific engineering design concepts (such as lift and thrust). As shown below, the home page of each Design Challenge features a simple interface that allows even young children to keep track of their progress as they (a) build a device in an effort to meet a specific engineering design challenge, (b) post a photo or a video as evidence, (c) get feedback from virtual mentors, (d) redesign their device, and (e) post a final reflection. Over 50 Design Challenges are nicely arranged on the Curiosity Machine website that keeps track of each family's progress. As I elaborated in a recent blog post, I helped Iridescent with a modest evaluation of a project aimed at helping families engage in inquiry-oriented science learning.  

Home Page of an Engineering Design Challenge on the Curiosity Machine

The weekly workshops were only two hours long; the real goal was getting families to complete design challenges from home. The evaluation revealed that only a third of the families reported submitting Design Challenges from home and fewer still appeared to be starting them while at home. Because most of the families reported having Internet access at home, this is mostly likely motivational issue. This raises a question extends beyond the context of these Design Challenges into the larger informal science community. For example, what could a museum put up on the web that might motivate parents and children working together on an investigation that builds on a special exhibit?
Given busy parents, compelling videogames, streaming media, social media, and all of the other potential attractors, this is a not a trivial motivational challenge. This challenge is heightened by the fact that competing theories of motivation suggest two very different approaches. In what follows I summarize these two approaches and offer an alternative based on newer situative theories of motivation that I have been exploring in my own research.

Applying Traditional Behavioral Models of Motivation
One class of approaches to motivation follows from tradition behaviorist models of learning. These approaches would focus on the "contingencies" in the environment that encourage or discourage children and families from engaging in Design Challenges from home. These perspectives would examine what likely suggest some sort of incentives to motivate initial engagement so that each member of the family would develop fluency with the necessary skills so that they could learn to work together sufficiently so that the act of mastering the Design Challenging would itself be a sufficient incentive so that the family would choose to begin and complete a new challenge from home. This might be a digital badge that contains evidence of success that families to could share over social media. This might be free passes to a science museum. While it seems unlikely, it certainly is worth imagining what sort of engagement would be motivated by cash incentives.
The nice thing about imagining cash incentives for learning is because it forces innovators and educators to consider the criteria for earning the incentives. Imagine if simply submitting a photo or a video of each device created in a Design Challenge and posting a reflection would deposit a $100 credit at Amazon. Lots of families would post lots of hastily completed devices. While some STEM learning might occur, much of the learning that would take place in those families would be about how to complete the Design Challenges as quickly as possible while still earning the credit. As nicely summarized by Paul Chance in (1992) researchers working in the behaviorist tradition have advanced guidelines for providing incentives to support learning. These guidelines encourage success-contingent incentives for meeting a specific criteria and discourage task-contingent incentives for merely completing the task. Recall that each Design Challenges includes a specific engineering design challenge. For example, the Suspension Bridge Design Challenge has families make a suspension bridge from plastic straws, string, and tape that can hold as many pennies as possible. After some testing, Iridescent might conclude that an appropriate criterion is 15 pennies. This would make it possible to only offer the incentive to families who upload a video or photo showing that their device met that criteria.
Chances article offers a number of useful guidelines for offering reward. Perhaps the most useful guideline is avoid using rewards as incentives. In other words, don't offer rewards for things that people are already motivated to do. Consider, for example, that perhaps parents don't need rewards to engage, but children do. Perhaps the families might themselves offer some sort of incentive (e.g., fast food, ice cream, five dollars) to their children for accomplishing each design challenge. Another useful guideline is use the weakest reward necessary. Clearly a $100 incentive for families is excessive. What about a $5 credit at the National Science Teachers Association bookstore?            
 Other reward guidelines summarized by Chance include reward at a higher rate early and then reduce frequency, set standards that are well within reach of the learners, and reward only the behavior you want repeated. One feature of the Applied Behavior Analysis (ABA) research that supports these guidelines is a reliance on single-subject designs. Such studies first examine baseline behavior. They then adjust the contingencies in the environment that reward desirable behavior and discourage undesirable behavior. Then the see what happens. Given that the Design Challenges are served from a very sophisticated website, it would clearly be possible to systematically tweak the success criterion and incentives to find which combination leads families to successfully complete the most Design Challenges.
Once obvious extension to this approach would be some sort of competition. What if Iridescent put up some sort of a "leader board" for each Design Challenge? Families could compete to meet or exceed the current leader. This alone might have a tremendous impact on family engagement. It is important to note that many strict behaviorists would argue against using competition. In his famous 1948 book Walden Two, B. F. Skinner argued that ABA could support a lifestyle free of war, completion, and social strife—much like Thoreau's Walden. But the continuing rise of market-oriented models of education and test driven reforms appears to have led many educators and innovators have to embrace competition. Additionally the emergence of "gamification" as an entire genre suggests many possible ways to gamify individual Design Challenges and the entire Curiosity Machine experience. Indeed proponents of gamification would likely suggest turning the Curiosity Machine into some sort of virtual STEM Family Feud.

The Debate over Rewards
A very different set of suggestions would follow from rationalist theories that came to dominate studies of learning and motivation after the "cognitive revolution" in the 1960s. This perspective argues against the use of "extrinsic" incentives to motivate family engagement. These perspectives argue that extrinsic rewards undermine the authentic learning that occurs as humans naturally try to make sense of the world around them. These assumptions are consistent with "constructivst" and "discovery learning" perspectives that came to dominate educational psychology in the 1980 and that are particularly influential in STEM education.

This perspective has been widely popularized by the writer Alfie Kohn, via book such as his Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes from 1993. In his 1993 response to Paul Chance, Kohn summarizes widely held concerns over the use of rewards for learning. These include that that rewards are controlling, ineffective, make learning less appealing, and ignore curricular questions. A major issue with rewards is the "overjustification" effect documented in hundreds of studies. This occurs when free-choice intrinsically motivated behavior decreases after the introduction of rewards. This debate has continued to simmered for decades. The opening sentence of Chance's (1993) response to Kohn's response nicely captures the disjoint between these two perspectives:

IT IS DIFFICULT to know how to respond to Alfie Kohn's critique. It is so disjointed and so full of misrepresentations of fact and theory that it is like a greased pig: one can scarcely get a grip on it, let alone wrestle it to the ground.

A series of meta-analyses were carried out on empirical studies of rewards in the mid-90s. In meta-analyses, the size and nature of the experimental effect across various conditions are themselves subjected to rigorous statistical analysis. While meta-analysis intended to resolve such enduring questions, a set of competing meta-analyses reaching opposite conclusions has only confirmed how irreconcilable these two perspectives are.

Supporting Intrinsically Motivated STEM Engagement
 Kohn and many other opponents of rewards draw on primarily on Self-Determination Theory (SDT), as defined by Edward Deci, Richard Ryan, and colleagues in the 1970s and which continues to be very influential. SDT incorporates Bandura's prior ideas about of modeling and self-effiacy in a much broader framework of innate human needs. As nicely summarized in Wikipedia, the theory focuses on three needs: 
  • Competence concerns the assumption that humans universally seek to control outcomes and experience mastery.
  • Autonomy concerns the universal urge to be causal agents of one's own life and act in harmony with ones integrated self (though not  necessarily independent of others)
  • Relatedness concerns the universal want to interact, be connected with, and experience caring for others.
Applying these theories would highlight the many things that Iridescent is already doing right to motivate families to encourage intrinsically motivated engagement in the Design Challenges, and suggest further refinements. Specifically, the three primary implications of SDT concern optimal challenge, performance feedback, and supporting relatedness. STD would argue against using rewards and competition, and instead suggest focusing on each of these three areas:
  • Optimal challenge: SDT argues the learning opportunities should be neither too easy nor too hard. If they are too easy learners don't feel a sense of accomplishment; if they are too hard, learners will have insufficient self-efficacy to engage. What I love about the Design Challenges is that they appear to be just hard enough so that younger children need help accomplishing the challenge. More specifically, the younger children need help in using the disciplinary concepts (such as load and tension) introduced in the videos to systematically (rather than randomly) redesign their devices to accomplish the challenge.
  • Offer useful performance feedback. SDT provides extensive guidelines in giving performance feedback, and it has been widely applied in the realm of classroom assessment and grading. The reason Kohn included praise in the title of his book is because of studies showing that teacher praise for behavior that is "externally-regulated" (rather than intrinsically motivated) undermines self-determination. This is one area where Iridescent might do quite a bit of refinement and research. The prior study showed the most of the feedback from virtual mentors was received by learners at least a day after they had submitted their challenge, long after the materials and devices were put away and some of the families had embarked on a new challenge. So further refinement to both the technology and practices for providing feedback to make that feedback more useful and used would be expected to improve engagement.
  • Help learners relate to others. I think that Iridescent is already doing a great job with this in the way that their challenges are designed. As I elaborated in my other post, in my one opportunity to observe families completing Design Challenges it was really endearing to watch a father gently but persistently use the principle of thrust to help his daughters redesign their "spinning machines" (made of a balloon, clay, straws, and a ping pong ball). I regret that it was so noisy in the room that I could not transcribe the recording, but got to witness the younger child use that term herself in talking to her sister. From my perspective, this kind of scientific relating is what is missing in many science classrooms. While I think Iridescent's intuitions have been remarkably effective, I suspect a more systematic application of SDT might point to additional innovations that might motivate engagement. For example, right now most of the relatedness that is supported appears to be within families. It might be very helpful search for strategies to foster relatedness between families.

As we see here, SDT offers clear suggestions for increasing intrinsic motivation and the high quality forms of engagement that have long been shown to follow. Many of these are included in sub-theory of SDT known as Cognitive Evaluation Theory. CET provides a practical framework for studying how social and environmental factors support or undermine intrinsic motivation. Most importantly, these principles would steer Iridescent away from any sort of incentives or completion, as these are assumed to undermine intrinsically motivated engagement.
            The challenge here is obvious. One set of motivational theories pushes in one direction, while another set of theories pushes in another. Because rationalist theories have long dominated educational and developmental psychology, teachers and parents are strongly discourage from using rewards. But schools continue to be dominated by grades and parents quickly learn that rewards and punishments appear to work. The problem is that teachers and parents are likely to use rewards and punishment in ways that even behaviorists would discourage. This problem seems to be getting worse in schools with the rise of market-oriented approaches to educational reform.

A Situative Alternative
            I have been fascinated by this tension between behaviorist and rationalist perspectives for my entire career. I first explored in the major area paper that wrote in 1993 in order to advance to candidacy in my PhD program. In that paper I argued that the sociocultural theories of cognition and learning first articulately by Vygotsky in the 1930s offered an alternative view for thinking about motivation. I had read Lave and Wenger's 1991 book on Situated Learning and was intrigued that this emerging theory might "sidestep" these nagging tensions over incentive. Thanks to the encouragement and patient editing of Paul Pintrich, the paper was eventually published in Educational Psychologist in 1997. That paper helped me land my first academic position and was nicely taken up by the new scholarly community that was emerging at that time.

            The 1997 article led to invitations participate in several symposia. These included on one in Sweden in 2000 that Simone Volet and Sanna Jarvela organized. The papers from that symposium led to an edited volume in 2001 that gave this new community a name: Motivation in Learning Context. This name reflects a larger shift in Educational Psychology that had begun in the late 1980s. Rather than applying basic theories that were first advanced by Cognitive Psychologists and Developmental Psychologist, educational researchers were increasingly interested in developing more useful "local" theories in particular educational contexts. Generally speaking, this shift is what continues to distinguish the research associated with the Learning Sciences from Educational Psychology.
            Around this time, I also began  collaborating with Mary McCaslin, including a chapter that explored  her theory of "co-regulated learning." My real opportunity came in in 2003 when Mary invited me to contribute to a special issue of The Elementary School Journal that she was editing. I got to really explore the motivation implications of "situative" theories of cognition that Jim Greeno and others had formalized in the 1990s. The title of the article included Greeno's notion of "engaged participation" as well as one of the most embarrassing (of many) typos in my work. While Mary asked me to "tone down" my argument, she allowed me to bypass some of the issues that one of the reviewers had with my "stridently situative" approach. The paper explored six aspects of engagement, and their implications for the design of learning environments. What follows is an initial exploration of their implications for Iridescent's goal of motivating extended family engagement completing Design Challenges from home.
            Internalization of goals and values. My first point was simply an extension of situative theories of cognition. These theories argue that knowledge primarily resides in social context and only secondarily in the minds of individuals. If we assume that this is true, then we can extend that assumption to the goals and value that motivate engagement and learning. While we obviously internalize goals and value from the context around us, assuming that they primarily live in the social context pushes us to focus on the social and cultural context for ways motivate engagement.
             In the case of Iridescent, this would lead to assume that most of goals and values that motivate the children actually come from their family, while most of the goals and values that motivate the family come from that larger sociocultural context in which that family functions. At general level, this means that Iridescent's efforts to motivate should be organized in terms of communityà family à child rather than the other way around. At a more specific level this suggest the need for systematic ethnographic work that examines how the larger cultural context motivates families to engage in scientific and how families motivate their children to engage. Brigid Barron at Stanford is doing some reallyi nteresting studies in this regard and could probably have some good ideas about what this would look like around family engagement in Design Challenges. I also just discovered what looks like a really nice paper by Middleton, Dupuis, & Tang (2013) that looks very promising as well.
      Engaged participation. Building particularly on Greeno's 1998 presidential address to the American Psychological Association, there seems to be unique value in characterizing engagement as meaningful participation in a context where to-be-learned knowledge is value and used. If participation in scientific inquiry is not value by the family, the Design Challenges will be hard pressed to motivate the children; likewise, if scientific inquiry is not valued by the larger sociocultural context in which the family participates, Iridescent has its work cut out for it. Of course this gets at why Iridescent, the National Science Foundation and other foundations and museum work so hard to reach out to women, racial minorities and other communities who continue to face significant obstacles such as sexism, racism and impoverishment in their pursuit of STEM education and careers. The design challenges already do a great job by emphasizing female and racial minority engineers in their videos and then offering their evening workshops in schools and libraries that serve inner-city and low income communities. One idea is that having one of those engineers actually come to the family science workshops might be really motivating for families and children. What might be very compelling would be having those same engineers serve as virtual mentors, particularly if the children and parents strongly identified with the visiting engineers.
The role of identity in engagement. The role of identity is crucial to engaged participation. While identity has traditionally been characterized in terms of internalized goals and values, situative perspective characterize identity in terms of our lived experience of participation in the social and cultural practices of communities. Such identities are negotiated and are assumed to reside primarily in those negotiations and only secondarily in the minds of the individual or the members of a particular community. In Iridescent's case this means self-report survey methods (like those that we used in the prior evaluation) capture a view of identity that is at best incomplete and possibly quite misleading. Rather, interpretive and observational methods are needed to study the way identities are negotiated between children and families and between families and communities. These negotiations are also carried out around displays of competence and recognition of that competence by the community. This means for example, that it is important to help children and families learn to use the disciplinary concepts (e.g., load and tension) introduced in the design challenges in ways that would be recognized as appropriate by STEM professionals.
   Legitimate peripheral participation. Lave and Wenger introduced the notion of peripheral participation in 1991 to capture the importance of trajectories of participation. As children and families negotiate their participation in the larger community of STEM expertise, they can do so in ways that define very different trajectories. Obviously, they are "peripheral" participants since they are not practicing scientists and engineers. Legitimate participation on the periphery defines an "inbound" trajectory. This means that as children and families participate more and more successfully, they are getting closer and closer to the community of practice. This further argues for supporting authentic use of disciplinary concepts in the design challenges. Rationalist "constructivist" models of instruction argue against explicit instruction on such concepts, particularly in informal settings. As elaborated above, these perspectives argue that overly explicit instruction will undermine the child's natural curiosity that will lets them discover what those concepts mean. From a situative perspective, it is more important to help children and families to understand and use those concepts in ways that experts would consider appropriate, in order to begin building an "legitimate" trajectory from the periphery towards the center of the STEM professional community. This perspective also suggests that many kinds of informal assessments can actually be quite motivating, so long as individuals are prepared for them and learners are able to succeed. For example, these theories suggest that a simple quiz that children would be asked to complete on their own (without their parents help) that lets children demonstrate their newly gained proficiency with the concepts that they mastered while completing the design challenges could have a substantial impact on most children's nascent identity as a future STEM professional.
            The importance of non-participation. Wenger's 1998 book on Communities of Practice helped me appreciate that the things we don't participate in define our identities just as much as what we do participate in. It helped me realize that in many STEM classroom students are not really participating in authentic STEM inquiry at all. As researchers like Leona Shauble and others have shown repeatedly, the only inquiry in many STEM classroom is figuring out how to get a good grade for the least amount of work. While STEM educators and parents often blame students for this, this perspective lays the blame squarely on the curriculum (and inappropriate use of tests of the learning from that curriculum). Consider the trajectory defined by what critics call "worksheet science." Students who participate more and more successfully in memorizing and recognizing shallow definitions of terms and completing simple procedure are unlikely to every find themselves in the center of a particular community of STEM practice.

A Personal Application
Mark Guzdial
These issues of trajectories and non-participation in STEM learning have personal relevance for me and my own family. I was quite pleased a few years ago when my son Lucas enrolled in a computer programming class in high school. I never learned to program myself and these days it I find it quite a handicap. While I bought an Apple II+ computer in 1982 (!) and taught myself BASIC, an instructional technology professor discouraged me from delving too deeply into technology or programming (because "it changes too often"). While I still want to learn how to code, my non-participation in programming clearly helped define my trajectory towards a Ph.D in Psychology and satisfying career as a Learning Scientist.
Unfortunately, the curriculum in my son's programming class was like the typical secondary computer science instruction that Mark Guzdial chronicles in his Computing Education blog. The coding worksheets seemed to have been haphazardly created to match various videos located on the web. My son wanted to use the much more professionally-produced videos and exercises that we were able to access via my university's account at, but his teacher insisted that my son complete the worksheets as well (so teacher could grade them).

After failing the first exam (which appeared to directly mirror the worksheets), my son dropped the programming class and instead enrolled in Journalism. His journalism teacher and curriculum were both outstanding and my son quickly developed into a strong writer. He was soon taking a turn each month on the weekly School Diary a monthly column in the local newspaper. As many of my friends and comments on the newspaper website pointed out, he seemed more thoughtful about topics like climate and college tuition than many politicians. His columns often sparked fierce debates on the paper's website. His class even elected him to represent the school newspaper at debate between the two candidates in Bloomington's 2015 mayoral race. The debate was moderated by the publisher of the local paper and the other panelists included a political science professor and reporter from the paper.
Lucas will be attending IU this fall and currently plans on majoring in Journalism and Political Science. Just as my own career trajectory was shaped by my own non-participation in programming, it seems quite likely that my son's trajectory has already been shaped in much the same fashion. Of course, as the child of two university professors, my son had opportunities and advantages that are unfathomable to most of the families that Iridescent targets in their programs. I think it is massively important to increase the number of children who establish a legitimate trajectory towards STEM careersearly on, and particularly do so for children of growing up in impoverished families. I think this is particularly important for mathematics and science. For many impoverished children, secondary-level math and science courses present such an obstacle that they never even get into college, much less STEM careers. I believe that Iridescent is on the right track to transforming these trajectories, and would love to see many more such efforts. (Here is a previous post about our initial efforts to fix this problem in freshman Calculus)

Putting it all Together
            So what should Iridescent do? As we saw above, the guidelines from behaviorist and rationalist perspectives were both quite straightforward routes going in two very different directions. However, one frustration that many skeptics have with these newer situative perspectives is that they are just that: perspectives. In contrast to the prior perspectives, situative and sociocultural perspectives fail to deliver straightforward prescriptive guidelines that promise to result in desired ways of behaving or thinking that are independent of context. In a subsequent post, I will elaborate on one possible path forward.

Chance, P. (1992, November). The rewards of learning. Phi Delta Kappan, 73 (10), 200-207.

Chance, P. (1993). Sticking up for rewards. The Phi Delta Kappan, 74(10), 787-90.

Greeno, J. G. (1998). The situativity of knowing, learning, and research. American Psychologist, 53(1), 5.

Hickey, D. T. (1997). Motivation and contemporary socio-constructivist instructional perspectives. Educational Psychologist, 32 (3), 175-193.

Hickey, D. T. (2003). Engaged participation versus marginal nonparticipation: A stridently sociocultural approach to acheivement motivation. The Elementary School Journal, 401-429.

Kohn, A. (1993, June). Rewards versus learning: A response to Paul Chance. Phi Delta Kappan, 74(10), 783-87.

Middleton, M., Dupuis, J., & Tang, J. (2013). Classrooms and culture: The role of context in shaping motivation and identity for science learning in indigenous adolescents. International Journal of Science and Mathematics Education, 11(1), 111-141.

Schauble, L., Glaser, R., Duschl, R. A., Schulze, S., & John, J. (1995). Students' understanding of the objectives and procedures of experimentation in the science classroom. The Journal of the Learning Sciences, 4 (2), 131-166.

Volet, S., & Järvelä, S., (2001) (Eds.) Motivation in learning context: Theoretical advances and methodological implications. London: Pergamon/Elsevier

Wenger, E. (1999). Communities of practice: Learning, meaning, and identity. Cambridge University Press.

Lave, J, & Wenger E. (1991). Situated learning: Legitimate peripheral participation  Cambridge University Press, Cambridge, UK.

No comments:

Post a Comment