4.b+Triangulation+of+Participant+Surveys,+Sociograms,+&+Website+Activity

The Take Action project is fairly simple to explain and typically generates a genuine interest with people. Much like the prospect of a new family pet which the kids promise they will take care of, typically the responsibilities of pet care eventually fall onto the shoulders of just one person. It's not that the other family members do not love the pet, but they lose interest in the day to day care of the animal. A similar story can be told about the Take Action project. The first week of the pilot, participants were very active in a variety of ways with the site. However, after the first week, much like the new family pet, interest waned. We found that participant engagement depended on the day of the week, and most especially the type of action.

Engagement of participants is examined in several different ways. Looking first at the traffic the Take Action site received tells us how many people were willing to take the first step in taking action: finding out what to act on. The first week of the pilot had an average of 81 visitors each week day. During the second set of weekdays of the pilot, visitors to the site dropped by about 40% to 48.8 visitors. After this initial drop in the average number of visitors to the site during weekdays, the numbers did not change as dramatically with an average of 43.8 visitors in week 3 and 34.4 visitors in week 4.

Getting people to the site did not guarantee they would necessarily act on the action of the day. Both the numbers of the people who acted and the number of visitors to the site peak during weekdays and drop on the weekends. In fact, the average number of visitors on weekends was 77.5% less than weekdays. Because of fewer visitors, completed actions by users on the weekends were significantly reduced as well. Friday, February 5th was the day with the greatest number of visitors according to Google Analytics, which ran successfully starting the third day of the pilot February 3rd. Despite the record number of visitors that day in the pilot, 88 visitors, the number of people taking action that day dropped by 42.6% from 54 people acting the day before to 31 people acting on Friday. (See Figure 1) In our observations journal for that week, this drop in completed actions was noted by both of us saying that we, "decided that we need more games, and websites with interaction in order to engage the students." (Take Action Journal, Week 1, Sunday) On February 13 and February 20 there are actually more actions completed than unique visitors to the site. This could be a result of one computer being used by several people or, as Ben noted in the Take Action Journal, "It would seem that people were participating on their own, but not necessarily on the current days' actions. They were either seeking out their own actions, or merely trying to find actions that suited them."



Taking Ben's observation, the actions were classified using Bloom's Taxonomy into their respective categories. Actions were classified according to the revised Bloom's Taxonomy Cognitive Dimension Process (Krathwohl, 212; 2002). The 28 actions for the pilot were identified as being an indicator of one of these dimensions with one of the dimensions getting broken down into more specific subgroups. The dimensions of cognitive learning that were identified include Remember, Apply and Create in the following ways: (Definitions adjusted from Krathwohl, 212, 2002)
 * Remember: retrieving relevant knowledge from a reputable source and being able to recall the information
 * Apply - Online: carrying out a procedure or "action" virtually via a computer
 * Apply - Offline: carrying out a procedure or "action" in the physical community. This may include, but is not limited to, the physical classroom, school, home, church, or local community.
 * Apply - Online, Apply - Offline: carrying out a procedure or "action" virtually via a computer which has a direct affect on a physical community. i.e. online games that directly result in giving money, food or other needed services to a community in need.
 * Create: Putting elements together to form an original product



Looking at the frequency of completed actions (Figure 2), participants were 28% more apt to act than average when asked to "Apply - Offline." Offline application actions asked participants to do things like reduce the amount of water used that day or read a book to someone else. Participants were least likely to complete an action when they were asked to create something, for example making an informational poster about something they've learned. Another significant difference was found between Apply - Offline and the other "Apply" sub-categories. Online actions included signing online petitions and informational online games like the Stop Disasters! game at http://stopdisastersgame.org. (For a complete list of daily actions see Appendix A)

One classification of actions was very unique because it uniquely fit both the Apply - Online and Apply - Offline classifications. There were only two actions that fit both classifications but this is because they were very similar. One action, occurring on a Monday, asked users to, "Visit www.freerice.com and help donate at least 100 grains of rice to the World Food Programme by playing the game." This asked students to apply their knowledge virtually in a game in order to physically give their earned rice to the hungry via the World Food Programme. The other action fitting the Apply - Online and Apply Offline classification was very similar to freerice.com. It asked participants to, "Donate at least 100 cups of water by playing a few games at www.freepoverty.com." The game is very similar and getting cups of water to those in need is also a pretty simple goal to grasp. However looking at the number of participants who acted on these to actions, the freerice.com action had nearly 9 times as many people complete the action as freepoverty.com and their cups of water. The difference is something we've already noted - weekend participation versus weekday participation. The freerice.com action was scheduled for a Monday and had 27 people act whereas freepoverty.com was scheduled on a Saturday and only had 4 people act.



A closer look at the Apply - Offline classification's ability to motivate users to act begs us to look at just the 20 weekdays of the pilot and cut out the low performing weekend days. (Figure 2b) A look at only those deeds that users saw on weekdays shows an even larger gap between the classifications. Completed actions that asked users to Apply - Offline had, on average, 36.8% more users acting. This is again evidence of users being motivated to act when they are asked to do so in their physical environment. One more look at this new weekdays-only table displays that all deeds which are classified as "Apply" have above average action completion. This data is a significant measure of user engagement and motivation.

Looking at the greatest sum of actions on Actions of the Day does not tell the entire story. Counting an action as meaningful because someone has merely clicked to say "I've Acted" is similar to taking attendance in a class and determining that all of those in attendance got something out of the day's lesson. To gain a better understanding of what students have learned, a teacher might facilitate a class discussion where students who have paid attention and engaged in the learning will most likely have more thoughtful responses and questions. This helps the teacher determine quality of student knowledge and not just who has been to class. The former is a better indicator of a student's future success in the class and understanding of content. Using karma points, points awarded from a scale of 0-5 by Take Action users on comments left by other users, the Take Action community as a whole determined which users were more creative, had a better understanding and engaged in each day's action - more points indicated higher quality comments. The users in the top ten percent of karma point sums are regarded as "super users" because their actions and engagement in the action are deemed by the Take Action community to be of higher quality. Looking at the actions which super users acted on shows that two actions in particular, "Use a gallon or 3.5 liters less of water today" on Feb. 3 and "Visit www.campaignforeducation.org/bigread and discover how many adults worldwide cannot read" on Feb. 8 were most often acted upon. (Figure 3) The first action, to use less water, fits in with the data which says actions that ask participants to apply offline are most likely to be acted upon. The next action which simply asked people to discover a new fact, falls into the remember classification. However, this day's action attracted a lot of traffic, fourth highest according to Google Analytics (Figure 1), and a lot of comments in the forum. This action had people talking and ultimately engaged in learning more about the Millennium Development Goal - Universal Education.



Another pattern of super users is the same as the regular users - less acting on the weekends. Despite quality actions that data suggests a user would be more apt to act on, super users were still less active on the weekends. (Figure 3)

Super users were also the hubs that controlled who received karma and in essence became super users. (Figure 4) Super users were more likely to give karma points to other super users than to give karma points to non-super users whereas many users who never even received karma themselves gave karma points to super users. Of the seven super users that were identified, five of these were teachers participating in the pilot. Although this seems like data we should reexamine without teacher involvement, we cannot ignore the immense impact our teacher-leaders had on the pilot. The fact that five teachers had a higher karma sum contributes to what we already know, students were more apt to act during the week when they went to class.

Users could choose if they wanted to act on a particular action or not. One of the goals of the project was to engage students in being active global citizens. We've looked at a few different ways of analyzing this by looking what types of actions users acted upon and the quality of those actions. Another piece of that goal is educate students on what is going on in the world. Our guide in this goal was using the UN Millennium Development Goals (MDGs). Every action of the day connected to at least one of these eight [|MDGs]. Dispersal of actions among the eight MDGs was not equal, but every goal did receive attention from the project. (Figure 5) However, when looking at the above average actions that were acted upon, as indicated by a user clicking "I've Acted," Environmental Sustainability rose to the top as a MDG that users were more likely to act upon. (Figure 6) Looking deeper into why a user may have acted upon Enviornmental Sustainability reveals that three of these five actions fell under the classification Apply - Offline. This is most likely because it is easier to write Actions of the Day which ask users to act within one's physical community on Environmental Sustainability than other MDGs like Combating HIV/AIDS, Malaria and Other Diseases.


 * [[image:MDGFrequency.PNG caption="Figure 5"]] ||  || [[image:MDGsAsActions.PNG caption="Figure 6"]] ||

Participant attitude toward being an active agent of change in the world was a goal of the project. In looking at participant responses to pre-pilot and post-pilot surveys about their ability to impact the world, the data is inconclusive. Surveys were anonymous and therefore we were unable to unsure that the same people were taking the pre-pilot surveys and post-pilot surveys. There were 76 people who completed the pre-pilot survey and only 45 who completed the post-pilot survey. The data leans toward telling us that students were more apt to believe they could make a small difference in the world, but since there are many uncontrolled variables involved in the survey it's hard to depend on this as concrete evidence of a change of attitude. (Figure 7)



I think the triangulation of all this data points to the fact that kids are more apt to act if they are doing something simple and something that's easy to understand what they are doing. They understand small may be small but it's also good enough which is a tough concept... or maybe they just think that because we are telling them to. I'm not sure!

media type="custom" key="5978723"