Search

How can educational platforms be instrumented to accelerate the use of research to improve students' experiences? We show how modular components of any educational interface - e.g. explanations, homework problems, even emails - can be implemented using the novel MOOClet software architecture. Researchers and instructors can use these augmented MOOClet components for: (1) Iterative Cycles of Randomized Experiments that test alternative versions of course content; (2) Data-Driven Improvement using adaptive experiments that rapidly use data to give better versions of content to future students, on the order of days rather than months. A MOOClet supports both manual and automated improvement using reinforcement learning; (3) Personalization by delivering alternative versions as a function of data about a student's characteristics or subgroup, using both expert-authored rules and data mining algorithms. We provide an open-source web service for implementing MOOClets (www.mooclet.org) that has been used with thousands of students. The MOOClet framework provides an ecosystem that transforms online course components into collaborative micro-laboratories, where instructors, experimental researchers, and data mining/machine learning researchers can engage in perpetual cycles of experimentation, improvement, and personalization.

Learners consume video-based learning content on various mobile devices due to their mobility and accessibility. However, most video-based learning content is originally designed for desktop without consideration of constraints in mobile learning environments. We focus on readability and visibility problems caused by visual design elements such as text and images on varying screen sizes. To reveal design issues of current content, we examined mobile learning adequacy of content with 681 video frames from 108 video lectures. The content analysis revealed a distribution and guideline compliance rate of visual design elements. We also conducted semi-structured interviews with six video production engineers to investigate current practices and challenges in content design for mobile devices. Based on the interview results, we present a prototype that supports a guideline-based design of video learning content. Our findings can inform engineers and design tool makers on the challenges of editing mobile video-based learning content for accessible and adaptive design across devices.

Team members commonly collaborate on visual documents remotely and asynchronously. Particularly, students are frequently restricted to this setting as they often do not share work schedules or physical workspaces. As communication in this setting has delays and limits the main modality to text, members exert more effort to reference document objects and understand others’ intentions. We propose Winder, a Figma plugin that addresses these challenges through linked tapes—multimodal comments of clicks and voice. Bidirectional links between the clicked-on objects and voice recordings facilitate understanding tapes: selecting objects retrieves relevant recordings, and playing recordings highlights related objects. By periodically prompting users to produce tapes, Winder preemptively obtains information to satisfy potential communication needs. Through a five-day study with eight teams of three, we evaluated the system’s impact on teams asynchronously designing graphical user interfaces. Our findings revealed that producing linked tapes could be as lightweight as face-to-face (F2F) interactions while transmitting intentions more precisely than text. Furthermore, with preempted tapes, teammates coordinated tasks and invited members to build on each others’ work.

Crowdsourcing is a common means of collecting image segmentation training data for use in a variety of computer vision applications. However, designing accurate crowd-powered image segmentation systems is challenging, because defining object boundaries in an image requires significant fine motor skills and hand-eye coordination, which makes these tasks error-prone. Typically, special segmentation tools are created and then answers from multiple workers are aggregated to generate more accurate results. However, individual tool designs can bias how and where people make mistakes, resulting in shared errors that remain even after aggregation. In this article, we introduce a novel crowdsourcing approach that leverages tool diversity as a means of improving aggregate crowd performance. Our idea is that given a diverse set of tools, answer aggregation done across tools can help improve the collective performance by offsetting systematic biases induced by the individual tools themselves. To demonstrate the effectiveness of the proposed approach, we design four different tools and present FourEyes, a crowd-powered image segmentation system that uses aggregation across different tools. We then conduct a series of studies that evaluate different aggregation conditions and show that using multiple tools can significantly improve aggregate accuracy. Furthermore, we investigate the idea of applying post-processing for multi-tool aggregation in terms of correction mechanism. We introduce a novel region-based method for synthesizing more accurate bounds for image segmentation tasks through averaging surrounding annotations. In addition, we explore the effect of adjusting the threshold parameter of an EM-based aggregation method. Our results suggest that not only the individual tool’s design, but also the correction mechanism, can affect the performance of multi-tool aggregation. This article extends a work presented at ACM IUI 2018 [46] by providing a novel region-based error-correction method and additional in-depth evaluation of the proposed approach.

With the advent of malware generation toolkits that automatically generate malware, anyone without a professional skill can easily generate malware. As a result, the number of new/modified malware samples is rapidly increasing. The malware generated in this way attacks vulnerabilities, such as PCs and mobile devices without security patch, causing damages involving malicious actions, such as personal information leakage, theft of authorized certificates, and cryptocurrency mining. To solve this problem, most security companies use the signature-based malware detection technique to detect malware, in which the signatures of known malware and files suspected to be malware are compared before detecting malware. However, the signature-based malware detection technique has a limitation in that it is not efficient for detecting new/modified malware which is generated rapidly. Recently, research is underway to utilize deep learning technology for detecting new/modified malware. In this study, we propose a SAT scheme that can detect not only known malware but also new/modified malware more quickly and accurately, thereby reducing malware-induced damages to PCs and mobile devices. The SAT scheme employs an open source library called Tensorflow in the GPU environment to learn malware signatures and then to statically analyze malware.

Motivation While each cancer is the result of an isolated evolutionary process, there are repeated patterns in tumorigenesis defined by recurrent driver mutations and their temporal ordering. Such repeated evolutionary trajectories hold the potential to improve stratification of cancer patients into subtypes with distinct survival and therapy response profiles. However, current cancer phylogeny methods infer large solution spaces of plausible evolutionary histories from the same sequencing data, obfuscating repeated evolutionary patterns. Results To simultaneously resolve ambiguities in sequencing data and identify cancer subtypes, we propose to leverage common patterns of evolution found in patient cohorts. We first formulate the Multiple Choice Consensus Tree problem, which seeks to select a tumor tree for each patient and assign patients into clusters in such a way that maximizes consistency within each cluster of patient trees. We prove that this problem is NP-hard and develop a heuristic algorithm, Revealing Evolutionary Consensus Across Patients (RECAP), to solve this problem in practice. Finally, on simulated data, we show RECAP outperforms existing methods that do not account for patient subtypes. We then use RECAP to resolve ambiguities in patient trees and find repeated evolutionary trajectories in lung and breast cancer cohorts. Availability and implementation https://github.com/elkebir-group/RECAP. Supplementary information Supplementary data are available at Bioinformatics online.

Similar to a design process for designing graphical user interfaces, conversation designers often apply an iterative design process by defining a conversation flow, testing with users, reviewing user data, and improving the design. While it is possible to iterate on conversation design with existing chatbot prototyping tools, there still remain challenges in recruiting participants on-demand and collecting structured feedback on specific conversational components. These limitations hinder designers from running rapid iterations and making informed design decisions. We posit that involving a crowd in the conversation design process can address these challenges, and introduce ProtoChat, a crowd-powered chatbot design tool built to support the iterative process of conversation design. ProtoChat makes it easy to recruit crowd workers to test the current conversation within the design tool. ProtoChat's crowd-testing tool allows crowd workers to provide concrete and practical feedback and suggest improvements on specific parts of the conversation. With the data collected from crowd-testing, ProtoChat provides multiple types of visualizations to help designers analyze and revise their design. Through a three-day study with eight designers, we found that ProtoChat enabled an iterative design process for designing a chatbot. Designers improved their design by not only modifying the conversation design itself, but also adjusting the persona and getting UI design implications beyond the conversation design itself. The crowd responses were helpful for designers to explore user needs, contexts, and diverse response formats. With ProtoChat, designers can successfully collect concrete evidence from the crowd and make decisions to iteratively improve their conversation design.

Planning a solution before writing code is essential in algorithmic problem-solving. However, novices often skip planning and jump straight into coding. Even if they set up a plan, some do not connect to their plan when writing code. Learners solving algorithmic problems often struggle with high-level components such as solution techniques and sub-problems, but existing representations that guide learners in planning, such as flowcharts, focus on presenting lower-level details. We use subgoal diagrams -- diagrams made of subgoal labels and the relationships between them -- as a representation that guides learners to focus on high-level plans when they develop solutions. We introduce AlgoPlan, an interface that enables learners to build their own subgoal diagram and use it to guide their problem-solving process. A preliminary study with seven students shows that subgoal diagrams help learners focus on high-level plans and connect these plans to their code.

Building a chatbot with human-like conversation capabilities is essential for users to feel more natural in task completion. Many designers try to collect human conversation data and apply them into a chatbot conversation, aiming that it could work like a human conversation. To support conversation design, we propose the idea of inviting the crowd into the design process, where crowd workers contribute to improving the designed conversation. To explore this idea, we developed ProtoChat, a prototype system that supports a conversation design process by (1) allowing the crowd to actively suggest new utterances based on designers' pre-written design and (2) visually representing crowdsourced conversation data so that designers can analyze and improve their conversation design. Results of an exploratory study indicated that the crowd is helpful in providing insights and ideas as designers explore the design space.

Conversation design is an essential step in building a chatbot. Much like visual user interface design, conversation design benefits from prototyping and user testing to allow for conversation exploration and improvement. However, it can be overwhelming to quickly iterate on the conversation design as the iterative process requires not only designing a conversation but also building and testing a working chatbot equipped with the conversation. We developed ProtoChat, a prototype system that supports an iterative conversation design by allowing designers to (1) prototype conversations, (2) test the conversations with the crowd, and (3) review and analyze the crowdsourced conversation data. Results of an exploratory study with four conversation designers show that the designers successfully iterated on their conversation design by reviewing how the crowd followed the conversation, which provided insights into concrete action items for improving their conversation design.

To help language learners achieve fluency, instructors often focus on teaching formulaic sequences (FS)--phrases such as idioms or phrasal verbs that are processed, stored, and retrieved holistically. Teaching FS effectively is challenging as it heavily involves instructors' intuition, prior knowledge, and manual efforts to identify a set of FSs with high utility. In this paper, we present FSIST, a tool that supports instructors for video-based instruction of FS. The core idea of FSIST is to utilize videos at scale to build a list of FSs along with videos that include example usages. To evaluate how FSIST can effectively support instructors, we conducted a user study with three English instructors. Results show that the browsing interactions provided in FSIST support instructors to efficiently find parts of videos that show example usages of FSs.

Collecting a sufficient amount of 3D training data for autonomous vehicles to handle rare, but critical, traffic events (e.g., collisions) may take decades of deployment. Abundant video data of such events from municipal traffic cameras and video sharing sites (e.g., YouTube) could provide a potential alternative, but generating realistic training data in the form of 3D video reconstructions is a challenging task beyond the current capabilities of computer vision. Crowdsourcing the annotation of necessary information could bridge this gap, but the level of accuracy required to obtain usable reconstructions makes this task nearly impossible for non-experts. In this paper, we propose a novel hybrid intelligence method that combines annotations from workers viewing different instances (video frames) of the same target (3D object), and uses particle filtering to aggregate responses. Our approach can leveraging temporal dependencies between video frames, enabling higher quality through more aggressive filtering. The proposed method results in a 33% reduction in the relative error of position estimation compared to a state-of-the-art baseline. Moreover, our method enables skipping (self-filtering) challenging annotations, reducing the total annotation time for hard-to-annotate frames by 16%. Our approach provides a generalizable means of aggregating more accurate crowd responses in settings where annotation is especially challenging or error-prone.

Slide-based video is a popular format of online lecture videos. Lecture slides and narrations are complementary: while slides visually convey the main points of the lecture, narrations add detailed explanations to each item in the slide. We define a pair of slide item and the relevant sentence in the narration as a reference. In order to explore the design space of reference-based interaction techniques, we present DynamicSlide, a video processing system that automatically extracts references from slide-based lecture videos and a video player leveraging these references. Through participatory design workshops, we elicited useful interaction techniques specifically for slide-based lecture videos. Among these ideas, we selected and implemented three reference-based techniques: emphasizing the current item in the slide that is being explained, enabling item-based navigation, and enabling item-based note-taking. We also built a pipeline for automatically identifying references, and it shows 79% accuracy for finding 141 references in five videos from four different authors. Results from a user study suggest that DynamicSlide's features improve the learner's video browsing and navigation experience.

Document authors commonly use tables to support arguments presented in the text. But, because tables are usually separate from the main body text, readers must split their attention between different parts of the document. We present an interactive document reader that automatically links document text with corresponding table cells. Readers can select a sentence (or tables cells) and our reader highlights the relevant table cells (or sentences). We provide an automatic pipeline for extracting such references between sentence text and table cells for existing PDF documents that combines structural analysis of tables with natural language processing and rule-based matching. On a test corpus of 330 (sentence, table) pairs, our pipeline correctly extracts 48.8% of the references. An additional 30.5% contain only false negatives (FN) errors -- the reference is missing table cells. The remaining 20.7% contain false positives (FP) errors -- the reference includes extraneous table cells and could therefore mislead readers. A user study finds that despite such errors, our interactive document reader helps readers match sentences with corresponding table cells more accurately and quickly than a baseline document reader.

Presentation slides play an important role in online lecture videos. Slides convey the main points of the lecture visually, while the instructor's narration adds detailed verbal explanations to each item in the slide. We call the link between a slide item and the corresponding part of the narration a reference. In order to assess the feasibility of reference-based interaction techniques for watching videos, we introduce DynamicSlide, a video processing system that automatically extracts references from slide-based lecture videos and a video player. The system incorporates a set of reference-based techniques: emphasizing the current item in the slide that is being explained, enabling item-based navigation, and enabling item-based note-taking. Our pipeline correctly finds 79% of the references in a set of five videos with 141 references. Results from a user study suggest that DynamicSlide's features improve the learner's video browsing and navigation experience.

While video has become a widely adopted medium for online learning, existing video players provide limited support for navigation and learning. It is difficult to locate parts of the video that are linked to specific concepts. Also, most video players afford passive watching, thus making it difficult for learners with limited metacognitive skills to deeply engage with the content and reflect on their understanding. To support concept-driven navigation and comprehension of lecture videos, we present ConceptScape, a system that generates and presents a concept map for lecture videos. ConceptScape engages crowd workers to collaboratively generate a concept map by prompting them to externalize reflections on the video. We present two studies to show that (1) interactive concept maps can be useful tools for concept-based video navigation and comprehension, and (2) with ConceptScape, novice crowd workers can collaboratively generate complex concept maps that match the quality of those by experts.

Distance Cartograms (DC) distort geographical features so that the measured distance between a single location and any other location on a map indicates absolute travel time. Although studies show that users can efficiently assess travel time with DC, distortion applied in DC may confuse users, and its usefulness "in the wild" is unknown. To understand how real world users perceive DC's benefits and drawbacks, we devise techniques that improve DC's presentation (preserving topological relationships among map features while aiming at retaining shapes) and scalability (presenting accurate live travel time). We developed a DC-enabled system with these techniques, and deployed it to 20 participants for 4 weeks. During this period, participants spent, on average, more than 50% of their time with DC as opposed to a standard map. Participants felt DC to be intuitive and useful for assessing travel time. They indicated intent in adopting DC in their real-life scenarios.

Crowdsourcing is a common means of collecting image segmentation training data for use in a variety of computer vision applications. However, designing accurate crowd-powered image segmentation systems is challenging because defining object boundaries in an image requires significant fine motor skills and hand-eye coordination, which makes these tasks error-prone. Typically, special segmentation tools are created and then answers from multiple workers are aggregated to generate more accurate results. However, individual tool designs can bias how and where people make mistakes, resulting in shared errors that remain even after aggregation. In this paper, we introduce a novel crowdsourcing workflow that leverages multiple tools for the same task to increase output accuracy by reducing systematic error biases introduced by the tools themselves. When a task can no longer be broken down into more-tractable subtasks (the conventional approach taken by microtask crowdsourcing), our multi-tool approach can be used to further improve accuracy by assigning different tools to different workers. We present a series of studies that evaluate our multi-tool approach and show that it can significantly improve aggregate accuracy in semantic image segmentation.

A distance cartogram (DC) is a technique that alters distances between a user-specified origin and the other locations in a map with respect to travel time. With DC, users can weigh the relative travel time costs between the origin and potential destinations at a glance because travel times are projected in a linearly interpolated time space from the origin. Such glance-ability is known to be useful for travelers who are mindful of travel time when finding their travel destinations. When constructing DC, however, uneven urban traffic conditions introduce excessive distortion and challenge user intuition. In addition, there has been little research focusing on DC's user interaction design. To tackle these challenges and realize the potential of DC as an interactive decision-making support tool, we derive a set of useful interactions through two formative studies and devise two novel techniques called Geo-contextual Anchoring Projection and Scalable Road-network Construction. We develop an interactive map system using these techniques and evaluate this system by comparing it against an equidistant map (EM), a widely used conventional layout that preserves the geographical reality. Based on the analysis of user behavior and qualitative feedback, we identify several benefits of using DC itself and of the interaction techniques we derived. We also analyze the specific reasons behind these identified benefits.

Single-cell analysis can uncover the mysteries in the state of individual cells and enable us to construct new models about the analysis of heterogeneous tissues. State-of-the-art technologies for single-cell analysis have been developed to measure the properties of single-cells and detect hidden information. They are able to provide the measurements of dozens of features simultaneously in each cell. However, due to the high-dimensionality, heterogeneous complexity and sheer enormity of single-cell data, its interpretation is challenging. Thus, new methods to overcome high-dimensionality are necessary. Here, we present a computational tool that allows efficient visualization of high-dimensional single-cell data onto a low-dimensional (2D or 3D) space while preserving the similarity structure between single-cells. We first construct a network that can represent the similarity structure between the high-dimensional representations of single-cells, and then, embed this network into a low-dimensional space through an efficient online optimization method based on the idea of negative sampling. Using this approach, we can preserve the high-dimensional structure of single-cell data in an embedded low-dimensional space that facilitates visual analyses of the data.

Thanks to the supply of smartphones, mobile phones have come to store increasingly more personal information of users. To protect users' information, the information stored in mobile phones should not only be managed safety but also be deleted not to allow restoration. In the filesystems that have been applied to existing mobile phones, even when users have deleted information, data are not completely wiped from the storage. Therefore, the data can be easily recovered by using forensics tools. Considering that mobile phones are always exposed to the risk of robberies and loss, this situation can be misused for personal information spill. The present paper points out problems in the methods of data deleting from smartphones, proposes an efficient data deleting method considering mobile device environments with limitations in battery and hardware performance such as smartphones, and analyzes the efficiency of the techniques in relation to the types of filesystems and data formats.

The Internet has enabled learning at scale, from Massive Open Online Courses (MOOCs) to Wikipedia. But online learners may become passive, instead of actively constructing knowledge and revising their beliefs in light of new facts. Instructors cannot directly diagnose thousands of learners' misconceptions and provide remedial tutoring. This paper investigates how instructors can prompt learners to reflect on facts that are anomalies with respect to their existing misconceptions, and how to choose these anomalies and prompts to guide learners to revise incorrect beliefs without any feedback. We conducted two randomized experiments with online crowd workers learning statistics. Results show that prompts to explain why these anomalies are true drive revision towards correct beliefs. But prompts to simply articulate thoughts about anomalies have no effect on learning. Furthermore, we find that explaining multiple anomalies is more effective than explaining only one, but the anomalies should rule out multiple misconceptions simultaneously.

While explanations may help people learn by providing information about why an answer is correct, many problems on online platforms lack high-quality explanations. This paper presents AXIS (Adaptive eXplanation Improvement System), a system for obtaining explanations. AXIS asks learners to generate, revise, and evaluate explanations as they solve a problem, and then uses machine learning to dynamically determine which explanation to present to a future learner, based on previous learners' collective input. Results from a case study deployment and a randomized experiment demonstrate that AXIS elicits and identifies explanations that learners find helpful. Providing explanations from AXIS also objectively enhanced learning, when compared to the default practice where learners solved problems and received answers without explanations. The rated quality and learning benefit of AXIS explanations did not differ from explanations generated by an experienced instructor.

In this paper we propose a protection mechanism for the debug port. While debug ports are useful tools for embedded device development and maintenance, they can also become potential attack tools for device hacking in case their usage is permitted to hackers with malicious intentions. The proposed approach prevents illicit use of debug ports by controlling access through user authentication, where the device generates and issues authentication token only to the server-authenticated users. An authentication token includes user access information which represents the user’s permitted level of access and the maximum number of authentications allowed using the token. The device authenticates the user with the token and grants limited access based on the user’s access level. The proposed approach improves the degree of overall security by removing the need to expose the device’s secret key. Availability is also enhanced by not requiring server connection after the initial token generation and further by supporting flexible token transfer among predefined device groups. Low implementation cost is another benefit of the proposed approach, enabling it to be adopted to a wide range of environments in demand of debug port protection.

Mobile devices such as tab devices (e.g. iPad, Galaxy Tab), smart phones, laptops, and ultra-mobile personal computers (UMPCs) have become very popular as an access point of important information because of benefits such as small size and mobility. However, because of their high probability of loss or theft, it can turn into a point of information leakage. This is the reason why protection mechanism for confidential information stored in mobile devices have become a need. In this paper, a confidential information protection system for mobile devices is proposed, which is composed of a secure file system and a set of protocols. To show the security of our solution, we have performed an analysis of possible attacks. Additionally, we have developed client/server application prototypes using C# to demonstrate its implementation feasibility for a real environment. Copyright © 2012 John Wiley & Sons, Ltd. (In this paper, a confidential information protection system for mobile devices is proposed, which is composed of a secure file system and a set of protocols. To show the security of our solution, we have performed an analysis of possible attacks. Additionally, we have developed client/server application prototypes using C# to demonstrate its implementation feasibility for a real environment.)

Defending from DDoS attacks have become more difficult because they have evolved in many ways. Absence of a specific predetermined pattern, increase of number of attack devices, and distributed execution of the DDoS attack makes hard the recognition of the attack sources and thus application of countermeasures. When the DDoS attack is being executed, most of the cases, the target cannot provide its services normally; this is not a significant problem for non-critical application, but, for availability critical services such as stock financial, stock market, or governmental, the effect of the attack may involve huge damage. In this paper, we propose a DDoS avoidance strategy to provide service availability to those preregistered important users. In the proposed strategy, we divide the attack scenario in different time points and provide alternative access channels to already authenticated and other valid users.

Wireless sensor networks are used in many applications in military, ecology, health, and other areas. These applications often include the monitoring of sensitive information making the security issue one of the most important aspects to consider in this field. However, most of protocols optimize for the limited capabilities of sensor nodes and the application specific nature of the networks, but they are vulnerable to serious attacks. In this paper, a Secure Energy and Reliability Aware data gathering protocol (SERA) is proposed, which provides energy efficiency and data delivery reliability as well as security. The proposed protocol's security was confirmed by a formal verification carried out using the AVISPA tool and analysis of the most common network layer attacks such as selective forwarding, sinkhole, Sybil, wormhole, HELLO flood, and acknowledgment spoofing attacks. Additionally, a visual simulation environment was developed to evaluate the performance of the proposed protocol.

JTAG (Joint Test Access Group) is a powerful tool for the embedded system development environments. The features of JTAG, however, can be exploited by malicious users as a backdoor for launching attacks, an approach which now constitutes a major threat in the domain of device hacking. To deny unauthenticated users access to the features of JTAG port, this paper proposes a novel JTAG security mechanism. The proposed solution uses authentication based on credentials to achieve improved security and usability over existing solutions. Our approach is easily applicable to all standard JTAG environments because its structure is designed to be independent from the application environment. Further, the approach has lower implementation cost than encryption/decryption-based solutions since only hash and XOR calculations are employed in its authentication protocol. The security of the proposed mechanism has been verified through analysis against all forms of expected attacks, and its functionality is demonstrated with a real-life implementation.

Prototyping allows designers to quickly iterate and gather feedback, but the time it takes to create even a Wizard-of-Oz prototype reduces the utility of the process. In this paper, we introduce crowdsourcing techniques and tools for prototyping interactive systems in the time it takes to describe the idea. Our Apparition system uses paid microtask crowds to make even hard-to-automate functions work immediately, allowing more fluid prototyping of interfaces that contain interactive elements and complex behaviors. As users sketch their interface and describe it aloud in natural language, crowd workers and sketch recognition algorithms translate the input into user interface elements, add animations, and provide Wizard-of-Oz functionality. We discuss how design teams can use our approach to reflect on prototypes or begin user studies within seconds, and how, over time, Apparition prototypes can become fully-implemented versions of the systems they simulate. Powering Apparition is the first self-coordinated, real-time crowdsourcing infrastructure. We anchor this infrastructure on a new, lightweight write-locking mechanism that workers can use to signal their intentions to each other.

Websites like YouTube host millions of how-to videos, but the interfaces are not optimized for learning. Previous research suggests that users learn more from how-to videos when the information from the video is presented in outline form, with individual steps and labels for groups of steps (subgoals) shown. We envision an alternative video player where the steps and subgoals are displayed alongside the video. To generate this information for existing videos, we propose a learnersourcing approach, where people actively learning from a video provide such information. To demonstrate this method, we created a workflow where learners contribute and refine subgoal labels for how-to videos. We deployed a live website with our workflow implemented on a set of introductory web programming videos. For the four videos with the highest participation, we found that a majority of learner-generated subgoals were comparable in quality to expert-generated ones. Learners commented that the system helped them grasp the material, suggesting that our workflow did not detract from the learning experience., Massachusetts Institute of Technology. Undergraduate Research Opportunities Program, Cisco Systems, Inc., Quanta Computer (Firm) (Qmulus Project), National Science Foundation (U.S.) (Award SOCS-1111124), Alfred P. Sloan Foundation (Sloan Research Fellowship), Samsung (Firm) (Fellowship)

In contrast to typical laboratory experiments, the everyday use of online educational resources by large populations and the prevalence of software infrastructure for A/B testing leads us to consider how platforms can embed in vivo experiments that do not merely support research, but ensure practical improvements to their educational components. Examples are presented of randomized experimental comparisons conducted by subsets of the authors in three widely used online educational platforms -- Khan Academy, edX, and ASSISTments. We suggest design principles for platform technology to support randomized experiments that lead to practical improvements -- enabling Iterative Improvement and Collaborative Work -- and explain the benefit of their implementation by WPI co-authors in the ASSISTments platform.

With an unprecedented scale of learners watching educational videos on online platforms such as MOOCs and YouTube, there is an opportunity to incorporate data generated from their interactions into the design of novel video interaction techniques. Interaction data has the potential to help not only instructors to improve their videos, but also to enrich the learning experience of educational video watchers. This paper explores the design space of data-driven interaction techniques for educational video navigation. We introduce a set of techniques that augment existing video interface widgets, including: a 2D video timeline with an embedded visualization of collective navigation traces; dynamic and non-linear timeline scrubbing; data-enhanced transcript search and keyword summary; automatic display of relevant still frames next to the video; and a visual summary representing points with high learner activity. To evaluate the feasibility of the techniques, we ran a laboratory user study with simulated learning tasks. Participants rated watching lecture videos with interaction data to be efficient and useful in completing the tasks. However, no significant differences were found in task performance, suggesting that interaction data may not always align with moment-by-moment information needs during the tasks.

Videos are a widely-used kind of resource for online learning. This paper presents an empirical study of how video production decisions affect student engagement in online educational videos. To our knowledge, ours is the largest-scale study of video engagement to date, using data from 6.9 million video watching sessions across four courses on the edX MOOC platform. We measure engagement by how long students are watching each video, and whether they attempt to answer post-video assessment problems.Our main findings are that shorter videos are much more engaging, that informal talking-head videos are more engaging, that Khan-style tablet drawings are more engaging, that even high-quality pre-recorded classroom lectures might not make for engaging online videos, and that students engage differently with lecture and tutorial videos.Based upon these quantitative findings and qualitative insights from interviews with edX staff, we developed a set of recommendations to help instructors and video producers take better advantage of the online video format. Finally, to enable researchers to reproduce and build upon our findings, we have made our anonymized video watching data set and analysis scripts public. To our knowledge, ours is one of the first public data sets on MOOC resource usage.

With thousands of learners watching the same online lecture videos, analyzing video watching patterns provides a unique opportunity to understand how students learn with videos. This paper reports a large-scale analysis of in-video dropout and peaks in viewership and student activity, using second-by-second user interaction data from 862 videos in four Massive Open Online Courses (MOOCs) on edX. We find higher dropout rates in longer videos, re-watching sessions (vs first-time), and tutorials (vs lectures). Peaks in re-watching sessions and play events indicate points of interest and confusion. Results show that tutorials (vs lectures) and re-watching sessions (vs first-time) lead to more frequent and sharper peaks. In attempting to reason why peaks occur by sampling 80 videos, we observe that 61% of the peaks accompany visual transitions in the video, e.g., a slide view to a classroom view. Based on this observation, we identify five student activity patterns that can explain peaks: starting from the beginning of a new material, returning to missed content, following a tutorial step, replaying a brief segment, and repeating a non-visual explanation. Our analysis has design implications for video authoring, editing, and interface design, providing a richer understanding of video learning on MOOCs.

One goal for massive open online courses is that the educational benefits they provide scale as a function of the number and diversity of learners interacting with the platforms, since an increasing amount of data is available about what interactions and content increase engagement and learning, as well as which educational interactions are effective for which learners, particularly since more and more is known about each individual. This paper presents the MOOClet Framework for tackling this goal, which recognizes a key relationship between randomized experiments and personalization of content. The Framework defines MOOClets as modular components of online courses that can be modified to create different versions, which in turn can be iteratively and adaptively improved through experiments and personalized to characteristics of users. We show how the MOOClet Framework provides guidance in identifying MOOClets and augmenting existing platforms with a platform-independent layer that enables experimentation and personalization even when platforms do not provide native support. We present a concrete usage scenario of the framework in an implementation for the EdX platform, showing how the addition of reflection questions and other content to a lecture video could be experimentally evaluated and personalized. A modeling simulation is also presented to show how using the MOOClet Framework allows data-driven decisions about experimentation and personalization to be made using existing machine learning models. Consideration of the MOOClet Framework could help researchers, instructors, and course designers in identifying, implementing, and improving modular components of existing online education platforms through experimentation and personalization.

Millions of learners today use how-to videos to master new skills in a variety of domains. But browsing such videos is often tedious and inefficient because video player interfaces are not optimized for the unique step-by-step structure of such videos. This research aims to improve the learning experience of existing how-to videos with step-by-step annotations. We first performed a formative study to verify that annotations are actually useful to learners. We created ToolScape, an interactive video player that displays step descriptions and intermediate result thumbnails in the video timeline. Learners in our study performed better and gained more self-efficacy using ToolScape versus a traditional video player. To add the needed step annotations to existing how-to videos at scale, we introduce a novel crowdsourcing workflow. It extracts step-by-step structure from an existing video, including step times, descriptions, and before and after images. We introduce the Find-Verify-Expand design pattern for temporal and visual annotation, which applies clustering, text processing, and visual analysis algorithms to merge crowd output. The workflow does not rely on domain-specific customization, works on top of existing videos, and recruits untrained crowd workers. We evaluated the workflow with Mechanical Turk, using 75 cooking, makeup, and Photoshop videos on YouTube. Results show that our workflow can extract steps with a quality comparable to that of trained annotators across all three domains with 77% precision and 81% recall.

Different microRNA target prediction tools produce different results. Motivated by this fact, here we present an ensemble-learning approach that combines the outcomes from multiple tools to reduce prediction error. We test this approach with a dataset derived from a public database containing human microRNAs and microRNA-mRNA pairs. According to our experimental result, using the proposed method tends to be significantly better than using individual prediction tools in terms of increasing the area under curve (AUC) defined on a receiver operating characteristic curve.

How-to videos can be valuable for learning, but searching for and following along with them can be difficult. Having labeled events such as the tools used in how-to videos could improve video indexing, searching, and browsing. We introduce a crowdsourcing annotation tool for Photoshop how-to videos with a three-stage method that consists of: (1) gathering timestamps of important events, (2) labeling each event, and (3) capturing how each event affects the task of the tutorial. Our ultimate goal is to generalize our method to be applied to other domains of how-to videos. We evaluate our annotation tool with Amazon Mechanical Turk workers to investigate the accuracy, costs, and feasibility of our three-stage method for annotating large numbers of video tutorials. Improvements can be made for stages 1 and 3, but stage 2 produces accurate labels over 90% of the time using majority voting. We have observed that changes in the instructions and interfaces of each task can improve the accuracy of the results significantly., Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. SuperUROP Program, Quanta Computer Incorporated

Video tutorials on the web have gained popularity in various domains, but most video repositories are not designed to support the unique content and structure of how-to videos. Learners face difficulty in finding relevant videos and applying the skills embedded in a video clip. We introduce ToolScape, a video browsing interface with a storyboard summarization and an interactive timeline. It allows learners to quickly scan, filter, and review multiple videos without having to play them. Learners can also jump to or repeat a particular step within a clip by clicking interactive indices on the timeline. In a within-subjects study where participants engaged in end-to-end design tasks with ToolScape and a control interface based on YouTube, the participants using ToolScape rated their design work higher and showed a higher gain in self-efficacy. External raters ranked designs using ToolScape higher.

Subgoal labeling [1] is a technique known to support learning new knowledge by clustering a group of steps into a higher-level conceptual unit. It has been shown to improve learning by helping learners to form the right mental model. While many learners view video tutorials nowadays, subgoal labels are often not available unless manually provided at production time. This work addresses the challenge of collecting and presenting subgoal labels to a large number of video tutorials. We introduce a mixed-initiative approach to collect subgoal labels in a scalable and efficient manner. The key component of this method is learnersourcing, which channels learners' activities using the video interface into useful input to the system. The presented method will contribute to the broader availability of subgoal labels in how-to videos.

Recently, several user authentication schemes for wireless sensor networks based on two-factor concept using the smart card technology were proposed. However, they have serious limitations in terms of security and usability. First, even though they are enhancements of other works, they still have several security flaws, such as vulnerability against parallel session, privileged-insider, and gateway-node bypassing attacks and lack of mutual authentication between user station and gateway node. On the other hand, they also present a usability constraint, in a sense that they do not consider the use case when sensor nodes cannot communicate with gateway node. In this case, data collected by isolated sensor nodes could not be accessed until they recover such communication, which is in many times not recoverable rapidly or forever (e.g., military applications, natural disaster monitoring). Due to all these reasons, this paper proposes a robust user authentication scheme which fixes the security weaknesses of previous solutions and provides wider usability considering the use case when the sensor nodes cannot communicate with the gateway node. Once the solution is described, its security is ensured by formal proof and analysis against attacks. Additionally, performance and cost analysis are executed to determine its level of feasibility for real implementation.

New threats to embedded systems have appeared lately and attacks such as physical hacking using high cost analysis tools have become popular. This is the reason why hardware component of devices managing confidential information cannot be considered as trusted anymore. In this paper, we propose a confidential data protection scheme from physical hacking. Our proposal provides encryption of confidential data stored in external memory with additional features such as anti key analysis/cloning and key destruction mechanisms.

The uses of wireless sensor networks have increased to be applicable in many different areas, such as military applications, ecology, and health applications. These applications often include the management of confidential information, making the issue of security one of the most important aspects to consider. In this aspect, a user authentication mechanism that allows only legitimate users to access the network data becomes critical for maintaining the confidentiality and integrity of the network information. In this paper, we describe and cryptoanalyze previous works in user authentication to illustrate their vulnerabilities and security flaws. We then propose a robust user authentication scheme that solves the identified limitations. Additionally, we describe how the proposed protocol is more suitable for a secure sensor network implementation by analysis in terms of security and performance.

Photoshop with Friends is an online community of learners exchanging just-in-time help on graphic design tasks. The system attempts to provide an interactive, visual, context-aware, and personalized mode of learning. Developed as a Facebook application, Photoshop with Friends allows users to help each other in live sessions, with built-in screen sharing, recording, and voice chat support. Major design decisions are guided by two laboratory studies that identified challenges in learning graphic design skills on the web.

Information systems manage assets that are critical for the business processes of organizations. Therefore, it is imperative that information systems be guaranteed and secured from the beginning of their development life cycle. Several approaches such as misuse cases, attack tree, and threat modeling have been proposed by way of security requirements. However, these approaches do not prioritize security requirements, though it is necessary in many cases. For example, when the security budget is insufficient, security requirements need to be prioritized to decide what will be developed and what will not. In this paper, we propose an extension to threat modeling by creating a process that allows the prioritization of security requirements via the valuation of assets, threats, and countermeasures modeled in a tree-like structured graph that we refer to as a “valuation graph.”

Wireless sensor networks are used in many applications in military, ecological, health, and other areas. These applications often include the monitoring of sensitive information making the security issue one of the most important aspects to consider in this field. However, most of protocols optimize for the limited capabilities of sensor nodes and the application specific nature of the networks, but they are vulnerable to serious security attacks. In this paper, a Secure Energy and Reliability Aware data gathering protocol (SERA) is proposed, which provides energy efficiency and data delivery reliability as well as a security scheme giving protection against the most common network layer attacks such as spoofed, altered, or replayed routing information, selective forwarding, sinkhole attacks, Sybil attacks, wormhole attacks, HELLO flood attacks, and acknowledgment spoofing attacks.

A numerous property and information are transferred to electronic and computerized while internet and networks are expanded. Behind the convenience, problems such as cyber terror, computer crime, and privacy trespassers are growing constantly. There are a number of evidences showing that damaged computers and the related systems are used for tracking the criminal or proving the crime. In the part of computer forensics, there have been many efforts to find the evidences in the network. A new efficient network forensic analysis using visualization, that supports panoramical views and human centralized analysis is proposed in this paper. In this work, the emphasis on maximizing the visualization effect related to network components is made to build more efficient forensic system. The experimental results show that visualized network forensic analysis utilizes resource and time more efficiently compared to the traditional methods.