CSD Home | SCS Home



Research Areas - Technology and Society Research in the Computer Science Department at Carnegie Mellon


CSD faculty: Emma Brunskill, Mike Christel (ETC), Roger Dannenberg, Ananda Guna, Alex Hauptmann, Raj Reddy, Manuela Veloso, Howard Wactlar, Eric Xing (ML/LTI)


Technology and society research encompasses those efforts whose outputs are focused on applications, or mechanisms enabling those applications, that when deployed, are directly beneficial to enriching and sustaining the conduct of our lives and the functioning of our society.

The domains are varied and presently include every level of society’s functioning: from personal interaction with cyberspace (Blum) to optimized business-to-business electronic commerce (Sandholm), from new forms of information and entertainment (Wactlar, Reddy) to new methods for manufacturing self-organizing micro-systems (Goldstein), and from improving individual healthcare diagnostics to detection of pandemics (Wactlar, Moore). The outputs vary from comprehensive systems to component technology, and the time frames for transition from research to field deployment may stretch from a year to a decade. These efforts are complemented by education programs of technology outreach (e.g., by consulting or insertion) by students to solve local problems in both domestic and foreign venues (Mertz, Veloso).


1 Achieving External Impact

A common characteristic of these efforts is that their impact is most often felt in disciplines outside the scope of computer science. Thus the first challenge is that of learning enough about the application domain and state of practice. This is most frequently accomplished through work with outside collaborators, either in academia or industry, who are domain specialists. Examples include geriatric psychiatrists, epidemiologists, and commodity brokers. Sometimes the learning must be immersive — through a sabbatical or a series of summers away.

The next challenge is in determining that application space, that niche, where the results we can achieve, often with underlying technology that is probabilistic and imperfect, produce valuable outcomes for its users despite its inherent error. This may be accomplished by applying multiple or error compensating methods, or by appropriately placing a human in the process to guide and inform the application.

This is naturally accompanied by the requirement to set and meet the expectations of the receiving community or discipline, for example, those of nursing home patients, caregivers and attending physicians in one case; practicing intelligence analysts in another, and business-to-business suppliers in a third. So the next critical set of challenges for this research relate to the demands of performance: speed, accuracy, reliability and scalability. Finally there are the challenges of technology deployment, acceptance and evaluation in the field.

The approach to this kind of research is often an integrative one: combining methods or results from multiple computer science areas, focusing on the performance of the end application, not that of the underlying techniques employed in isolation. An implementation challenge often arises here as one needs to integrate legacy computation written in different languages for different environments and not necessarily intended to be interoperable. The development may spawn focused research in the contributing sub-areas, but based on enabling and improving the application’s validity, scalability, and breadth of applicability. This is most often a highly iterative experimental approach, as much hypothesis driven as exploratory, requiring evaluation in the field or on real data of sufficient volume and breadth to be surrogates of field trials.


2 Research Thursts

Searchable Web Universal access to information through the World Wide Web has been fertile ground for application of our science and technologies. Wactlar pursued the goal of making video a searchable medium by integrating errorful speech, image and natural language understanding, while Reddy engineered the establishment of a Universal Library and an international Million Book Project to create the corpus. The former generated the 8000-hour Informedia Digital Video Library and spawned commercial enterprise but found its most eager audience in the intelligence community for monitoring world-wide broadcasts. The latter has established major scanning efforts in India and China and is well on its way to its ambitious goals.

Internet Hygiene Whereas universal access is a good thing for people, when machines masquerade as users the intent is often malicious. Manuel Blum’s innovative CAPTCHA for “completely automated public Turing test to tell computers and humans apart” has been adapted by major portals to prevent automated registrations by providing interactive tests only humans are likely to be able to pass. Olston on the other hand has worked to alleviate the shortcomings of the major search engines with mechanisms for ameliorating their bias against new content and enabling topic-specific web-crawlers.

Patient Care Societal relevance and impact of our research is often most positively viewed when it improves our health and safety in significant ways. Several efforts in computer science research have been focused on those goals. Wactlar’s work in video search gave rise to efforts in machine understanding of human behavior that was captured in video. The resulting project, called CareMedia, targets nursing home patients and improving the appropriateness and outcomes of treatment interventions: behavioral interventions, environmental states, and drug treatments. This work is carried out jointly with Ashok Bharucha, a geriatric psychiatrist at the Western Psychiatric Institute and Clinic and the University of Pittsburgh Medical Center.

Our approach is to capture a continuous audiovisual record of the patients’ activities, and automatically process and summarize that voluminous record to achieve more accurate metrics of activity and behavior to better enable optimal interventions. The research is conducted along three dimensions: machine understanding and classification of aural and visual data recorded in the nursing home environment, categorizing and dynamically summarizing antecedents and consequences of behavior time-aligned to various interventions, and creating information visualizations that limit redundancy and highlight episodes useful for evaluating changes in a patient’s physical and cognitive condition while addressing privacy and policy restrictions. Figure 1 illustrates video data capture and automated analysis of characteristic optical flow to measure eating behavior. The long term goal of the research is to allow these techniques to be fielded in other settings like an individual’s home, providing a greater level of monitoring and maintenance for elderly residents so that they can enjoy optimal interventions in their own homes, as well as independent living and delayed admittance to skilled care facilities.

Figure 1: Video-based Optical flow Data for Measuring Eating Behavior in Nursing Home

Early Epidemic Detection More globally, Moore’s Auton Lab has deployed systems that help detect early signs of unnatural disease outbreaks. For example, every day their algorithms search national pharmacy retail data for signs of emerging “hotspots” of disease activity. Their methods are used in large part because of new algorithms that make it practical to search over a trillion regions in a few minutes for the most statistically significant increase. In another example, their WSARE (What’s Strange About Recent Events) algorithm, which involves a combination of Bayesian network structure learning and search for significant deviations from the anticipated distribution of properties of Emergency Department admits, has also been used for monitoring in many states. It was recently responsible for detection of an entirely new (but fortunately natural) pediatric outbreak in Israel. This work is in collaboration with Mike Wagner and his RODS lab at the University of Pittsburgh.

Other Thrusts Artificial intelligence has also been successfully applied to electronic commerce by Sandholm in his work to automatically design and build electronic marketplaces (e.g., “smart auctions”) that lead to more efficient outcomes. The applications range from designing rules for business-to-business commerce to those for referendums selecting alternative public works projects.

Newly spawned research by Goldstein and Mowry crosses multiple computer science areas to realize Claytronics: a form of programmable matter that can create a physical replica of an arbitrary moving 3D object that can be updated in real time. Potential applications abound from self-adapting devices to synthetic interactive environments, but the challenge of even programming millions of such cooperating, sub-millimeter scale units is daunting.


3 Challenges

Though still in its early stages, these prior and ongoing efforts have led to the emergence of a Carnegie Mellon Quality of Life Technologies Initiative whose mission more broadly is the life-long assurance of human function and performance through advanced technology, such as environmentally aware robotic assistive devices, communicating networks of neural implants, and wearable cognitive assistants. The goal is to significantly improve and sustain the quality of life for all citizens by integrating information, engineering and medical innovations for the prevention, rehabilitation and compensation of human ailments. The initiative, though formulated and centered in CSD, transcends all SCS units and many other departments and colleges across the University. Much of this work is intended to be conducted with strong participation from clinical medical researchers and institutions both in our region and across the country.

Impediments to this type of research are varied. First and foremost is the need to engage in field deployments in clinical settings where outcomes can be documented and measured. Anecdotal evidence of success and failure is woefully inadequate. Complementary to that is the need for iterative deployments, refiecting what is learned in field trials with affected subjects.



      CSD Home   Webteam  ^ Top   SCS Home