|Title||Management, Analysis, and Visualization of Experimental and Observational Data - The Convergence of Data and Computing |
|Publication Type||Conference Paper |
|Year of Publication||2016 |
|Authors||E. Bethel, W, Greenwald, M, van Dam, KK, Parashar, M, Wild, SM, Wiley, HS |
|Conference Name||IEEE 12th International Conference on e-Science |
|Date Published||10/2016 |
|Abstract||Scientific user facilities—particle accelerators, telescopes, colliders, supercomputers, light sources, sequencing facilities, and more—operated by the U.S. Department of Energy (DOE) O ce of Science (SC) generate ever increasing volumes of data at unprecedented rates from experiments, observations, and simulations. At the same time there is a growing community of experimentalists that require real-time data analysis feedback, to enable them to steer their complex experimental instruments to optimized scientific outcomes and new discoveries. Recent efforts in DOE-SC have focused on articulating the data-centric challenges and opportunities facing these science communities. Key challenges include di culties coping with data size, rate, and complexity in the context of both real-time and post-experiment data analysis and interpretation. Solutions will require algorithmic and mathematical advances, as well as hardware and software infrastructures that adequately support data-intensive scientific workloads. This paper presents the summary findings of a workshop held by DOE-SC in September 2015, convened to identify the major challenges and the research that is needed to meet those challenges.