Manish, computational and data-enabled science and engineering (CDS&E) has traditionally used dedicated platforms and specialized high-end systems – why should clouds be considered for them?
Clouds are already having a significant impact by enabling enterprises to outsource IT on demand. Clouds enable the outsourcing of many of the tedious aspects of research, such as deploying, configuring, and managing infrastructure, and thus enable scientists to focus on science.
Cloud services improve productivity, facilitate the sharing of research results, and enable the reproducibility of associated computations. This has democratized access to computational and data resources for researchers who don’t have adequate local infrastructure.
What role do you see for clouds in computational and data-enabled science and engineering (CDS&E)?
While clouds can provide many of the same advantages to CDS&E that they provide enterprise IT, it is critical to also look beyond these benefits, and understand application formulations and usage modes that are meaningful in a hybrid infrastructure with grids, high-performance computing, and clouds.
Cloud computing can support CDS&E applications by providing infrastructure; when, for example, local infrastructure is not available or is insufficient. It can also supplement existing infrastructure to provide additional capacity or complementary capabilities to meet heterogeneous or dynamic needs. Clouds can serve as accelerators, or provide resilience to scientific workflows by moving execution to alternative resources when a failure occurs. Cloud abstraction can alleviate some of the challenges that scientific applications face in current high-performance computing environments.
Which aspects of computational and data-enabled science and engineering (CDS&E) and clouds does your research address?
Our research has been exploring cloud concepts and supporting infrastructure for CDS&E. Our experiments include a broad set of federated resources spanning existing grids, high-performance computers, clusters, public clouds, and applications. We have been using the CometCloud framework, developed at Rutgers, to explore application formulations and hybrid infrastructure usage modes that are meaningful for CDS&E application workflows.
This includes three usage modes: ‘HPC in the cloud’, where researchers outsource entire applications to current public and - or - private cloud platforms; ‘HPC plus cloud’, which focuses on exploring scenarios where clouds can complement grids and high-performance computers to support science and engineering application workflows; and ‘HPC as a cloud’, that’s focused on parallel resources using elastic on-demand cloud abstractions.
Can you tell me a little bit about CometCloud?
CometCloud is an autonomic computingengine that enables a dynamic and on-demand federation of clouds and grids, as well as deployment and execution of applications on federated environments. It supports diverse parallel infrastructures, clouds and grids, enabling on-demand integration of public and private clouds. And it provides autonomic cloudbursts, i.e. a solution for extreme requirements and spikes in demand. The CometCloud programming layer provides platforms for application development and management. It supports MapReduce, Workflow, and Master-Worker, or Bag-of-Tasks.
What do you see as the key research challenges for computational and data-enabled science and engineering (CDS&E) and clouds?
First, algorithms and application formulations can benefit from elasticity and hybrid cloud, grid, and high-performance computing usage modes. Second, programming abstractions and systems can enable CDS&E applications to simply and effectively take advantage of the elastic access to resources and services. Third, middleware stacks and management policies will be able to support the new CDS&E application formulations and services.
Note, that these are in addition to core cloud challenges such as programming models, security, standardization, etc. These are discussed in more details in our recent white paper on Cloud Paradigms and Practices for CDS&E.
Do you have any concluding remarks for readers?
Aggressive cloud computing technology development has resulted in many classes of cloud services that provide attractive solutions for a range of business applications. It is also clear that there are real benefits in using clouds as part of a hybrid cyber infrastructure for supporting CDS&E, e.g. simplifying the deployment of applications and the management of their execution, improving efficiency, productivity, and providing more attractive cost to performance ratios. Furthermore, clouds can support new classes of algorithms and enable new application formulations, which can potentially revolutionize CDS&E research and education.
Subscribe to the blog to receive updates about:
AltaFlux understands what you and your organization need to excel, and can deliver rapid innovation to unleash your full workforce potential. Together, we can empower your business by streamlining, transforming, and optimizing your key HCM and talent processes with industry-leading SAP SuccessFactors technology—enabling you to adapt at the speed of change.
AltaFlux Corporation is a global HCM cloud consulting partner based in Troy, Michigan. We empower organizations by streamlining, transforming, and optimizing key human capital management (HCM) processes with industry-leading HCM cloud solutions like SAP SuccessFactors, Benefitfocus, WorkForce Software and Dell Boomi.