Designing Multi-Cloud Data Platforms for Large-Scale Enterprise Workloads
DOI:
https://doi.org/10.15662/IJEETR.2023.0501008Keywords:
Multi-Cloud, AWS, GCP, Data Platforms, Cloud Architecture, Distributed SystemsAbstract
The first-time workloads of large-scale businesses also demand increasing amounts of data platforms that can support elastic scalability, capacity to support high availability, interoperability, and resilience to heterogeneous clouds. The design of such platforms to have a multi-cloud environment presents significant architectural problems related to data integration, location of workloads, distributed processing, security, governance, performance, and cost optimization. This study introduces an architectural framework for crafting multi-cloud data platforms incorporating services and capabilities throughout Amazon Web Services (AWS) and Google Cloud Platform (GCP). The given architecture constitutes one data architecture that has ingestion, storage, processing, orchestration, metadata management, analytics, security, and monitoring layers. It can pool distributed data processing capabilities and can policy-driven workload placement to in addition to scaling to enterprise-scale apps, reduce the reliance on a single cloud provider. The architecture is also interoperable as standardized interface is offered to provide automated governance and centralized observability across boundaries of cloud. The key evaluation aspects to ensure that the proposed platform is efficient are the performance, scalability, availability, and resource usage. The paper explains how a well-planned multi-cloud environment can be used to help power and agile enterprise data solutions and deal with the distributed infrastructure challenges. The proposed architecture provides a powerful foundation where interested companies seek to implement transforming enterprise workloads, on scalable, regulated and cloud independent data system bases.
References
[1] M. S. Aslanpour et al., “Performance evaluation metrics for cloud, fog and edge computing: A review, taxonomy, benchmarks and standards for future research,” Internet of Things, vol. 12, 2020.
[2] S. S. Gill et al., “Transformative effects of IoT, blockchain and artificial intelligence on cloud computing: Evolution, vision, trends and open challenges,” Internet of Things, 2019.
[3] A. H. Ali et al., “Recent trends in distributed online stream processing platform for big data: Survey.”
[4] A. A. Alli et al., “The fog cloud of things: A survey on concepts, architecture, standards, tools, and applications,” Internet of Things, 2020.
[5] M. Bendechache et al., “Simulating resource management across the cloud-to-thing continuum: A survey and future directions,” Future Internet, 2020.
[6] E. A. Brewer, “Kubernetes and the path to cloud native.”
[7] Y. Chen et al., “Exploring the use of synthetic gradients for distributed deep learning across cloud and edge resources.”
[8] R. N. Calheiros et al., “CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms,” Software: Practice and Experience, 2011.
[9] D. P. Abreu et al., “A comparative analysis of simulators for the cloud to fog continuum,” Simulation Modelling Practice and Theory, 2019.
[10] R. A. Cherrueau et al., “Enoslib: A library for experiment-driven research in distributed computing,” IEEE Transactions on Parallel and Distributed Systems, 2021.
