Trusted partner for your Digital Journey

Modern Data Architecture using Snowflake

2

Table of Contents

1. Abstract 3

2. The challenges of data architecture using traditional technologies 4

3. Modern data architecture with cloud data warehouses 5

4. A truly modern data architecture using Snowflake 6

5. Summary 7

3Modern Data Architecture using Snowflake

For many enterprises, Snowflake is a compelling option for a cloud-based data platform that can meet the organization’s data analytics needs. In this paper, we will examine three typical enterprise data architecture approaches, introduce Snowflake, explain its architecture and features, and discuss the advantages of migrating on-premises data platforms to Snowflake.

Abstract

4

Until recently, on-premises Data Warehouses (DWs) were the logical home of all enterprise data. However, in recent years, an explosion in the volume, velocity and variety of data has put tremendous pressure on DWs, which are unable to scale to meet this demand.

In response, many organizations have turned to Hadoop to manage the enormous increase in data while maintaining the coherence of the data warehouse. However, Hadoop requires hard-to-find skills and requires additional overhead to operate. It has served the data science community well, but most business users are unable to make direct use of the data stored in a Hadoop data lake.

The typical data architecture that pairs Hadoop with on-premises data warehouses looks something like this:

The challenges of data architecture using traditional technologies

Streaming

Un-structured

Loader

Structured

Extr

act &

Lo

ad

Stage

Data LakeAI / ML

Ad hoc Query

Ad hoc Query

BI Reports

DW

5Modern Data Architecture using Snowflake

The current market leaders in the Big Data space (including many popular cloud vendors) offer an alternative, cloud-based data architecture similar to the one depicted below:

These types of systems are a significant improvement over the traditional data architecture seen above. They address the scalability challenges associated with on-premises data warehouses, but cloud data warehouses still cannot support all the varied workloads of the enterprise. “Modern” data architectures that employ cloud data warehouses still require separate data lakes to be built using object storage to store and process big data (in terms of volume, variety, and velocity).

With its unique multi-cluster shared data architecture, Snowflake is a dynamic and highly scalable data platform that offers complete separation of compute (virtual data warehouse) and storage on cloud. For superior data processing, Snowflake runs one or more compute resources which can be automatically resized to support an increase or decrease in the concurrent workloads. During query execution, these clusters retrieve data from storage and keep data and query results cached to improve the performance of current and future queries. The compute clusters can also operate on the same data while fully enforcing transactional integrity and ACID compliance.

Modern data architecture with cloud data warehouses

Structured

Streaming

Un-structured

Server-less Loader

Extr

act &

Lo

ad

Data Lake

AWS/Azure/GCP

Data Warehouse

ModeledData

Data Marts

Analytics(AI/ML)

Ad hocReporting

BI ReportingObjects/Files

Object Storage

Cloud Data Warehouse (e.g. Redshift, BigQuery, Synapse, etc.)

6

In our experience, we have observed that most data architects working with Snowflake continue to fall back on the “modern” data architecture design discussed earlier. The typical installation uses Snowflake as a data warehouse only, while maintaining a file-based data lake outside of Snowflake — even though Snowflake can replace all these systems.

To harness the true potential of Snowflake, architects must stop thinking about data in terms of existing types of systems such as data warehouses, data marts and data lakes. This approach imposes an unnatural and artificial boundary in the enterprise data landscape. As a result, data continues to be siloed in disparate data systems, which makes it very challenging to reap the full business value and governance rewards that Snowflake can deliver.

With Snowflake, there is no need to divide the enterprise data landscape into “data zones.” All enterprise workloads can be supported with a single platform, delivering benefits such as:

• Rapid data provisioning and real-time insights

• Nearly infinite scalability

• Separation of storage and compute

• Loosely coupled architecture

• Ability for users to leverage all types of data using existing SQL skills

• Easy exchange of live data inside and outside the organization without copying data or developing an ETL

• Easy data monetization using Snowflake Marketplace

Instead, the following architecture approach is designed to take full advantages of Snowflake’s innovative features.

A truly modern data architecture using Snowflake

Interested in learning how your organization can take advantage of Snowflake?

Contact our Snowflake CoE at dl-snowflake-da-coe@atos.net to set up a consultation.

Streaming

Storage Compute

Structured

Un-structured

Data Exchange

AI/ML

BI Reporting

Real Time Analytics

Stage

Data Lake

DW

SQLELT

BI Report

Analytics

AWS / Azure / GCP

Server-less Loader

Extr

act &

Lo

ad

7Modern Data Architecture using Snowflake

Most cloud data warehouses were originally designed for on-premises use, then moved to cloud. As such, they are unable to fully leverage the scalability of cloud storage and compute.

Unlike other cloud data warehouses on the market, Snowflake’s unique architecture provides a cohesive platform for all your data needs — storing data in its native format with industry-leading security and compliance standards and other unique features. It allows an enterprise to focus on obtaining every possible insight from its data, enabling data-driven decision making for all types of users and workloads.

At Atos, we have leveraged our experience helping enterprises migrate to Snowflake to build a complete toolkit that helps customers quickly and securely migrate their on-premise data warehouses to the Snowflake platform.

Please contact our Snowflake CoE at: dl-snowflake-da-coe@atos.net to set up your consultation and learn more about our Snowflake migration offerings.

Summary

Copyright © 2020 Atos. Atos, the Atos logo, Atos|Syntel, and Unify are registered trademarks of the Atos group. © 2020 Atos. Confidential information owned by Atos, to be used by the recipient only. This document, or any part of it, may not be reproduced, copied, circulated and/or distributed nor quoted without prior written approval from Atos.

Atos|Syntel is a leading provider of integrated digital and knowledge process services and a member of the Atos Group, a global leader in digital transformation with 110,000 employees in 73 countries and annual revenue of €12 billion. We help enterprises accelerate their digital journeys, increase agility and business performance, evolve to “Digital native” standards, and deliver scale and flexibility for the Digital Age.

Atos|Syntel unites Atos’s scale and world-class technology capabilities with Syntel’s industry focus, global delivery model, and services powered by intelligent automation.

Learn more at: www.atos-syntel.net

Let’s start a discussion together

About Atos|Syntel