www.datameer.com
Open in
urlscan Pro
162.159.134.42
Public Scan
Submitted URL: https://mail.datameer.com/e3t/Ctc/GD+113/cGr5k04/VVY-P43Xr-H8W8hfB__4_343ZW3V7XmG4GFzKpN4lQrLB3q3pBV1-WJV7CgQ5QW35fVp23lR-...
Effective URL: https://www.datameer.com/blog/how-to-transform-your-data-in-snowflake-part-1/?utm_campaign=Datameer%20Announcement%20Flow...
Submission: On March 15 via api from SE — Scanned from DE
Effective URL: https://www.datameer.com/blog/how-to-transform-your-data-in-snowflake-part-1/?utm_campaign=Datameer%20Announcement%20Flow...
Submission: On March 15 via api from SE — Scanned from DE
Form analysis 2 forms found in the DOM
GET https://www.datameer.com/
<form role="search" method="get" action="https://www.datameer.com/" data-hs-cf-bound="true">
<input type="search" id="search" placeholder="Search..." name="s" aria-label="Search" value="">
<input id="search_submit" value="Search" type="submit">
</form>POST https://forms.hsforms.com/submissions/v3/public/submit/formsnext/multipart/5578595/c9af33da-cc1a-4d62-9488-4d851beef63c
<form accept-charset="UTF-8" action="https://forms.hsforms.com/submissions/v3/public/submit/formsnext/multipart/5578595/c9af33da-cc1a-4d62-9488-4d851beef63c" enctype="multipart/form-data" id="hsForm_c9af33da-cc1a-4d62-9488-4d851beef63c" method="POST"
class="hs-form stacked hs-form-private hsForm_c9af33da-cc1a-4d62-9488-4d851beef63c hs-form-c9af33da-cc1a-4d62-9488-4d851beef63c hs-form-c9af33da-cc1a-4d62-9488-4d851beef63c_ae5b441e-8648-4f08-9af4-7ccc58cbb009"
data-form-id="c9af33da-cc1a-4d62-9488-4d851beef63c" data-portal-id="5578595" target="target_iframe_c9af33da-cc1a-4d62-9488-4d851beef63c" data-reactid=".hbspt-forms-0" data-hs-cf-bound="true">
<div class="hs_email hs-email hs-fieldtype-text field hs-form-field" data-reactid=".hbspt-forms-0.1:$0"><label id="label-email-c9af33da-cc1a-4d62-9488-4d851beef63c" class="" placeholder="Enter your Business Email"
for="email-c9af33da-cc1a-4d62-9488-4d851beef63c" data-reactid=".hbspt-forms-0.1:$0.0"><span data-reactid=".hbspt-forms-0.1:$0.0.0">Business Email</span><span class="hs-form-required" data-reactid=".hbspt-forms-0.1:$0.0.1">*</span></label>
<legend class="hs-field-desc" style="display:none;" data-reactid=".hbspt-forms-0.1:$0.1"></legend>
<div class="input" data-reactid=".hbspt-forms-0.1:$0.$email"><input id="email-c9af33da-cc1a-4d62-9488-4d851beef63c" class="hs-input" type="email" name="email" required="" placeholder="Email" value="" autocomplete="email"
data-reactid=".hbspt-forms-0.1:$0.$email.0" inputmode="email" style="padding-right: 40px;"></div>
</div>
<div class="hs_utm_medium hs-utm_medium hs-fieldtype-text field hs-form-field" style="display:none;" data-reactid=".hbspt-forms-0.1:$1"><label id="label-utm_medium-c9af33da-cc1a-4d62-9488-4d851beef63c" class="" placeholder="Enter your Channel"
for="utm_medium-c9af33da-cc1a-4d62-9488-4d851beef63c" data-reactid=".hbspt-forms-0.1:$1.0"><span data-reactid=".hbspt-forms-0.1:$1.0.0">Channel</span></label>
<legend class="hs-field-desc" style="display:none;" data-reactid=".hbspt-forms-0.1:$1.1"></legend>
<div class="input" data-reactid=".hbspt-forms-0.1:$1.$utm_medium"><input name="utm_medium" class="hs-input" type="hidden" value="email" data-reactid=".hbspt-forms-0.1:$1.$utm_medium.0"></div>
</div>
<div class="hs_utm_source hs-utm_source hs-fieldtype-text field hs-form-field" style="display:none;" data-reactid=".hbspt-forms-0.1:$2"><label id="label-utm_source-c9af33da-cc1a-4d62-9488-4d851beef63c" class=""
placeholder="Enter your Channel Detail" for="utm_source-c9af33da-cc1a-4d62-9488-4d851beef63c" data-reactid=".hbspt-forms-0.1:$2.0"><span data-reactid=".hbspt-forms-0.1:$2.0.0">Channel Detail</span></label>
<legend class="hs-field-desc" style="display:none;" data-reactid=".hbspt-forms-0.1:$2.1"></legend>
<div class="input" data-reactid=".hbspt-forms-0.1:$2.$utm_source"><input name="utm_source" class="hs-input" type="hidden" value="hs_automation" data-reactid=".hbspt-forms-0.1:$2.$utm_source.0"></div>
</div>
<div class="hs_utm_campaign hs-utm_campaign hs-fieldtype-text field hs-form-field" style="display:none;" data-reactid=".hbspt-forms-0.1:$3"><label id="label-utm_campaign-c9af33da-cc1a-4d62-9488-4d851beef63c" class=""
placeholder="Enter your Campaign UTM" for="utm_campaign-c9af33da-cc1a-4d62-9488-4d851beef63c" data-reactid=".hbspt-forms-0.1:$3.0"><span data-reactid=".hbspt-forms-0.1:$3.0.0">Campaign UTM</span></label>
<legend class="hs-field-desc" style="display:none;" data-reactid=".hbspt-forms-0.1:$3.1"></legend>
<div class="input" data-reactid=".hbspt-forms-0.1:$3.$utm_campaign"><input name="utm_campaign" class="hs-input" type="hidden" value="Datameer Announcement Flow Nurture" data-reactid=".hbspt-forms-0.1:$3.$utm_campaign.0"></div>
</div>
<div class="hs_utm_content hs-utm_content hs-fieldtype-text field hs-form-field" style="display:none;" data-reactid=".hbspt-forms-0.1:$4"><label id="label-utm_content-c9af33da-cc1a-4d62-9488-4d851beef63c" class=""
placeholder="Enter your Content UTM" for="utm_content-c9af33da-cc1a-4d62-9488-4d851beef63c" data-reactid=".hbspt-forms-0.1:$4.0"><span data-reactid=".hbspt-forms-0.1:$4.0.0">Content UTM</span></label>
<legend class="hs-field-desc" style="display:none;" data-reactid=".hbspt-forms-0.1:$4.1"></legend>
<div class="input" data-reactid=".hbspt-forms-0.1:$4.$utm_content"><input name="utm_content" class="hs-input" type="hidden" value="176120105" data-reactid=".hbspt-forms-0.1:$4.$utm_content.0"></div>
</div>
<div class="hs_utm_term hs-utm_term hs-fieldtype-text field hs-form-field" style="display:none;" data-reactid=".hbspt-forms-0.1:$5"><label id="label-utm_term-c9af33da-cc1a-4d62-9488-4d851beef63c" class="" placeholder="Enter your Term UTM"
for="utm_term-c9af33da-cc1a-4d62-9488-4d851beef63c" data-reactid=".hbspt-forms-0.1:$5.0"><span data-reactid=".hbspt-forms-0.1:$5.0.0">Term UTM</span></label>
<legend class="hs-field-desc" style="display:none;" data-reactid=".hbspt-forms-0.1:$5.1"></legend>
<div class="input" data-reactid=".hbspt-forms-0.1:$5.$utm_term"><input name="utm_term" class="hs-input" type="hidden" value="" data-reactid=".hbspt-forms-0.1:$5.$utm_term.0"></div>
</div>
<div class="hs_product_interest hs-product_interest hs-fieldtype-checkbox field hs-form-field" style="display:none;" data-reactid=".hbspt-forms-0.1:$6"><label id="label-product_interest-c9af33da-cc1a-4d62-9488-4d851beef63c" class=""
placeholder="Enter your Product Interest" for="product_interest-c9af33da-cc1a-4d62-9488-4d851beef63c" data-reactid=".hbspt-forms-0.1:$6.0"><span data-reactid=".hbspt-forms-0.1:$6.0.0">Product Interest</span></label>
<legend class="hs-field-desc" style="display:none;" data-reactid=".hbspt-forms-0.1:$6.1"></legend>
<div class="input" data-reactid=".hbspt-forms-0.1:$6.$product_interest"><input name="product_interest" class="hs-input" type="hidden" value="T++" data-reactid=".hbspt-forms-0.1:$6.$product_interest.0"></div>
</div><noscript data-reactid=".hbspt-forms-0.2"></noscript>
<div class="hs_recaptcha hs-recaptcha field hs-form-field" data-reactid=".hbspt-forms-0.3">
<div class="input" data-reactid=".hbspt-forms-0.3.0">
<div class="grecaptcha-badge" data-style="inline" style="width: 256px; height: 60px; box-shadow: gray 0px 0px 5px;">
<div class="grecaptcha-logo"><iframe title="reCAPTCHA"
src="https://www.google.com/recaptcha/enterprise/anchor?ar=1&k=6Ld_ad8ZAAAAAAqr0ePo1dUfAi0m4KPkCMQYwPPm&co=aHR0cHM6Ly93d3cuZGF0YW1lZXIuY29tOjQ0Mw..&hl=de&v=85AXn53af-oJBEtL2o2WpAjZ&size=invisible&badge=inline&cb=w4fyd386lfxv"
width="256" height="60" role="presentation" name="a-j6d2su8u4baj" frameborder="0" scrolling="no" sandbox="allow-forms allow-popups allow-same-origin allow-scripts allow-top-navigation allow-modals allow-popups-to-escape-sandbox"
data-lf-form-tracking-inspected-ywvko4x2ple8z6bj="true" data-lf-yt-playback-inspected-ywvko4x2ple8z6bj="true"></iframe></div>
<div class="grecaptcha-error"></div><textarea id="g-recaptcha-response" name="g-recaptcha-response" class="g-recaptcha-response"
style="width: 250px; height: 40px; border: 1px solid rgb(193, 193, 193); margin: 10px 25px; padding: 0px; resize: none; display: none;"></textarea>
</div><iframe style="display: none;" data-lf-form-tracking-inspected-ywvko4x2ple8z6bj="true" data-lf-yt-playback-inspected-ywvko4x2ple8z6bj="true"></iframe>
</div><input type="hidden" name="g-recaptcha-response" id="hs-recaptcha-response" value="" data-reactid=".hbspt-forms-0.3.1">
</div>
<div class="hs_submit hs-submit" data-reactid=".hbspt-forms-0.5">
<div class="hs-field-desc" style="display:none;" data-reactid=".hbspt-forms-0.5.0"></div>
<div class="actions" data-reactid=".hbspt-forms-0.5.1"><input type="submit" value="Sign Up" class="hs-button primary large" data-reactid=".hbspt-forms-0.5.1.0"></div>
</div><noscript data-reactid=".hbspt-forms-0.6"></noscript><input name="hs_context" type="hidden"
value="{"rumScriptExecuteTime":1164.4000000059605,"rumServiceResponseTime":1472.300000011921,"rumFormRenderTime":2.0999999940395355,"rumTotalRenderTime":1475.9000000059605,"rumTotalRequestTime":255.6000000089407,"lang":"en","renderRawHtml":"true","embedAtTimestamp":"1647338978480","formDefinitionUpdatedAt":"1638202684550","pageUrl":"https://www.datameer.com/blog/how-to-transform-your-data-in-snowflake-part-1/?utm_campaign=Datameer%20Announcement%20Flow%20Nurture&utm_medium=email&_hsmi=176120105&_hsenc=p2ANqtz-_U3c9a8e8SAve9h3YZL9GUH0y1G1gKxyRyd2nybpgRFlcrLHlOQUwVI1DS_rv9zmKwpF-qBesrCczJdtTfBNBrVf49-Ix1Y7lxVT4J4fEtbZXY0Io&utm_content=176120105&utm_source=hs_automation","pageTitle":"How to Transform Your Data in Snowflake: Part 1 - Datameer","source":"FormsNext-static-5.458","sourceName":"FormsNext","sourceVersion":"5.458","sourceVersionMajor":"5","sourceVersionMinor":"458","timestamp":1647338978480,"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36","originalEmbedContext":{"portalId":"5578595","formId":"c9af33da-cc1a-4d62-9488-4d851beef63c","target":"#hbspt-form-1647338978389-4651237233"},"urlParams":{"utm_campaign":"Datameer Announcement Flow Nurture","utm_medium":"email","_hsmi":"176120105","_hsenc":"p2ANqtz-_U3c9a8e8SAve9h3YZL9GUH0y1G1gKxyRyd2nybpgRFlcrLHlOQUwVI1DS_rv9zmKwpF-qBesrCczJdtTfBNBrVf49-Ix1Y7lxVT4J4fEtbZXY0Io","utm_content":"176120105","utm_source":"hs_automation"},"renderedFieldsIds":["email"],"formTarget":"#hbspt-form-1647338978389-4651237233","correlationId":"73318d75-ea2c-42f2-8150-216dac5adf2c","hutk":"45c03435c39b4e0fb1087dfdff460cef","captchaStatus":"LOADED"}"
data-reactid=".hbspt-forms-0.7"><iframe name="target_iframe_c9af33da-cc1a-4d62-9488-4d851beef63c" style="display:none;" data-reactid=".hbspt-forms-0.8" data-lf-form-tracking-inspected-ywvko4x2ple8z6bj="true"
data-lf-yt-playback-inspected-ywvko4x2ple8z6bj="true"></iframe>
</form>Text Content
* Capabilities
* Customers
* Resources
* Docs
* Blog
* Search
* Request Demo
HOW TO TRANSFORM YOUR DATA IN SNOWFLAKE: PART 1
* John Morrell
* October 11, 2021
HOW TO TRANSFORM YOUR DATA IN SNOWFLAKE
The data modeling world is constantly evolving as new technologies are
introduced. First, it was data warehousing, then MPP data warehouses, followed
by Hadoop and data lakes. Now we are in the cloud data warehouse era.
The advent and popularity of cloud data warehouses, such as Snowflake, have
altered the way we think about data transformation and modeling. The new ELT,
or extract, load, and transform, process extracts, and loads raw source data
into Snowflake, which is then transformed into final form for your analytics.
This allowed organizations to take advantage of the inexpensive and scalable
compute and storage services of Snowflake and created agility by separating data
loading and data transformation processes and workload with data engineers
performing the former and data analysts the latter. Organizations could create
any number of subject-specific analytical data models that are optimized for
their own needs and can use modern organization techniques such as Snowflake
virtual data warehouses.
The ELT model also allows organizations to share the data modeling and
transformation workload. A new role has emerged over the past few years – the
data engineer – whose primary responsibility is creating and implementing data
pipelines and data models, part of which is data transformation. In Snowflake,
the EL and the T are separated with the data engineer responsible for the EL and
shared responsibilities between the data engineer and analytics teams for the T.
STEP 1: GETTING THE DATA INTO THE CDW
In the modern data landscape, data comes from a wide variety of sources. The
faster-growing sources of data for analytics are from SaaS applications and
cloud services. These sources have extremely complex data structures and APIs.
The modern EL or “data loader” tools, such as Fivetran, Stitch, and others,
focus on eliminating this complexity and replicating “objects” from these
sources into Snowflake.
Thus, the initial data model your team will work from is a set of tables in the
cloud data warehouse that look like objects from your data sources, are grouped
similarly, and contain all the same fields. But because the data is still in a
form similar to the SaaS application or cloud service objects, it could be very
cryptic and not understandable by a data analyst.
There is one very important data transformation step that needs to be applied to
the data on its way into Snowflake. If any of the data is private or sensitive,
it needs to be anonymized or masked. This is critical to maintain the privacy
of the data and ensure regulatory compliance.
After the raw data is loaded, the data engineering team may apply a first pass
at data cleansing. In this first step, data engineers could apply general,
standardized cleansing to (a) find and correct missing or invalid values, (b)
transforming incorrectly formatted fields, and (c) extracting individual fields
from complex, multi-faceted columns.
STEP 2: CANONICAL DATA MODELING
Once the data is in the CDW and has gone through the first pass of data
transformation, the data engineering team can transform the raw data into
canonical data models that represent specific subjects. Examples of these would
be data models representing customers, contacts, leads, opportunities,
activities, and more.
The primary rationale for canonical data models is to create shared, reusable
components for multiple use cases. Along with this comes added benefits:
* The creation of a single version of the truth for each subject and field
within that subject,
* Providing shared and standardized definitions and documentation about the
data for each subject,
* Transparency into the data models and how they are built to build trust in
the analytics community.
The data engineering team will gather requirements from the various business and
analytics teams to build the canonical data models. These data models will
typically be supersets of the requirements to maximize reuse and consumption.
The data models will also continuously evolve as new requirements or data
sources come into play.
Since the raw data that came from the data sources is often normalized (in some
cases mildly normalized and others highly normalized), the canonical data models
will typically blend (JOIN, UNION, etc.) data from multiple objects to create a
rich and complete set of fields to represent the subject. In addition, the
canonical data models may also have some data enrichment to calculate new fields
for standardized use in different use cases.
STEP 3: USE CASE DATA MODELING
The final step in data modeling is to create datasets that are specific to the
analytics use case. For modern data modeling in Snowflake, this task is
typically done by the data analyst. Why? It boils down to roles and skills:
* Data engineers tend to know more about the data itself – where it resides,
how it is structured and formatted, and how to get it – and less about how
the business uses the data. This makes their ideal role in getting the data
into Snowflake and first-pass data modeling.
* Data analysts know less about the raw data but have a complete understanding
of how the business would use the data and how it would be incorporated into
analytics. This makes their ideal role to be use case data modeling and
transformation.
Data analysts may have varying technical skills but would prefer to spend more
time on what they are good at – analysis – and less on coding data
transformation. This is where a low-code or no-code data transformation UI
becomes important, eliminating the need for analysts to write complex SQL code
and Python-like scripts.
Use case data modeling and transformation will typically involve:
* Data cleansing that is specific for the use case, such as identifying and
fixing outliers or deduping records,
* Data shaping and reduction such as sorting and organizing the data,
eliminating unneeded fields, or narrowing the scope of the data to time
periods or specific dimensions, and
* Data enrichment to add new calculated fields specific to the analysis or
uploading local files specific to the use case, such as external or
department-specific data.
The optimal, final form of the data model will be a single flattened data
structure – a very large, wide table. This, along with materialization,
eliminates the need for expensive JOINs to be performed each time a query is
performed for the analysis.
SNOWFLAKE IMPLEMENTATION
In part 2 of this series, we will explain how to optimize your data
transformation in Snowflake.
WRAP UP
Modern data stacks using cloud data warehouses and ELT processes have created
the need for modernized data modeling within the data stack. A highly modular
approach to data modeling and transformation is required, as is a highly
collaborative process between data engineering and analytics teams where each
can best use their skills and knowledge.
Datameer provides data transformation solutions that enable data engineers and
analysts to transform and model data directly in Snowflake to solve complex
analytical projects. Are you interested in learning more about transforming
data in Snowflake? Read our complete how-to guides, what is data modeling and
how do you do it, how to transform your data for analytics, and how to optimize
your data transformation for Snowflake.
NO-CODE DATA TRANSFORMATION
Model data directly in your Snowflake instance with easy-to-use functions and
visual workflows
Learn More
TOPICS
CONTENT TYPE
Best Practices Customer Stories Data Essentials FAQ Press Product News Product
Update Use Cases
INDUSTRY
Financial Services Healthcare & Life Sciences Media & Telecom Retail &
E-Commerce
TYPE
Customer Success Analytics Finance Analytics HR Analytics Marketing Analytics
Product Analytics Sales Analytics Supply Chain Analytics
SOLUTION
BI and Analytics Cloud Migration Data Collaboration Data Engineering Data
Governance Data Integration Data Modeling Data Preparation Data Privacy &
Security Data Transformation DataOps ETL/ELT
TECHNOLOGY
Alteryx Fivetran Looker Qlik Snowflake SQL Tableau
FOLLOW US ON
* Facebook
* Twitter
* Linkedin
TRANSFORM & MODEL DATA IN SNOWFLAKE WITH NO CODE
Learn More
TRANSFORM & MODEL DATA IN SNOWFLAKE WITHOUT WRITING A SINGLE LINE OF CODE
Learn More
MORE RESOURCES WE THINK YOU MIGHT LIKE
THE FUTURE OF DATA ENGINEERING IS NO-CODE
Talent shortages, a disintegrated modern data stack, and a continued
programmatic approach to dat...
* John Morrell
* February 28, 2022
LARGE-SCALE PIVOT TABLES IN SNOWFLAKE WITH DATA...
Pivot tables are an everyday operation all Excel users perform. Business
analysts using Excel wi...
* John Morrell
* February 28, 2022
SNOWPARK FOR THE SCALA AND PYTHON DEVELOPERS; D...
In June of 2021, Snowflake introduced Snowpark, a new dataframe style developer
experience for Sn...
* John Morrell
* February 14, 2022
SOLUTIONS
* Energy and Utility
* Financial Services
* Healthcare
* Retail
* Telecom
* Travel & Hospitality
PRODUCT
* Documentation
* Request Demo
* Support
COMPANY
* About Us
* Contact Us
* Careers
* Legal
* Press
* Write for Datameer
RESOURCES
* Blog
* Customer Stories
* Events
* Resources Library
LEARN MORE
* Data Transformation
* Data Catalog
* Data Governance
* Data Mining
* Data Modeling
* DataOps
* Data Preparation
* Data Profiling
* Collaborative Analytics
* Analytics Engineering
SIGN UP FOR OUR NEWSLETTER
Business Email*
Channel
Channel Detail
Campaign UTM
Content UTM
Term UTM
Product Interest
FOLLOW US ON
* Facebook
* Twitter
* Linkedin
Copyright ⓒ 2022 Datameer, Inc. All rights reserved.
* Privacy Policy
* Terms of Use
PRIVACY PREFERENCE CENTER
When you visit any website, it may store or retrieve information on your
browser, mostly in the form of cookies. This information might be about you,
your preferences or your device and is mostly used to make the site work as you
expect it to. The information does not usually directly identify you, but it can
give you a more personalized web experience. Because we respect your right to
privacy, you can choose not to allow some types of cookies. Click on the
different category headings to find out more and change our default settings.
However, blocking some types of cookies may impact your experience of the site
and the services we are able to offer.
Allow All
MANAGE CONSENT PREFERENCES
STRICTLY NECESSARY COOKIES
Always Active
These cookies are necessary for the website to function and cannot be switched
off in our systems. They are usually only set in response to actions made by you
which amount to a request for services, such as setting your privacy
preferences, logging in or filling in forms. You can set your browser to block
or alert you about these cookies, but some parts of the site will not then work.
These cookies do not store any personally identifiable information.
Cookies Details
TARGETING COOKIES
Targeting Cookies
These cookies may be set through our site by our advertising partners. They may
be used by those companies to build a profile of your interests and show you
relevant adverts on other sites. They do not store directly personal
information, but are based on uniquely identifying your browser and internet
device. If you do not allow these cookies, you will experience less targeted
advertising.
Cookies Details
PERFORMANCE COOKIES
Performance Cookies
These cookies allow us to count visits and traffic sources so we can measure and
improve the performance of our site. They help us to know which pages are the
most and least popular and see how visitors move around the site. All
information these cookies collect is aggregated and therefore anonymous. If you
do not allow these cookies we will not know when you have visited our site, and
will not be able to monitor its performance.
Cookies Details
FUNCTIONAL COOKIES
Functional Cookies
These cookies enable the website to provide enhanced functionality and
personalisation. They may be set by us or by third party providers whose
services we have added to our pages. If you do not allow these cookies then some
or all of these services may not function properly.
Cookies Details
BACK BUTTON BACK
Vendor Search Search Icon
Filter Icon
Clear
checkbox label label
Apply Cancel
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label
* 33ACROSS
HOST DESCRIPTION
VIEW COOKIES
* Name
cookie name
Confirm My Choices
We use cookies to enhance user experience and to analyze performance and traffic
on our website. By clicking “Accept All Cookies,” you agree to the storing of
cookies on your device to enhance site navigation, analyze site usage, and
assist in our marketing efforts. To learn more about how we use cookies, please
see our Privacy Policy.
Accept All Cookies
Cookies Settings