www.databricks.com Open in urlscan Pro
2606:4700::6812:3b3  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Skip to main content
 * Why Databricks
    * * Discover
         * For Executives
         * For Startups
         * Lakehouse Architecture
         * DatabricksIQ
         * Mosaic Research
      
      * Customers
         * Featured Stories
         * See All Customers
      
      * Partners
         * Cloud Providers
           Databricks on AWS, Azure, and GCP
         * Technology Partners
           Connect your existing tools to your Lakehouse
         * Data Partners
           Access the ecosystem of data consumers
         * Built on Databricks
           Build, market and grow your business
         * Consulting & System Integrators
           Experts to build, deploy and migrate to Databricks
         * C&SI Partner Program
           Build, deploy or migrate to the Lakehouse
         * Partner Solutions
           Find custom industry and migration solutions

 * Product
    * * Databricks Platform
         * Platform Overview
           A unified platform for data, analytics and AI
         * Sharing
           An open, secure, zero-copy sharing for all data
         * Governance
           Unified governance for all data, analytics and AI assets
         * Artificial Intelligence
           Build and deploy ML and GenAI applications
         * DBRX
           The new standard for high-quality and efficient LLMs
         * Data Management
           Data reliability, security and performance
         * Data Warehousing
           Serverless data warehouse for SQL analytics
         * Real-Time Analytics
           Real-time analytics, AI and applications made simple
         * Data Engineering
           ETL and orchestration for batch and streaming data
         * Data Science
           Collaborative data science at scale
      
      * Integrations and Data
         * Marketplace
           Open marketplace for data, analytics and AI
         * IDE Integrations
           Build on the Lakehouse in your favorite IDE
         * Partner Connect
           Discover and integrate with the Databricks ecosystem
      
      * Pricing
         * Databricks Pricing
           Explore product pricing, DBUs and more
         * Cost Calculator
           Estimate your compute costs on any cloud
      
      * Open Source
         * Open Source Technologies
           Learn more about the innovations behind the platform

 * Solutions
    * * Databricks for Industries
         * Communications
         * Financial Services
         * Healthcare & Life Sciences
         * Manufacturing
         * Media and Entertainment
         * Public Sector
         * Retail
         * See All Industries
      
      * Cross Industry Solutions
         * Customer Data Platform
         * Cyber Security
      
      * Migration & Deployment
         * Data Migration
         * Professional Services
      
      * Solution Accelerators
         * Explore Accelerators
           Move faster toward outcomes that matter

 * Resources
    * * Training and Certification
         * Learning Overview
           Hub for training, certification, events and more
         * Training Overview
           Discover curriculum tailored to your needs
         * Databricks Academy
           Sign in to the Databricks learning platform
         * Certification
           Gain recognition and differentiation
         * University Alliance
           Want to teach Databricks? See how.
      
      * Events
         * Data + AI Summit
         * Data + AI World Tour
         * Data Intelligence Days
         * Event Calendar
      
      * Blog and Podcasts
         * Databricks Blog
           Explore news, product announcements, and more
         * Databricks Mosaic Research Blog
           Discover the latest in our Gen AI research
         * Data Brew Podcast
           Let’s talk data!
         * Champions of Data + AI Podcast
           Insights from data leaders powering innovation
      
      * Get Help
         * Customer Support
         * Documentation
         * Community
      
      * Dive Deep
         * Resource Center
         * Demo Center

 * About
    * * Company
         * Who We Are
         * Our Team
         * Databricks Ventures
         * Contact Us
      
      * Careers
         * Working at Databricks
         * Open Jobs
      
      * Press
         * Awards and Recognition
         * Newsroom
      
      * Security and Trust
         * Security and Trust


 * Login
 * Contact Us
 * Try Databricks




CATEGORIES

 * All blog posts
 * Company
   * Culture
   * Customers
   * Events
   * News
 * Platform
   * Announcements
   * Partners
   * Product
   * Solutions
   * Security and Trust
 * Generative AI
   * Mosaic Research
 * Engineering
   * Data Science and ML
   * Open Source
   * Solutions Accelerators
   * Data Engineering
   * Tutorials
   * Data Streaming
   * Data Warehousing
 * Data Strategy
   * Best Practices
   * Data Leader
   * Insights
 * Industries
   * Energy
   * Financial Services
   * Healthcare and Life Sciences
   * Media and Entertainment
   * Retail and Consumer Goods
   * Manufacturing
   * Public Sector


AI REGULATION IS ROLLING OUT…AND THE DATA INTELLIGENCE PLATFORM IS HERE TO HELP

by Scott Starbird and Jonathan Frankle

March 15, 2024 in Company Blog

Share this post

Policymakers around the world are paying increased attention to artificial
intelligence. The world’s most comprehensive AI regulation to date was just
passed by a sizable vote margin in the European Union (EU) Parliament, while in
the United States, the federal government has recently taken several notable
steps to place controls on the use of AI, and there also has been activity at
the state level. Policymakers elsewhere are also paying close attention and are
working to put AI regulation in place. These emerging regulations will impact
the development and use of both standalone AI models and the compound AI systems
that Databricks is increasingly seeing its customers utilize to build AI
applications.

Follow along our two-part “AI Regulation” series. Part 1 provides an overview of
the recent flurry of activity in AI policymaking in the U.S. and elsewhere,
highlighting the recurring regulatory themes globally. Part 2 will provide a
deep dive into how the Databricks Data Intelligence Platform can help customers
meet emerging obligations and discuss Databricks’ position on Responsible AI.


MAJOR RECENT AI REGULATORY DEVELOPMENTS IN THE U.S.

The Biden Administration is driving many recent regulatory developments in AI.
On October 30, 2023, the White House released its extensive Executive Order on
the Safe, Secure and Trustworthy Development and Use of AI. The Executive Order
provides guidelines on:

 * The use of AI within the federal government
 * How federal agencies can leverage existing regulations where they reasonably
   relate to AI (e.g., prevention of discrimination against protected groups,
   consumer safety disclosure requirements, antitrust rules, etc.)
 * How developers of highly capable “dual-use foundation models” (i.e., frontier
   models) can share results of their testing efforts, and lists a range of
   studies, reports and policy formulations to be undertaken by various
   agencies, with a notably important role to be played by the National
   Institute of Standards and Technology, within the Commerce Department (NIST).

In quick response to the Executive Order, the U.S. Office of Management and
Budget (OMB) followed two days later with a draft memo to agencies throughout
the U.S. government, addressing both their use of AI and the government’s
procurement of AI.


THE ROLE OF NIST & THE U.S. AI SAFETY INSTITUTE

One of NIST’s primary roles under the Executive Order will be to expand its AI
Risk Management Framework (NIST AI RMF) to apply to generative AI. The NIST AI
RMF will also be applied throughout the federal government under the Executive
Order and is increasingly being cited as a foundation for proposed AI regulation
by policymakers. The recently formed U.S. AI Safety Institute (USAISI),
announced by Vice President Harris at the U.K. AI Safety Summit, is also housed
within NIST. A new Consortium has been formed to support the USAISI with
research and expertise – with Databricks¹  participating as an initial member.
Although $10 million in funding for the USAISI was announced on March 7, 2024,
there remain concerns that the USAISI will require additional resources to
adequately fulfill its mission. 

Under this directive, the USAISI will create guidelines for mechanisms for
assessing AI risk and develop technical guidance that regulators will use on
issues such as establishing thresholds for categorizing powerful models as
“dual-use foundation models” under the Executive Order (models requiring
heightened scrutiny), authenticating content, watermarking AI-generated content,
identifying and mitigating algorithmic discrimination, ensuring transparency,
and enabling adoption of privacy-preserving AI.


ACTIONS BY OTHER FEDERAL AGENCIES

Numerous federal agencies have taken steps relating to AI under mandate from the
Biden Executive Order. The Commerce Department is now receiving reports from
developers of the most powerful AI systems regarding vital information,
especially AI safety test results, and it has issued draft rules applicable to
U.S. cloud infrastructure providers requiring reporting when foreign customers
train powerful models using their services. Nine agencies, including the
Departments of Defense, State, Treasury, Transportation and Health & Human
Services, have submitted risk assessments to the Department of Homeland Security
covering the use and safety of AI in critical infrastructure. The Federal Trade
Commission (FTC) is heightening its efforts around AI in enforcing existing
regulations. As part of this effort, the FTC convened an FTC Tech Summit on
January 25, 2024 focused on AI (including Databricks’ Chief Scientist-Neural
Networks, Jonathan Frankle, as a panelist). Pursuant to the Executive Order and
as part of its ongoing efforts to advise the White House on technology matters
including AI, the National Telecommunications and Information Administration
(NTIA) has issued a request for comments on dual-use foundation models with
widely available model weights.


WHAT’S HAPPENING IN CONGRESS?

The U.S. Congress has taken a few tentative steps to regulate AI thus far.
Between September and December 2023, the Senate conducted a series of “AI
Insight Forums” to help Senators learn about AI and prepare for potential
legislation. Two bipartisan bills were introduced near the end of 2023 to
regulate AI — one introduced by Senators Jerry Moran (R-KS) and Mark Warner
(D-VA) to establish guidelines on the use of AI within the federal government,
and one introduced by Senators John Thune (R-SD) and Amy Klobuchar (D-MN) to
define and regulate the commercial use of high-risk AI. Meanwhile, in January
2024, Senate Commerce Committee Chair Maria Cantwell (D-WA) indicated she would
soon introduce a series of bipartisan bills to address AI risks and spur
innovation in the industry.

In late February, the House of Representatives announced the formation of its
own AI Task Force, chaired by Reps. Jay Obernolte (R-CA-23) and Ted Lieu
(D-CA-36). The Task Force’s first major objective is to pass the CREATE AI Act,
which would make the National Science Foundation’s National AI Research Resource
(NAIRR) pilot a fully funded program (Databricks is contributing an instance of
the Databricks Data Intelligence Platform for the NAIRR pilot).




REGULATION AT THE STATE LEVEL

Individual states are also examining how to regulate AI, and in some cases, pass
and sign legislation into law. Over 91 AI-related bills were introduced in state
houses in 2023. California made headlines last year when Governor Gavin Newsom
issued an executive order focused on generative AI. The order tasked state
agencies with a series of reports and recommendations for future regulation on
topics like privacy and civil rights, cybersecurity, and workforce benefits.
Other states like Connecticut, Maryland, and Texas passed laws for further study
on AI, particularly its impact on state government.

State lawmakers are in a rare position to advance legislation quickly thanks to
a record number of state governments under single-party control, avoiding the
partisan gridlock experienced by their federal counterparts. Already in 2024,
lawmakers in 20 states have introduced 89 bills or resolutions pertaining to AI.
California’s unique position as a legislative testing ground and its
concentration of companies involved in AI make the state a bellwether for
legislation, and several potential AI bills are in various stages of
consideration in the California state legislature. Proposed comprehensive AI
legislation is also moving forward at a fairly rapid pace in Connecticut.


OUTSIDE THE UNITED STATES

The U.S. is not alone in pursuing a regulatory framework to govern AI. As we
think about the future of regulation in this space, it’s important to maintain a
global view and keep a pulse on the emerging regulatory frameworks other
governments and legal bodies are enacting.


EUROPEAN UNION

The EU is leading in efforts to enact comprehensive AI regulation, with the
far-reaching EU AI Act nearing formal enactment. The EU member states reached a
unanimous agreement on the text on February 2, 2024 and the Act was passed by
Parliament on March 13, 2024. Enforcement will commence in stages starting in
late 2024/early 2025. The EU AI Act categorizes AI applications based on their
risk levels, with a focus on potential harm to health, safety, and fundamental
rights. The Act imposes stricter regulations on AI applications deemed
high-risk, while outright banning those considered to pose unacceptable risks.
The Act seeks to appropriately divide responsibilities between developers and
deployers. Developers of foundation models are subject to a set of specific
obligations designed to ensure that these models are safe, secure, ethical, and
transparent. The Act provides a general exemption for open source AI, except
when deployed in a high risk use case, or as part of a foundation model posing
“systemic risk” (i.e., a frontier model).


UNITED KINGDOM

Although the U.K. thus far has not pushed forward with comprehensive AI
regulation, the early November 2023 U.K. AI Safety Summit in historic Bletchley
Park (with Databricks participating) was the most visible and broadly attended
global event so far to address AI risks, opportunities and potential regulation.
While the summit focused on the risks presented by frontier models, it also
highlighted the benefits of AI to society and the need to foster AI innovation. 

As part of the U.K. AI Summit, 28 countries (including China) plus the EU agreed
to the Bletchley Declaration calling for international collaboration in
addressing the risks and opportunities presented by AI. In conjunction with the
Summit, both the U.K. and the U.S. announced the formation of national AI Safety
Institutes, committing these bodies to closely collaborate with each other going
forward (the U.K. AI Safety Institute received initial funding of £100 million,
in contrast to the $10 million allocated thus far by the U.S. to its own AI
Safety Institute). There was also an agreement to conduct additional global AI
Safety Summits, with the next one being a “virtual mini summit” to be hosted by
South Korea in May 2024, followed by an in-person summit hosted by France in
November 2024.


ELSEWHERE

During the same week the U.K. was hosting its AI Safety Summit and the Biden
Administration issued its executive order on AI, leaders of the G7 announced a
set of International Guiding Principles on AI and a voluntary Code of Conduct
for AI developers. Meanwhile, AI regulations are being discussed and proposed at
an accelerating pace in numerous other countries around the world.


PRESSURE TO VOLUNTARILY PRE-COMMIT

Many parties, including the U.S. White House, G7 leaders, and numerous attendees
at the U.K. AI Safety Summit, have called for voluntary compliance with pending
AI regulations and emerging industry standards. Companies using AI will face
increasing pressure to take steps now to meet the general requirements of
regulation to come.

For example, the AI Pact is a program calling for parties to voluntarily commit
to the EU AI Act prior to it becoming enforceable. Similarly, the White House
has been encouraging companies to voluntarily commit to implementing safe and
secure AI practices, with the latest round of such commitments applying to
healthcare companies. The Code of Conduct for advanced AI systems created by the
OECD under the Hiroshima Process (and released by G7 leaders the week of the UK
AI Safety Summit) is voluntary but is strongly encouraged for developers of
powerful generative AI models.

The increasing pressure to make these voluntary commitments means that, for many
companies, various compliance obligations will be faced fairly soon. In
addition, many companies see voluntary compliance as a potential competitive
advantage.


WHAT DO ALL THESE EFFORTS HAVE IN COMMON?

The emerging AI regulations have varied, complex requirements, but carry
recurring themes. Obligations commonly arise in five key areas:

 1. Data and model security and privacy protection, required at all stages of
    the AI development and deployment cycle
 2. Pre-release risk assessment, planning and mitigation, focused on training
    data and implementing guardrails - addressing bias, inaccuracy, and other
    potential harm
 3. Documentation required at release, covering steps taken in development and
    regarding the nature of the AI model or system (capabilities, limitations,
    description of training data, risks, mitigation steps taken, etc.)
 4. Post-release monitoring and ongoing risk mitigation, focused on preventing
    inaccurate or other harmful generated output, avoiding discrimination
    against protected groups, and ensuring users realize they are dealing with
    AI
 5. Minimizing environmental impact from energy used to train and run large
    models


WHAT BUDDING REGULATION MEANS FOR DATABRICKS CUSTOMERS

Although many of the headlines generated by this whirlwind of governmental
activity have focused on high risk AI use cases and frontier AI risk, there is
likely near-term impact on the development and deployment of other AI as well,
particularly stemming from pressure to make voluntary pre-enactment commitments
to the EU AI Act, and from the Biden Executive Order due to its short time
horizons in various areas. As with most other proposed AI regulatory and
compliance frameworks, data governance, data security, and data quality are of
paramount importance.

Databricks is following the ongoing regulatory developments very carefully. We
support thoughtful AI regulation and Databricks is committed to helping its
customers meet AI regulatory requirements and responsible AI use objectives. We
believe the advancement of AI relies on building trust in intelligent
applications by ensuring everyone involved in developing and using AI follows
responsible and ethical practices, in alignment with the goals of AI regulation.
Meeting these objectives requires that every organization has full ownership and
control over its data and AI models and the availability of comprehensive
monitoring, privacy controls, and governance for all phases of AI development
and deployment. To achieve this mission, the Databricks Data Intelligence
Platform allows you to unify data, model training, management, monitoring, and
governance of the entire AI lifecycle. This unified approach empowers
organizations to meet responsible AI objectives to deliver data quality, provide
more secure applications, and help maintain compliance with regulatory
standards. 

In the upcoming second post of our series, we’ll do a deep dive into how
customers can utilize the tools featured on the Databricks Data Intelligence
Platform to help comply with AI regulations and meet their objectives regarding
the responsible use of AI. Of note, we’ll discuss Unity Catalog, an advanced
unified governance and security solution that can be very helpful in addressing
the safety, security, and governance concerns of AI regulation, and Lakehouse
Monitoring, a powerful monitoring tool useful across the full AI and data
spectrum.

And if you’re interested in how to mitigate the risks associated with AI, sign
up for the Databricks AI Security Framework here.

 

¹ Databricks is collaborating with NIST in the Artificial Intelligence Safety
Institute Consortium to develop science-based and empirically backed guidelines
and standards for AI measurement and policy, laying the foundation for AI safety
across the world. This will help ready the U.S. to address the capabilities of
the next generation of AI models or systems, from frontier models to new
applications and approaches, with appropriate risk management strategies. NIST
does not evaluate commercial products under this Consortium and does not endorse
any product or service used. Additional information on this Consortium can be
found at: Federal Register Notice - USAISI Consortium.

Try Databricks for free
Get Started


RELATED POSTS


FURTHERING OUR COMMITMENT TO RESPONSIBLE AI DEVELOPMENT THROUGH INDUSTRY AND
GOVERNMENT ORGANIZATIONS

February 8, 2024 by Scott Starbird in Company Blog
At Databricks, we've upheld principles of responsible development throughout our
long-standing history of building innovative data and AI products. We are
committed to...
See all Company Blog posts


 * * Why Databricks
   * Discover
     * For Executives
     * For Startups
     * Lakehouse Architecture
     * DatabricksIQ
     * Mosaic Research
   * Customers
     * Featured
     * See All
   * Partners
     * Cloud Providers
     * Technology Partners
     * Data Partners
     * Built on Databricks
     * Consulting & System Integrators
     * C&SI Partner Program
     * Partner Solutions
   * Why Databricks
   * Discover
     * For Executives
     * For Startups
     * Lakehouse Architecture
     * DatabricksIQ
     * Mosaic Research
   * Customers
     * Featured
     * See All
   * Partners
     * Cloud Providers
     * Technology Partners
     * Data Partners
     * Built on Databricks
     * Consulting & System Integrators
     * C&SI Partner Program
     * Partner Solutions
 * * Product
   * Databricks Platform
     * Platform Overview
     * Sharing
     * Governance
     * Artificial Intelligence
     * DBRX
     * Data Management
     * Data Warehousing
     * Real-Time Analytics
     * Data Engineering
     * Data Science
   * Pricing
     * Pricing Overview
     * Pricing Calculator
   * Open Source
   * Integrations and Data
     * Marketplace
     * IDE Integrations
     * Partner Connect
   * Product
   * Databricks Platform
     * Platform Overview
     * Sharing
     * Governance
     * Artificial Intelligence
     * DBRX
     * Data Management
     * Data Warehousing
     * Real-Time Analytics
     * Data Engineering
     * Data Science
   * Pricing
     * Pricing Overview
     * Pricing Calculator
   * Open Source
   * Integrations and Data
     * Marketplace
     * IDE Integrations
     * Partner Connect
 * * Solutions
   * Databricks For Industries
     * Communications
     * Financial Services
     * Healthcare and Life Sciences
     * Manufacturing
     * Media and Entertainment
     * Public Sector
     * Retail
     * View All
   * Cross Industry Solutions
     * Customer Data Platform
     * Cyber Security
   * Data Migration
   * Professional Services
   * Solution Accelerators
   * Solutions
   * Databricks For Industries
     * Communications
     * Financial Services
     * Healthcare and Life Sciences
     * Manufacturing
     * Media and Entertainment
     * Public Sector
     * Retail
     * View All
   * Cross Industry Solutions
     * Customer Data Platform
     * Cyber Security
   * Data Migration
   * Professional Services
   * Solution Accelerators
 * * Resources
   * Documentation
   * Customer Support
   * Community
   * Training and Certification
     * Learning Overview
     * Training Overview
     * Certification
     * University Alliance
     * Databricks Academy Login
   * Events
     * Data + AI Summit
     * Data + AI World Tour
     * Data Intelligence Days
     * Full Calendar
   * Blog and Podcasts
     * Databricks Blog
     * Databricks Mosaic Research Blog
     * Data Brew Podcast
     * Champions of Data & AI Podcast
   * Resources
   * Documentation
   * Customer Support
   * Community
   * Training and Certification
     * Learning Overview
     * Training Overview
     * Certification
     * University Alliance
     * Databricks Academy Login
   * Events
     * Data + AI Summit
     * Data + AI World Tour
     * Data Intelligence Days
     * Full Calendar
   * Blog and Podcasts
     * Databricks Blog
     * Databricks Mosaic Research Blog
     * Data Brew Podcast
     * Champions of Data & AI Podcast
 * * About
   * Company
     * Who We Are
     * Our Team
     * Databricks Ventures
   * Careers
     * Open Jobs
     * Working at Databricks
   * Press
     * Awards and Recognition
     * Newsroom
   * Security and Trust
   * About
   * Company
     * Who We Are
     * Our Team
     * Databricks Ventures
   * Careers
     * Open Jobs
     * Working at Databricks
   * Press
     * Awards and Recognition
     * Newsroom
   * Security and Trust

Databricks Inc.
160 Spear Street, 13th Floor
San Francisco, CA 94105
1-866-330-0121

 * 
 * 
 * 
 * 
 * 
 * 

See Careers
at Databricks

 * 
 * 
 * 
 * 
 * 
 * 

© Databricks 2024. All rights reserved. Apache, Apache Spark, Spark and the
Spark logo are trademarks of the Apache Software Foundation.

 * Privacy Notice
 * |Terms of Use
 * |Your Privacy Choices
 * |Your California Privacy Rights
 * 





WE CARE ABOUT YOUR PRIVACY

By clicking “Accept All Cookies”, you agree to the storing of cookies on your
device to enhance site navigation, analyze site usage, and assist in our
marketing efforts.
Reject all cookies Accept all cookies
Manage Preferences



PRIVACY PREFERENCE CENTER




 * YOUR PRIVACY


 * STRICTLY NECESSARY COOKIES


 * PERFORMANCE COOKIES


 * FUNCTIONAL COOKIES


 * TARGETING COOKIES

YOUR PRIVACY

When you visit any website, it may store or retrieve information on your
browser, mostly in the form of cookies. This information might be about you,
your preferences or your device and is mostly used to make the site work as you
expect it to. The information does not usually directly identify you, but it can
give you a more personalized web experience. Because we respect your right to
privacy, you can choose not to allow some types of cookies. Click on the
different category headings to find out more and change our default settings.
However, blocking some types of cookies may impact your experience of the site
and the services we are able to offer.
More information

STRICTLY NECESSARY COOKIES

Always Active

These cookies are necessary for the website to function and cannot be switched
off in our systems. They are usually only set in response to actions made by you
which amount to a request for services, such as setting your privacy
preferences, logging in or filling in forms. You can set your browser to block
or alert you about these cookies, but some parts of the site will not then work.

PERFORMANCE COOKIES

Performance Cookies


These cookies allow us to count visits and traffic sources so we can measure and
improve the performance of our site. They help us to know which pages are the
most and least popular and see how visitors move around the site.

FUNCTIONAL COOKIES

Functional Cookies


These cookies enable the website to provide enhanced functionality and
personalization. They may be set by us or by third party providers whose
services we have added to our pages. If you do not allow these cookies then some
or all of these services may not function properly.

TARGETING COOKIES

Targeting Cookies


These cookies may be set through our site by our advertising partners. They may
be used by those companies to build a profile of your interests and show you
relevant advertisements on other sites. If you do not allow these cookies, you
will experience less targeted advertising.

Back Button


COOKIE LIST

Filter Button
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label

Clear
checkbox label label
Apply Cancel
Confirm My Choices
Reject All Allow All