❄️Snowflake

Instructions for installation and configuration of Snowflake agent

Snowflake is a cloud-native data warehouse and general-purpose data platform.

Use this integration to build Indicators and Virtual datasets on top of your databases defined in Snowflake

Pre-requisites:

git
terraform-cli (> v1.2) : detailed installation instructions here
You Lariat API Key and Application Key. Contact someone on the Lariat Team to get these.

Download the Snowflake Terraform Installer

git clone [email protected]:lariat-data/terraform-lariat-snowflake-agent.git

Run the installation

LARIAT_API_KEY=<your_api_key> LARIAT_APPLICATION_KEY=<your_app_key> terraform apply

During installation you create a yaml config. With this config you specify the databases, schemas and tables to track and a unique source id to identify metrics that come from this source.

The installer uses this to scope permissions to only the subset selected. The installer puts the file onto your object storage and you can update this file directly to add further tables and databases and run the update command with the installer. The location is in a bucket that starts with lariat-snowflake-default-config and has a timestamp suffix.

You may use the wildcard * to match any table names with a chosen pattern.

The * wildcard will also accommodate any future tables that follow the same naming pattern, so it is convenient for use in systems where tables are often rolled over, or created automatically with a common prefix.

The YAML configuration for Lariat Snowflake Agents require the following structure:

databases:
  my_snowflake_database:
    my_schema:
      - my_table
      - my_other_table
    my_other_schema:
      - a_third_table
  my_second_database:
    my_schema:
      - user_*
      - transaction_*

source_id: a_unique_source_id  # e.g. lariat-aws-snowflake-us-east-1

Note: The source_id should be unique across your entire organization to make sure you don't conflate metrics from two different sources.

PreviousAWS Athena NextGCS Object Storage

Last updated 1 year ago

hashtagPre-requisites:

hashtagDownload the Snowflake Terraform Installer

hashtag

hashtagRun the installation

Pre-requisites:

Download the Snowflake Terraform Installer

Run the installation