# Iceberg

**Note:** We currently only support Iceberg for our AWS Customers 

[#instructions](#instructions "mention")

[#post-install-setup](#post-install-setup "mention")

## Instructions

{% tabs %}
{% tab title="Installing" %}
We recommend that you follow the Installation instructions from the UI. This can be found on app.lariatdata.com by clicking on the Integrations tab and going to "Add new integration" as outlined in the [Installation & Configuration ](/fundamentals/installation-and-configuration.md)page.  

To describe what the install entails, we outline how you could do this outside of the UI. If you are interested in the code that powers the install, please take a look here[ ](https://github.com/lariat-data/install-aws-iceberg-agent/)[https://github.com/lariat-data/install-aws-iceberg-agent](https://github.com/lariat-data/install-aws-iceberg-agent/)

#### What do you need for the install? 

* docker 
* Your Cloud Account ID and region
* Your own Cloud account keys 

Note: If you are running the install outside of the UI, you will need your Lariat API Key and Lariat Application Key and Lariat generated Cloud access keys (e.g. AWS key and secret key)

#### Installation command 

If using the UI, copy the installation command and fill in the unpopulated fields. 

Here is what the command looks like: 

{% code overflow="wrap" %}

```bash
docker run -it --pull=always --mount type=bind,source=/local/path/to/config/iceberg.yaml,target=/workspace/iceberg_agent.yaml,readonly -e AWS_REGION={YOUR_AWS_REGION} -e AWS_ACCOUNT_ID={YOUR_AWS_ACCOUNT_ID} -e AWS_ACCESS_KEY_ID=$(aws configure get aws_access_key_id) -e AWS_SECRET_ACCESS_KEY=$(aws configure get aws_secret_access_key) -e AWS_SESSION_TOKEN=$(aws configure get aws_session_token) -e AWS_ACCOUNT_ID={YOUR_AWS_ACCOUNT_ID} -e LARIAT_TMP_AWS_ACCESS_KEY_ID={PREFILLED_BY_UI} -e LARIAT_TMP_AWS_SECRET_ACCESS_KEY={PREFILLED_BY_UI} -e LARIAT_API_KEY={PREFILLED_BY_UI} -e LARIAT_APPLICATION_KEY={PREFILLED_BY_UI} lariatdata/install-aws-iceberg-agent:latest install
```

{% endcode %}

If you do not have access to the UI and need LARIAT\_TMP\_AWS\_ACCESS\_KEY\_ID and LARIAT\_TMP\_AWS\_SECRET\_ACCESS\_KEY you will have to reach out to <support@lariatdata.com>.
{% endtab %}

{% tab title="Configuring" %}
The config defines catalogs, databases and tables that will be tracked by the Iceberg agent. The agent installer sets up the correct permissions to access Glue Catalogs and ensures that the agent is running and collecting the relevant health metrics from the Iceberg agent.&#x20;

```yaml
catalog:
  glue:
    databases:
      DATABASE_NAME:
        - TABLE_NAME:
            uri: "s3://location_of_data/prefix/to/location/"
        - TABLE_NAME_2:
            uri: "s3://location_of_data/prefix/to/location/"
  postgres:
    user: $POSTGRES_USER
    password: $POSTGRES_PASSWORD
    host: $POSTGRES_HOST_NAME
    port: $POSTGRES_PORT
    databases:
      DATABASE_NAME:
        - TABLE_NAME:
            uri: "s3://location_of_data/prefix/to/location/"
source_id: unique_source_id
```

The catalogs available are `glue` , `postgres` , `rest` , `hive` and others defined in <https://py.iceberg.apache.org/configuration/#glue-catalog>
{% endtab %}
{% endtabs %}

## Post-Install Setup

In order to configure tracked tables outlined in the configuration, go to the datasets section in the homepage and click on the ellipses next to the newly tracked Iceberg Datasets. Select "Configure" and move through the menu to rapidly set up indicators. You can also click on "Indicators" from the sidebar and select "Create an Indicator" to create fully customized indicators. That is covered in [Working with Datasets and Indicators](/fundamentals/working-with-datasets-and-indicators.md) - This page summarizes the details of the Rapid Set Up path.  The rapid path to setting up indicators should take no longer than 5 minutes. If you need to batch set up indicators for 100s of datasets, we have a beta feature that does. Just reach out to <support@lariatdata.com> and we will activate it for you.&#x20;

<figure><img src="/files/AAdayeVIIBItHVgjJxVk" alt=""><figcaption></figcaption></figure>

Select a timestamp field that either has an ISO formatted date or is a unix timestamp. You can also use duckdb sql to define a computation that creates a unix timestamp from existing fields. We encourage you to use partition fields as the timestamp field.&#x20;

You can also select dimensions, these are columns that you can later break metrics out by as seen below. You can also use these dimensions to filter health metrics for a more granular view.&#x20;

<figure><img src="/files/E9jeZFl6Z6YutR0glucr" alt=""><figcaption></figcaption></figure>

You can then select the frequency with which Metrics should be collected.&#x20;

<figure><img src="/files/oSFYtbusMMpoLT5ovku7" alt=""><figcaption></figcaption></figure>

Finally, you can rapidly select the columns and type of health metrics prior to complete metric definition. You will start seeing indicators populate based on the backfill schedule or general schedule you select. Note: Only select summary stats for numeric fields.&#x20;

<figure><img src="/files/PDtgImlahgBDmnUWJH5f" alt=""><figcaption></figcaption></figure>

Reach out to <support@lariatdata.com> if you have any further questions!&#x20;


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.lariatdata.com/integrations-data-storage/iceberg.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.