Skip to main content

Firebolt setup

Some core functionality may be limited. If you're interested in contributing, check out the source code for the repository listed below.

  • Maintained by: Firebolt
  • Authors: Firebolt
  • GitHub repo: firebolt-db/dbt-firebolt
  • PyPI package: dbt-firebolt
  • Slack channel: #db-firebolt
  • Supported dbt Core version: v1.1.0 and newer
  • dbt Cloud support: Not Supported
  • Minimum data platform version: n/a

Installing dbt-firebolt

Use pip to install the adapter. Before 1.8, installing the adapter would automatically install dbt-core and any additional dependencies. Beginning in 1.8, installing an adapter does not automatically install dbt-core. This is because adapters and dbt Core versions have been decoupled from each other so we no longer want to overwrite existing dbt-core installations. Use the following command for installation:

Configuring dbt-firebolt

For Firebolt-specific configuration, please refer to Firebolt configs.

For other information including Firebolt feature support, see the GitHub README and the changelog.

Connecting to Firebolt​

To connect to Firebolt from dbt, you'll need to add a profile to your profiles.yml file. A Firebolt profile conforms to the following syntax:

profiles.yml
<profile-name>:
  target: <target-name>
  outputs:
    <target-name>:
      type: firebolt
      client_id: "<id>"
      client_secret: "<secret>"
      database: "<database-name>"
      engine_name: "<engine-name>"
      account_name: "<account-name>"
      schema: <tablename-prefix>
      threads: 1
      #optional fields
      host: "<hostname>"

Description of Firebolt Profile Fields​

To specify values as environment variables, use the format {{ env_var('<variable_name>' }}. For example, {{ env_var('DATABASE_NAME' }}.

FieldDescription
typeThis must be included either in profiles.yml or in the dbt_project.yml file. Must be set to firebolt.
client_idRequired. Your service account id.
client_secretRequired. The secret associated with the specified client_id.
databaseRequired. The name of the Firebolt database to connect to.
engine_nameRequired in version 0.21.10 and later. Optional in earlier versions. The name (not the URL) of the Firebolt engine to use in the specified database. This must be a general purpose read-write engine and the engine must be running. If omitted in earlier versions, the default engine for the specified database is used.
account_nameRequired. Specifies the account name under which the specified database exists.
schemaRecommended. A string to add as a prefix to the names of generated tables when using the custom schemas workaround.
threadsRequired. Set to higher number to improve performance.
hostOptional. The host name of the connection. For all customers it is api.app.firebolt.io, which will be used if omitted.

Troubleshooting Connections​

If you encounter issues connecting to Firebolt from dbt, make sure the following criteria are met:

  • You must have adequate permissions to access the engine and the database.
  • Your service account must be attached to a user.
  • The engine must be running.

Supporting Concurrent Development​

In dbt, database schemas are used to compartmentalize developer environments so that concurrent development does not cause table name collisions. Firebolt, however, does not currently support database schemas (it is on the roadmap). To work around this, we recommend that you add the following macro to your project. This macro will take the schema field of your profiles.yml file and use it as a table name prefix.

-- macros/generate_alias_name.sql
{% macro generate_alias_name(custom_alias_name=none, node=none) -%}
    {%- if custom_alias_name is none -%}
        {{ node.schema }}__{{ node.name }}
    {%- else -%}
        {{ node.schema }}__{{ custom_alias_name | trim }}
    {%- endif -%}
{%- endmacro %}

For an example of how this works, let’s say Shahar and Eric are both working on the same project.

In her .dbt/profiles.yml, Sharar sets schema=sh, whereas Eric sets schema=er in his. When each runs the customers model, the models will land in the database as tables named sh_customers and er_customers, respectively. When running dbt in production, you would use yet another profiles.yml with a string of your choice.

0
Edit this page