Skip to main content

List of FAQs

1. Why would I need Metriql if I'm already using dbt Core?#

dbt Core lets you transform, test, and document your data, but its output is still the database tables. You can analyze these modeled tables in your BI tool, use them as the data source for your analytical applications, but you need to define metrics in each tool that you're using. Most of the BI tools provide ways to define metrics either using their GUI or programmatically. However, it can't be shared with the other tools that you're using. Metriql standardizes the metric definitions and serves as the metadata layer for all your company data.

A more comparable alternative would be LookML, Looker's data modeling language. Tristan, one of the dbt Labs founders, has a blog post explaining the differences between LookML and dbt Core. Typically, if you need to perform complex transformations, you can use dbt Core materializations. Otherwise, you can use metriql Aggregates for roll-up tables, define your metrics in dbt Core resource files without learning a brand new data modeling tool, such as LookML.

2. Why Kotlin when we already have Python?#

Most of the data people feel comfortable with Python, and some even hate the JVM world; we get it. That being said, Metriql has fundamental differences compared to dbt Core. It runs on schedule to transform your data, whereas metriql is usually used as an HTTP server. Also, metriql needs to understand the SQL dialect to be able to generate ad-hoc queries. Therefore we need battle-tested database drivers. We use JDBC, the technology behind many other BI tools, such as Looker, Tableau, and Metabase. While we agree that Java is verbose, Kotlin is worth learning.

3. How is this different from [x]?#

Minerva is Airbnb's internal metrics platform. We only know quite a little information as there is only one blog post about it, and it's not open-source yet. It's an end-to-end tool, from data ingestion to serving layer, and people interact with the metrics via an API. While it looks like a great idea, not all companies have the luxury to implement an architecture like Airbnb. metriql works on top of your data warehouse, and it makes use of Aggregates for rollup tables that don't move the data out of your data warehouse. It's just the metadata layer for your data, not an end-to-end tool.

Cube.js is a backend for embedded applications. It lets data people define the metrics in Javascript and build an API for their data. Cube has integrations with most of the front-end frameworks so that you can build custom user interfaces for your internal applications. They also have a two-level caching mechanism similar to Metriql's Aggregates, but they ingest the data into its own storage, built with Rust. Cube is more suitable if your data is already modeled and you're building a high concurrency analytical application as it doesn't provide data lineage and ingest the data through its own storage.

4. What's Rakam, and how it's related to metriql?#

Rakam, Inc. is the company that built metriql. We have been using dbt Core internally for the last 2 years, and we believe in their viewpoint. We tested metriql for our customers and found out that it can also be used outside of Rakam. We're also using tons of open-source technologies, and the founders also have open-source background. We decided to open-source metriql under a vendor-neutral organization using the same open-source license as dbt Core.

5. Do I need to use dbt Core for metriql?#

If your data is already modeled, you can use dbt's sources to create datasets in metriql. Here is an example project that doesn't need any transformation.

6. Does metriql run my dbt Core models?#

No, it does not. If you have table or incremental models, you need to run dbt yourself. We suggest using dbt Cloud from Fishtown Analytics, but you can also run dbt locally or use a CI system such as Github Actions or Gitlab CI if you don't want to use a Cloud IDE.

If you have other questions, join the conversation on Slack.#