Protect your LookML: Continuous Integration for reliable data

Looker Continuous Integration (CI) Explained

Key Concepts:

• Continuous Integration (CI): A development practice where code changes are frequently merged into a central repository, followed by automated builds and tests.
• LookML: Looker's modeling language used to define dimensions, measures, and relationships in data.
• Looker CI: An automated tool within Looker that validates LookML code and content before deployment to production.
• Suite: A collection of validators configured within Looker CI to check for different types of errors.
• Validators: Individual checks within a Looker CI suite, including LookML validator, content validator, SQL validator, and assert validator.
• Pull Request: A request to merge code changes from a branch into the main repository.

1. Introduction to Looker CI

• Looker CI is presented as an automated safety net to prevent breaking existing dashboards when deploying LookML updates.
• It helps catch issues arising from changes in upstream database tables that could invalidate LookML.
• The primary goal is to provide confidence in deploying code and accelerating data insights for the team.

2. What is Continuous Integration?

• Definition: Continuous integration involves developers frequently merging their code changes into a central repository.
• Automated Process: After each submission, an automated process builds and tests the code.
• Benefits:
  • Enables multiple developers to integrate code for the same LookML project simultaneously.
  • Prevents conflicts and errors in the production environment.
  • Ensures clean, validated data for business users to analyze.

3. How Looker CI Automates Tasks

• Looker CI helps automate tasks to find mistakes in LookML, ensuring code reliability and maintainability.
• It acts as an extra set of eyes, reviewing code before it goes live.

4. Setting Up and Configuring Looker CI

• The setup involves creating a "suite" with four configurable validators.
• Understanding these validators is crucial for improving the workflow.
• Users can configure validators individually or start with default configurations.

5. The Four Validators

• LookML Validator: Checks for syntax errors in the LookML code.
• Content Validator: Identifies dashboards and looks that will have errors upon deployment.
• SQL Validator: Looks for inconsistencies between LookML dimensions and database table metadata.
• Assert Validator: Runs tests written in LookML to catch data anomalies upstream.

6. Using Looker CI in the Development Workflow

• Process:
  1. Make changes to LookML.
  2. Create a pull request.
  3. Looker CI automatically runs the suite of validations.
  4. If errors are found, they are flagged.
  5. Fix the issues and rerun CI.
  6. Repeat until all validators pass.
  7. Merge with confidence.
  8. Deploy changes to production.
• Benefits: Thoroughly tested and validated code before deployment.

7. Benefits of Enabling Looker CI

• Provides an extra layer of protection for content.
• Keeps users safe by preventing broken dashboards and inaccurate data.
• Enables building amazing things with confidence.

8. Conclusion

• The video encourages viewers to try out continuous integration by having an admin enable Looker CI.
• It invites feedback on how Looker CI has eased the development process.
• The main takeaway is that Looker CI is a valuable tool for ensuring the quality and reliability of LookML code and content, leading to faster data insights and a more confident development process.