Make Looker Continous Integration work better for you

Key Concepts

• Looker Continuous Integration (CI): The automated process of testing LookML code changes before they are merged into production.
• Incremental Validation: A method to isolate errors unique to a specific development branch, ignoring pre-existing production errors.
• Explore Targeting: A configuration method to limit CI testing to specific LookML explores rather than the entire model.
• Explore Exclusion: A configuration method to omit specific, large, or problematic explores from the CI testing process.

Optimizing Looker CI Performance

The video addresses a common bottleneck in Looker development: the inefficiency of retesting an entire LookML model when only a single dimension or minor change has been implemented. By default, Looker’s CI is configured to be exhaustive, which can lead to significant delays in large projects containing hundreds of explores.

1. Incremental Validation

Incremental validation is presented as the primary solution for developers who need to isolate errors introduced by their specific branch.

• Functionality: It filters out pre-existing errors that already exist in the production environment.
• Benefit: It allows developers to focus exclusively on the errors they are responsible for, significantly reducing the time required for validation in complex, large-scale projects.

2. Targeting and Excluding Explores

While incremental validation handles error isolation, targeting specific explores manages the scope of the testing process.

• Targeting Explores: To restrict testing to a specific area of the model, developers must use the syntax: model_name/explore_name. This ensures that only the relevant portion of the model is validated.
• Excluding Explores: In scenarios where an explore is exceptionally large or is currently being modified by another team member, it can be excluded from the CI process. The same model_name/explore_name syntax is used within the "explores to exclude" configuration option.

3. Strategic Workflow

The video emphasizes a clear distinction between these two methodologies:

• Incremental Validation is for error isolation (distinguishing new errors from old ones).
• Targeting/Excluding is for scope management (defining which parts of the model are tested).

By combining these two approaches, developers can avoid the "perfectionist" default behavior of the CI system, which attempts to validate the entire model, and instead create a streamlined, efficient deployment pipeline.

Synthesis and Conclusion

The core takeaway is that Looker developers do not need to accept the default, time-consuming CI process. By implementing incremental validation, developers can ignore legacy errors and focus on their own code changes. Furthermore, by utilizing explore targeting and exclusion, teams can manage the computational load of their CI pipelines, ensuring that testing remains fast and focused even as projects grow in size and complexity. This dual-layered approach—isolating errors and narrowing scope—is essential for maintaining a clean, efficient, and productive development workflow in Looker.