Starting in Looker 7.12, you can deploy any Git commit SHA, tag, or branch to production with advanced deploy mode. This helps consolidate repositories in multi-environment developer workflows, where each environment points to a different version of a codebase. It also gives one or a few developers or administrators greater control over the changes that are deployed to production.
Motivation
By setting up a Git workflow across multiple Looker host instances, you can create a dedicated development environment for developers while preventing end users from accessing experimental code. All development and code review is done in a development or staging instance. Once changes pass your team's quality assurance review, they are deployed to a staging or production instance as desired.
This article describes how to set up a workflow among multiple instances and multiple Git repositories. If you have multiple instances and one Git repository, see the Git Workflow Using One Repository Across Multiple Instances — Development, Staging, and Production Help Center article.
Looker recommends using a single repository whenever possible.
Advanced Git Setup for Multiple Repositories
This setup requires multiple Git repositories and knowledge of Git commands.
You may want to set up multiple Looker hosts to achieve a development workflow that looks something like:
[1] Development > Production
[2] Development > Staging > Production
Each of these stages resides on a dedicated Looker host instance. This configuration uses a manual Git process to move code between two separate GitHub repos that can be automated or kicked off by a button.
Although this method shows a development workflow configuration [1] based on an existing single Looker project, looker_project
, it could easily be adapted for [2] or any number of intermediate hosts between Development and Production as needed.
Overview
The general process is as follows:
- Create two GitHub repos,
dev
andprod
. - Associate each repo with its respective Looker instances, Development and Production, according to the Setting Up and Testing a Git Connection documentation page.
- Create a local Git repo that is a clone of
dev
, perhaps on a server owned by the approver. Add theprod
GitHub repo as a second remote:git remote add production <<ssh_url>>
- Create a webhook trigger in the production GitHub repo to notify
prod
Looker to sync with GitHub as new code is pushed. In essence, set up a webhook on apush
event to notify the Production Looker instance. - Implement some kind of quality control around the second item in the migration process shown above, based on what works best for your team.
Step-by-Step Guide
To designate the existing dev
repo and set up a prod
repo on a second host, follow these steps:
- In the Github UI, go to the existing project, select Settings, and ensure that
master
is set as the default branch. If a different branch is set as default, change the default tomaster
. - Create a local folder and clone the
looker_project
:mkdir ~/Desktop/looker_project cd ~/Desktop/looker_project git clone git@github.com:MyOrgNameInGithub/looker_project.git
- Switch to the cloned project and make sure you're on the
master
branch:cd looker_project git checkout master
- If we view the Git remotes for the cloned repo, we should see
origin
as the original repo location:git remote -v
-
Create a new empty repo via the GitHub UI. For our example, let's name the repo
looker_project_prod
to house ourprod
project: - Add our new empty repo as a new remote for the cloned repo, as below:
git remote add prod git@github.com:MyOrgName/looker_project_prod.git
In this case,prod
is simply a name we assign to the remote; it can be whatever we want. We can verify the remote repo with this command:git remote -v
- Perform an initial push to our new remote of the
master
branch:git checkout master
This will ensure we're on the correct branch:git push prod master
This will push themaster
branch of our cloned repo to our new remote,prod
. In the GitHub UI, we should see that the cloned LookML has been pushed to our new project on themaster
branch.
Now, create a new empty project on the Production Looker instance: -
Enter Development Mode, select the Develop menu, then Manage LookML Projects:
-
On the LookML Project page, click New LookML Project.
-
Choose a name (such as
looker_project_prod
in our example), and select Blank Project. -
Click Create Project.
-
Follow Looker Git setup instruction prompts to connect to the new repo. For a complete tutorial, see the Setting Up and Testing a Git Connection documentation page. Ensure that you provide the deploy key read/write access.
- Follow the prompts in the Looker IDE to deploy to production. Looker will pull in the code from the new
prod
repo to the local Looker file system. -
Next, configure the new repo via the Github UI to hit Looker's deploy webhook on the production host.
In the GitHub UI, select Settings, then Webhooks, then Add webhook.
-
Enter the URL in the following format:
[instance_name.looker.com]/webhooks/projects/[project name]/deploy
For example:
https://instance_name.looker.com/webhooks/projects/looker_project_prod/deploy
This will cause the Looker host to pull the
master
branch from the Git repo to its local file system, making the changes visible.
Once configured per the above, go to your Project Settings page and reset the Git connection using the Looker UI on the prod
host:
In the Github UI, we can make the SSH key read-only for the prod
repo by not selecting Allow write access, so that no development can happen on the Production instance.
This means that any user entering Development Mode (including during setup) will see the error below while syncing their Development Mode. This, in effect, is what we want.
Proposed Development Workflow
- LookML development takes place on the
dev
Looker instance, and all code is committed and deployed via the Looker UI. This pushes the LookML to themaster
branch of the development repo. - We can enable pull requests to allow for code review before commits are merged into
master
, per the instructions on the Setting Up Version Control documentation page. - To move LookML from the
dev
repo to theprod
repo, we just need to run the following in the cloned repo (while on the master branch):git pull git push prod master
- When the
prod
repo detects the push, GitHub will hit the deploy webhook on the Lookerprod
instance to pull in these changes.