Overview

In Looker 4.0, we introduced a new LookML syntax that improved upon the original YAML-style LookML with familiar but easier syntax and a rich and assistive IDE to make LookML development easier. From Looker 4.4 onward, all new LookML projects have had to be created in new LookML; however, some projects that were created before the conversion may still be modeled in old LookML.

Looker provides an automated way to convert an "old" or YAML LookML project to new LookML. This article explains how to perform that process. Each LookML project needs to be converted individually, and developers must prepare and coordinate efforts, so be sure to read all the way through this article before converting a project.

Prepare to Convert

First and foremost, the task of converting a YAML LookML project to new LookML must be performed by only one LookML developer. The conversion takes place within that developer's Development Mode and the changes are then committed and deployed like any ordinary LookML change.

That being said, the process requires the cooperation of all LookML developers. Because the conversion alters essentially every line of LookML, it is important to make sure other LookML developers have committed and deployed all of their changes to the project. Otherwise, there will be some very nasty Git merge conflicts.

Although the process of converting is automated, new LookML is much more thorough about error checking than YAML LookML. This means that, after the conversion, there may be errors showing up that weren't previously surfaced. Don't worry! These errors are real and resolving them will improve the quality of the project. The number will vary from project to project, so fixing them could take only a few minutes, but be prepared for the process to take a few hours, during which development on the project will need to remain paused by all LookML developers.

Steps to Coordinate before Conversion

To prepare for converting to new LookML:

1. Choose one LookML developer to run the conversion process.
2. Make sure the project is error-free by running the LookML Validator. Fix all errors, then commit and deploy the changes before continuing to the next step.
3. Designate a time to perform the conversion.
4. Communicate to all other LookML developers that they need to have any changes they're working on committed and deployed by the designated time. That is, their Development Mode copy of the project should be in the Git state Up-to-Date with Production.
5. Communicate to all other LookML developers that they should not make, commit, or deploy any changes until the conversion is complete.

Run the Converter

To convert to new LookML:

1. Enter Development Mode.
2. Open the project that is to be converted.
3. Make sure the Git status in the upper left corner shows Up-to-Date with Production.
4. Run the LookML Validator to ensure there are no errors in the project — the result should be No LookML Issues.
5. In the drop-down menu to the right of the Git status, choose Convert Project to New LookML.
6. Select Convert to New LookML in the confirmation dialog that opens.

Validate the New LookML

When the converter has finished running, the left pane should look like this:

To validate the new LookML:

1. Press the Validate Again button to run the LookML Validator. If there are no errors, you are good to go. Skip ahead to the Commit and Deploy section of this article.
2. If you had no errors before and now you do have errors, don't panic. As previously noted, new LookML is more thorough about checking for errors than YAML LookML. Go through and fix the errors. If you get stuck resolving them, refer to Appendix: Resolving Errors at the end of this article.

Commit and Deploy

At this point, commit and deploy as you would with any ordinary LookML change. Be aware: this commit will trigger a rebuild of your persistent derived tables (PDTs). Once the commit is deployed, let all the other LookML developers know the conversion is done and that queries relying on PDTs may take a little longer than usual to run, at least until everything's rebuilt. They can pull from production in their Development Mode to bring in the changes and then continue developing — now with an improved and more intuitive LookML syntax, and a much more helpful and powerful IDE!

Appendix: Resolving Errors

As a result of new LookML's more thorough model validation, you may see errors after you convert that weren't surfaced before. Additionally, new LookML's on-the-fly error checking requires a bit more information in some cases; the converter does its best to figure this out for you, but in some cases you may need to add an extra includes line in a view file or two.

Invalid Field References

In some contexts, "old" YAML LookML does not ensure that the fields or sets you reference are actually defined somewhere in the model. The most likely place this will surface is in drill field lists and sets.

For example, if you had drill_fields: [id, name, email], but at some point in the development of your model you removed the name field in favor of a first_name and last_name, YAML LookML will not warn you that you are now referencing a field (name) in this drill field list that does not exist. This can also occur if you are referencing a field in a dimension group and your YAML LookML didn't specify the field the way new LookML does.

New LookML, on the other hand, will tell you about these types of errors. So, after converting your project and running the LookML Validator, you may see errors about invalid field references that you never saw before. This is a good thing! Fixing these errors will improve the overall quality of your model.

Missing Includes in a View File

In some cases, new LookML's on-the-fly error checking requires a bit more context than was necessary in YAML LookML. Specifically, it may be necessary to place an include line in some view files, most commonly when one view extends another.

When there is no ambiguity about which file needs to be included in a view, the converter will automatically add the appropriate include declaration in the new LookML view file and you won't see any error.

In some cases, however (such as with projects containing multiple models), there may be two or more views with the same name declared in separate view files. The converter won't know which one is the correct one to include, so it will add a comment at the top of the new LookML file indicating that one of those views should be included. Resolving the resulting error is as simple as going to the view file and uncommenting the correct suggestion.

Stranded extension_required Explores

Like the invalid field references issue above, in new LookML the LookML Validator will let you know about Explores that are declared with extension: required, but not actually being extended. These Explores should either be reconnected to their extended objects or, if unused, removed.

Version Control

Because the LookML conversion process replaces your .lookml view and model files with .lkml ones, if one or more developers commit to your project during conversion, you will end up in the middle of a merge conflict. As such, it is extremely important that only one developer run the converter and they should coordinate to ensure that everyone's development modes are clean and up to date prior to conversion.

To avoid version control issues, there should be a code freeze during conversion. Other developers should not check in changes during the time blocked off for conversion.

If the developer executing the conversion runs into any issues during the conversion process but prior to committing changes and pushing to production, they may find it helpful to revert to production. This option from the drop-down (pictured in the Run the Converter section of this article) will undo the conversion, allowing them to start the process over.

Further Reading

For more information, check out the topic on New LookML — Deep dive into the new syntax and the new IDE in the Looker Community.