When a query times out or gets cancelled, it can be tricky to pin down the cause. A key piece of info: Looker does not timeout queries once they have been sent to the database. That means that all the timeouts we discuss here will be either before Looker sends the query to the database, or occurring somewhere outside of Looker.
Timeouts in Looker
Looker does not timeout queries once they have been sent to the database. We do, however, timeout queries that have been waiting in the Looker queue.
Per-Connection Query Limit and Timeout
You can set a concurrent query limit and timeout in the connection panel.
The “Max Connections” really means “max concurrent queries”. In the example above, Looker will only allow 75 queries to be registered as “waiting on database”. These will appear as “running” in the Admin > Query page. Any queries beyond this will be queued in Looker, and will appear as “Queued” in the Query Panel. Looker will timeout queries after 120 seconds of being in this queue.
Note: This is different from your database queue! Though Looker may have 75 queries that all appear as “running” in the Admin > Query page, that does not mean that all of them are running on your database. Your database may have its own queueing rules - see “Database Timeout” below.
Per-User Query Limit and Timeout
Each Looker user is allowed to run 15 queries simultaneously by default. Any extra queries are queued in Looker until one of the 15 running queries finishes. If they stay in queue for over 600 seconds, then Looker drops them from the queue.
Scheduler Query Limit and Timeout
Scheduled queries are run by the scheduler as a given user. Thus, they are subject to the connection rules above, and the per-user rules. In addition, there is a scheduler concurrent query limit of 10 and a timeout of 1200 seconds.
Renderer Query Limit and Timeout
The PDF dashboard renderer process has a job limit of 2. If a job times out, you’ll see a
RendererTimeout error message in the Admin > Schedule > History page. We’re trying to cut down on these but if it happens to you, please email firstname.lastname@example.org with as much details as possible.
Changing the Defaults
If you are self-hosted, you can change some of these defaults using the
The relevant options are:
--per-user-query-limit=<i> Limits number of concurrent queries per user (default: 15) --per-user-query-timeout=<i> Length of per-user timeout to wait for connection (default: 600) --scheduler-query-limit=<i> Limits number of concurrent scheduled queries (default: 10) --scheduler-query-timeout=<i> Length of scheduler timeout to wait for connection (default: 1200) --concurrent-render-jobs=<i> Number of simultaneous phantomjs processes (default: 2)
If you are hosted by Looker, you can request these defaults to be changed by emailing email@example.com. We’ll need authorization from a technical contact on your instance to make the changes.
Timeouts Outside of Looker
Besides the Looker product, there are a few other places where queries can time out. These can kill queries that have been running on the database!
Anything that comes between your Looker instance and browser has a potential to timeout running queries that are run in the browser. Looker-hosted instances use a proxy that times out queries after 60 minutes. Self-hosted Lookers often have a proxy that times out connections in much less time, like 60 seconds. You can read more on Looker proxy timeouts here.
The database can have its own rules for queueing and timeouts that are completely independent of Looker. For instance, Redshift has a default concurrent query limit of 5. So let’s say your “Max Connections” is set to 75, and you run 100 queries in Looker. Then 25 of those will be queued in Looker, and 75 sent to Redshift. Of those 75, only 5 will be run by Redshift and the remaining 70 will go in the Redshift queue.
Check the documentation for your database to see what kind of options you have when it comes to query timeouts and queueing.