View the original community article here
Last tested: Mar 27, 2019
So you want to load test… why? Common reasons include:
- Determine cluster sizing (bulk testing against different cluster/node sizes and configurations, behavioral tests)
- Determine optimal network architecture
- Determine which database to use, how to structure queries, and sizing database (bulk testing against different databases / configurations and queries)
- Determine estimated concurrency threshold (behavioral tests)
- Determine expected performance degradation past optimal concurrency thresholds and where bottlenecks will occur first
- Other considerations:
- Testing Looker Only: structure tests to read from cache only to isolate Looker performance. LocustIO offers a quick way to test this.
- Testing Database Only: a common tool for this is ApacheBench
- Testing Looker + Database: structure tests to run through Looker and not read from cache. LocustIO offers a quick way to test this.
- Testing Looker + Database + Client (End to End): Tests will incorporate full range of Looker use cases such as dashboard loading, space navigation, and even exploration. Probably needs a thick headless browser to conduct such testing.
The goal is to isolate your concern and structure a test to specifically alleviate that concern.
Sample questions:
- How many concurrent users / queries in a given timeframe?
- What types of activities will be tested? (e.g. API calls, schedules, etc.)
- Are we concerned or flexible about the database chosen for our use cases?
- How will you measure success? (e.g. high availability / uptime, overall response times, Looker’s overhead vs. running directly in database, etc.)
Okay, you have a test structured. Now what?
- Configure instance: Depending on the use case, scaling vertically (upsizing nodes to XL or XXL) or horizontally (clustering 3 or more nodes) might make sense.
- Review query timeouts and queueing here to understand if any default settings need to updated before testing (e.g. connection limits, per-user query limits, etc.)
- Run a load test: There’s a whole host of different types of performance testing (a handful of common ones include stress testing, spike testing, endurance testing, etc.), this section focuses on behavioral testing. Follow these steps to install LocustIO (note that you may need to generate an SSH key for the first step). You will need to update the config.yml and locustfile.py with the host, keys, and content (e.g. Look IDs) you’d like to simulate.
- Launch Locust, and select “new test” in the upper right. Input the following:
- Number of users to simulate: ultimate number of users to launch calls. The default config file is set up to simulate users repeatedly executing a single Look via API.
- Hatch rate (users spawned / second): how quickly the users are ramped up.
- Launch Locust, and select “new test” in the upper right. Input the following:
Monitor and review results:
- LocustIO: offers some great charts, statistics, etc. to monitor as you’re testing.
- Datadog (used internally at Looker) is used to monitor servers during tests (e.g. CPU utilization, garbage collection spikes, etc.).
- System Activity history: use this on the instance you’re testing to understand if queries are throwing errors, being returned from cache, etc. during tests.
- “Status” will highlight any errors
- “Result source” will tell you if a query returned from cache
- “Message” returned the error message for any queries that resulted in an error
- Recommend scheduling an alert to proactively monitor for errors
This content is subject to limited support.