首页  思维导图  详情

Full Chain Load Test Platform

2019-08-14 17:30:44   3  举报





AI智能生成

Full Chain Load Test Platform Phase 1

perf

作者其他创作

大纲/内容

Test Scenario(Suite)

P1: Can pick up multiple load test scripts

P1: Can set Virtual User distribution (in % format or in absolute value format ) across multiple load test scripts

P1: Can set different ramp up strategy/ramp down strategy/load test duration for different load test scripts

P1: Can set up different load test schedule for different load test scripts (e.g, execute script A between 8 am and 8:30 am, execute script B between 8:10 am and 8:40am)

P1: Can set default runtime property for all load test scripts (e.g, http connection timeout, http download timeout, etc)

P2: Can set different runtime property for different load test scripts (e.g, http connect timeout, http download timeout, etc)

P2: Can set load test termination condition (e.g, stop the load test if error rate > 90%, etc)

P2: Can schedule load test execution on a future date

P2: Have GUI for configurating above settings

Test Planning

P2: Have centralized place to present planned load test

P2: Have booking system for coordination with multiple test engineers

Test Preparation

P1: Can execute test data preparation/configuration scripts

P1: Load test env health check before load test

P1: If there is other traffic in app/DB, eliminate these traffic (manual or auto) before kick off load test

P1: Test Env Deployment

P1: Can deploy pre-defined # of apps/DB

P1: Can define server hardware spec (default is same as production) and roll out to load test env

P1: Can define load test specific parameters (e.g, java opts) and roll out to load test env

P2: Have GUI for above settings

Test Execution

P2: Can scale out test client if there is too much load on test client (e.g, CPU util in K6 test client > 90%)

P1: Can increase traffic (# of virtual users) on the fly duration test exection

P2: Can decrease traffic (# of virtual users) on the fly duration test exection

Monitoring

P1: Resource consumption monitoring

P1: Test Client

P1: Application servers and DB

P2: Container level resource consumption

P1: Can persist monitoring raw data and load test result raw data in centralized place for analysis

P1: Alert when several metric(s) exceed threshold

P1: Action

P1: Keep Live data

P1: Dump thread dump/heap dump automatically if several metric(s) (e.g. cpu util) exceed threshold (e.g, 90%)

P2: Have interface to dump other business specfic info (e.g, stack trace holding Redis client connections)

P2: If several metrics exceed threshold, auto scale out test client/application server if needed

Reporting

P1: Have portal to present load test result and test client/app/JVM/DB resource monitoring info

P1: Graphs

P1: Basic graphs: 0. Summary 1. Running Users over time. 2. Hits per second over time 3. Throughput over time 4. Average Response time over time. Reference: https://www.guru99.com/how-to-use-analyzer-in-loadrunner-12-0.html

P1: For the original graph, can generate sub graph for different time period/# of virtual users

P2: Can have graphs to show multiple metrics for correlation (e.g, average response time over time, hits per seconds, test client/app/JVM/DB resource consumption over time)

P2: Can have different perspective for resource analysis (e.g, different percentile, median, etc)

P2: Intelligent analysis

P2: Can correlate several spike with overloaded resouce consumption (e.g, cpu overloaded) automatically