Talking Load and Performance Testing with Rob Thevenet

We chatted with Rob Thevenet our Sr. QA Manager about his experience and challenges with performance and load testing for enterprise software. Rob is a seasoned QA guru and has built automation and continuous frameworks to support performance automation and performance testing for HP, Virgin Mobile and Sun Run.   At Apexon he leads the automation and performance competencies, rolling out new projects all the time.

Interviewer – Rob, thanks for taking the time to talk to us today. We wanted to do a technical deep dive into some of the issues that you come across repeatedly during performance and load testing preparation and planning.

Rob — No problem, I will be happy to answer any questions you might have. Thanks for having me.

Q1. What are Enterprise, Load and Performance (L&P) test objectives?

Rob – Some or all of the below can be included as test objectives and each require their own type of approach, metrics and tool sets

  • Understand application or subsystem/component behavior under different operating conditions
  • Disaster Recovery (DR) preparation and/or Business continuity (BC) assurance
  • Model (mirror or extrapolate)
  • Implement (tune)
  • Benchmark (pre-test, pre-prod, etc.) specific subsystems or modules

 

Q2. What issues / constraints often need be overcome to setup, execute, measure, extend the test?

Rob – Technical gap or debt, environment contention and/or dependency

 

Q3. Which Non-Functional Requirements (*NFRs) and related conditions are in play for the test effort?

Rob – The following potential conditions apply to most if not all NFRs (most listed below) and serve as possible parameters for applying to common usage cases by particular user types including system and/or operational activities including database re-indexing, backups, replication, etc.:

(a)       Minimal usage (standup, idle, acquiesce then activate minimally)

(b)     Normal operation usage levels

(c)       Peak operation usage levels

(d)     Stress/Breakage usage levels

*List of NFRs – Availability, Business Continuity, Capacity, Configuration, Data Integrity (Data Currency, Data Integrity, Data Restoration, Data Retention

Disaster Recovery), Error Handling, Globalization, Internationalization, Localization, Logging, Longevity, Manageability, Messaging

Portability, Reliability, Scalability, Security, Testability, Usability, Volume

 

Q4. What are some Key Metrics to collect and what makes their analysis interesting,

Rob – Throughput, Latency, Transaction roundtrip time and Response times (for system, subsystems*, components); per SLA or Industry accepted thresholds**

*subsystems = Operating System, Application, Hardware, Software, Driver, Firmware, Memory, Disk I/O, Cache, Network, Processor, etc.

**SLA or industry accepted thresholds;

 

Q5. Which tools/methods can ensure a successful test effort?

Rob – Virtualization and Database snapshots help to provide s/w, h/w, f/w and state interchangeability, as well as decreased setup time enabling shorter Root Cause Analysis (RCA) cycles for issues found in test or production

-Full system infrastructure monitoring using Splunk, HP Mercury Sitescope, Topaz, IBM , Oracle Empirix, and/or Silk Central

-Unique Instance Report (UIR)

-Sorted Message type files (contain # of instances per source, message data, log entry of its containing log or event file)

 

Q6. What acceptance criteria are essential to a successful test with valid results?

Rob – Access to Application / System Under Test (AUT/SUT), REST, SOAP, etc. APIs to drive, check status, etc.

-Framework design and implementation supporting L&P test infrastructure client approval sign off

-Functional Stability

-NFR acceptance criteria defined (Agile, at least 1 NFR acceptance criteria per card)

-Implement an ongoing L&P test effort to reoccur monthly to quarterly in frequency supporting regular release cycle certification

 

Q7. What are protocols encountered in L&P testing and special conditions or methods to consider for handling?

Rob – Web(HTTP/HTTPS/HTML), LDAP, COM/DCOM, OBIEE, IEEE 802.11 (mobile standards) <OSI model>

 

Q8. What types of L&P issues manifest? How can failure analysis pinpoint issues?

Rob –  Database deadlocks, failed/stopped transactions, refused connections, HTTP level 200, 300, 400 or 500 messages, load generation agent script/code compilation errors.

 

Interested in our Testing Services?

Please enable JavaScript in your browser to complete this form.
Checkboxes
By submitting this form, you agree that you have read and understand Apexon’s Terms and Conditions. You can opt-out of communications at any time. We respect your privacy.

Other stories you may enjoy...

Healthcare Apps and the Need for Security

One of the most exciting areas in tech right now promises to be “the most personal” ever. A key aspect of making wearable devices like the Apple Watch personal is through...

Developing an App for the 2020 General Election?

Here is a thought: With the UK General Election having just finished, could the next one in 2020 be the first to use a mobile app to allow people to vote? The polling...

When Digital Is Everywhere, Performance Testing is Everything

Software is eating the world, as they say. You don’t need to be a tech company for software to run your universe. Last year, 3.7 billion people were affected by software...