Log analysis is one of the most important aspects of performing effective and efficient software testing. End-to-end functionalities and user interfaces may look right but a QA can never be sure about the product until they make sure that there are no errors or exceptions or something unexpected in the application logs.
We at Fleek, make sure that we go through each and every log file generated by our applications and supporting components and analyze the possibility of any hidden bugs or issues that can only be revealed by examining logs thoroughly. One should possess a high understanding of product and technologies used in order to examine the logs effectively.
Few important points while reviewing the logs
- Make sure that logs are writing only meaningful details, otherwise, logs will lose their usefulness.
- Errors or exceptions must be captured with a full stack trace which can help in understanding and troubleshooting.
- Details in the log files should have correct levels like the error, warning or info. If these levels are not defined correctly then logs may mislead the troubleshooting process.
- Log file must NOT have any sensitive information. The sensitivity of data can be defined based upon the nature of the application business. However, there must be strict guidelines on what data need to be captured in the logs.
- Every log entry must have a timestamp associated with it. The timestamp would be handy to deep dive the investigation on any production incident.
- Make sure logging is not impacting the performance of the application in any scenario.
How log analysis can help in the QA process
- It helps in analyzing the production issues and identifying a possible root cause of the defect. This will help in improving your test plan and reproducing production issues on your QA environments.
- If you attach the correct log details along with your defect then it would reduce at least 50% of the developer effort in reproducing the defect and fixing it.
- Effective log analysis will always help you during integration testing and reporting quality defects. During integration testing, it is very important to identify the problematic component. If you are good at reading the log files and analyzing the error/exception then it makes things quicker in finding the exact root cause.
- Sometimes QA comes across a defect which is not always reproducible and QA seems unable to find the exact pattern or steps which could reproduce a defect. In such cases, application logs can come to the rescue and could help QA to better understand what is causing the defect. QA can easily determine the exact scenario to reproduce defect by analyzing the logs.
Conclusion
- QA should always include logs analysis in their test plan checklist
- QA should understand the details captured in the logs because it will help a lot in troubleshooting and reproducing the defects
- QA should make sure that only meaningful details are captured to keep logs useful
- If you don’t have access to the log files, request the access today and start looking into the log files.
- There are many tools available in the market which provide easy interface to access and analyse the logs, for example kibana, cloudwatch, grafana, and datadog etc.
- Wherever possible, QA should attach logs with the reported defects.
