How do we sustain in Software Test Automation career?

I am sure that this question often strikes every test automation professional, irrespective of their experience level and skills proficiency. So, in this blog, I am going to do a deep dive with you to share the industry experience which I have gained, and that you can easily relate with your experiences.

The rise and transformation of test automation roles:

From around year 2002, Software Testing as a profession got significant acceleration in India. And within few years, the market demand for Test Automation professionals risen scrumptiously. Popular testing tools of that decade was Quick Test Pro, WinRunner, Silk Test, Rational Robot, SoapUI, Rational integration tester and Ruby on rails. I am not covering ETL test automation tools in this blog, as I do not have practical experience with any of those tools.

Nevertheless, Microsoft Excel based macros and VB Scripts were very popular those days, mostly used for some windows based repetitive tasks automation, including manipulation of Databases. PERL and Shell Scripting provides similar capabilities in more secured Unix/Linux/Solaris platforms.

Many training centres emerged across India, to utilise the opportunity to upskilling a mass and make huge profits out of it. Those days, all training centres used to conduct classroom trainings, some even offered placement assistance. Many who could afford such trainings those days, got benefitted either by switching to new role or by securing a new job. These days, the traditional training centres evolved too, offering hybrid trainings (including classroom, virtual, offline), but now they are facing direct competition from more organized Online training platforms and free training vlogs available on the internet.

In that decade, many did certifications in QTP, WinRunner, Silk Test and RFT tools. Whoever did the certifications earliest, mostly got benefitted.

So, what is challenging our sustainability in test automation roles?

In simple terms, the technological advances in electronics and semiconductor industry, coupled with substantial advances in Cloud computing, mobile platforms, AI, ML, IoT, Responsive Web, Single page/React applications, and new generations of high-speed internet, opensource tools has been destabilising our careers over the past decade, say 2013 to 2023.

Well, one might debate, that the technological advances have impacted all types of job roles, and human lives. Absolutely!!! Our lives have transformed to Digital; Quality Assurance and Test Automation are directly responsible to maintaining and raising the quality of our Digital transformation journey.

Between 2002 – 2012, many organizations used to conduct trainings on test automation tools, beside other technologies, for their employees, and it was optional to participate. They did use to conduct some skill-based exams, offering a certificate to their employees whoever scores 72% (for example) or above. Since, its optional, only people whoever was serious in building a career out of it or who want to be a future manager, were the only participants. In that decade, most roles demanded a manual tester, who can do automation if needed.

From 2012 onward, the optional training started becoming mandatory, and they started featuring in the career paths, yearly learning goals and performance appraisal discussions. From 2015 onward, many organizations revamped the role of manual testers, by upskilling that population to Automation Tester roles, who can perform manual testing if required. In this period, we mostly transformed from using Excel based test data sheets to BDD Examples, API driven data (JSON, XML, and recently GraphQL), Configuration based data (Properties files) etc. Many of us were tasked with doing complex test automation's, for example Mainframe applications, PDF files, image recognition and comparison, Big Data, running Unix batches part of integration test and reading console logs etc.

From 2020, the automation roles now demand knowledge and experience of CI/CD tools (GitLab’s, Jenkins, TeamCity, Terraform, Ansible, Ant, Docker, Kubernetes, Azure/AWS), deployment, Elasticsearch backed log server. That means, an automation tester who can perform DevOps tasks too.

Additionally, introduction and rise of new automation tools (mentioned below) started challenging the test automation professionals who spent most of their career with UFT (QTP), RFT, Selenium (IDE, 2.0 or 3.0) and Silk Test to upskill in these tools, to sustain and stay relevant in the job market.

• Open source: Playwright, Cypress, WebdriverIO, Puppeter, Selenium Webdriver 4.0, PyTest, Robot Framework, RestAssured
• Licensed tools (Including SaaS offerings): Copado Provar, AccelQ, Tricentis TOSCA, Test Complete, Ranorex, Katalan Studio, Karate
• Test Infrastructure providers: BrowserStack, SauceLab, LamdaTest

Accompanied by the language antagonisms Java, C#, Python, JavaScript, and TypeScript.

For hardcore techies, who are having 12+ years of experience, there are other additional challenges to deal with in order to sustain, managing an agile or e2e test automation team, preparing and implementing test strategies, enforcing design patterns and architectural decisions, code reviews, mentoring, managing conflicts, test environments, test data, test logging (Elasticsearch, New Relic), test metrics and test reporting.

Finally, what actions would help us to sustain?

•  Positive attitude and openness towards learning.
• A systematic approach, balancing our life priorities and managing at least 4 hrs a week for our personal skills development and practice, with devotion.
• Taking advantage of our organizations provided/supported learning platforms to identify, register and complete relevant training.
• Earning digital badges
• Keeping an eye on market relevant certifications, and earning those
• Getting accustomed to NLP based machine learning algorithms and trying out tools like ChatGPT
• Build skills in at least one RPA tool (UiPath, Microsoft Power Automate etc.)
• Finally, mastering JavaScript. What??? Yes, you read it right. If you master JavaScript, all these tools will become easy for you to learn: Cypress, Playwright, WebdriverJS, WebdriverIO, Test Complete and Tricentis TOSCA.

 

Test Automation tools evaluation - process overview

Preface

In this blog, I have attempted to summarize the important aspects of test automation tools selection process, based on my personal experience, and industry insights. I hope you will find this read interesting, and unique in nature, and it will help you enrich your knowledge and widen your views. Please share your opinion using the comment section and share it with your friends and colleagues.

Introduction

If you are working or had worked in an IT organization, and ever worked/interacted with a QA/Test team, you will often hear these terms “Automation POC”, “Tools selection”, “tool evaluation” etc. The “test automation tools evaluation and selection process” is an integral part of any organization who is either a consumer or producer of IT services or products.

Initiation

Before initiating a test automation tools evaluation and selection process, first we need to brainstorm and understand what are the business needs, what are the goals and objectives, what is the budget, what are the timelines, how many team's representatives needed (depending upon stake in the tools evaluation), are there any existing organizational process to obtain the trial licenses from the tools vendors, the degree of freedom given to the Test Architect/Automation Architect / Automation Manager / Test Manager (to shortlist the tools, procuring trial licenses, selecting and entrusting the team members for the evaluation and reporting, steering SLA's with inter-operating teams, i.e. internal Cyber security teams to get timely compliance clearance for installation of required software's / cloud tenant subscription etc., DevOps team to set up integration with CI, CD, or tools like JIRA, setting up demo test servers/data etc.), are any of these tools used by our parent or sub-organizational entities in other geographies at present or past, do we have any of these tool vendors in our existing/preferred suppliers list etc. 

Next, draft a plan that will detail about the goals and objectives, list the test automation tools that will be evaluated, mention the test scenarios to be covered, document the trial license procurement procedure, specify any needed test infrastructure/software, specify security compliance procedure, commitment needed from various teams, activities to be carried out, by whom and when, synchronizing or parallelization of tasks, defining the communication channels and control flow, timelines, reporting mechanism, frequency of review meetings, data management process in compliance with appropriate data-privacy law (including sensitive data, such as bespoke documents, pricing details, test data etc), risks and mitigation, expected outcomes (Questionnaires, Checklists, Reports, Activity Diagrams, Review logs etc.). Then circulate it among the stakeholders for their review and collect feedback. Ideally always place the plan under a configuration management tool for versioning.

Once all parties are satisfied with the plan and they sign it off, then an official kick off for the tool evaluation project can begin. I have categorically used the term "project" here, as it will be a time-bound piece of work with specific objectives and goals to achieve and will be used as a strategic instrument. In general, test tool evaluation is considered as a process as it is repetitive in nature, but each iteration may have different objectives to achieve.

Team selection and team building (can overlap with planning)

This is pivotal to the success of a tool evaluation and selection project. Ideally 2 to 3 committed personnel are required, additionally keep 1 person as buffer to cover up any absenteeism or reassignment of any personnel for any other priority work. The entrusted personnel should have passion for learning new tools, experienced enough to deal with the technical complexities, strong analytical, communication and reporting skills, self-motivated, know when and how to deep dive to gather crucial insights keeping in mind the objectives the tools are expected to meet and the end goal of the project, can deal with timeline pressure, identify and communicate the risks/issues on time.

It is always a good practice to allocate some personnel with diversified skills and experience to shadow the committed personnel on this project. This will help to develop the workforce and will spread positivity. Remember, in Agile projects, it’s hard to find such available personnel.

Tools shortlisting (can overlap with planning)

This must be aligned with the overall organizational strategy of an organization’s digital journey and the vision for quality. Test automation tools can be broadly categorised into these categories:

UI, API, CLI, Mobile, RPA, AI/ML (including BOT), Embedded, Platform Automation (aligned towards DevSecOps), IoT, Visual Testing, Accessibility.

Depending on the category, prepare a questionnaire, keeping in mind the objectives and goals, to gather and capture important and necessary information about these tools. Create two copies. Allocate a team with a timeline, handover one copy to them, to do research and find top 3 tools that satisfies majority of the questions. In parallel, take opinion from other experts within the organization, who might have been used any such tools in the past. Once this activity is concluded, collect the report from your team, and compile all gathered information. Next, have a meeting with your team and assert their confidence levels on the 3 tools that they have shortlisted and recommended.

Remember, sometimes, tools are shortlisted at a strategic level and passed on to a team for evaluation and feedback, based on that final procurement decision is made.

Procuring automation tools trial licenses

Depending upon whether the organization has existing procurement process or not, this can be initiated by entrusting Automation Architect / Automation Manager / Test Manager to steer the communications with either the procurement department or the tools vendors.

 

In this process, the first step is to contact the tools vendor through their official website's request a demo license page. Once you submit a form, the vendor organization's representative will connect with you to gather details about the business needs and expectations. At this point, you could provide them a copy of the questionnaire to capture their responses. Focus on any hardware, software, network requirements for the tool. Involve representatives from your organization’s cyber security compliance, DevSecOps, and procurement departments, and include Automation Architect/Test Manager and Technical Architect/Principal Engineer in these discussions. Be rational during these discussions, don't ask for too many free licenses, as some may be wasted due to lack of commitment/unavailability of people, set right expectations, request for bespoke offer and pricing details. Sometimes a non-disclosure agreement needs to be signed, as pricing is a sensitive information. If you have strict timelines, tell them upfront, build upon mutual trust and establish some SLA for communication turnaround time, it will be helpful in expediting procurement of demo licenses. 

Installation, exploration, and reporting

These days, many automation tools are offered as SaaS (‘Software as Service’ cloud offering). So, you may not be required to download and install something, like traditionally. But if it is required for the chosen tool, then read their installation guidelines carefully, do the installation and setup. Most vendors allocate a SPOC to reach out for help or a ticket-based service desk or a BOT to contact support, use it wisely.

 

Once done, start the exploration by automating the scenarios as per the test plan. At this stage, team members are expected to do multitasking, they should use their committed time in reading/watching videos of the tutorials/user docs and then they should apply the gained knowledge immediately to progress automating the scenario. It’s a good practice to fill in a query tracker and send it to the TA/TM, so that they can communicate with the tools vendor and expedite removing any impediments. Also, keep making notes, from the experience you gain out of this activity, as you will need it at the end to write up a report. Highlight any risks immediately to the TA/TM so that appropriate actions can be taken in good time.

Once, the activity is concluded, send a report in the predefined format, to the TA & TM, including your recommendation.

Note, sometime multiple tools are explored in parallel. It is the TA / TM's responsibility to combine the reports and producing a master report for the higher management.

Final procurement decision

Multiple factors influence this decision, majorly pricing model, and budget trade-off on a long term, it plays a crucial role. Apart from it, Vendors reputation and credibility, cyber and data security aspects, turnaround time for procurement process and all other important communication, the tools fitment for the pipeline projects across the organization portfolios, existence of talent pool in local job market, ease of use, training and support, software & patch updates, ease of migration from old to newer version etc.

Sometimes, the decision gets deferred, it favours an open-source solution, or the organization outsource that automation work to an entrusted service-provider for short term to observe the potential benefits coming out of using the selected automation tool.

 

Interview preparation series: Write a Java Program to calculate the angle formed between the hour and the minute arms in an analog clock

In this post, I am going to solve this programming puzzle following a simple approach. If you have any better approach in mind, please mention in comment. If you are liking my posts, then please share it with your friends.

Steps to solve the problem during interview or online test:

1. Patiently analyse the problem statement for 2-3 minutes, conceptualize the expected calculations. In our case, we know that analog clocks have overall 360 degree angle. The hour hand moves 30 degrees between each hour. The minute arm moves 6 degrees between each minutes. The hour hand also moves 0.5 degrees each minute. On 60th minute, the hour arm moves to a new hour, and the meeting arm gets reset to zero. When hour ==12, then there is no angle formed between hour and minute arms. 

2. Use, any of your favorite Java IDE. (Prerequisites: Java 6 or above installed, Set JAVA_HOME)

3. Create a Java class file, name it following the best practices

4. Create the main method

5. It's always a good practice to take inputs from the user, rather than using hard-coded input values. This creates a positive impact in the interviewers mind.

6. Write the logic in simplistic way possible

7. Ensure you are displaying the expected output

8. Do a proof read, before executing for the first time, if the program fails at first run, it may create a negative impact in the interviewers mind

9. Run the program against different inputs to proof the logic is correct. For online tests, upon the code submission it will auto trigger unit tests, and highlight any failures. You can fix the issues before the test gets timeout.

public class CalculateHourAndMinuteArmAnglesInAnAnalogClock {
    public static void main(String[] args) {
        /**
         * Taking two inputs from the user at runtime
         * hour: Represents the hour arm value of an analog clock
         * minute: Represents the minute arm value of an analog clock
         */
        int hour = new Scanner(System.in).nextInt();
        int minute = new Scanner(System.in).nextInt();
        //Declaring a local variable which will hold the final value of the
        // computed angle
        double angle;
        //We need to ensure that the user has entered correct data, otherwise
        // we will
        // throw a RuntimeException
        if (hour > 0 && hour <=12 && minute >= 0 && minute<=60) {
            //if user enters minutes=60, then we will add 1 to the hour value,
            // and reset minute to 0
            if(minute==60 && hour<12) {
                hour += 1;
                minute=0;
            }
            //if hour=12, then we need to set hour = 0, and compute the
            // angle between 12 and the minute value
            if(hour==12) {
                hour = 0;
                angle = Math.abs(hour - (minute * 6));
            }//Otherwise, we will calculate the angle as follows
            else {
                angle = Math.abs((60 * hour) * 0.5 - (minute * 6));
            }
            System.out.println("hour = " + hour);
            System.out.println("minute = "+minute);
            System.out.println("Angle between the hour and the " +
                    "minute arms of an analog clock is: " + (int) angle
                    + " degrees");
        }else{
            throw new RuntimeException("Invalid inputs, hour should be" +
                    " >=1 and <=12, and minute should be >=0 and <=60");
        }
    }
}

Interview preparation series: Write a Java Program to count the occurrences of each character in a String value

In this post, I am going to solve this programming puzzle using two approach. First, using Regular Expression and Second, using String functions. Given below the code, kindly go through it, if you don't understand something then leave a comment, I will be happy to answer. If you have any suggestion for me to improvise the contents, please leave a comment. If you are liking my posts, then please share it with your friends.

Steps to solve the problem during interview or online test:

1. Patiently analyse the problem for 2-3 minutes, conceptualizing the needed calculations. You may rephrase the problem statement and check with the interviewer if you have fully understood the problem, as otherwise your program will not output the expected result.

2. Use your favorite IDE (if you are giving a remote interview and using own device) for Java programming, like InteliJ, Eclipse etc. Ensure that you have Java 8 or above installed and the JAVA_HOME path is set in the environment variable.

3. Create a java class file, use appropriate visibility (private, public, protected) and follow naming best practices. For online interviews/coding challenges, if there is no specific ask to show inheritance, then no need to create any interface or abstract class to complicate the excercise.

4. Create a main method to execute the program. This is also required by most of the online coding test platforms, as they run different unit tests to measure accuracy of the program.

5. It is a good practice to seek inputs from the user using meaningful prompts to make the program interactive, rather than hard coding it inside code. The inputs can also be accepted from commandline arguments.

6. Put meaningful names for the variables, but remain careful and avoid creating unnecessary variables.

7. Write the code in a simple manner. Often the code could be written in different ways, in such a situation, follow the way you are more comfortable with. Focus on writing correct logic than spending time in ornamentation or showcasing your coding prowess.

8. Before running the program for the first time, always do a proof reading and make any needed corrections, since if the first run fails, it could create a negative impact to the evaluators mind. Often we do mistakes in anxity, so keeping calm is a necessity. In organizations and enterprizes, the real-time programming will be much complex and there could be pressing deadlines to achive shorter milestones, therefore during recruitment, they observe the behaviour of the programmer to assess whether the candidate could handle work pressure, stress and the quality of work in a seamless manner.

9. Run the test with different inputs, to prove that the logic is working fine. If you are writing the code in any of the online test platform, then don't worry, it will fire unit tests upon the code submission and highlight the failures, you could correct them before the test meet its time outs. 

import java.util.Scanner;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * Java program to count occurrence of each characters within
 * a String valueBelow program accepts an input from user, 
 * then counts the occurrence of each character in that String
 * There are two implementations, first one is using Regular
 * Expression and the other one is using String functions
 * @Author: Susnigha Chatterjee
 */
public class CountEachCharOccurrenceInAString {
    public static void main(String args[]) {
      String inputText = new Scanner(System.in).nextLine().toLowerCase();
      int counter=0;
      char[] inputTextCharacters = inputText.toCharArray();;
        //First implementation: Using Java Regular expressions
         for(int x=0;x<inputTextCharacters.length; x++ ) {
            Pattern objPattern = Pattern.
                    compile(""+inputTextCharacters[x]+"");
            Matcher objMatcher = objPattern.
                    matcher(inputText);
            counter=0;
            while(objMatcher.find())
            {
                counter++;
            }
            System.out.print(inputTextCharacters[x] +" => ");
            System.out.println(counter);
        }
        System.out.println("---------Separating the outputs----------");
        //Second implementation: Using Java String functions
        for (int i = 0; i < inputText.length();i++) {
            counter = 0;
            System.out.print(inputTextCharacters[i] +" => ");
            for( int j=0; j<inputText.length(); j++) {
                if((inputText.charAt(j)==inputTextCharacters[i]))
                    counter++;
            }
            System.out.println(counter);
        }
    }
}

Given below the outputs:

ulupolupolo
u => 3
l => 3
u => 3
p => 2
o => 3
l => 3
u => 3
p => 2
o => 3
l => 3
o => 3
-------------- Separating the outputs---------------
u => 3
l => 3
u => 3
p => 2
o => 3
l => 3
u => 3
p => 2
o => 3
l => 3
o => 3

Recruiting Test Automation talent vs Upskilling existing talent

Problem statement:

In IT organizations or Software Engineering units of corporate organizations, a question seldom strikes the mind of leadership, whether they recruit a test automation talent from the labour market or make internal arrangements to upskill their existing talent pool to fill up any open positions in their projects. 

It’s not an easy decision to make, since hiring process for both permanent or contractual talent could take up significant time, jeopardising the project needs and schedule. The recruitment process will involve creating an open position, advertising the job, collecting, and filtering the resumes that best matches the job criteria, identifying an able interview panel to conduct the technical evaluation (sometimes, it is outsourced to consultancies), scheduling managerial rounds, negotiating notice period and salary as per fitment, making an offer, offer acceptance, onboarding etc. Filling the position from an existing vendor organization may reduce the lead time, but it will increase OPEX, and provide less degree of freedom on that resource’s performance management and over time it will establish dependency. More often vendor organizations are unable to source an automation resource ticking all boxes of the skill and experience requirements for the job.

Over past few years, many organizations conducted inhouse upskilling programs to migrate all manual testers/quality assurance professionals to test automation skill sets. Some did partnerships with external training providers to design a customised training plan suiting their organizational strategy and resourcing forecasts for upcoming years. They did lots of investments in their learning and certification budget apart from investing time, but still they find that many of the upskilled resources are not able to fulfil the promise when they are deployed into challenging projects, most of which are following Agile lifecycles.

So, what's common among these options, is identifying the right talent for the job.

Recommended Solutions:

ü Recruiting a candidate from job market:

All recruitment decisions originate from the organization strategy to support the organizational vision. For example, if the vision is “delivering IT projects on or before the end date, within budget, meeting all quality standards”, then recruiting the people with right skills and experiences must be part of their strategy, as they may not be interested investing time in training, which would increase their CAPEX and OPEX, and not help much in achieving their vision on short term. However, if the vision is for long term, and the mission is to earn recognition of say to be in the list of top 10 employers, then investing in training would be required.

 

As every project undertaken, have an impact on the organization’s profitability, vision and mission, any recruitment for the open position in the project should be taken seriously.

1.     Clearly draft the job description (JD) - elaborate on the job roles and responsibilities, the specific test automation skills and relevant experiences that are expected out of a candidate. Always involve the projects Technical Architect, Delivery Lead, Project Manager Test Manager/Architect in the JD drafting and review process. If the role going to be crucial, it’s advisable to mandate having some specialised professional certifications on certain skills/technologies. Example, if the project requires an Automation Architect/Automation Lead/Automation Manager, then "ISTQB certified Automation Engineer" certification could be mandated. This will ensure that the candidate is having professional knowledge on test automation principle and practices. Optionally include any programming language certification in Java, JavaScript or Python that will ensure the candidate has good coding background.

2.     Structure the selection process in 4 stages: Every organization have different recruiting process. My recommendation is a 4-stage process:

a.     Assign technical people to screen resumes.

b.     Online screening: max 7 multiple choice questions on test automation and related technologies, plus couple of small programming puzzles to solve.

c.     Technical Round: wrap up within 90 minutes, focus on the technical challenges, debugging skills and innovation that the candidate has carried out, his contribution towards the automation optimization and its impact on ROI. Consider any contribution to Git-Hub or any other opensource platforms, any publications on test automation journals/blogs. Most importantly, asses the candidates’ level of passion for automation, coding, problem solving and learning.

d.     Managerial Round: max 30 mins, understanding their motivation, whether its financial rewards, achieving a certain designation or getting wider recognition. Their capability to perform under tight timelines balancing their stress, ability to work as a team player. 

Most importantly, carefully deal with those who are not passionate and self-motivated towards automation. They may not be a good fit in the long run.

ü Upskilling existing workforce:

This is an excellent idea, for sustainability, earning reputation, balancing geo-political impacts, and display commitments towards people development. Since, upskilling involves both CAPEX and OPEX, organizations should:

1.     Start by gathering statistics on the skills their customers and clients are looking for, the ones that their competitors are using. In terms of test automation skills, the tools are aligned with the layers of testing. Like, UI Automation using specific tools and languages (i.e., playwright, cypress, webdriverio, selenium with JavaScript/Java/Python, robot-framework, Copado, Provar, AccelQ, UFT, Test Complete, TOSCA, RFT etc.), API Automation (i.e., ReadyApi, RestAssured, Postman, SoapUI, Swagger, Pact, rest, soap, http/https, ftp, ssh, json, xml, GraphQL, Mulesoft, Kong, RIT, karate etc.), Mobility Automation (i.e. Appium, Perfecto, SeeTest etc.), Robotic Process Automation (RPA tools, i.e. Uipath, BluePrism, Robot Framework RPA etc.), Database Automation (i.e. using Java, JavaScript, Python etc.)

2.     Do a survey on existing testing talent pool, identify the sample populations, those are willing to upskill in test automation skills, and those are willing to upskill for a different job role that they wish to move to (example: Scurm Master, Business Analyst, PMO etc.). Upskilling all manual testers to automation without giving any choice, may not be the wise idea, and I personally believe this type of move never make much impact for the organization in the long term. Some people are phobic towards coding, as they are to Mathematics. No matter how simplified the coding materials are written, they won't develop much interest to learn and nurture it. These people may be more interested to upskill as such they could take up roles like:  Scrum Master, Business Analyst, Project Managers, Content Writer, Product Manager. So, Choosing the right candidates to upskill to automation skills is the key here.

3.     Next, develop training plans, schedule trainings, conduct trainings, gather feedbacks, and feed the inputs to revisit the the training plan. According to me, when developing training plan, beside the basics, advance concepts (like dynamic xpaths in case of Selenium, usage of Regular Expressions, JSON handling, executing DB queries from code etc) with practical case studies and programming tasks (both guided and un-guided) should be include. At the end of the training, it is advisable to conduct one online test (including programming) to ascertain that all participants have successfully met the training goals and objectives. Always identify actionable items on the feedbacks received on the training plan, material and the trainers. This will generate key performance indicators for the training program.