In Depth with Federico Toledo: “More and Better Testers Will be Needed”

What does the future of software testing look like? Why is the role of a tester crucial in our current society? Will artificial intelligence ever replace the human eye? We talked in-depth about all of this and much more with Federico Toledo, Ph.D. in computer science and COO of .

”Science, my boy, is made up of mistakes, but they are mistakes which are useful to make, because they lead little by little to the truth” is a phrase by the French writer Jules Verne that Federico Toledo knows well. As a relentless tester, Toledo knows that each finding in testing is key to optimizing systems that could be relevant on a large scale in a globalized society largely mediated by technology.

Raised in a humble countryside family, he grew up growing vegetables and dreaming of becoming a carpenter. He went a long way to get to where he is. He knows effort and sacrifices firsthand: when he began studying engineering, he commuted for 4 hours each day to make it happen.

Now, he is a leader and household name in software testing who offers conferences, workshops and speaks at multiple international events. He is one of the founders of Abstracta, a world-leading testing company with offices in the US and Latin America. He has a  and a , through which he shares valuable information and insights.

His book  has been an inspiration to the tech community all over the globe and he is currently working on its new edition, which is more than promising.

Increasingly disruptive technologies are coming up, and artificial intelligence brings with it new paradigms, with complex challenges such as the need for more testers in the industry, with better preparation. In an in-depth dialogue with Toledo, we delved into relevant issues to testing, and in a heartfelt conversation had some revealing insights into his life story.

Is it possible to ensure that a system never fails?

No, there will always be failures. It’s impossible to demonstrate their absence. So it is impossible to say that a system doesn’t have them. There are even occasions when it is a decision to go into production even with known bugs because perhaps not all of them have a severely negative impact, or because the negative impact weights less than delaying delivery. There is a wide variety of users, with different priorities and quality perceptions. Therefore, determinating when a system is already good enough is one of the most difficult decisions to make. It depends on the objective, the context, the severity of the errors, and how well the testing has been done. The horizon is to reach perfection and it has to continue being like this, but we know that we cannot achieve it. There will never be perfect software.

Why is the proper functioning of systems key in this era?

A catastrophe can occur due to the failure of a system. There are many examples: from space rockets that exploded to systems that have caused serious health problems. These are very extreme but real cases. There are also situations that affect people’s daily lives and therefore their quality of life. Today almost everything is mediated by systems and that is why software quality is so important for the daily development of life itself.

To reach the so expected software quality you mentioned, the automation of some tests can be transcendental. Achieving it efficiently requires a lot of knowledge but also judgment. In this regard, you quoted this sentence from James Bach in your book: “Automating chaos just gives faster chaos”. How do you avoid chaos and learn to distinguish which are the most important tests for automation?

Knowing the project that is being tested is essential to understanding its priorities. In turn, time and experience provides important notions about their needs. It is essential for testers to work as a team with other testers, analysts and developers, to carry out their work always considering all the risks, both from the technical side and from the business side. It is in this collaboration that we will learn to decide what is most important to try.

Why is it not advisable to automate 100% of the tests?

The pesticide paradox applies for this question: the insects that a pesticide cannot kill become stronger. An automated test will always test the same thing, it does not explore things that we didn’t ask to be tested. It finds the biggest bugs on the first pass, but the ones that were not found get stronger. That’s why it’s good to have a combined approach: performing automatic testing but also “manual” testing, with a share of curiosity. Of course, it would be ideal to do only manual testing! But that is inefficient, expensive and, if a person must test the same thing over time, eventually it ends up being even boring. Also, if new versions come out every day, there will come a time when the eyes stop seeing and curiosity will be lost. It’s important to automate everything that must be checked periodically in the same way.

So the role of the manual tester is crucial…

Absolutely. Time ago, manual testing was given to people without much experience in many places. But it is actually a very important role. Getting it right definitely makes a difference. If the person who executes a test manually does so only following instructions, without curiosity, judgment or commitment, he will not be able to find all the necessary errors. He is not going to know how to explore and understand how important the different components are for the business. Automation can never replace the human eye. But it is an excellent tool to expand what we can achieve with our efforts.

Which is the biggest turning point in a tester’s career?

The career isn’t linear. There are many who believe that there is a single path, starting with functional testing, and then moving on to automation. Manual testing is fundamental and not necessarily just the beginning of the career, some prefer to dedicate themselves completely to it. On the other hand, automating is super playful, it’s like a game in which you go through stages, trying to make the application believe that there is a person on the other side of the screen when in reality there is a script. Achieving that is satisfying! Curiosity guides the way. You begin to wonder: what if now I try to make the system believe that there are 10 people instead of 1? And what will happen if each of them provides different data? It’s quite a challenge. At the beginning of the journey, it’s a time to absorb everything you can but, in reality, you always keep learning. There are many ways to develop in testing, but the biggest turning point is when you realize that you can help other testers advance in their career. Regardless of the type of testing that is done, that is the moment when someone can start leading projects.

How is it possible to maintain the focus of testing associated with risks over time?

I’m going to associate this with personal time management. Every day there are more tasks to do, and they accumulate. We must accept that we are incapable of doing everything, that our time is finite and we have to decide what not to do. That’s where you have to ask yourself: what happens if I don’t do this? Will the system continue to work? Are there going to be angry users? Are we going to lose money? Are there going to be lives at risk? This analysis must always be present, in every decision that we make in testing. We should look forward to stopping urgent tasks. If there is something urgent it is because we are doing something wrong. The focus on risk-based testing is a mindset, and we should maintain it over time, adapting it to the new risks that arise in each context.

Your book is one of the first books in Spanish dedicated to practical aspects of the subject, with an incredible level of coverage. Where did your interest in doing this come from?

I always liked to write. I wanted to make  known and I thought of doing it by contributing something useful. I wanted to build a bridge between the testing industry that was growing in North America and Europe, bring all that knowledge to Uruguay and position Abstracta as a thought leader. I also always liked to bridge the gap between academia and industry. Since 2007, I have written a lot of articles about testing on the Abstracta blog. When I finished my doctorate in 2014 in Spain, I decided to collect everything I had already done and develop new topics to be able to offer a complete and free material, with good practices and techniques, so that more people could dedicate themselves to testing in Uruguay and create a community.

Why is it important that more people dedicate themselves to testing?

First, the industry needs more testers, developers, analysts and many other roles. On the other hand, testing is a good first step into the software industry, which has one of the best salaries in the market, more growth opportunities and better work environments. Testing is a good channel for people to join through functional testing and then develop in other areas if they wish. Having more testers responds to a need in the software industry but through it, people’s quality of life can be improved.

What led you to dedicate yourself to testing?

It is often said that those who started in testing, started there by accident or chance. In my case, it was because it gave me the opportunity to work a few hours a day at the university and be able to continue studying. The person who interviewed me, who was my first boss, asked me if I was sure to work as a tester since that would mark my professional future. I answered that I was sure, thinking that he was exaggerating and it wouldn’t be that decisive. 15 years later, here we are! I was really lucky to have chosen this profession and that I liked it! I grew up in the countryside in a humble family near Atlántida in Uruguay, 50 kilometers from the capital, Montevideo. I went to a rural school of only 50 children. I enjoyed building my toys with wood and growing crops, and I dreamed of being a carpenter. My mom was left alone raising her two kids when I was 10 years old and my sister was 8. She had to go out looking for work and started cleaning houses. We never lacked the basics, but it was a very strained situation, we got by day to day.

Federico Toledo with his sister and mother, when he was 9 years old

How was the path to continue studying?

The teacher of my last year in elementary school, Marta, was very important to me. One day, she told my mother: ‘You have to send your kid to high school. She insisted on the idea and my mother was firm with it, otherwise, perhaps my father would have put me to work in the fields. It’s amazing the influence a teacher can have on a person’s life. After high school, I went to the University of the Republic, I was able to do it since it is free and also because I received scholarships from the  and : one consisted of a monthly allowance to be able to dedicate myself to studying, and another gave me access to a University community dining room. But it was not easy, the first year and a half I traveled 4 hours a day to study. I traveled 5 kilometers by bicycle from my house in the countryside to Atlántida, there I took a bus in which I traveled 1 hour to Montevideo, and then I walked 30 minutes to the university. Then the same path in reverse to return. Fortunately, we were able to rent a house in Montevideo with a friend, very cheap and not in very good condition, but a house nonetheless. In my 3rd year of college, I got a job at the Software Testing Center (CES), and that generated an economic change that gave me more stability and I gave up scholarships to make room for others who needed it at that time. I continued studying and developing myself, I did my PhD in Spain, with a scholarship from the National Agency for Research in Innovation (ANII). All my mom’s sacrifice was worth it.

What is it that keeps you on the path of testing today? What do you dream for the future in this sense?

When I started in the sector, few people knew about testing and many were not sure of its usefulness. This evolved a lot, today it is already quite established in the industry. Testing is a means to an end, which is to have quality products that provide excellent user experiences. But what keeps me in testing is much more than that. Today at Abstracta we know what we can generate in people: offer an enjoyable, flexible and well-paid job that is a before and after in their quality of life. With this clear purpose, today my dream is Abtracta to continue growing. We have built a sustainable ecosystem, capable of achieving a positive impact on society, generating synergies and also supporting other initiatives. What moves me the most is being able to create that bridge for the growth of communities and the improvement of people’s quality of life.

How do you see the future of testing? What’s coming up?

We are in a very interesting and stimulating moment, very disruptive issues are emerging in terms of technology, everything that has to do with ‘crypto things’, decentralized organizations and artificial intelligence. Many people are working on artificial intelligence to offer tools for programming and even testing. All this generates, of course, anxieties, many people wonder if testers are going to be replaced by artificial intelligence. In reality, the more technology there is, the more testers we are going to need.

Why?

Artificial intelligence has much more complex technology than systems that don’t use it. For this reason, more and more testers are going to be necessary, and with better preparation. Software is becoming more and more complex and we are going to have better tools, learn more about mathematical models and statistics, and learn more about computational thinking. There is a connection between technology and the human experience, and that is impossible to automate. That bridge needs a human consciousness that can assess and empathize with the user. Everything related to emotions is impossible to automate. We will have difficulties and different challenges. It is an encouraging future, if it were easy it would be boring. Well-executed testing is going to have a radical impact in the coming era.

Mini BIO

Federico Toledo is a Computer Engineer from the University of the Republic of Uruguay and a PhD in Computer Science from the University of Castilla-La Mancha, Spain.In 2008, he co-founded , a software testing company, which today is a world leader in the area and is constantly growing. Currently, it has 3 offices in Uruguay and has expanded to Chile and the United States.He also founded , , the social and educational project , and  the most important testing conference in Latin America.International speaker and author of the book “Introduction to Information Systems Testing”, he graduated from Stanford + LBAN SLEI in 2021, a program dedicated to Latin entrepreneurs with a focus on scaling their companies.Since 2019, he has lived in California with his wife, who works as a researcher at the University of Berkeley. In 2022 they plan to move back to Uruguay.Also Published