On AI and Testing Tools
With the emergence of publications and public access to tools like ChatGPT, there has been a lot of excitement. Managers are dreaming of a new digital workforce. Many tool providers and product developers are actively looking for ways to make their products more attractive with generative AI. Sometimes, this search can feel a bit forced. While AI tools are innovative and attract budget allocation because they bring modernization, the added value of generative AI remains limited in some scenarios. In general, Large Language Models (LLMs) should be used with caution, especially when complex work is involved.
“As a trainer, I explain how to use such generative AI models, available in both free and paid versions, to accelerate your testing processes. ChatGPT & Co have huge potential to increase our productivity if they are used correctly. However, I doubt that they will serve to improve product quality with regards to requirements, because budget pressures persist. In fact, I fear that we will soon witness attempts to cut budgets even further because we can now do the work with AI. But this is my personal opinion,” says Dr. Anne Kramer.
On the Role of Testers
“There is a misconception among some managers that with tools like ChatGPT, we will no longer need testers or even junior developers,” adds Dr. Kramer. “They imagine that a few specifications are sufficient and that AI will do the rest. This perspective undervalues the crucial role of testers. We still need the experience of senior developers and testers to check the results of generative AI. Who will replace those seniors, when they are retiring and no juniors exist to take their place? Testing remains an essential and invaluable profession.”
Driven by the adoption of agile methodologies, many organizations focus on unit tests performed by developers and rely on users for the rest. But system and end-to-end tests are also important! “For example, I encountered an issue on a website requiring two-factor authentication. It directed me to another application for a code, but the code contained characters while the field only accepted numbers. This kind of mismatch could pass unit tests but fail system-level testing. Such errors illustrate the need for comprehensive testing to ensure seamless integration and usability,” highlights the expert.
On AI’s Impact and Future Directions
AI can simplify our lives by taking over repetitive task , but the creative aspect of development and testing will still require human expertise. “While AI-generated content can be impressive — producing text in English better than what I could write — it struggles with genuine invention,” shares Dr. Anne Kramer. “I don’t believe it will replace testers. Instead, we should think of it as a helpful digital assistant.”
“My tutorial at the LSTE event focused on learning how to write effective prompts to make the AI assistant useful without adding unnecessary rework,” details Dr. Kramer. “During the second session, I adopted a more philosophical approach, discussing the interplay of frustration and enthusiasm for these technologies, and the long-term direction this could take. It is about navigating between challenges and finding productive solutions.
Reflections on AI and Science Fiction
The future of AI might feel like science fiction — with AI being capable of critical thinking and intelligent judgment of what we ask it to do. “But just because it is science fiction does not mean it will not come true one day. Take the example of the journey to the moon. Initially, it was just science fiction, but then it took place. Sure, the technical means were different, but the return to Earth was quite similar to Jules Verne’s novel. The same could be true for AI advancements and the dreams of an AI capable to act on its own on large scale. However, for now, AI is just a tool and we must master its use. It is not yet capable of fulfilling every expectation, but with careful application and with due caution, it can be a valuable resource”, concludes Dr. Anne Kramer.
