Questions to Ask Your App Developer About Testing
By: Carol Vercellino, CEO & Co-Founder
Mobile app testing sounds really important, but as an entrepreneur with an app idea, what do you really need to know about it? What questions do you need to ask the developer you’re going to work with to build your app to ensure it’s being tested thoroughly, especially if you don’t have a software development background?
I’m Carol Vercellino, the cofounder of Oak City Labs, and this is my cofounder Jay Lyerly, who also serves as our chief technology officer. He’s going to share his knowledge of testing.
Watch the video or read on below to learn more.
Carol: Alright Jay, let’s start with the basics. Why is it even important to test your mobile app?
Jay: Testing is all about risk management and how you can manage any changes to your application. So as your app grows and develops, it becomes more and more complicated. It’s easier to introduce problems when you make changes. So, a good solid testing foundation minimizes those kinds of issues.
Carol: So, risk management. Does it also help with the expense later down the road? Does it make it cheaper later?
Jay: Yeah because since you’re doing that risk management, you can feel safer in making changes later. And those later changes are less likely to cause catastrophic failures that sneak up on you.
Carol: You wouldn’t know anything about that! (laughing)
Jay: (Laughing) No, no no!
Carol: So if you were an entrepreneur and you were going to hire an app developer, what advice or what things would you ask about testing?
Jay: So the first thing I would ask is just really generally, “What kind of testing do you do?” Hopefully they don’t say “None!” So what you’re looking for after that is what kind of automated testing they do. Some developers will just do it all by hand. Manual testing is an important step, but you also want to have some automated tests to go with it.
[You’ll also want to] talk about test coverage. That’s how much of your application gets tested in an automated way. It’s really hard for an end user application like a mobile app to have 100 percent test coverage just because there are a lot of weird cases that aren’t worth testing. But you probably want at least half, 50 percent. Up to 70 or 80 percent would be great numbers to hear.
Carol: Okay, so if a developer said yes, they do automated testing, and have over 50 percent coverage – or they say 100 percent coverage – are there any red flags someone should look for? Especially if they really don’t know what automated testing is, or what test coverage really even means?
Jay: Well, 100 percent test coverage is a little bit scary because there are always some bizarre cases that are really difficult to reach, and that’s not really an efficient use of a developer’s time. As far as other red flags, you might ask about how those tests get run. Is it all part of an automated, continuous integration kind of thing where they have a system that reproduces tests and builds the product for them? Or is it by hand, just sort of on the fly from their laptop?
Carol: Okay, or you can do what I do. Just ask what tools they use and madly Google them afterward to see if they’re actually legit. (Laughing)
Jay: (Laughing) That’s also a good idea, yeah!
Carol: Alright Jay, before we close – just one question I have to ask. Can you share a nightmare, or a war story from where you should have had a test, but maybe you didn’t test, and things didn’t go so well?
Jay: So one of the hard things is [deciding] how much to test. So we had an app we built one time where people made a list of stuff and could share the list with their friends. And their friends could pin those lists so they could come back to them later. And we had an issue where if a user pinned somebody’s list and they came back and unpinned or deleted that pin later, there was a database mistake and it would cascade that deletion. It would actually delete the list, which was obviously super bad.
Jay: We had tests to check if the list would go away if the [author deleted his own pin]. But we were not checking for those sort of secondary effects. So, that was a surprise and not good. But you know, the first thing we did was go back and add the test for it. So that’s an interesting way you can use testing to do bug fixes. If you get a bug, you can write a test that illustrates that bug. Then you go fix the bug and that test turns green, which means it passes. That’s a safety check in the future so when you make more changes down the line, you don’t reintroduce that bug back.
Carol: That also could be a good interview question for an entrepreneur who is hiring a developer. You could ask them, if you get bugs, what do you do? Ask about their process with bugs and if they’re writing tests.
Carol: Jay, thank you so much for your wisdom on testing. I know you have helped us a lot at Oak City Labs with that, and we’re very thankful for that. If anyone has any questions or would like to hear more about testing or any other topic, just feel free to leave a comment, like this video, follow us on social media or go to our website www.oakcity.io. Thank you!