What do you do when you produce an app that is used by tens of thousands of businesses, and you need to make sure it still works after you’ve made changes?

You test it, right?

Yes, you test it. And that’s all well and good if you can write automated tests — programs to test your programs — and kick them off whenever you like. Daily, nightly, multiple times a day, however frequently you like — because it’s just a machine doing the testing, and the machine is only too happy to do what you tell it to.

But what if you have to test a physical hardware device, such as the credit card readers attached to your point of sale? You cannot simulate those in software and easily involve them in automated tests.

So, testing the swiping of credit cards is much trickier than something that can be tested completely in software, but it is no less important because you still have to make sure that your users can actually charge customers properly and take in money to run their businesses. It’s arguably the most important aspect of a point of sale.

For years at ShopKeep, we have been faced with the tedious challenge of manual testing for credit card swipes. Someone had to take a stack of credit cards, and run through a bunch of different kinds of credit card transactions on our point of sale, to make sure that whatever changes we had made to our app hadn’t broken anything pertaining to charging a credit card via a swipe. (This doesn’t even mention the need to test EMV chip insertion and contactless payments, but those will perhaps be covered in future blog posts.)

Every single time we cut a release, we had to conduct manual testing.

That’s hella tedious, brah, you might say. (If you were in California, at least.) And you’d be right. (You’d also sound like you were from California.)

Humans don’t tend to be very good at tedious tasks. Heck, we hate them. They bore us and suck out our souls with every single iteration. And with that boredom comes attention gaps, which begets human error. (“Did I test swiping with a Discover card? I don’t remember. Meh, let’s just say I did and consider it done.”) So, not only do you have unhappy humans, you also end up with a costly and ineffective testing process, which doesn’t get executed as often as you would like, and bugs slip through.

There has to be a better way!

What loves nothing more than performing boring, repetitive tasks? A robot. And so, to help us gain more control, better results, and take back time better served on other projects, we made a freaking robot.

Here is an overview of the steps we took:

  1. Come up with a basic, back-of-the-envelope design for what we wanted to achieve.
  2. Build the robot according to the design.
  3. Test it out: can we run automated tests on it?
  4. Refine the design and iterate accordingly: what can we improve on it?

If you want to experience more about how we did it, check out the video below. It contains all the detail you could hope for.


Encouraged by this success, we can now continue this innovation along many different paths, to test more of the hardware integrations we offer:

  • Credit cards versus gift cards
  • Various card brands
  • Credit versus debit
  • Swipe versus chip insert (we already had a similar machine for testing contactless taps)
  • Various credit card gateways and processors with which we integrate
  • Barcode scanning
  • Opening a cash drawer
  • Load testing (how many times can we swipe a credit card, and how fast, before it gives up?)
  • Combinations of the above

This new way of testing enables us to redirect efforts from tedious, happy path testing to deeper, more exploratory testing of our app. Which will catch more edge cases, and make for an even more reliable app. Which makes for happy engineers and happy customers alike.

 

APPENDIX: MOAR DETAILED STEPS

Below is a list of more detailed steps of how we executed this project.

  1. Come up with a basic, back-of-the-envelope design for what we want to achieve.
  2. Order all the parts we think we’ll need, including an Arduino, a stepper motor, a stepper motor driver, a power supply, and one of those slides used in file cabinets.
  3. On a 1-foot-by-2-foot platform, attach the components listed above, as well as a Mac Mini and one of the aforementioned credit card readers.
  4. Construct a crank arm and wheel that uses reciprocating motion (very much like piston action) to push and pull the file cabinet drawer slide back and forth.
  5. Attach a credit card to the file cabinet drawer slide in just the right position.
  6. Connect the Mac Mini to the Arduino via a USB cable.
  7. Wire the Arduino to the stepper motor driver, and the stepper motor driver to the stepper motor and power supply.
  8. Write a simple Node server to receive HTTP calls and rotate the stepper motor.
  9. Call that Node server over HTTP from an Appium script.
  10. Kick off the Appium script on a schedule or when triggered by a new push of our code. Sit back (or, in our cases, sleep soundly, since it runs overnight) and revel in the fact that a robot minion is now performing our tedious testing for us. Consume the test reports that it generates and use our precious human powers for greater things.

 

Gabe_Heafitz

Gabe Heafitz

Gabe is the technical lead for ShopKeep Engineering's Team Denarius, which focuses on the payments and hardware aspects of the company's flagship iOS point of sale app. In addition to developing features, his responsibilities include guiding technical decisions, shepherding and facilitating effective sprints, coaching developers' skills growth, and serving as Denarius' technical liaison. Prior to ShopKeep, Gabe served as a senior software engineer at BuyWithMe. Gabe spent the 6 prior years at HBO, where he developed and maintained systems that scheduled exactly what video media would be broadcast on each of HBO's dozens of satellite feeds on a daily basis. Gabe earned both a bachelor's and a master's of engineering in computer science from Cornell University.