How Albertsons Is Harnessing the Power of AI and Automation to Transform Consumer Relationships

Friedel: Albertsons Companies represents 24 banners across the country, some of the biggest ones being Albertsons and Safeway, although we also have some large regional players throughout the country. There are almost 2,300 stores across markets with more than 34 million customers every week. In addition, we have about 400 fuel stations that we're heavily testing and making sure the software is working well. We've implemented this test automation solution across both the retail POS as well as fuel environments.

Guffanti: We’re going to zero in on how you transform the checkout experience. Can you give us a bit of background on why you decided to focus on that? Why did you decide to embark on this transformation journey? This dates back to pre-pandemic times, as well, so where you were and where your head was at that time.

Friedel: What we’ve seen, as I'm sure others have, the demand, the need for getting software capabilities out to where the customers and associates are has become an increasing priority. The rate and quality of expectations that people have come to expect and desire demanded something that would give us the ability to be more consistent, or faster in our ability to do that testing. Automating much of our regression testing through the use of Keysight’s Eggplant Technology, as well as the robotics from IntelliQA has given us that capability. It's allowed us to get solutions out more quickly, as we were hearing loud and clear from business stakeholders.

Guffanti: What were you hearing loud and clear from some of your customers in terms of their preferences and behaviors? What do we need to transform in terms of how we do things? And how has that changed from pre-pandemic to post-pandemic?

Friedel: Everybody does some version of grocery shopping. What we hear, more often than not, is that many people actually thoroughly enjoy many aspects of the grocery shopping experience: walking around, looking at the food, squeezing their vegetables, etc. However, the part that people consistently like the least is the checkout experience. The goal at that point is to get out of the store as fast as possible, and our goal is to help them do that while executing all the steps in that final stage.

There is significant focus on how we can make that final process in the store more seamless and frictionless through the use of additional technology and capabilities such as contactless technology for identifying themselves with loyalty or for payment. Quite simply, making sure it flows well — it's not bound with errors or inefficiencies in the flow — and as we release new capabilities, we're not introducing a step in that flow that complicates the process causing friction for those customers.

As far as what changed in the pandemic, again, the customer expectations continue to grow as well as associate being treated individually, getting a smooth experience on the way out. 

We still had to release and test new software. Fortunately, we had already started down the path of automation and remote control capabilities for our POS within the lab. This allowed us to continue testing while also developing automation capabilities at the beginning stages of the pandemic, but also throughout, by having some of that foundation in place. We’ve been leveraging resources, not only on-site in our Pleasanton, Livermore, California offices, but also in places like the Philippines and India, where we've had people that can actually develop and test from remote locations.

Guffanti: It seems to me that the pandemic created innovators of us all, out of necessity. But it sounds like you already had laid a lot of that foundation beforehand, so you were able to hit the ground running when crisis hit. I’d like to bring Josh and Paul into the discussion. This is a technology story, and I’d like to get your insight into how technology plays a role in achieving customer centricity from a higher level.

Kaye: As Mike mentioned, there’s a customer journey. Having that smooth customer journey is important, but it doesn't happen by magic. It’s designed by people understanding what the processes are and how they can improve the technology to be able to do those extra things. Contactless is reasonably new, it's been around in Europe for longer than the U.S., but there's contactless out there all the time. Other aspects, such as QR codes, are widespread in the U.S. There are other aspects that are there, but they don't just happen. Retailers, especially the guys at Albertsons, they really focus on those paths. Then, it's our job to work out how we test those paths.

Guffanti: How did your early conversations with Albertsons turn from a business discussion into a conversation about robotics and AI?

Kaye: Albertons was in a good place because they knew exactly what they wanted to achieve. The early conversations with Mike and others were about how we would, not what we would. We’d talk about how we would speed up testing, how we would fill in the gaps. There was an ability to do things like cash transactions and other forms of transactions, but by adding robotics and card payment technologies, it made it a full story. When we first came across to Livermore, Mike, Josh, and the team were very welcoming. We got the robots out, did some demos, then they then told me that the CEO of Albertsons was coming in two days. We got up and running, did a demonstration with the robots — Archie, that you'll see soon, I believe — it all happened quickly. It was a really great time, wasn't it, Josh?

Wilber: It was, but I don't know if “great” is the right word to use there. It was rather hectic. That first week there, the first couple of days they were showing us demos and getting everything on the network. Then, shortly after, they're saying the CEO is coming. We were all nervously trying to get everything else working. We did a really good job with that. It was a fun experience, but it was a little stressful at the same time.

Kaye: We did it.

Guffanti: What were the comments from the CEO when he saw it in action?

Wilber: He definitely had a huge smile on his face. He was very ecstatic and overall happy to see what was going on.

Friedel: As a follow-up to his tour around some of the offices, he ended up having a bit of a playback, or presentation, back to the rest of the company. In that, he referenced Archie, the robot, so we knew we made a bit of an impression, at least.

Guffanti: Fantastic to get buy-in and support from the very top. Let’s shift gears to the meat of this discussion: the practical applications. Shifting towards the business need, what you want to accomplish, the goal of frictionless commerce. How did you make it happen? We started the discussion on technology, of course it's key to roll out technology testing. You mentioned the lab, tell us how you developed the testing lab, the role that it played, and what type of testing is performed in the labs.

Friedel: As a company, we've taken testing and our lab environment very seriously, as evident by the size and scale that some folks have come and seen. Paul's been pretty impressed with it.

Some of the robots more directly control the pin pads, while others work with IntelliQA to create a patent-pending solution around fuel tests and fuel robotics, where we're able to test fuel pump transactions within that lab.

We’ve been working on integrating these robotics-based flows into our CICD pipeline to get that first look of intake of new code. There’s an opportunity to run that through some base regression tests. When you talk about what are the types of tests that we've automated, again, there are almost 550 tests automated within the core retail POS regression suite. There are another 200 in the payment certification flow to make sure that we don't introduce any issues into the environment when changes in the payment environment are made.

We've also been increasing the number of tests within our fuel and self-checkout environments. Those are newer areas for us, but are going well. Lastly, we do what we refer to as enterprise certification, where we have a program that any change that comes into the retail environment has to get validated and certified through this enterprise certification environment. We've done a good bit of automation in that area as part of the baseline, and then some of the core execution for tests that run through that environment as well.

Guffanti: That’s a great overview of your testing philosophy and methodology. Paul, in your dealings with other retailers, are they as aggressive on testing as Albertsons, or is Albertsons an anomaly? How important is testing?

Kaye: All retailers take testing seriously — they have to. One of the special things about Albertsons is the vision that is there. Most companies would think of fuel payment testing and decide they can't really do that, or they aren’t sure about it. Mike, Josh, and others on the team are always asking: What can we do next? We meet weekly with the team — each Monday evening for me, each Monday morning for these guys — to talk about what can be next. It's that approach, that desire to go and do, that willingness, that makes Albertsons special. It's the comprehensiveness, the approach. It’s why I love working with Albertsons.

Guffanti: Paul, can you give us a taste of what the testing lab looks like? You’ve mentioned the Eggplant solution a few times, how does that play a role in the testing lab?

Kaye: Eggplant is the tool that orchestrates all of this testing. It is a test automation tool and test automation solution. Eggplant drives both the emulation of the merchant — driving the cash register, creating the basket — there's some really cool stuff in there because the basket is being created with barcode scans. These guys have devised a mechanism to move the barcode scans on so that you could create any basket they choose. Then, from Eggplant, the robot is called to be able to go and do the various actions that emulate the customer. In this case, the robots enter their cell number, which links to the loyalty account, then does a dip transaction, enters the contribution or donation that the customer wishes, and then through to the end of the transaction. Eggplant is then back on the register.

In this case, it's a Toshiba register, but Eggplant has the capability to automate many registers and other brands are available. All of that pulls together to make a real solution — you've got the merchant, the person working on the till being emulated, the customer being emulated — and it's bringing into play different card technologies. They can dip, tap, or swipe with either physical cards or simulated cards from tools such as the UL brand test tool and the ICC solutions tool.

Guffanti: Mike and Josh, anything to add to that on what we just saw or any kind of adjustments that needed to be made along the way from your initial vision of the testing lab?

Wilber: When we first started using the Keysight's tool for Eggplant for automation, we did not have the robotic solutions. We had a scanner there as well, scanning a barcode. Initially, we started out with simple transactions and simple cash payments. However, we were able to evolve that into being able to scan items, and now we can scan any item we want as well as coupons. With that, we’ve been able to evolve that into IntelliQA's robotic solutions. 

Friedel: When we first started this, we assumed, as we have with most of our testing, that the only way to effectively test is by being present in the lab. In a retail POS environment, you have to put your hands on things and actually execute the test that way. This unlocked the ability to do not only the test automation, but to leverage the stack for manual testing, remotely. Particularly for the automation side, we were pleasantly surprised to figure out that we could leverage folks in other parts of the country and world to be able to build this and create this “follow the sun” mentality, where some folks are developing test cases and after they're done, somebody else in a different time zone is able to review them.

Conversely, we have automated testing that we're able to run through the night, over weekends, etc. Being able to leverage folks in other locations during their daytime and during their shifts to restart tests — if they run into an issue, to evaluate something that may go on, or start a new set of tests — has been powerful for us to reduce time-to-market. That’s an ever-needed capability where everybody wants it out there faster. It’s helped us in that regard, also.

Guffanti: That's a perfect segue to my next question, which is time to market and as it relates to COVID. How did COVID affect your testing timing and urgency, and how did you pivot to address that need for speed?

Friedel: I referenced it briefly earlier, COVID did fundamentally change things and increase the expectation, or reduce the expectation, around time-to-market. Our business, our customers expected things to be done faster. Being able to run these robots overnight and over weekends, as new code drops come in, allowed us to be testing those. Then, when folks come into the office in the morning, they've already got an amount of validation and testing completed. 

Unfortunately, you do find issues with code on occasion and have to retest or re-run some regression tests once that's done. We've found this to be helpful in that regard because there’s bound to be a hit when you have to re-run testing. In this case, being automated, we don't lose as much time because now we can execute that during off hours when we wouldn't have normally had folks able to test.

It has certainly reduced the amount of time that it takes for us to get change and capabilities out to the stores. I make reference to our core releases, but then we run a lot of what we refer to as “off cycle,” which are smaller changes and releases that we can get out more quickly that don't require as much of the regression. While at this point we're not running a ton of the automation on those, by being able to run the automation on some of the larger major releases we're working on, it frees up resources to be able to put focus on some of those other smaller changes. That allows us to get them out to the stores much faster than we were able to in the past.

Guffanti: I was going to save Q&A for the end, but some of the questions play into our conversation pretty well, and are very specific. Paul, does Eggplant validate physical and digital receipts?

Kaye: Eggplant definitely validates digital receipts, which is the preferred method. We've worked with a number of retailers where some aspects of the receipts are only generated on paper. To do that, we've developed receipt validation units that use a camera and OCR technology to be able to validate those paper receipts. It's fantastic in some respects, but as this has been developed, we've taken lots of paper — it's surprising how much paper an automated test run that goes overnight, repeatedly generating and validating the receipts, takes. If you've got large waste paper baskets, you can do it, but in the interest of looking after the world, paper, and forest, I'd suggest using it only when you have to, and we'll help you do it if you need to.

Guffanti: Does IntelliQA and Eggplant have the ability to also have robotics removing product for weights of products to determine cost for the basket?

Kaye: You can move products onto the self-checkouts or onto anywhere else. In most cases, there's no real need to because what you're testing is the payment. Most stores can turn off the weight security checks on each of the items and focus on the payment. Josh, that's a fair statement, isn't it? That's probably the way to go about it, to segregate checking the weight of the items to checking the payment.

Wilber: That’s right. This goes into our self-checkout testing that we just started working on the automation for. As you were saying, Paul, we have the capability to disable that functionality for the checking of weight. Our main focus is more so on the release itself, the actual new functionalities we have coming in with these new code drops, and more importantly, the payments as well.

Friedel: There's less of a desire to test the physical hardware in that case. 

At one point, we did consider: Should we do that? Do we have to create something that picks up the items and moves them around? We decided that juice wasn't worth the squeeze on that, at least for us.

Kaye: You're absolutely right, Mike. You can do it, but there's no point.

Guffanti: Mike and Josh, how long does it take, on average, to build out an automation script, leveraging Eggplant in IntelliQA?

Wilber: That's a good question. Before we can even start automating any test case or making any script, we have to lay out the overall groundwork first. For us, we have to develop what is called a snippet. There's multiple snippets for multiple actions of what we want to do, such as signing into the POS system, robotic functionalities, entering items, and such. Once you have those core snippets down — depending on the overall complexity of the test case — it could take us 15 minutes, or it could take a couple hours. It depends on the test case and what we're testing, but that's about how long it should take for us.

Friedel: Josh, along those lines, or another core part. When we first started with the automation it was more of a directed or functional test case. Then, we moved in to leverage Keysight, Eggplant technologies, AI-based, model-based tests. In that case as well, there's the notion of having to build out that model first, tie the snippets to it, and then that allows a lot of different permutations. Correct me if I'm wrong since you're closer to it than I am.

Wilber: That’s right. Another nice functionality of Eggplant DAI, or digital automation intelligence, is that it is AI-driven and uses machine learning. For us, we have those directed test cases. Mike, you were saying earlier, we have about 550 front-end point-of-sale test cases that we have automated. When it comes to the exploratory testing that we're talking about, we have different actions and different states where instead of following a directed path, the Eggplant DAI can actually go ahead and enter an item first, then do loyalty, and then payments after. If we can change it up a little bit where we can enter loyalty first, then item entry, and then payment. Then, we can not only do direct test cases, but exploratory testing as well.

Guffanti: What percentage of regression tests are automated, and how many days of testing has the automation saved?

Friedel: This has been one of those things, candidly, that frustrated me because every time I ask the number, I get a different number back. I finally realized that there's so many more capabilities, features, and functions that come in that not only does the number of automated test cases increase at a percentage level, but the total number of the demand for total needed test cases is increasing on a daily basis, too. It's a bit of a moving target, at this point, we're probably at around 60% of total.

We do recognize that there's some that, franking a check, and there's some random ones that we're not going to be able to automate. For some, it's probably not worth it at some point — probably around 15-20% are not good candidates, not worth the time to go after. However, we're adding 100-200 test cases every year. Of course, some will go away over time, but it's become, again, a bit of a moving target of a question over time.

Guffanti: Got it. Moving on to the business results and lessons learned segment of the discussion. Mike, with your focus on AI, robotics, and everything that ensued with the testing lab, how did that ultimately affect your business results?

Friedel: I made some reference earlier to the demand, the sense of urgency, the need to support associates and customers in the stores — there was a very strong desire to get change out there faster. Of course, there wasn't much tolerance for that change not working well. Where this helped us is being able to accelerate good quality, positive change into our retail environment.

Guffanti: I’d like to talk a little bit about lessons learned and allow you each to comment on what lessons have you learned from this entire process. This also relates to any challenges faced during test script maintenance. Any lessons learned or challenges that you had to overcome throughout the entire process?   

Wilber: Definitely one of the biggest things we had to learn and figure out is that with new functionalities that we do have coming in sometimes we will have a change in certain prompts or the overall flow of certain functionalities. When it comes to that, one thing that we had to quickly learn is how to develop our scripts. 

Friedel: On my side, it ties back to keeping track of the metric and being able to track progress. That gets a little challenging as both sides are increasing and changing over time, but that becomes critical in a couple ways: To bring visibility that you do have new test cases and to have a way to work them into the automation flow. 

One of the things we looked at as new projects came in, was that there was a bit of a tax to projects that they needed, not only to build test plans and some tests as part of new capability introduction, but also fund some of the automation associated with making them repeatable. While there is value, and certainly it's helping us, it sometimes becomes difficult to quantify to keep the investment going.

That's one of the things we've spent time on, trying to make sure that we have resources both internally, as well as the ability to leverage external resources in order to help us continue with automating tests as they go along, so that we don't end up with a big backlog.

Guffanti: Mike, to that point, you mentioned your CEO poking his or her head in on the process. How is this being viewed by the executive staff, your efforts? Is it more of a profit center that drives a ROI or a cost center that provides efficiencies and allows other parts of the business to do their things?

Friedel: At a business level, they're probably not, in general, quite as aware or interested in the testing, so to speak. They're more interested in the value that's getting delivered out to the stores. This is a little bit of the magic that happens behind the scenes to help enable some of the other capabilities, and the timing of getting the capabilities out to the stores faster. Within our technology organization, automation and testing is seen as a very valuable and critical capability. Again, anything we can do to take repetitive tasks away from humans and automate them, is seen as a very high priority within our organization.

Guffanti: Why don't we talk a little bit about — we have retailers that are clearly leaning in right now with a lot of questions. What advice do you have for others who are embarking on their AI and robotics rollout journey? 

Kaye: We repeatedly get questions about how we would start, how we would do. There's a tendency to build a picture and make something really complex, that is actually not that complex. As Josh was saying, we've made it relatively simple to be able to use technology. The key thing is to focus on the scenarios that matter, such as the payment scenarios, get that going, and then be able to understand, get the return on investment, and continue to grow and evolve. That's absolutely key.

The other aspect of this is not to give up. The story with fuel and other aspects that we've done together with Albertsons demonstrates the need to be able to break problems down, be able to work out aspects, overcome each challenge within that journey, and then work at it consistently, collectively, and collaboratively to be able to overcome things. Talk to people who've done it before, it’s quite straightforward.

Guffanti: Mike, Josh, what's next in terms of your business vision and how you're going to apply learnings from the AI and robotics testing capabilities? What other business applications are you looking at now?

Friedel: We're still in the early stages within the fuel, we're probably about halfway through fuel now, and our self-checkout. Those are some of the core capabilities. We also have integrated our point-of-sale with our pharmacy system,which is high on our list to get that automation in place. We've already automated integration with Starbucks kiosks within the stores.