|
Audio
|
Visual
|
|
Daniel Faggella (Host):
0:08
This is Daniel Faggella and you're listening to the AI in Business podcast where non-technical executives stay ahead of the AI curve. When it comes to AI in retail,
0:18
there's two big categories that we've covered a lot here on the show and that tend to dominate some of the venture funding and the press releases around how AI is overhauling retail.
0:28
The first of which is anything personalization and recommendation.
0:32
Most of this is in an e-commerce environment and the second is in the supply chain.
0:37
Especially after COVID, more people are shopping online and supply chains have been forced to evolve because of changing circumstances.
0:44
This episode doesn't focus on either of those.
0:46
In this case, we focus on using computer vision to understand the behavior of customers inside our stores or the kinds of purchasing behavior and maybe activities of people who are walking by our store being able to collect customer level intelligence.
1:04
Amazon knows who you are based on what you shop for.
1:07
They know every single click you make.
1:09
Tracking people in the physical world is certainly much harder, but there's also potentially a lot of value to be gleaned from doing it well.
1:16
Our guest this week is Kelly Harlin.
1:17
She's a director of commercialization for Sharp NEC.
1:21
Sharp and NEC together collectively have something like 160,000 employees.
1:26
These are two of Japan's largest electronics manufacturers, and Kelly speaks specifically about their computer vision applications in retail.
1:34
In this episode, she opens up with some of the information that retailers wish they knew about people who shop with them physically and in person.
1:42
Secondly, about what AI can drink in about a user, from the cost of their shoes to the way that they walk, et cetera.
1:49
And what that can tell to people who are responsible for stocking the shelves or marketing to their audience in different geographic regions where there are going to be different customers and maybe we should respond to them in a different way.
2:02
Certainly a more nascent use case for computer vision, but likely to be one that's eventually ubiquitous in retail, at least in places where cameras are permitted.
2:10
I hope that whether you're in retail or not, this episode helps to open up your eyes as to where computer vision can add value to your business.
2:18
That is the goal of every episode, to crack open insights that can help business leaders, consultants, professionals see where AI is adding value in the world and give them fresh ideas.
2:27
A big thank you to Kelly for being able to do just that in this episode.
2:30
So without further ado, this is Kelly Harlin with Sharp NEC here on the AI in Business podcast.
[intro music plays]
2:46
So Kelly, welcome to the program.
|
|
|
Kelly Harlin (Director of Solutions Marketing and Commercialization at Sharp NEC):
2:48
Thank you.
|
|
|
Faggella:
2:48
Yeah, glad to have you on.
2:49
I I'm shocked we haven't had anybody from Sharp NEC on the show, certainly not in the last three or four years.
2:55
And you've got some interesting use cases to chat about.
2:57
We're diving in on the topic of retail today and we're solving some pretty specific business problems about understanding the customer.
3:04
Before we unpack the AI, Kelly, can you talk a little bit through what retailers want to know about their buyers?
3:10
We'll talk about the business problem they're trying to solve.
|
|
|
Harlin:
3:13
We're finding, because we talked to customers that a lot of our, our business buyers want to understand more about the customer's lifestyle and and who they are.
3:20
And it's what brands are they wearing, how are they spending their time in the store, where they spend their time in this space and and just more about them as a person, which is about object detection, right?
3:30
So you can detect a logo, you can detect the kind of clothes they're wearing.
3:33
You can detect the color of clothing they're wearing.
3:35
There's just a lot that they want to understand that they just can't get with the current tools that they have.
3:40
And that's what the problem we're trying to solve for them is learning more about their customers and their attributes.
|
|
|
Faggella:
3:44
So at the end of a, at the end of a given month or a week or even a day, we can say, hey, who is in the store, what do we know about them and how could that inform it. Is is the goal here
3:52
basically, let's inform our everything from marketing strategy to our product layouts to anything like what, what kind of decisions does that customer info kind of inform?
|
|
|
Harlin:
4:01
Yeah, absolutely.
4:02
I think from what we're looking to provide is insight into what type of product assortment that you want to provide, whether it's regionally or nationally.
4:09
Certainly every, every area of the country has a different preference for clothing and likes etcetera.
4:15
And then also more about where customers are spending their time.
4:18
So are there peak times people are coming in and spending time, you know with staffing and then advertising, right.
4:24
So, So what kind of clothing, what kind of brands do you want to advertise based on what you're learning.
4:29
So there's a lot that we can we can solve from a marketing perspective, from staffing perspective and a product certain perspective.
4:35
And that's really our goal is how do we provide that information to make those decisions.
|
|
|
Faggella:
4:38
Got it.
4:39
So really cool to to to put on blast here that we are informing many different elements of the business.
4:44
It's not just, hey, what's the rack that we're putting in the window.
4:47
It's so many other things that we can know if we understand foot traffic, we understand things about the people where they're spending time in the store etcetera.
4:53
OK.
4:54
So now we can dive in a little bit on artificial intelligence.
4:57
Obviously computer vision fits into the mix here.
4:59
You guys are in this space pretty robustly.
5:02
Walk us through a little bit about what kind of equipment is involved, what kind of processing happens to turn that I guess footage into insight.
|
|
|
Harlin:
5:10
Yeah, so I'd be happy to.
5:11
So we, we have developed a solution that uses, it’s at the edge.
5:15
So we wanted to start with a low cost computing device.
5:18
So it's an easy entry level just to test out the AI and learn how it you know can benefit from you, benefit you as a company.
5:25
So we are working with Raspberry Pi at the edge initially with existing cameras.
5:30
You can use your own IP camera or you can off the shelf.
5:33
We're using a Logitech camera for a lot of the use cases that we're working with.
5:37
So really looking at inexpensive equipment and be able to do that right there at the edge.
5:41
So we're not providing the cloud services, we're providing the data points and the building blocks for a business or an integrator to pull that data and actually create their own dashboard.
5:51
So, so our goal is really to to provide the solution and then allow you to use your existing equipment or use our computing devices to come with our displays.
5:59
The camera feed is actually being captured by the camera.
6:01
The intelligence is done on the Pi, the computing is done on the the Pi or the SDM that we're using.
6:06
So it's it's not a smart camera.
6:08
The feed goes into the computing device, it's processed very quickly.
6:12
We can do it, our models can do it at the edge on a Pi in your real time.
6:16
So if you're looking at some near real time information you want to gather or you want to do trends, we've got the option to do that as well.
6:23
So it feeds directly in there.
6:25
The aggregation is done as it goes out into the the database.
6:28
And so we're just providing that that analysis tool and then working with the customer to help provide that the database.
6:34
And we have an API, it's a REST API.
6:37
So we tried to make it as simple as possible.
6:39
If you can make AI simple.
|
|
|
Faggella:
6:40
Yeah.
6:41
Yeah.
6:41
Well, I I think a lot of the ways, I mean you probably can speak to this at greater depth, but a lot of the ways that these retailers, particularly folks that don't have gigantic, you know, R&D and data science teams and that's most retailers even enterprise or mid sized folks that they're not necessarily going to want to get their hands dirty, but they do want to have a dashboard where they can have the insights, they want to understand how it works but maybe not have to do all the the hardware and software themselves.
7:04
So now that we talk about getting this set up, we've got cameras.
7:08
I I imagine part of the complexity here and and maybe you can speak a little bit to this Kelly, is where do we set up our array for kind of visual data intake?
7:17
You know, I can imagine there's like an entry, maybe there's something on the outside where we're looking at who's, who's peeping in our windows, maybe we need to put them in specific positions looking down aisles so we get as much rich information from 1 camera shot as possible.
7:30
What does it look like to orchestrate a store to actually drink in insights we can use?
|
|
|
Harlin:
7:34
OK and all depends on what the data you're trying to capture, to your point.
7:37
So if you want to capture dwell and traffic, you could use existing IP cameras that are often mounted in the ceiling because we're detecting objects, so we're not looking at a face so it'll detect that object walking in.
7:47
So you could use your existing equipment at entrances.
7:49
So you could use the an a ceiling mounted IP camera if you want to understand the logos people are wearing or anything about that person from an object perspective you'd want to have.
8:00
That's why the digital really comes in strong here.
8:02
For a lot of the digital displays that are there, you have the monitors and they're right
8:06
as people are stopping and looking at monitors, whether they're looking at wayfinding, they're looking at digital messaging.
8:11
That's just the optimal mounting height to put a camera to see logos people are wearing or the type of clothing that they're wearing or what's happening from an object detection perspective of the customer.
8:21
If you want to be able to see down aisles, again if it is traffic, you can mount them in the ceiling.
8:26
If it's trying to get something from a perspective of a human object, it's got to be something more at that eye level or or head level.
8:35
So we've really been seeing the value and having on the displays because that's where a lot of people are gathering and then a lot of these retailers have displays or transportation has displays.
8:45
So it's really based on the use case, but our goal is to provide as many options as possible.
|
|
|
Faggella:
8:49
Got it.
8:50
OK.
8:50
So yeah, it'll maybe some people say okay dwell time really matters most for us or some of them is we want to understand foot traffic.
8:56
We've opened up a lot of new locations.
8:58
We don't know a ton about demographics.
8:59
We want to look outside.
9:00
So it's going to depend on our on our goals that would that would alter the way that we want to orchestrate what we're up to.
9:05
I guess now we can talk a little bit about how the AI goes to work kind of discerning what we want to discern.
9:11
I'm going to kind of give you my very naive outside perspective and then maybe you can add some luster to it.
9:16
But what I presume here is when it comes to kind of foot traffic and dwell we we can count human beings, maybe we can count even adults versus children, maybe there's some retailers for that matters or something, I don't know.
9:26
But regardless, we can count humans, you know, pairs of feet and we can, we can count what time they're passing in front of our store.
9:34
We can also count where they are in the store with maybe there's a lot of people that are going down aisle one or two, but for some reason for the last two weeks, aisle three and four are pretty barren.
9:43
We can look at that kind of information and that's simply kind of counting people, you know, relatively simple object detection.
9:48
The tougher part here is getting a sense of lifestyle.
9:51
Many of these retailers, they need to know what are the budgets of our buyers, where where are they spending their money?
9:56
We can tell a lot about how much somebody spends on clothes based on what they're wearing.
10:00
Also, different brands mean different things.
10:02
Maybe some are more athletically oriented, some are more luxury oriented, some are, you know, representing some kind of culture.
10:08
What does it look like to train a system, to know that these are Air Force One shoes and that is a shirt with a Drake logo on it for his newest album or something like that?
10:19
Like what.
10:19
What is it?
10:20
What does it take to to build all that complexity into these machines?
|
|
|
Harlin:
10:23
You know, it takes a lot of data.
10:24
So we know when we're building out those data sets, it takes a lot of data and visuals to train on.
10:29
So that's why a lot of the training will be done through video and identifying those objects and showing videos of of the logos etcetera or people or the clothing etcetera.
10:39
So it takes a lot just flooding that algorithm with all this data so it can learn.
10:44
And so we will build out the pipeline and when the pipeline's ready to go, it's continuous learning.
10:49
So once we build it out for a customer, we'll have trained all these data sets and shown the the program and let it analyze like New Balance has two types of logos, they have an N and then they have the NB training on all these different variations, right.
11:00
It takes time and it takes data and you have to flood it with a lot of video data training and then really where it learns and grows.
11:07
And and we saw this when we did it at Infocom.
11:09
When you actually have it in a retail environment or in an environment where you have actual human beings walking around and it's getting a much larger volume of data being thrown at it over a period of time, it it learns and it gets smarter.
11:22
So there is a learning period
11:24
for the smart AI part of the business and especially when you have large data sets that you have to train, it takes a lot of visual data to flood that algorithm to learn about the different intricacies of that object.
11:37
And so it it is.
11:39
And that's why I think people don't necessarily understand or appreciate for accuracy to be there, it has to have learned over time and been flooded with that image to understand its accuracy and and what it is.
|
|
|
Faggella:
11:51
Yeah.
11:51
And I guess, you know, if I think about how this would get solved logistically, there are some vendors in the computer vision space who do all of the CV training themselves and they basically say now we'll plug it in, you're all set.
12:05
And then there's other vendors who say, hey, you, Mr.
12:07
Buyer, Mrs. uyer, you're going to have a different circumstance.
12:10
We're going to have to train off of your information for a bit and make this bespoke for you guys.
12:16
Are you guys going the route of of doing a lot of this apparel analysis centrally and then offering it or is it more different from store to store?
|
|
|
Harlin:
12:25
So right now what we're experiencing, it depends on the needs of the customer.
12:28
We want to make this as flexible, customizable as possible.
12:31
So we're doing so in the case of of logo detection and you have a base of logos that you've trained on, there's going to be some retailers or customers that have specific logos or maybe even their own logos.
12:42
That's where we're going to have to work with them to actually do part of the training.
12:47
So if you have a new brand that's coming out and they've got a new logo, we're working with a brand agency that came to us and asked us, could you do this?
12:52
Well, it's a brand new logo, a brand new brand.
12:53
There's not a lot of it out there.
12:55
That's going to take a little bit more work.
12:56
And there's going to have to be work with a customer directly on that to get the data set trained.
|
|
|
Faggella:
13:00
Got it.
13:00
Yep.
13:01
So it'll be interesting to see how this ecosystem evolves.
13:03
I'm sure there's going to be some maybe use cases that you guys can have out-of-the-box, others that you've got to customize.
13:09
It's going to be sort of a space that that builds overtime.
13:12
But regardless I guess once we once we can understand these things from an end user perspective,
13:17
do you folks sort of work on the dashboards or or are you working with partners that kind of build out the interface?
13:22
Because being able to know how many people had Air Force Ones versus a certain kind of Reebok versus you know flip flops on or whatever.
13:30
And then being able to know how many people had a blazer on versus a this versus a that maybe I can imagine many customers.
13:36
I mean I don't know if they'd want to say this overtly they would literally want to know the dollar value of the clothing on a human skeleton like what is the dollar value.
13:43
And so people are going to want to crunch all sorts of interesting information.
13:46
You know, maybe the people with more money tend to walk down these aisles.
13:49
People with a little bit less might go down these aisles.
13:51
These are all useful bits of info. The visualization of that to support decisions.
13:55
At the beginning of the episode, you brought up some great points.
13:58
You said, hey, this might help us with product placement in in the store.
14:01
This might help us with how we want to stock our shelves in a national, regional level.
14:04
That's very important.
14:05
This might help us with marketing in order to inform that decisions.
14:08
We have to see this stuff in a way that's going to be insightful.
14:11
What does that dashboard look like?
|
|
|
Harlin:
14:13
And that's a great question and that's one of the most important pieces about AI and data is the visualization and really being able to tailor it to your business goals so you can take it and make it actionable.
14:24
So we, we are working currently with partners that are providing that dashboard and visualization for their customers.
14:31
So we've got a couple integrators we're working with.
14:33
We also have worked with a company that was at Infocom that was part of this lift and learn.
14:38
They actually have a cloud dashboard and the ecosystem of partners.
14:42
So they're doing all of the providing all the dashboarding tools and working with us to to get all the inferences set up and get all of that algorithm so they can actually figure out their queries.
14:51
So they've built out a dashboard that we're going to work with them that you could buy off the shelf if you bought their dashboard and their pick and watch program.
14:59
So we're looking at different angles because that is a really important part of this because the data means nothing if you can't visualize it and make it actionable.
15:06
And we also are looking at providing just a basic dashboard with the program.
15:10
So if you're just a smaller retailer, you just want to know what your accounts are, you would have a facial visualization and a dashboard on our Navisense product when it launches.
15:19
But more complexity is going to have to be someone that really like an integrator or somebody that understands how to pull those data points together.
|
|
|
Faggella:
15:25
Understood.
15:26
And and part of the, the fun part of our show here, Kelly, is that we get to look at how vendors are evolving and adjusting to the needs of their customers.
15:33
You know, there are some folks in the hardware space that also are going to be pretty involved in the software.
15:37
There's others that are really going to work with partners for certain kinds of the solution.
15:41
You guys it sounds like might be you might have something off the shelf, but you might work more on some of the the data and the hardware.
15:48
But actually the custom way that this store versus that store versus that store actually uses it might end up being built out by a partner.
15:54
So that's useful for our listeners to know that sometimes the way vendors kind of interface with the market is, is through channels along those lines.
16:01
So I guess Kelly this will take us into our last question.
16:03
We've painted a pretty strong picture here of these initial pilot programs you guys are running in this domain.
16:09
What it potentially opens up for retailers.
16:12
Obviously there's a dashboard and insight element here.
16:15
At some point this stuff will become a little bit more normal in the retail world and it's it's potentially going to change strategy.
16:21
When you look ahead to the future around where these use cases are going to fit in and how they might change what retail looks and feels like, what are some of the things you wish more leaders understood about the future we're headed into?
|
|
|
Harlin:
16:33
I wish they understood,
16:34
I think that there's it could be overwhelming to think about AI and data and in the past the way the solutions were available they were very heavy and expensive and complicated and and they were very they're proprietary. So they're we're providing building blocks and I think that it's important for leaders to know, objects are extremely valuable and understanding those objects.
16:54
So detecting objects from a security perspective or logo perspective there's so much richness in your environment and and it's it's not invasive and it's it's not a security concern because it's really just objects but objects tell a big story and so computer vision can detect an object and we can bring that into a dashboard and we can collect all this information.
17:14
But it's all about what is your use case what business problem are you trying to solve like we talked about at the beginning of the podcast and and I think that data can help solve those.
17:22
It's just being really crisp and clear.
17:23
And what do you want to solve?
17:25
Yeah and yeah and and understanding what's possible.
17:28
And that's what our conversations we've had with our customers.
17:31
We start basic with dwell and people count.
17:33
And as we go through that, the light bulb kind of goes off and they say, oh gosh, we can understand for collaboration, maybe conference rooms or transportation, what kind of bags people are carrying, how many bags are going through the turn.
17:42
Like there's so many things that come out and you start to talk to them and business leaders and then they value, right.
17:48
It's, I guess it's flexible and it's more possible to evolve and customize than ever was before.
|
|
|
Faggella:
17:54
Got it.
17:55
Well.
17:55
And again, as a vendor, the hey, this stuff is great and it's super easy.
17:59
You know, it's it's somewhat of a message that isn't horrendously shocking.
18:03
But the takeaways here that I'm kind of picking up from you #1 is that there is really a lot of customization leader to leader, and it's going to depend on what they want to understand.
18:12
And I think one of the things you did really well here today, Kelly, is basically paint the picture of some of the things that we can detect and what that would mean.
18:19
And hopefully that will open up the imagination doors for some of our listeners to say okay, what are the things that I can't really quantify right now, but I would love to quantify?
18:26
It almost sounds to me, Kelly, on some level like and and it's such a long time coming and it's not going to be an easy road.
18:32
But at some point retailers will be able to do some of the cool stuff that digital retailers can do. Physical retailers,
18:39
in other words, they'll be able to say how many people were in the store, what did this user buy?
18:42
What did this user buy?
18:43
It will know things about those users.
18:45
You know, Amazon has this great advantage that I have to log in when I'm using Amazon and they get to customize my whole experience.
18:51
Retailers obviously there's so much more hardware, software to actually get them up to the same level.
18:55
But it sounds like shifting towards that level of responsiveness is kind of where we're going.
|
|
|
Harlin:
19:01
Absolutely.
19:01
It's how do you provide for brick and mortar the same kind of metrics and data and insight that we're getting online.
19:07
And and that's been it's a complex problem to solve but I think we're heading in a good direction.
19:11
And and I also feel like my goal is in road
19:14
mapping this out is really listening to the customer and what do they need because if we don't have use cases and and provide data sets and need something to somebody to provide value, it's not worthwhile.
19:24
And I think that's a big part of it too is in getting engaged in the customer.
|
|
|
Faggella:
19:27
Figuring out., yeah, exactly how would they use it and where.
19:30
And there's so much still to explore and evolve in this space.
19:32
You know, taking the physical world and making it as instrumented and information rich as the digital world is a is a high challenge.
19:38
But there's much, much, much more at kind of a SAS accessible level now.
19:42
And I think hopefully today some of our listeners have a better idea of what that stuff is.
19:45
So Kelly, this has been a great Tour de force of some new cool use cases around computer vision.
19:50
Thank you so much for being able to join us.
|
|
|
Harlin:
19:53
Thank you for having me.
19:53
I enjoyed it.
|
|
|
Faggella:
[outro music plays]
20:03
So that's all for this episode of the AI in Business Podcast.
20:06
A big thank you to Kelly, and thank you to you, our listener, for tuning in all the way through to the end of this episode.
20:10
I hope that.
20:11
Some of the use cases around how data can be collated and visualized about moving customer traffic, foot traffic that is, are things that maybe are jostling some ideas about in your own mind.
20:22
I imagine from physical bank teller locations to retail locations to gas stations, this kind of technology might come into play when it comes to what we're going to stock the shelves with and how we're going to market to our customers.
20:33
Every geography is different, and Kelly makes a good point in being able to shed a light on how this technology might be useful in the years ahead.
20:41
If you're listening in and you want to know more about retail use cases of artificial intelligence, you can download our executive CHEAT SHEET for AI and retail, which you can find at emerj.com/RET1.
20:53
One that's RET like retail and then the number 1, emerj.com/RET1.
21:00
There you can download our AI and Retail Executive CHEAT SHEET, which covers two things.
21:04
Not only a quick lexicon of some of the most important terms to understand for AI and retail, but also a nice handful of use cases to give you a representative splay of where AI is making its way into the retail ecosystem today, both online and offline.
21:19
It's a synced resource.
21:20
Hopefully it's one that's useful for you.
21:22
emerj.com/ret1
21:25
That's all for this episode.
21:26
Thank you so much for being able to be with us.
21:27
As always, I look forward to catching you on the next one.
21:30
You're on the AI in Business Podcast.
|
|