You have heard of Large Language Models, but what can we do with Small Language Models and edge computing? While visiting the Cisco Innovation Center, I had the opportunity to chat with Joel Jacob about innovative ways to bring Machine Learning to the edge with Splunk Edge Hub. Joel provides information about the technology and use cases

Transcript

Auto-generated captions, lightly cleaned. Speakers are not separately labeled.

hey everybody we’re out here at the Cisco Innovation Center and I’ve been talking to Joel about machine learning models and running those on the edge now why would you want to do something like that yeah so to me I it’s very a problem that’s really close to my heart because I actually started uh as a manufacturing engineer in an automotive Factory at Toyota so you’d always be thinking about oh I wish I could bring some intelligence to this production line to help me know if I’m building it better like for example you’re connecting these wire harnesses in the car one of the most common quality issues is that it’s not fully clicked and you want to actually hear that click noise and it’s a loud noisy environment and often time these operators can you know Miss that noise okay so what if I could have a model that could listen for a specific click and be my you know assistant in the factory so it’s yeah it’s very exciting to be like hey this is a space where we’re going to start helping people uh solve problems with machine learning models that’s really cool so we were talking earlier about machine learning models and Edge hub from splant can we do that today can we run these models on a hub yeah so that’s why and we did launch this uh EDG Hub device last year we were thinking okay what what is holding back a lot of uh these Enterprise customers from actually being able to deploy AI at the edge you know everyone’s talking about it but you need some sort of edge Gateway device that has a compute capable to run an AI model and I think you know in the broader space right now everyone’s talking about large language models and these large uh models but there’s also a new emerging Trend really this year we’re going to probably see a breakout in in small language models and that’s really around well we can actually take something more optimized for the use case more def defined and more narrow and actually run it closer to the environment um and now you’re going to say well why do you need to run it closer to the environment um one of the critical things is you know latency if you think about self-driving cars and you’re trying to make a decision based on video inputs you don’t want to be sending that to the cloud and then now you have a network issue and then there’s reliability issues you want to do that on the edge so Edge AI is really going to be uh a big thing that we see emerging more and more this year and also moving beyond just language models into multimodal models right we heard a little bit about it last year but um we’re really going to see more of an noww vision and sound recognition and you know metrics or logs or text is all going to come together to really give you that assistance in your you know Enterprise environment so you’ll be able to build and deploy these models on the EDG Hub directly yeah so um the EDG Hub connects to the Splunk platform so we have a you know software platform it can really like crunch all this data do the analytics um and what we have is the ability to take models from Splunk and push it down to the edge you know this gets into this whole concept of how do you really bring together like you know the cloud and the edge and the continually like updating new models that are coming out and and all that so you need some sort of ml Ops platform um and that’s really what we’re working with Splunk the platform to do is expand that further and further today uh we have shown with several customers ability to take those models that you train and Splunk and you know push it to the edge and also give developers ability to add more function with vision models and other things at the edge even like models you can download from from hugging face so some pretty cool stuff happening right on so that sounds really cool and Innovative where can I go get some more information about how to do all this yeah I think for me as a product manager I would love to hear you know directly to understand the pain points about what people want so I would say hey just send you know me or you a note on LinkedIn um we’d love to to hear from from people you heard it right right here so contact Joel directly um DMS are open [Music]