DRUID Talks Ep #12 - Part 2 - The Power of Conversational AI with Generative AI Combined: Practical Tips for Organizations

Episode #12 - part 2 of our DRUID Talks Podcast features the same amazing guests: Sorin Peste, Cloud Solution Architect, Data&AI at Microsoft and Daniel Balaceanu, Chief Product Officer at DRUID, this time exploring how companies can develop AI applications with excellent technology while ensuring data security/privacy and regulatory compliance, and offering advice on adopting generative AI to improve organizational processes. They also debate the potential of Generative AI in creating value and transforming the future of work and talk about trends and their vision for how the technological advancements will shape their industry and impact the world in the company of our host and SME, Kieran Gilmurray.

Kieran Gilmurray: Welcome back to the power of Conversational AI with Generative AI, combined - part two, today! We have Sorin, the Cloud Solution Architect at Microsoft, who has been involved in IT in one way or another since he was ten years old. He has seen everything that has happened over the last 20 years and built games for fun. He's been a leader, a consultant, a solution architect. He's told it all. He loves the latest technology: Artificial Intelligence, Virtual Reality, and the Internet of Things. And today, we get to pick his brain for a second time on all of that tech and the future of work.

Dan is the Chief Product Officer of DRUID AI. He has had a long and distinguished career. It doesn't mean he's old. He's just had a long and distinguished career with technologies and specialities in Microsoft Dynamics, CRM, business intelligence, document management, HR, and of course, Conversational AI. Now, fuzing Conversational AI with Generative AI is part of DRUID’s product roadmap. What Dan does not know about technology is not worth knowing.

And today, I get to interview both Sorin and Dan on Artificial Intelligence and how it's impacting business, society, and people's lives for a second time. Welcome, Sorin and Dan! It's such a pleasure to have you on the DRUID podcast once more!

Sorin, how do you see or how do we ensure that companies approach developing AI applications so that they deliver excellent technology?

And to go back to earlier on, but they do this in a way that's secure, and that privacy and regulatory compliances are all followed.

Sorin Peste: Yeah, that's a great question. That's a great question, especially in the context of Generative AI, which kind of requires some very, very large models behind it. And it kind of depends. The answer to this kind of depends on whether you're using your Generative AI as a service from somebody like OpenAI, or you try to roll your own models inside your company. Both approaches have advantages and disadvantages, but I think using them as a service will be the way to go for a lot of companies in a lot of scenarios, simply because this is a much lower barrier to entry, and the time to value is almost instant compared to other types of projects or projects in general.

So, again, if you're using a service, you need to understand, you need to do your homework, you need to understand what data is collected, where the data is stored, whether you have an option to opt out of data collection at all or completely just to reduce the potential data exfiltration in the scenario, and how the data is actually being used by the models that you employ. So, this needs to be a process that should happen upfront. This is not something that you can just say, "Okay, we'll figure out later," it doesn't work like that. You should also consider some other mitigation techniques like anonymization of data, removing personally identifiable information, anything like that should ideally be scrubbed from your inputs before you present that data to these models. These services are backed by those large language models.

Obviously, if you try to roll your own models, you may be able to keep that data completely within your organization, but then you have the disadvantage of not being able to iterate as quickly. It may take longer to reach value, and there's always a tradeoff to that, right?

Kieran Gilmurray: Is there, sorry, because we mentioned here the two parts just... if everybody uses globally available data at the globally available models, then technically they're going to get the same answer subject to how good a project engineer they are. If you want to be distinct and gain a competitive advantage, then you use your own IP, you use your own knowledge that's been built up over the years, and you use your own team in the processes or the methods that create a distinct competitive advantage.

Is there a deciding guideline or set of rules that you've come across that says „go generic” or „go specific” and „go unique” or „go hybrid?” How would you advise companies at the very, very start as to which to choose?

Sorin Peste: Yeah, I wouldn't say this is a very friendly decision tree chart for you to look at, but usually, in order to make use of that technology in your business scenarios, you will need to expose it to your internal data. Now, there are multiple ways to do this, and what's been happening recently in the space is that you no longer need to expose that data to the model directly or say, maybe train the model with your data. You can just let it know that the data exists, and the model can take a look at it and then incorporate that knowledge into its answers. And you can also tell it to only use that information, your private information, when generating that answer rather than just relying on the global knowledge. This is a process that's called "rounding," and it's usually accomplished through prompt engineering and a few other techniques. And it is, in my opinion, basically the deciding factor between a great solution and a merely passable one.

So, pretty much everyone that I've seen will eventually go to expose their own data to the model. But how you actually ground the model into only using that data and giving you what you want from it is a process. It's not something that you can just take for granted.

Kieran Gilmurray: And that's maybe where people need that little bit of guidance at the start to help them set things up correctly or configure correctly.

Sorin Peste: That’s right.

Kieran Gilmurray: Dan, what would your advice be for organizations wishing to adopt Conversational AI and Generative AI in their organizations?

Daniel Balaceanu: So, from my perspective, the first thing is to be okay with consuming Generative AI from security, governance, and data privacy. So, put the technology in place to allow anyone in the company to use this in a governance-managed way. So, live this data privacy and concerns aside.

And then, to unleash this technology to anyone interested, not to keep it for a few technical people to experiment because the technology is so powerful that with an enabler like a Conversational AI platform or DRUID virtual assistant, it can be in the hands of any person in the business. And start experimenting and discovering use cases.

Myself, I discovered, like a couple of tenths of use cases, but now I am reading books, presenting 20,000 use cases. So, I would allow any person in the organization to try to use Generative AI in their daily tasks and build, like a knowledge inside the organization, like a library of prompts and use cases that can be shared. It's a tool. It's a gift that can be delivered to any employee.

Kieran Gilmurray: Let me ask. I'm going to separate the next two questions, like just very quickly to both of you and quick answers, if you don't mind.

Generative AI, we hear a lot about job losses or value creation. Where do you sit on the fence, Sorin? Which is it, job creation or job destruction with generative AI?

Sorin Peste: Yeah, that's a good question. The way I look at it, the way I like to frame it, is that I don't think that AI will replace people, but I think that people using AI will replace people not using AI. Uh, so it's going to be an enabler rather than a destroyer. I think that some time-consuming tasks will become less so. They will be much easier to accomplish by the same people, much faster with fewer errors, and much less need to redo things. There will be instances where specific tasks will be attainable by non-specialists. I've given the example of, you know, conversational language to an app, and I think that's one scenario that I'm really excited about.

There are also new jobs that will be, and are, created at this point. Even right now, we have the prompt engineer, which I think is becoming a sort of a job in itself with its own skills and maybe qualifications.

I won't say that there will be nothing that will disappear. There will probably be some things that will go the way of the, you know, carriage builder or those kinds of activities that will be performed better by some of the AI.

But overall, I think this is just the main outcome of this, making us a whole lot more productive.

Kieran Gilmurray: Dan, same question to you, Generative AI, job, destroyer or value creator?

Daniel Balaceanu: I am sharing from my projects and working with my customers. When we started DRUID, one of our concerns was how the employees will look at DRUID and automatization. They will be afraid that it will replace the job, and that will push back the technology.

And what is happening is the opposite. There is a queue of employees saying, "When will I get my robot?" Agents in the contact centre are not pushing the DRUID bots, being afraid that they will lose their job. It's the opposite. I want a robot because I can handle more calls. The HR persons, the same. Procurement, financing, legal.

So, it's a demand to have these tools. It is not a worry that this tool will replace my job. Not at all.

Kieran Gilmurray: Yeah, I love hearing that. I have to say, my use of it is it's just augmented what i'm doing, allowing me to be a lot more productive and better at what I'm doing. And that's exciting, exciting for everyone.

Final question. We've seen the year 2000, we have seen AI, we've now got Generative AI, we've now got conversational business applications and a whole host of exciting technology that just wasn't around a decade ago.

What other emerging trends in technology to you both find the most exciting and how do you envision them shaping the future of the industry in the world? And if I begin with Sorin in first answering that question, what tech's exciting you and what's coming that we should be aware of?

Sorin Peste: So, I'm going to stay in this AI field since that is the one that I'm basically living my life in for a few years now. I think that Generative AI has the potential to unlock some of the capabilities of some of the other AI trends and some of the other AI technologies that were actually in the spotlight just a few years back, just before Generative AI started to take over a little bit.

So, the example I'm thinking of is reinforcement learning, which is a technique where you have this agent, this autonomous agent that is learning to do a task, a general task, just by trying and failing at it multiple times. The most well-known example of this is AlphaGo, which was a go-playing robot or go-playing program developed by DeepMind that basically taught itself to play a very, very complex game, which is a, which is go, during several weeks of self-plays. Essentially, it just played the game with itself many, many times and learned the rules of the game and how to play better and better just by doing it. And that's the program that got to such a level that at some point it defeated the reigning Go World Champion.

So that technology has not gone away. It's still around, but it was essentially kind of lost the spotlight a little bit, I think, because of the fact that there are also some limitations to that technology. There are some, let's say, some things that it's not very good at. And I think Generative AI can complement that very well.

So things like Generative AI and reinforcement learning are techniques that can work together now to mitigate each other's weaknesses and build something that we can just imagine for right now. Just imagine some agent that learns to do a physical task like a robot, a physical robot that really learns to do a physical task not just by trying and failing at that task, but also getting verbal input from a trainer or being able to talk to a specialist in natural language about that particular task. So just imagine that kind of scenario. I think that's really exciting, and I think we'll see more of that during this decade for sure.

Kieran Gilmurray: Someone said recently, „If you can imagine it and you can describe it, then Generative AI can create it.” And that sounds like what you're describing there.

Sorin Peste: Yeah, exactly.

Kieran Gilmurray: Dan, what technologies are exciting the life ahead of you at the moment, and what do you see as an emerging technology in the next number of years, getting you even more excited?

Daniel Balaceanu: I am very deep in the Conversational AI space, and in my opinion, the conversational will be the new operating system, the new operating way. So, always when we will interact with a device, a system, a data, an application, we will just talk and chat as we do with humans. And this will unleash all the existing functionalities and all the current tasks to accomplish them in a different way.

Generative AI and the large language model will continue to be better and better. And not only that, they will understand what I want to ask, but they will be able to complete more and more tasks, and all the providers in that space will just use this new technology. So, all the existing applications will transform and become conversational.

Kieran Gilmurray: It's interesting you're saying that because we're so used to using keyboards, but it's not the most natural thing in the world. The most natural thing in the world is a conversation, although I like both because someone who isn’t able to speak can actually...

Daniel Balaceanu: You see what happens with the mobile phones, they used to have keyboards and then they had touchscreens and there is no way back.

Kieran Gilmurray: There is no way back, and now we're talking to them.

Daniel Balaceanu: I remember my kids, they go to the TV and start sliding with their fingers on the TV, and nothing happens. They go, „The TV is broken”.

Kieran Gilmurray: Now people talk to the TV, and nothing will happen. And then they'll think the TV's broken as well.

Daniel Balaceanu: I will go and talk with my HR Application on the desktop. This application is broken. It's not reacting to what I'm saying to it.

Kieran Gilmurray: It's an exciting time to be involved in technology!

Sorin, Dan, thank you so much indeed. Like what we've discovered today, there is a whole new world of technology exploding beneath this, and everybody is exploring this technology in their own unique way. Sorin, I think you mentioned, or Dan, there are 20,000 use cases so far. That book you are reading must be a very big book.

I think the future is exciting. It's what we make of it, as long as we do it in an ethical, compliant, risk, and regulatory secure way. The times like we are now doing that, then giving everybody access to this technology to allow them to explore, develop, create, and do new and better things in a way that augments, not replaces them, leads us to a very exciting future over the next couple of years.

Thank you both very much indeed!

Daniel Balaceanu: Thank you for having us. It’s a pleasure to be with you all.

Sorin Peste: Yeah, thank you, Kieran, and good talking to you and Dan. This was a really fun podcast.