DRUID Talks Ep #12 - Part 1 - The Power of Conversational AI with Generative AI Combined: Practical Tips for Organizations

Episode #12 of the DRUID Talks Podcast features two amazing guests: Sorin Peste, Cloud Solution Architect, Data&AI at Microsoft and Daniel Balaceanu, Chief Product Officer at DRUID, exploring the impact of the generative AI fused with Conversational AI on the tech landscape and uncovering new opportunities for companies, while addressing challenges like data bias, privacy concerns, and finding suitable business use cases; they share essential insights on the short and long-term benefits for organizations, offering invaluable advice to ensure exceptional tech adoption with data security and regulatory compliance.

All these topics are orchestrated by our own Subject Matter Expert, Kieran Gilmurray.

Today you are watching part one of this episode, with a second part coming up on August 9th! Watch the full content and transcript below.

Kieran Gilmurray: Today, we have not one, but two special guests!

Sorin is a Cloud Solution Architect at Microsoft, and he has been involved in the IT industry in one way or another since he was ten. From building games for fun to becoming a developer, team leader, consultant, and solution architect, he has done it all. He loves the latest and greatest technologies, including Artificial Intelligence, virtual reality, cloud computing, and the Internet of Things. And today, we get to pick his brain on each.

Dan is the Chief Product Officer of DRUID AI. He has had a long and distinguished career in technology, specializing in Microsoft Dynamics, business intelligence, document management, and of course, fuzing Conversational AI with Generative AI as part of DRUID's Product Roadmap. What Dan does not know about technology is not worth knowing.

And today, I get to interview both Sorin and Dan, on Artificial Intelligence and how it's impacting business, society, and people's lives.

Welcome, Sorin and Dan! It is such a pleasure to have you both on DRUID's podcast. How are you today?

Daniel Balaceanu: Amazing, Kieran. Thank you very much. It's always a pleasure being on this podcast, and looking forward to the discussion.

Kieran Gilmurray: Fantastic. Well, Dan, I'm going to kick off with you. A question popped into my head when I was working on this new podcast episode.

With everything that's been going on since our last talk, will Generative AI actually replace Conversational AI?

Daniel Balaceanu: Definitely not. So, Generative AI is a new source of energy that all of us, humans, receive. It's a gift for all of us. And you know, when a new source of energy comes, there is no replacing the existing machines. It's just making them better. So, totally, it's not a replacement but an empowerment. So, new use cases, new speed, and better speed are in front of us.

Kieran Gilmurray: It's exciting times! Sorin, hello, sir! A pleasure to meet you, by the way. And we were discovering that you've been in technology since you were ten. I'm assuming you're not ten anymore, so it's good to see. We're going to pick your brain.

The first question: you've seen, obviously, a lot of things coming in, going over the last number of years since you've been in tech. How do you see today's AI and the derivatives that have come off the back of that technology? How does it compare to the year 2000.com era?

Sorin Peste: Yeah, hi, Kieran. Thanks for having me on. Yeah, definitely not ten anymore, but it's been a fast, fun past couple of decades, wasn't it since the 2000s? So, I mean, I see a lot of people going to that comparison, you know, the hype and all the kind of go-to the unreasonable promises, maybe, that are being made in relationship to Generative AI. In some ways, it mirrored the expectations of the .com era. And while there may be some similarities to that era, I think there are also some very important differences.

First of all, this is not AI's first time in the spotlight. And AI has this tendency of going through like boom and bust cycles and has been here before. It has been in the spotlight before with different generations. And in fact, during the 2000.com boom, AI was actually in the midst of what we call "winters", whereas there was comparatively little interest in AI, there were comparatively little investments being made in AI during the late nineties and early 2000s.

And indeed, it didn't even start to pick up again until maybe 2012 or so with the new computer vision models making use of the new, you know, deep learning frameworks that were being developed. So, in some sense, AI has been here before. It's come and gone, and now it's back. For one thing, I think this is more of a kind of long-term, longer summer if you want to call it that. It's already about ten years since Alex Nathan and the kind of computer vision models around started to spur interest and investment into AI again. So, it's already like twice as long since the .com bubble went.

We are also in a very different economic environment than the 2000s. We are already in an era where the interest rates are quite high, and investors are actually quite careful currently as to what kind of startups and what kind of ideas they invest in. So, even the fact that we're in this scenario and there's still a huge amount of interest in Generative AI and Generative AI scenarios tells me that we're quite in a different place than we were in the .com years.

Kieran Gilmurray: They were indeed interesting times, Sorin, and I remember them well. And I think that some people listening in will not, obviously, the younger part of the audience as well. For me, it's personally excellent to see that companies are really looking at investment opportunities because, for too long, they just threw money at things hoping it would work. But I think a lot will be surprised that AI has gone through various "winters" because now it feels like the biggest technology in the world. So let's see how it turns out. But let me ask a question to both of you.

Microsoft is one of the biggest players in the technology industry, and it's made a big bet on Generative AI. It complements Conversational AI and brings new opportunities to businesses. But how do you see it impacting companies over the near term and the long term as well?

And Dan, if I begin with you if you don't mind, and then Sorin, if you wouldn't mind answering that same question afterwards.

Daniel Balaceanu: We are discussing right now about an amazing technology - Generative AI, which is very powerful and can be used, in my opinion, in almost any process. Any use case can benefit from it. I've been practising this for a couple of months already, and I am telling you, I am discovering new use cases that I can apply each week. The discussions I have with enterprises, our customers, and our partners, are amazing. There is no discussion of "Can I use it or not?" All the discussions start from, "I want to use it; I have to use it; Help me to use it with risk management, with enterprise governance." And in the projects we have, actually, the respective processes that start to onboard Generative AI are transformed. The agents in the contact centre solve tasks faster. The people in HR complete documents faster; in the procurement, in the legal add these documents faster.

So, companies are transforming their processes and making their processes more effective using Generative AI. So, it's a must thing to do.

Kieran Gilmurray: Aha. Sorin, sir, your thoughts?

Sorin Peste: I agree with Dan. It's just a fact that Generative AI makes Conversational AI a lot more approachable, so it's more of like a boost to Conversational AI. It just unlocks a lot more scenarios just because it's actually a lot better than previous ways to understand what the end user wants, and what their intent is. The previous ways to understand exactly what the user needed to do were quite brittle.

So, a lot of times, Conversational AI and the people who are implementing Conversational AI tended to avoid them whenever possible, and they were steering the solutions towards a more guided approach where, you know, the conversational agent will ask you a bunch of questions to understand what you want to do, and you would have like very limited options.

Whereas now, with something like Generative AI, that is a lot more capable of understanding what you need without having to go through a huge amount of training, which is also very important because it lowers the barrier to entry. These scenarios are now possible, and not only are they possible, but they're actually feasible to be implemented in a way that is still reliable and safe and secure, which was actually not the case previously.

Kieran Gilmurray: Well, with that in mind, again, a question to both of you, and I think you've mentioned security, but there are challenges for CIOs and Chief Product Officers when implementing Generative AI in an enterprise. You know, there are some risk challenges, potential data bias challenges. Data privacy has come up as a concern. And also, there are issues finding business use cases as well.

What are the challenges that you're both seeing and do those resonate with you?

And Sorin, if we could start with you, this time?

Sorin Peste: Right, Kieran. So I would say that right now, finding use cases is not the problem. We're still kind of in the early adoption stage of this technology. The low-hanging fruit has not been picked yet. There are still quite a few areas, and I think Dan already mentioned a few, like HR and procurement and a bunch of others, where it is possible to get moving quickly and get results equally quickly. So, finding use cases currently is not necessarily the main challenge, but I would agree that there are potential issues with data privacy, security, and bias.

Data privacy is one issue that is gathering the most attention. Pretty much everyone I talk to understands that as a potential data privacy issue here, especially if your approach involves models that someone else hosts for you or offers for you, like OpenAI, for instance. There's always the question of, okay, "What does this model retain from the data I presented to it? Is it used to train itself? Is that a potential data leak for me as a company?" Those kinds of questions.

So, I would say that there are some mitigation techniques available, like anonymization of the data and so on, but they only go so far. At the basic level, some level of trust in your providing or Generative AI provider is necessary. You need to understand very well what the AI provider offers in terms of privacy, what data is collected and is not collected, and how the data is processed.

In terms of things like the bias of the models, this problem is less well understood. There is not much tooling out there to test these for biases. There are research papers out there that point out that these models do have quite a lot of biases adopted from the training datasets. But how well those biases or how much they impact your own use case is a trickier question to answer. That's why I would say that having a good practice for adopting Generative AI and having a sort of a continuous improvement approach, something like DevOps for Generative AI if you want to call it, is kind of essential at this point.

And the tooling for stuff like that is only now being developed and made available. So, it's still early days, I would say.

Kieran Gilmurray: Early days, but it's great that we're actually having the conversation around risk and ethics and everything else because it wasn't there 20 years ago, and that's where we were probably creating more problems ourselves. We're not deliberately, no developer sets out to create a biased dataset or biased product, but now we're having these conversations. I'm hoping the awareness helps with that.

And Dan, what about from your point of view? What are the risks that you're seeing when it comes to Generative AI and implementing it in technology or AI solutions?

Daniel Balaceanu: I mean, the facts in front of us, as described by Sorin, are there. Now, the DRUID team, what we are doing, we develop the technology to respond to these concerns and the challenges in front of us. So, we are building the tooling, the toolset, to use in the company under enterprise governance, the Generative AI. So, we provide the AI DevOps, we provide the auditing and looking, we provide support for the entire lifecycle, we provide prompt engineering and architecture for different use cases. So, we understand these challenges well, and we build the tools to respond.

And if just to take, for example, data privacy, I was so happy when Microsoft released the hosting of OpenAI and added their legally binding statements that they are not using the data we shared with them for training the models. They have the option to not look at all; they use it for a particular OpenAI service. So, they are responding exactly with the enterprise for the enterprise needs when it comes to data privacy and things like this.

Regarding data bias and the data context and the knowledge, in my experience, I noticed that in the majority of the projects in the enterprise, the organization looks to surface the data inside the organization, not the data open knowledge from the internet. So, again, a tool like DRUID, the Conversation AI Platform, preparing the enterprise context in a secure way and isolating the problem and stopping the model from hallucinating, it's key.

So, there are tools and solutions to absorb the Generative AI aligned with each particular use case.

Kieran Gilmurray: I love this. This is going to seem like an odd conversation, somebody with risk and governance and privacy and everything else, but these are important points that organizations can't miss too often. AI technology, you know, great to design that, great to throw it out there, and then we forget about all these other items. It's a real sign for me, and Sorin, I was around in the year 2000 as well, if not before. It's a real boon to hear that there's a business case behind it, there's ethics behind it, there's risk management behind that, there's DevOps with security built in by default behind these things because you cannot forget these as a company. Otherwise, you will end up in trouble. And that's never a position that you actually want to be in.

To both of you, though, one more time, because I'm enjoying that there's a similarity in the answer that you are both giving, and there are also subtle differences, which really excites me as well. I love hearing two different experts coming out with their views on this.

What do you see as the short-term and long-term benefits of implementing Generative AI in a company?

Sorin Peste: Yeah, so thinking about the benefits of Generative AI in a company, a short term is going to be a lot of areas where you're boosting employee productivity and just make it easier for people to do their current jobs. It's just much quicker to find out how to do stuff. The way I personally use Generative AI currently is just find answers to stuff, right? I don't want to need to go and find the actual documentation or try to read whatever article about something. I just ask an agent that's already been exposed to that information, and I can just ask them, you know, "I'm trying to do this; what's the best way to do it?"

One of my colleagues was saying that, essentially, you just have your very own stack overflow on your internal knowledge. So, it's just something like, you can ask an expert that's already read the entire documentation of all your products ever, and could just give you the answer like you would be asking someone who has worked with that technology for ten years, with references, with everything.

So, the short term is basically, a lot of the tedious tasks will become much easier, much faster to solve. Long term, I think we will begin to rethink about how we approach some tasks, some current tasks, right? And some approaches that were probably unfeasible previously are now feasible or are available to more people. It's like making an app; for instance, there was a lot of push in the past few years on about low code applications, right? No code/local applications where everybody could make an app just by clicking a few buttons and, you know, maybe configuring it. Usually, that would delve into some code at some point because the application would become a bit more complex, and you would need to understand the platform to make it work better. But now you can just make an app by just describing what you want. And we already have started to see some early examples of this. You just tell this agent in natural language what you want, and they will make it for you. And if they don't make it the way you want, you can just go back and tell them they actually want you to change this application in some ways, right?

So, some of these tasks are going to be approached differently, and they're going to be available to a lot more people that were previously kind of kept out of this loop just because they didn't have the requisite skills.

Kieran Gilmurray: I don't think people realize it. You and I were talking, preparing for the podcast, we were going way back 1990s and one year when you didn't have this expert system in place. You had to phone a friend and hopefully you had a good book of friends to get you the answers. It must be amazing for people who are starting out in technology now, just to have this by default. It's the same with the Internet being able to ask a question. Now you've got Generative AI, a brain the size of the world, able to support you in answering questions for everything that you might need.

Dan, what do you see is the short and medium to long-term benefits of this new technology?

Daniel Balaceanu: In a short time, like right now, any enterprise can give their employees this personal butler. So they can give a DRUID bot for each employee, and that bot is just a simple proxy to the world knowledge embedded in a private open AI subscription. So each employee in the organization, whatever tasks he needs to complete, whatever open acknowledge needs to access, can immediately ask a question or give a phone to DRUID and have humankind knowledge on the phone. So, immediately. And this, in my opinion, will amaze the employees and will grow the appreciation of the employees for their employer that they give them that tool. So, this can be accomplished immediately.

Then, you can transform the customer experience. So any data that you have in the organization and processes can now be faced by the customer in a conversational way. So, for existing applications and data, you just transform them with the conversational experience. And again, this can happen immediately. I'm not saying short admits, immediately, and then, seeing the benefits, you change your strategy, you approach everything you do differently, and you will be amazed to see that things that you did specially, now you can do it differently, much faster, much more productive.

And this is what amazes me about Generative AI, not just how well it understands natural language, but it's able to produce content, and it’s able to execute tasks. I am asking about some insights around my data, it’s not only that my question is understood, but I get a reasonable answer. An answer with intelligence, with computation, and I am presenting with insights that otherwise it's very difficult for me to discover. So it's part of solving the task.

Kieran Gilmurray: I really like that. And I love the definition of a personal butler, which you both mentioned, and this isn't a lot of money at all. This is the extraordinary bit for me, make up a number, but it's not far...

Daniel Balaceanu: And also, until then, you need to be on a particular level in the organization charter, social level, to have your personal assistant. Now everybody gets theirs.

Kieran Gilmurray: I hate to use the word "democratized" technology, but we literally have given, for $20 a month, a butler that's a brain the size of the globe and put it into every application and every process that we've got. My one, at the moment, is just if I was running an organization that makes up a number that was a thousand people, then a thousand people have access to this technology straight away. Because before, we talked about people, processes, and technologies as that triumvirate of things that we need to impact. Now, it's almost people, process, technology, business strategy. Change that based on Generative AI and how you work and the change management program that allows you to incorporate this inside of the organization and then keep iterating those things to create a new and better business. We are in great times. We really, really are!