Christian Hemmeshøj Runge: Automating Success - The Transition from Manual to AI-Driven Testing

This transcription is AI-generated and may contain errors or inaccuracies.

Matt

My name's Matt and I will be talking with Christian Hemmeshøj Runge about transitioning from manual to automated testing and how different tech ecosystems approach problem solving. Hi Christian. I'm really happy to have you here and I don't like really like long introductions, but I think in your case, in case of a SimCorp, the company that you work for, it would be really important for the listeners to understand like what project, the product do you build and what is your role about?

Christian Hemmeshøj Runge

Yes. So starting with SimCorp, it's a investment management technology firm. So it's been around for more than 50 years. We're doing software where the point is to simplify our clients on operations and make them help make better decisions faster. And the clients in this context are some of the world's largest financial institutions like national banks, asset managers, so large financial companies with a lot of money they manage for their clients.

Matt

So highly regulated, let's say fintech banks. So quite a challenging topic. And your area is around the AI. I mean it's the hottest topic and the hottest area right now. So probably everybody is paying a lot of attention on what you are doing.

Christian Hemmeshøj Runge

Yes, yes. Or what we're doing. Part of my philosophy is not. It's not me doing it, it's us doing it. So for me, I'm working in a center of excellence in our product development. So we're about 10 people in my team specifically and we're organized around this safe with the product manager, a director of engineering, which is me, these kind of scrum and safe roles. But the point is not so much for me what are we doing?

The point is how can we get our 1200 engineers to be doing AI? How can we use AI in the company? So I don't want everything AI to be in my team. I want my team to help the others get good at what they're doing. We're still doing stuff, but yeah, that's the point.

Matt

And do you see like a shift in the company itself during the last two, three years around the AI and how the initiatives are pushed and what. Maybe you could elaborate what changed? Right.

Christian Hemmeshøj Runge

Yes, yes. We started with AI or machine learning about nine years ago. And that was mostly experimenting. It was machine learning models. It was figuring out what are the right use cases, what can we do. And that kind of failed because we couldn't get the data we wanted. There were use cases and technology was there, but we couldn't get the data because we couldn't get client data.

Then a few years ago when generative AI came Out those models were pre trained. So that means we didn't have that problem for those use cases and then all the hype around it. So in the beginning it was like we need to look at what is this, what can we use it for. And then going more and more into a production mindset from experimenting onto what can we actually get out there and create value with so much more attention, much more hype, but also actually more value. That's what I will say in the last few years.

Matt

And you mentioned that you have a team and you like to push the whole organization, like all engineering teams and engineers inside the organization to think about the AI and how they maybe they could apply. So maybe you could talk about like your team structure, how do you operate.

Christian Hemmeshøj Runge

So we have, as I mentioned briefly inspired by the SAFE Framework, we have a product manager who's responsible for what should we be doing, what's the business value. We have a director engineering, which is me, who's responsible for making that happen. Then we have an engineering manager for the team who makes sure of the daily operations. In that team we have six full time engineers, four students in that team working together on whatever we need to do. And what we need to do is talking to our legal department for instance about what can we do and not do what should be in our contracts, talking to compliance, talking to platform, coordinating with our partners, co innovating with for instance Microsoft Alchemy. There are several of those. So having that and then the team also works on.

It's called an AI components. For instance around now we're working with a Rack Solutions document search, so searching with LLM in documents and that is a very generic component. So that is something we're doing as the AI team and then we're supporting the other areas with how do you set something up in Asia? What kind of model do you use?

How do we evaluate it? What do you need to consider with cost, with performance, these kind of things. And if they have an idea, is that a good idea or a bad idea? From a technical AI perspective And I.

Matt

Assume in Europe it's quite challenging with the legislation part on AI and data and especially in a highly regulated environment in which you are. And how do you approach it? Like first do you talk with legal and say like hey guys, I have a few ideas and they, you know, drop the bombs saying like this is not possible, this is not possible, you know. Or how do you, how do you approach. Or you first do the proof of concept.

Christian Hemmeshøj Runge

Yeah, it's. I think I'm lucky in this aspect because As I said, SimCorp has been in this business for 50 years. So AI is just a new technology. And what the clients say is you need to keep our data secure and you need to make it available and all this availability, security, don't share it, have some good governance around it. We've done that for 50 years, so we now just need to do that with AI as well. So from that point of view it's not really that hard. So most of my discussions with clients when I'm being called into that is saying that, that yes, we of course need to consider new attack vectors, we need to consider training models and how do we train and share and all that.

But it's not that different from what we have been doing for decades. But yes, when I have a crazy idea, I just go to legal and I say I have a crazy idea, I want to do whatever, when do I need to stop. And then fortunately they see their role as helping me and helping our clients. They don't have this no to everything approach. They have a let's see if we can say yes in a responsible way. And I think that's a very good way to cooperate.

Matt

You mentioned really interesting point before regarding the development of AI, right. So when the new models appeared, we've already pre trained data so you don't have to train your model because before you had the problem that you of course cannot use the data from the client. But do you see some changes in this area that there are already some, some close environments where you can feed the models with the data and create something big? Because this is the huge thing, right? If you have the model trained around the data which client is using or you are personally using inside the organization.

Christian Hemmeshøj Runge

I think there are different use cases. There are the easy ones like usually you and me using ChatGPT for something and then you do don't need to train because it's really good enough to do that. And then there are the other cases where you need to do some prompt engineering or you need to do some fine tuning or you need to do some rag or whatever to make sure it understands the context you're in better, to give you better answers. And then I think we should also not forget about the machine learning models because generative AI, yes that's a hype, that's cool, that can do all these feints in your stuff. But a lot of what we want to do like also outlier detection, seeing patterns in things that, that is machine learning and that is far more efficient and it's Better at it. So we should not discount these use cases and they can be combined. For instance, you are something in natural language that's processed by the LLM.

What you're trying to do is an outlier detection that is routed to a machine learning model and then in combination they give you a good answer.

Matt

And regarding SIM Corp itself and AI initiative and machine learning initiative, because you mentioned new technologies and AI, let's say it's one of those new things that you use and you need to validate with legal team. And during the last three, four years the AI and machine learning like went to mainstream all in with the OpenAI. Right? But I recall my studies when I was doing my master's and it was 15 years ago, we already, you know, 15 years or 17 years ago we already were working in machine learning and AI. And back then it was there. Right. And I'm wondering at the SIM Corp, probably you explore some different solutions around the AI and machine learning for quite a while. Right? So this is not very new topic to you.

I wanted to ask like when you look inside the organization, do you see like a huge, you know, this huge leap in the last three years regarding development of AI inside the organization? I'm not talking about the market, but inside the organization.

Christian Hemmeshøj Runge

Yes, because things are that hype and when market says things like this is going to revolutionize the way we work and jobs are not going to be the same in three years and stuff like that. People who are interested in technology listen, but also business leaders, they go, oh, I need to figure out what is this about? So it has been a lot of this, what is this about approach. So saying, okay, it's going to revolutionize the market, it's going to revolutionize what we're doing. How? Show me, let me try it out, let Me Start Up ChatGPT or Microsoft Copilot as we're using, what can it do? What can it not do?

What does that mean for my use cases? Let's make an experiment, the things we believe in. Let's build it in a simple version, let's give it to our clients because then they can see for themselves what is that? Is that a good idea? Is it a bad idea? What does it cost to operate? And I think a lot of our product areas have started doing that and we have now an AI board, we have the center of Excellence, I'm in legal disengaging in our compliance departments is engaged in it.

So I think there's a lot more senior management is engaged in that. So I think There's a lot more traction, a lot more interest in, in it. So it's, it's not just a, a mathematical component in, in the far back end anymore. It's. Yeah, it has a lot more attention.

Matt

And how about the goals for this year, next year or upcoming years regarding AI implementation?

Christian Hemmeshøj Runge

Yeah, the way we see it, we have kind of three legs. So we have internal efficiency, so making ourselves work more efficiently. So that's things like GitHub Copilot for instance. We're using that, Microsoft M365 Copilot, these kind of solutions where we buy something and we use it internally or build something. Then we have in our interactions with clients, that's another leg, like support cases when, when a client has a blog or a question or whatever, can we process that faster and better? And we now have built solutions for that as well. And then in the product.

So basically how can we make our clients more efficient and better and make better decisions? So what we're focusing on is a client part. Again, we're not discounting the others, but we're saying, okay, clients is the, is the main focus, at least for now. And I think we were really lucky actually a few years ago because we decided that what we had, we have this big, this big platform, end to end platform that is actually quite complex is maybe the right word, at least big. And we're building a new user interface for that. And when we started doing that in earnest, LLMs came around. So we can build that into the way we build the new user interface.

So instead of patching it onto Something we did 20 years ago, we can actually build it in from the start. So that is what we're doing. We're building this copilot approach to this new user interface and say right now you can ask a question and it answers. And then the next step would be things like actually doing things for you, like running simulations or whatever. So you ask you to do something and impose that in the system and next step from that would be more autonomous agents. Maybe not a fully autonomous agents, but maybe you can tell I'm considering investing in, I don't know, Microsoft, what is the sentiment? And then it tells you it's a positive one.

And then you can do whatever you want and then you can tell it, for instance, let me know if that changes. And then maybe in three months it come back to you and say, Matt, you asked me about sentiment from Microsoft, now it's going negative, you might need to reconsider your investments or whatever it might be. So that's the main roadmap for where we're going with that.

Matt

And how do you see, like, what is your personal opinion on the AI in five, 10 years? Because there is a lot of them and I'm wondering, how do you see it?

Christian Hemmeshøj Runge

I might guess on five, 10 months. It's really a good question, because if you asked 10 years ago, the answer would be radically different. Right. I think it's moving really, really fast right now. And some of the industry experts, like Sam Altman and people like him say we will have agents in 25. So AI agents that can work alongside team members in your teams. So they can be, for instance, a developer, like this year.

And then they're talking about, they're talking about artificial general intelligence maybe this year, maybe in a few years, and maybe super intelligence by the end of the decade. So, I mean, that's, that's one end of the spectrum. Then I talk to some people who are doing research in universities and they're saying, nah, there are so many things that these models can't do. There are limitations to their architecture. Yes, they can do impressive things, but try asking them simple questions and they just fail. So maybe, but maybe not. So I think the truth is somewhere in between.

And I think what we need to do is to keep an eye on it. So don't just lean back and say, oh, whatever, this AI is probably going away, but boot up an LLM, see what it can do. Start using Copilot or ChatGPT or Gemini or whatever. Consider, what are you doing at your job? How can AI help you do that? And then play with it, see for yourself, what can it do? I think that's probably the best advice I would give, which is also, again, coming back to what we're doing as an organization.

So start doing it, because then you know, what's it about? And if it really goes bananas in two years, you're more prepared because you know, what is it about?

Matt

And I think the users of the system are really important because they are paying your bills usually. Right. And I'm wondering, in your case, how do you, how do you, how do you get those problems? How do you talk to the users? How do you approach testing your solutions with the, with the users?

Christian Hemmeshøj Runge

Yeah, that's not so much an AI question as it's, how do you run your business question. And that is actually a challenge with the size we have, because we're about 3,500 people in different product areas. Some are people in sales, some are in support, some are consultancy, some are Development. So getting that, we definitely have that knowledge as a company because we're talking to the clients all the time. It's more how do we get that to the right people in the company and who are the right people in a given case? But it's simple answer is we have product managers who for a given area like portfolio analysis, for risk, for order management, stuff like that, it's their job to understand what is the market doing. So they're going to conferences, they have meetings with individual clients and they look at the support cases coming in.

What are, what are they saying? They look at market trends like for instance, AI and then fit all that together. And then when they have something they think is a good idea, they describe it. Maybe if you make a few mock ups, make a demo and then they go to the clients and say, hey, I have an idea, what does that look like in your eyes? What's would that be useful? And then have that discussion. And then we of course trying, of course we are trying to implement that in a simple way and then give it to them so they can start pilot that functionality and give us feedback on is it actually good or bad.

Matt

So usually the, let's say the client's problem is coming from the product manager. So product manager has an idea of a feature and he's like discussing with you like hey, could you contribute to that?

Like could we use the AI? And this is like a typical road, so to say.

Christian Hemmeshøj Runge

Yeah, so the product manager is the person to make the roadmap. And of course if I have an idea because I see something in technology, I go to the product manager and tell him, hey, I have this kind of input. If the clients see that or strategic initiatives within SimCorp that has to go into that planning as well. But they are kind of the, the hub for that information, turning that into what do we need to build.

Matt

And what is your current biggest challenge or the biggest pain point in your role that this may be not visible from the outside?

Christian Hemmeshøj Runge

I think it is visible from the outside. But I think to me I'm initially coming from a consultancy background and what happened there was I went to clients, I talked to them, I heard their problems, implemented a solution and they had it within a month. And that's just not happening in enterprise. I mean again we have 3,500 people, we have literally tens of millions lines of code. It's running in these big banks, investment companies and so on and so forth. They have a lot of integrations to what we're doing. There's a lot of compliance we need to comply with.

So things just take time. And sometimes it's like how hard can it be to add a field to this form? I mean just do it. And then, but what I'm doing is I'm asking how can that be so hard? I mean actually asking. And then people tell me why it can be so hard. And it's fair because you have to put it in there.

It has to be translatable. So I don't know how many languages it has to be configurable, it has to be fit it into the APIs and the whatnot and it's like, yeah, okay, fair enough. It takes that long. And yeah, I mean on the plus side it's more stable for the clients. Again, if you operate a system, how many changes do you want really? I mean do you want Windows to change on a daily basis? Probably not.

So it also has a plus side but sometimes you can just go, ah, get it done.

Matt

Yeah, I, I, I get it. It's like a difference between a startup and when you get going bigger it's the complexity is there and regulation, it's completely different mindset that you need to change and yeah, but I really like to ask the question about the toughest challenge in your career. So I'm wondering in your case, do you recall the toughest challenge that you've got See from.

Christian Hemmeshøj Runge

Yes. Yes. Yeah. After I did my MBA in, in the US for a year, I came back, I took a job in Denmark as a development manager and I came to this department of about 25 people and then being responsible for, for the people there, being responsible for deliveries and so on and so forth. And that seemed fine, it was fine. But at that point in time I joined January 1st, the day before the company did a major reorganization from this area. So that was a change.

Then the project manager for our biggest project, she quit the day before. Then we decided, oh, as part of this reorg, we want to go from a project organization to a product organization. That's a different talk, but that's also a big thing. It's a big mindset shift. And then we need to have this all a new quality assurance system. And by the way, one of our products is a Windows product, so we should probably change that into a web product. So basically like everything was up in the air.

So I don't think I'm being unfair if I said nobody knew what they were doing, they knew what they were doing two months earlier because I mean that was their job. They knew that job well and how to do it. But I mean, so many things changed at the same time and they got a new manager and I was new. So no matter where it looked, it was just, things are not working or we need to define how they should be working or who should do it and how they should do it. And people were stressed out and the clients were, when are you going to deliver? And like, I don't know. I don't know.

I need to figure out. And before that, I was used to being reliable. I like to be reliable. I like to have things under control, at least at that point in time in my life. And I was used to having that and then coming to this place where it's unfair to say it was not under control, because I think it was. But there was so many uncertainties and so many new things that people had to learn and do, including me. So that was hard.

Matt

And like, the lessons learned out of it. How do you like the next time? How would you approach this huge change?

Christian Hemmeshøj Runge

There was definitely some, some time management. I think I was good at time management before I got better. And then this control thing, I think I went from I like to have things under control to accepting they're not. I mean, few things are actually under control and. But that doesn't mean you can't.

Can't change them. You can, you can still influence them, you can still push them in different direction and you can do that more or less.

Matt

But.

Christian Hemmeshøj Runge

But that need for things to be under control is far less with me right now. So more, I'm more inclined to say yes to do something I don't really know how to do. It's just like, I don't know how to do that. It doesn't really feel comfortable, but okay, we'll make it work. And I think that was a big learning from that. I don't know how I would have done it differently now. Maybe been less hard on myself in the beginning.

Matt

And the last but not least, the question that I like to ask all of my guests about some inspirations. I mean, any books or conferences or resources that have been particularly influential, influential on yourself. And you recall the titles that you could share.

Christian Hemmeshøj Runge

Yeah, there, there's been many recently around, how do you work agile and stuff like that, and what, what makes a good company and things like that. But I especially remember the seven Habits of Highly Effective People by Stephen Cowie. That was back when I started many years ago. I really like those. All of those are good. And I have revisited it recently just to double check if they're still good and I still think they're good. I especially like the one with thinking win, win.

So when you go into any kind of collaboration, working with other people, trying to negotiate, say how can we come out of this with both of us being winners? There's no good enough.

You win or I win. It's we need to win because then this will work going forward. And another one is called Sharpen the Soul, which means you need to keep getting better and keep refining what you're doing all the time. And if you're really busy, then you especially need to do it. And that just keeping that in mind that you need to take time to learn and adjust.

Matt

Awesome. Christian, thank you so much for the conversation. I think most of our topics today were around AI, but it is really valid thing and especially important I think for the listeners to understand how the bigger enterprises are approaching it. So thanks for your thoughts and thanks for your time. Thank you so much for today.

Christian Hemmeshøj Runge

Sure. You too. Follow Matt and Leshek on LinkedIn.

‍