Empower 2021: Fireside chat - Model-driven DevOps – What Is It?

Hello, everybody. This is John Pocknell with Quest Software and welcome to this Fireside Chat. I've tried to do my best here given that it's a virtual fireside chat. So I've got a background here. Hopefully, you can see this with a fire. Vamshi and I are ready to get started. We've got our mugs of hot chocolate and we're going to get going. So first of all, I want to thank Vamshi Damera for participating in this Fireside Chat. And we're going to talk about model-driven DevOps at Quest Empower. And we also want to thank you, the audience, we want to have you participate in this discussion as well. So feel free to ask questions of either Vamshi or myself as we go through this discussion on model-driven DevOps. And we'll have some questions and answers at the end of this session. But feel free to drop questions in the chat because we know what it's like. You think of a question at the beginning and then you forget towards the end, right? So if you think of a question you want to ask, just drop it in the chat and then we'll go through them at the end. So just to lay out a little bit of structure, first of all. We're going to start by talking about Vamshi's background, his current experience, and his previous experience, which is heavily weighted towards DevOps and Enterprise architecture. And specifically, we're going to start talking about some of the business goals of DevOps in general. But some of the other challenges that companies face when they're trying to align business and IT together. We hear about DevOps a lot, but we tend to hear it in terms of involving IT people, and not so much the business people. And that can lead to some challenges, which we'll get on to. And then, we'll then feed that into the conversation around what model-driven DevOps actually is. And by the way, you may have also heard the term biz DevOps, which is a term that's been around for a while, otherwise known as DevOps 2.0. You may have heard that term as well. And they are all pretty much the same thing in terms of the need to bring business and IT together when we're talking about software delivery, OK? So Vamshi, how are you? Good. Welcome to this session. It's great that you're back and talking to Quest and customers again. This isn't the first time we've done this, right? Right. Thank you so much for inviting me. It's a pleasure to talk about these topics. Pleasure's all ours. So Vamshi, just for the audience, could you first sort of describe your current role? I mean, we've known each other for a few years now. And you're in a new role-- relatively new role as Enterprise architect at the Washington Metropolitan Area Transit Authority. Can you describe what your current role involves? Because it's a little different to your previous role, right? Right, so my role at Washington DC Metro is to evaluate their ERP solution. So they've been heavily invested in their ERP solution for past 15, 20 years. And now, they're in that stage where they want to reevaluate their ERP roadmap as to where they want to go from there. Switch the vendors or get-- is it worthwhile to switch the vendor? Or they stay current and keep their ERP versions current or migrate to the newer version or migrate to Cloud. And to do that, I was-- I'm chartered to capture all the business processes, capture all their interfaces, which go to and from ERPs. Capture all the customizations in their ERP systems. And any third party, how they integrate into the ERP system. So my role was to get a entire picture from business perspective to the implementation perspective. And how do we move towards-- and the marketplace has several other ERPs-- competitive ERPs, which would be a fit for their needs. So my role was to evaluate this-- capture all the data-- metadata from functional to technical to implementation. And then, draw a road map, which fits them. Right, so you're actually talking to both the business community and the IT community, right? Correct. Right, and I understand that there's a team of enterprise architects there as well at DC Metro. Yes, there are. OK, good. So we'll circle back to your current role in a second. The other thing I wanted to focus on as well-- because I think the audience will be really interested in this. Because I'm not assuming that the people listening in are sort of [INAUDIBLE] with DevOps particularly. They may be involved in beginning the path towards adopting DevOps. DevOps is not something you just switch in overnight. That's obviously something which is a cultural change within an organization. So you've got to have people in different functional areas collaborating together. But then, you've also got to combine that with automated processes. And along with those automated processes, you generally have-- you look for tools that will help you build an efficient software delivery pipeline with automated processes that can get the whole thing going. So it's typically around the cultural change involving people, processes, and technology. So just on the subject of DevOps, let's go back to your previous role as well. Because you were involved with a number of DevOps projects as well, right Vamshi? Right, I was. And with those projects, there were obviously some business drivers behind that because you were involved in implementation of-- particularly the database side into-- which I think was an existing DevOps infrastructure. But the piece that was missing was bringing the database change management processes into that. What were the biggest drivers behind that? Why did they go down the road of DevOps in the first place? So I was working for Hartford Insurance, a 200-year-old company. In Connecticut, right? In Connecticut. And then, had a huge applications-- a huge number of applications and their life cycle was very slow. And they were very-- the applications were 20 years old, 15 years old. And change was really tough. And the Hartford management team decided to move to Agile. And it was always a challenge to do a change for those systems because there's the change-- the application development team would do some changes-- the database team. There were several silos of these teams. Not able to talk to each other. So it was a challenge to do a change. And even the change which was propagated had to-- had a lot of issues because there is no consolidated pipeline to do these changes. So there was significant progress made by the application development team to do their DevOps, but the loose link in the chain was always the database side. Me being a manager for a significantly larger database team, I was chartered to incorporate the database changes as part or move the database changes into the DevOps pipeline. That's where my role got-- So that's a pretty significant challenge for anybody, right? To try to do that. Because typically, we see with database development, particularly Oracle side, where you've invested in database code. Trying to orchestrate that inside of a fast moving pipeline is a challenge requiring some pretty significant changes on the part of developers and DVAs, right? Correct. A significant challenge because always database behaves-- the understanding of database behavior or database changes versus a code change was lacking. So the perception of that change vise versa, there has to have a common ground between those development areas, operations, and database teams to be able to think in line and have some kind of creative ideas as to how do we balance all these things. There are security concerns from the database side. There's fast changes requirements from the business inside. with application teams. They are concerned about the break fixes or the fix needs to be tested thoroughly before they get implemented. So there were conflicting requirements, which are all for the good-- for the goal was good to implement a change. But there are conflicts while doing that change. So there was a common understanding-- common language needs to be spoken across all these teams to make that change or adopt the DevOps culture in the team. Yeah, I mean, coming back to the business needs. So as I understand it, the typical release cycle for these database changes was somewhere around eight weeks. Is that right? Right. But the business goal was to align the releases so that they were getting value every two weeks. So you were basically looking for a release cycle of two weeks, which is requiring a 4x improvement on your delivery time. Is that right? Correct. [INTERPOSING VOICES] [LAUGHTER] That's a huge change. So there were in-built tools available, which we had from [INAUDIBLE] where we could increase the testing process, increase the efficiency, bring all the repositories together. And that way we could-- we were bringing all the changes into one pipeline instead of having two different pipelines, one pipeline for application changes, one pipeline for database changes. And eliminating all the manual process of doing the change. Put everything into one single pipeline. Have the checks and balances as part of the pipeline instead of having manual checks and balances. That really built the confidence in the teams that the change can be automated. The changes can be faster. So over several iterations we achieved that goal of two weeks [INAUDIBLE]. Phenomenal. And you're already using version control. But this kind of change required your-- [INAUDIBLE] effort on your developer's part to actually set about doing automated unit testing and automated QA. Because there was some pretty significant goals there, too, in terms of code delivery. Correct. So to improve the-- there was-- the defects on the code-- code defects that-- we've measured the code defects. And after a change goes through, how many defects are noticed or recapture-- capture the technical debt, the percentage? How much technical debt is there? How many are obsolete code exists in the database? How much code is not? There is no unit testing for those code. So by capturing all that information, we could reduce the technical debt in the code. So one other question, then we'll switch-- start switching back to your current role. But just one final thought because one thing we often hear about is when implementing CI/CD pipelines for database changes, we get a lot of skepticism from DBAs, who are, after all, they're the gatekeepers for the production systems. And they're a little bit suspicious about automating things that are going to ultimately change data structures and data in the production system. How were you able to overcome that at the Hartford? So we implemented-- for that, we implemented several different database schemas so that the changes that-- we increase the number of environments in the lifecycle so that a change-- environment is for that change, right? So we have a QA environment dedicated to QA. We have a unit testing environment dedicated to unit performance testing to perform and testing. So we isolated all the SDLC life cycles so that the environment cannot be interchanged. That was happening so that way the sanctity of that environment was retained. So that built the confidence on the database. And then, of course, the unit testing, the checks and balances. Audit logging. That all increased quite a bit. And there are tools available there and the changes are captured and the logs are captured and deciphered as to whether the change took place was the relevant change. That's what we increased of that monitoring through the using of pipelines or the delivery pipelines. That built the confidence in the database-- DBA community. Absolutely. Because I mean, the DBA is obviously-- the data is the sanctity, right? That's the crown jewels of the company, right? So anything that's going to ultimately risk the data. And you can understand their concerns. So it's good that you have these different systems and that gradually built that more and more confidence. So one of the things that got me thinking about model-driven DevOps, biz DevOps, and so forth is this DevOps approach where we have, obviously, cultural changes bringing together of people working collaboratively. As I said at the beginning, generally only involves technical people. And we still have this issue of the business being left out of the equation. So one of the things I want to transition to now is in terms of what we hear around accelerating the delivery of value to the business. We still hear about some projects fail to fully deliver what business wants because of misinterpretation or misunderstanding of what was required in the first place. So given your current role within DC Metro, tell us how things currently work there in terms of how the business articulates what they need in the first place. But then, subsequently what sort of changes they want. And how does that feed into the DevOps pipeline from an IT perspective? So there is a huge gap in the way the data is consolidated, right? So Enterprise architect, he draws the pictures in Visio, defines where the process-- business processes are. Where the infrastructure items, where-- which objects are dependent. It's a high level pictorial of a solution. And then, there are functional requirements which are defined by business people, OK? I want a feature or this feature or that feature, whatever it is. And they reason out why that features. Then, that gets translated into-- somebody has to evaluate, OK? These changes should go or not go. OK, if they go, then the flow goes to the an application development team. And they will decide on the technical aspects of it. OK, this is the changes. What are the technical specs? What is the impact of that changes? So generally, this pathway is for enhancements, OK? Now, there's another different types of changes where bug fixes where it doesn't go through that whole process, right? They start from the technical side of things. Right. So now, there is a well-documented enhancement-- technical design document. Everything is written. Now, then the next step is what happens to the bug fixes, OK? The bug fixes, they've identified there's a bug and that has to be fixed. It needs to be fixed in a faster fashion at the time because the timeline for a bug fix is much shorter than an enhancement. So now, that's where the shortcoming comes in, OK? The development team goes and fixes the bug. There is a tendency of not updating the entire documentation from the architecture level to-- so that gaps have been evident and have been found. And now, the bug fixes always don't look at the entire scope or entire-- what kind of research-- [INTERPOSING VOICES] --because of that bug fix. So that's definitely a gap in most of our organizations I've worked for and especially the one which we have. So that's where we are trying to get a one place or one source of truth where end to end everything can be visualized. There are no tools in the marketplace that I know-- only few-- I would say, only few tools in the marketplace which would give end to end visual of an application. All right, so just summarize what you just said, though. So as I understand it, then, you've got business architects that describe what the business needs in a business language. And then, yourself and the Enterprise architecture team work with IT to help interpret that information into a technical language so that they can develop a technical specification which they can then give to their development teams in order to understand what the business actually needs. But at the same time, you've got to help enable the necessary technology to operationalize those capabilities value chains as well, right? So you're very involved with the infrastructure. But one thing you did say is that there is no central repository of information. There's no sharing information. You've got the business user. Can't believe they're using Visio. I guess there are other visualization tools as well. And then, you've got the IT folks using their own tools as well. And never the twain shall meet. So that's a problem. And the other thing as well is that the business uses-- the business community is using business semantics, right? So they can't express, actually, what they want in a technical language because they don't understand the technical side of things and vise versa. So you've got language disparity as well, which is kind of accelerating the problem. And then, you hit on another interesting thing, which is a flowing of information both ways, right? So you've got the business changes going one direction. And then, you got bug fixes and other things flowing in the opposite direction and they're not meeting in the middle. So you get this disjointed approach. Now, have I sort of understood that right? Correct. So let's talk about some of the consequences of that. I mean, in your experience, and you mentioned that you've seen this in other companies as well. What's the consequences to business of this disconnection and no sharing of information between the two different communities? So not having a consolidated information always leads to a misinterpretation of the requirements or the code changes. So that makes the change-- being accurate. There's always that room for breaking or missing-- getting represented or changes which are not intended gets-- goes into the production system. So that then causes disruption downstream that I've seen. Because when the business actually sees the end result and they say, well, that's not exactly what we asked for or what we want changed. Then it has to go back again. It kind of-- does that defeat the whole object of needing to go to Agile? Correct. So there is a significant percentage of changes if they don't go through that process. There is another subsequent change to the change. The number of changes increase. And at Hartford, that's what we wanted to do is reduce the number of changes, OK? So that's what we achieved doing. OK, so we've talked about the business requirements feeding into IT so IT has a better understanding of what the business needs so we minimize that disconnect. But in the world of cybersecurity and regulation compliance and data access and things like that. What about things like data governance? So making sure that business IT have their own perspective of the data so they understand how data flows throughout the enterprise. What the interdependencies are, I guess, between the data and the applications that they serve? And how to ensure regulation compliance. So these have to conform with CCPA or GDPR or something like that. In your current role, how does-- is that something that's being addressed as part of everything else? Yeah, I mean, that's the regulation guidelines that we have to adhere. Is it a challenge? Yes, it is a challenge to be able to adhere to all this data privacy regulations. And it's the challenge that there is a raise of this-- the breach in data. And always, it's the Holy Grail, as you said, right? So there are several strategies that you've been using. There are several technologies available. I don't know whether they've matured enough to do some of them. Most of the organizations are using in-house scripts to do masking data. There's no standard way of identifying confidential data and providing entire Enterprise wide view of risk, at least in our place. I don't see that being done. But there is a huge need or a huge gap. Yeah, but are we doing some masking and some kind of governance to adhere to this data privacy regulations? Yes, we did. But is it a comprehensive industry standard? I'm not sure that we are doing that. And of course, the fact that you've got this disconnect between the business and IT accentuates the problem because, frankly, both communities need to understand where there could be at risk, right? But the business needs to understand it from their side. IT needs to understand the physical problems of identification and protection of sensitive data, right? So again, there's a need to share information and you need to have a picture, a model, if you like, of the entire enterprise viewed at from both perspectives so that you've got collaboration between the two to help minimize that risk. Yeah, and have some kind of masking algorithm frameworks, or [? to create ?] data masking, so that all the critical business workflows have incorporated that framework so that any data comes in, goes out, fall under those algorithms, or use those algorithms and fall under the framework of that. That should do it. Yeah. In the industry, there is a gap. Yeah, definitely. And it's not just about looking at databases and trying to figure out where the sensitive data is. You've got to have the business community need to have eyeballs on where the potential risk is as well, right? So it's not just something that's confined within IT. And again, it's sharing of knowledge, isn't it? One of the interesting aspects as well-- we talk about DevOps and DevOps infrastructure, particularly around hybrid Cloud infrastructures and running workloads in the Cloud and that sort of thing. When we think about DevOps and you've got all these different sort of environments and moving changes, what role do you think performance monitoring plays when it comes to DevOps and CI/CD when we're talking about hybrid Cloud infrastructure? Is that something you think is significant? Being able to monitor performance impact on database changes before it happens is very critical. Because now, on Cloud, the expectation is every application performs optimally and there's subsequent-- or second response. Beyond that is not tolerable. And the resources on the Cloud-- the Cloud seems extensible. But you buy a certain amount of resources on the Cloud. So you are limited to that-- those resources. Yeah, they can expand. [INAUDIBLE] for expanding also is costing the company more. So optimal use of performance is key. So having this performance step in the CI/CD pipeline is very essential and having the checkpoints for any performance issues. We have to resolve all the issues before they get magnified into production. Cool. OK, so we've-- just a quick recap, then. So we talked about some of the business drivers behind DevOps, in particular. Particularly from a database side of things. Looking at a conversion pipeline. Some of the things you have to think about moving from a traditional development process methodology to automating a lot of the tasks you mentioned, right? But then, as well is this disconnect between the business and IT, which is huge when it comes to not only defining the needs and articulating those needs, handling changes bidirectionally. And then, bringing in things like data governance and data security and things like that. And then, of course, the infrastructure itself, monitoring that. That's a lot of different things, right? I mean, so given everything that we've just said in the last 25 minutes, what do you think an ideal solution should look like both in-- particularly around bringing together as this subject is about, of course, model-driven DevOps. Aligning the business on IT, reducing number of change requests, like you said, and ultimately, provide much more value to the business. What would something like that look like in your mind? I think having a centralized shareable metadata repository that can provide business and IT data. Stakeholders with a view into where the data is across the Enterprise. Having data catalogs that provide data and data lineage in a presentational form that they can understand. That's very critical. Right, so effectively, having an abstraction of the metadata that is in a language that they can understand. Like we said before, business semantics versus technical semantics. But it's the same data. Is that what you're saying? Correct. All right, and how would that help, ultimately, do you think? So that gives a visual representation of end to end. For any change, you can either plug that information into your pipelines, into your change or enhancements or migration to a different solution altogether. So this is the key for an application to access the way-- before there is a key, you need to have your ER diagrams and you need to have a technical design documents. Have your functional specs. That was a requirement before now. It's changing. Have a unified visual of all these documents in together. That's the new need because of the Agile-- the fast changing. We need one repository. So the metadata repository, effectively, that provides capability to model the enterprise with views into both the business side of things and also the technical side of things. But on the technical side of things, we're really talking about data model, right? Data model that could ultimately start as a conceptual model. Because at this point, if it's a start up project, you might not necessarily know what database platform you want to deploy onto. So you want some sort of a conceptualized model. But that model, then, is able to interact with the metadata repository both ways. So you can reconcile any changes that come from IT or from the business. But from the IT community standpoint, they're basing all their downstream changes based on whatever happens to that data model. Is that right? Correct. Yeah, because we've-- the reason I think this title-- although it goes by different sort of pseudonyms, the notion of model-drive DevOps, I think, ultimately comes from model-driven development if you remember model-driven development years ago. The aim was to drive all of your changes using an Enterprise modeling tool. And sadly, it didn't really happen. But I think now that we've got DevOps and we've got automated pipelines, I think because of things changing so rapidly, I think we've gone full circle. And we've gone back to a point at which we really need to have driving changes coming from a model. Would you agree with that? Yes, I would definitely agree with that. Great. So we're up on our 30 minutes, Vamshi. So I really want to thank you again for providing us with some really interesting insights. And we hope you, the audience, have also found this discussion helpful as you think about your DevOps initiatives or your current DevOps initiatives. And really, whether you call it mobile-driven DevOps or biz DevOps or DevOps 2.0. I think the problems are the same, right? How do you effectively align business and IT? That's really what we've been talking about, right, Vamshi? Right. In order that you drive software changes faster. You minimize rework, as Vamshi said. You enable the business to work more effectively with IT by sharing this common metadata. And by leveraging data intelligence, it's able to then visualize that landscape so that you avoid people using Visio and other tools. But because you're sharing information using a tool that actually gives you different types of visualization based on your background in order that you can then avoid any miscommunication, misinterpretations Vamshi alluded to earlier on. And make it much more likely that you will get business agility and value and securely deliver change more quickly into the business. So thank you again, Vamshi, for that discussion. It's been really helpful insightful. Have you anything else to add, Vamshi, before we start wrapping up? I know we're got some Q&A in a bit. Hopefully, we're going to get some questions. So we'll deal with those in a second. But have you anything to add first of all? No, I don't have anything to add yet. Thank you for the discussion. Excellent, right. One thing to note as well, we've got a session coming up tomorrow with Tim Emmanuel. It's at 12:00 to 12:30 EST. And it's called Enterprise Impact Analysis. And it touches on some of the things that Vamshi and I have been talking about in terms of interconnectivity between applications and the data. And this is a big problem with people and with organizations and people trying to figure out where these problems are. Because applications and infrastructure are so complex these days. It's very easy to make a change in one area, and then have an impact somewhere else. And you don't know where that is until you suddenly realize there's a problem. So I'd recommend-- if you have-- join one of-- any of the sessions tomorrow, join that one because that's a useful complementary session, I would say, to what we've been talking about here as well. We've got some resources here as well. So we can get these slides to you as well. And I also believe that these resource links will be made available separately, anyway. But these are some helpful resources. If you want more information about what Quest does in terms of solving the problems that we've been talking about, you can see some solutions there from Erwin Evolve. This is an enterprise architecture and business process modeling tool. So this would be typically used by people like Vamshi to develop out the Enterprise architecture, build out business process modeling, and then you would have data stewards using data intelligence tool-- data intelligence to build out that data catalog with a metadata repository and build out the data literacy so you've got a good understanding of how the data flows around the Enterprise. And then, that would then feed into Erwin data model, which is a data modeling capability. And then, ultimately, deploying those physical data models into your target environment. It feeds those changes into the whole CI/CD pipeline. And then, finally, in terms of orchestration, you could-- again, you could use Erwin data intelligence for that. So you can actually build out a CI/CD pipeline as well. Or as Vamshi did at the Hartford, you could use Toad DevOps toolkit, which is a CI/CD orchestration for Oracle. We also have a CI/CD orchestration for SQL server and Azure SQL DB. It's called Apex SQL DevOps toolkit. So these are tools that take the sort of standard way or the historical way, if you will, of doing database development on these two platforms and helping you orchestrate those changes by automating a lot of these different tasks and feed those into a building deployment process. So that's some useful resources for you as well. So we're going to turn it over to questions and answers now. So if you-- let me see if we have any questions. That'd be great.