DevOps Trends in the Age of Generative AI

Brooke Jamieson

Hi everyone! Thank you so much to Netlify for having me here. It's really exciting to be here to support an AWS partner, Netlify, as well as all the other AWS partners that I've gotten to meet during the event.

In an interview yesterday, I was asked what composability meant to me. I heard afterward that the interviewers got different answers from everyone, which is a good sign. But to me, it brings together the modernization strategies and the maturity that I've seen across the industry and the world as I get to talk to different developers. One of the big key themes that ties that all together is DevOps.

DevOps is how you can ship products. It's because your software developers and your IT teams need to do something that's difficult for all humans but especially tech people, and that's communicating effectively. There are lots of trends that come and go with DevOps, but as we're working with generative AI, there are different things that come out of the woodwork as well. So there are different ways we can ship products faster, more securely, and write better quality code so that we're getting to all of these cool, fancy use cases that we've seen and heard from during the other presentations.

DevOps combines cultural philosophies, practices, and tools to increase the speed and delivery for apps and services. It really does come down to people, though. It comes down to what's important to people, how they might work together, what they're doing now, and then how that will transform to what they want to be doing in the future. But we're just trying to get faster evolution and improvement of those products compared to traditional software development, which is what gives you these really increased outcomes for your customers, which is what we're here for at the end of the day.

As traditional software has moved to more AI, people just expect more from everyone all the time, and generative AI is a really big factor in pushing them because people are now seeing and dreaming so much more with what they think they might be able to build and should be building in the future.

Generative AI, this slide is from an article I wrote called "The ABCs of Generative AI" (the link is in the bottom corner of the slides). If you get asked at parties what a Transformer is and you don't know what to say, I really recommend looking at "T is for Transformers" in this article. But generative AI is a subset of deep learning, and it's a type of AI that can create new content and ideas, conversations, stories, videos, and music.

Like I said, I am an AI person through and through, so I can't not talk about generative AI. But one piece that really gets missed out is you've got a layer of AI on the top. Machine learning is a subset of that. Deep learning is another subset of that, and generative AI is nested down in there as well. It's powered by machine learning models called Foundation models, which enable you to work with large models that are pre-trained on truly enormous datasets. It's really hard to even visualize how big these are, but this is what enables that content creation and curation without human involvement.

There are lots of ways you can use AI in DevOps, but overall, it's helping you to get to a state of operational efficiency, which is what DevOps is all about in the first place. It can improve the quality of code that you're working on. If you want to preemptively spot bugs, something like moving towards a proactive method for this, rather than just waiting to see a bug and then figuring out how to deal with it, a really big benefit of AI that you're going to see across this presentation is being able to move from a reactive state to a proactive state to squash some of these things that were ruining your developer experience before. It also gives you much more scalable security, so you can automate code reviews and then enhance your security measures as well. All of this is just helping you to move faster but with confidence because you have more checks in place to know that you're on the right track doing the right thing.

Alright, Amazon Code Whisperer is now generally available. It helps you to build faster and more securely with an AI coding companion. I have had such a good time working with Code Whisperer, and it's something that's really matured even as it's moved through from preview. When it went general availability in Amazon speak, that means when it's actually available, not in a previous round, they added so many more languages, which you're going to see on the next slide.

But it helps you to generate code suggestions in real time. It's also going to help you to scan for hard-to-find vulnerabilities and flag code that resembles open-source training data. Code Whisperer is a coding companion that was trained on Amazon internal source code as well as open-source code, and it will actually also flag this for you. There's a slide in a few that's even just been announced this week for a new feature, which is very exciting.

But the moral of the story with Code Whisperer is that deep down, I know exactly how to put a file into S3. I know exactly how to do it. I've done it so many times. But I Google it every single time I do this. So I close down my IDE, then open Google, check that the thing I thought I knew exactly how to do is indeed correct, and then I go back. So I've broken my flow state. I've wasted time, and it's taken me out of doing something that I was really concentrating on. So figuring out how to move back into a flow state is something that's really helped me to get the most value out of Code Whisperer, and I feel like that's how it really slots in with DevOps as well.

You can use it with your tools that you already know and love. There are 15 languages together. Rust was recently added, which I know people were very excited about. But there are lots of favorites there as well. You can also use it in VS Code in the IntelliJ family of IDEs, and in AWS Cloud9. But also, you can use it within the SageMaker Studio. Now, that was an announcement that really snuck under the radar but it's been very helpful to lots of machine learning developers. You can also use it within the Lambda console, as well as other things, which is very helpful. So it's just fitting in the way that you work. You're not having to take in other things.

This is what the code generation looks like, which is really helpful. So what happens is you write a comment as you normally would in natural language, so you just write in English what you want to be doing, and then Code Whisperer can suggest comment completions to help developers formulate what they're trying to do. So at the end, it's trying to figure out what you're there to do and then how it can best help you to do that. Sometimes, like in this example, you get a full code function, sometimes you get a single line of code. It depends on what you're doing. My big tip with code generation is to think about the comments you're writing as prompt engineering. It's something that you're going to be communicating more clearly if you do it that way.

But then I've noticed with the developers that we've been seeing around the world, treating it like prompt engineering is something that helps them to get bigger benefits from this. A really important key point is that because it's been specifically trained on all of the APIs for all of the AWS Services, it's going to give you really good quality results for AWS services. So if you're someone that's working with a lot of AWS, it will help you out. But obviously, it's working across all the other languages for all sorts of different coding tasks that you're working with.

The reference tracking is something that I think I hand the product team a lot because I want them to talk more about this. I think it's a really important feature. When it spots code that it's giving you in an answer that's similar to the open-source code it was trained on, it gives you a reference log. So you can see here, it's showing you what it was from, it's licensed under an MIT license, and you can also accept or reject that. So if you don't want to use something that appears in open source, you can toggle through with your arrow keys to see what you actually might like to use instead. But it's just helping you to do appropriate attribution with what you're working with.

Security scanning is also a very unsung hero of the product. It can scan what you're doing to check for vulnerabilities, things like the OWASP Top 10. It comes up with enforcement of crypto library and also AWS security standards and best practices, and lots of other things. But it won't just tell you that there's a problem; it shows you how to fix that problem, which is something that's really going to help your team scale and grow as you do this working alongside your team.

So this is working; the security scans are supported in VS Code and JetBrains IDEs for Python, Java, and JavaScript, which I think covers a lot of the Netlify developers that we've seen today.

Okay, this is the slide I put in that says "available soon," and between the slide cut-off date for this conference and today, it's now in public preview, which is really exciting. So you can customize Code Whisperer for even better recommendations. You can now get more relevant recommendations by customizing on your own as an organization – your own internal libraries, APIs, standards, patterns for architecture, things like that. It's going to help you to write better code that's more suited for exactly what you're doing, but it's also going to help you onboard new developers faster because you're going to be able to say, "This is not how to write just correct code, but correct code for your organization." Additionally, you can protect your valuable IP. Security is very important to AWS, obviously, so you can protect your IP with secure customizations, but there are also really good admin controls. I only just started digging down into this now that they've made the announcement. There are very, very good admin controls if you would like to be an admin overlord. Please, this is your time to shine, and it's isolated from the base model. It's going to help you just to settle people into your organization, especially if there's very specific things that you, as an organization, need care about and what you value, even because so much of development comes down to there's lots of things you can do and what should you do, and that can and should trade-off is unique to developers all around the world. And this will help you do that at a scale that we haven't seen do it work.

How you can really get the best results when you're leveraging AI to improve DevOps. The first of all is reducing your time to recovery. That makes sure you're leveraging these machine learning insights to really quickly diagnose, but not just say something's wrong, showing you how to actually fix it so you can remediate those issues as well. Also, as we've seen, you can proactively, so not reactively but proactively improve your code quality. So you can find those hard-to-find bugs, critical issues, and those OWASP Top 10 security vulnerabilities, but then have really intelligent recommendations on how to fix those things. The third one is, we'll see later on, you can even catch your most expensive line of code. So you can eliminate costs of writing custom integration code with integration functionality optimized for machine learning. So Code Whisperer is integrated with CodeGuru, and this is one of the things that really helps it to function. You can detect, track, and fix security vulnerabilities anywhere in the development lifecycle using machine learning and automated reasoning. There's a security feature that's finding those security vulnerabilities, and there's also a profiler that's continually searching for application performance optimizations. This is what's going to be finding those most expensive lines of code and recommending ways to fix them so you reduce your CPU use, compute costs, and other things like that.

DevOps Guru, so we had CodeGuru previously, and now we're on DevOps Guru. This uses machine learning to analyze application metrics, logs, and events to detect abnormal behavior and then provide insights, context, and then those remediation strategies so teams can focus on responding to those issues rather than sifting through noise. This is called DevOps Guru. It's because it's good for DevOps, but this is one of the things that AI is not just like a thing that could help you with DevOps; this is a really concrete example of how it does help teams in so many different industries to identify those operational issues before they start impacting your customers.

I get to speak to lots of different developers across different industries, and this is what I've come up with as sort of the best practices for incorporating AI and DevOps. But I think it really comes back to incorporating AI in anything.

The first one is accountability, so you're trying to bring development and operations closer together. It's not just like throwing against the wall, but you want to really stress the importance of maintaining a balance between AI automation and human oversight. It's the balance between and those trade-offs where your company values will really shine and how you can then bring through the DevOps culture that we talk about a lot, that's really where this is coming into play.

Automation, so AI is going to help you speed up your delivery, reduce human interaction and errors, integrate into your existing workflows without a system overhaul, and as you've seen, even automatically go through and provide suggestions based on your internal code bases now. So that's a very big key of how it's going to help you to bring it together.

There's also awareness. So when we were talking about the metrics and logging that you can analyze on the previous slide, AI is really well placed to do this because it's bringing back that reactive to proactive switch. So instead of just waiting for something to go wrong, you can then proactively deal with that when it does happen. So you're working out the state of your systems at all times.

Autonomy is really important. So this is developer autonomy, figuring out how to strike that balance between automation and oversight. Things like Code Whisperer really help developers get that autonomy when they're working, because it's working where they are in their IDE. But then you can also have that autonomy that's enabled by centrally enforced standards and governance, which is something like the new part of Code Whisperer that we were talking about. Because there's these new layers of controls that are beyond this, not just developers have autonomy but also their product teams as well.

I really also recommend getting in touch with the partner community. Things like the AWS Community Builders program, please get involved in that. I was a Community Builder before I worked at AWS. If you know anyone fantastic, they can work their way up to being an AWS Hero. I was an AWS Machine Learning Hero. Also attending things like AWS Re:invent or local user groups are fantastic to see.

Everyone asks what is the best way to learn AWS or the best way to learn to implement pieces of tech within AWS, and it really all comes down to getting involved with the people which is what devops is all about as well. I used to run a video series where I would pretend to do a weather report for a week, and I would say where it was cloudy. There were user groups all over the world, so I would say it's cloudy in Munich for the Munich AWS User Group. And during those videos, I found there were more than 50 user group meetups in the world every week, everywhere around the world. They're happening so often and you can find your local community. They're going to help you to hire other developers in your area. They're going to see what's trending with what people are seeing in the marketplace, and it's going to help you to bring scale to what you're doing across the community not just in what you're doing. That's the biggest place you're going to get these benefits from AI and devops.

I'm extremely online so I'm sure some of you will find me. My name is Brooke Jamieson. The slide with my information is not here, but if you look on my Twitter there's a video of me and a cat carrier explaining at AWS. Now that you're left with that, it's great to see anyone in person. Please come and talk to me about AWS. I would love to hear any and all of your feelings. Thank you so much to Netlify for having me.