ChatGPT and the downside of digital democracy

There has been much talk since the launch of ChatGPT, and the subsequent rush of other generative AI options, about how one of the main things it will do is “democratize” several industries – including coding and programing.

What that means is that it will allow people with no experience in a programming language to use generative AI as a substitute for learning or skill. It will allow them to write code and apps in any language – though not necessarily to know if it’s right, or why it works, or what to do if it doesn’t.

That’s the potential downside to digital democracy – the loss of specialized skillsets and standards that programmers and coders developed over decades in the span of a generation or two of “citizen coders.”

We spoke to Kevin Bocek, VP of Ecosystem and Community at Venafi, a company that specializes in machine identity management and application outage prevention, to ask whether we were just being too gloomy about the downside of digital democracy – or whether there really is a downside to a future of AI-generated, democratized coding after all.

THQ:

So – what’s the risk of AI generative coding? Come to that, is there a risk to AI generative coding?

KB:

It used to be the case that if you were going to develop code, you needed to have some expertise, just like learning any language. If you were going to learn a machine language, you needed to have some expertise, some learning.

And more than likely, when you were developing code, if you were part of a business, you were part of an organized developer team. Back in the day, that was 100% what I’d do, I’d sit down and draw architecture, then write code. And I had special expertise in Java.

That’s the way it has been.

Show me the risk!

And in business, you could be a Go developer, a Java developer, you might even be a data scientist specializing in Python coding.

The rise of generative AI that can write code, whether it be something like ChatGPT or GitHub Copilot, means you can write code with prompts. I can explain what I need ChatGPT to write, and it will do it. I can ask GitHub Copilot in Visual Studio, I can add in a language I’ve never coded in, and something that I don’t know how to do, and say “Write me some code in this language, it should do this, that, and the other.” And it will.

And the point about democratized coding is that anyone can do that. An almost total newbie can ask ChatGPT to write them a PowerShell script to, for instance, make some change on some type of service that their team uses.

That opens the world up not only to a whole bunch of more code, but a whole bunch of more coders.

Where’s the risk?

THQ:

We were just about to ask that.

KB:

I’m glad. A whole bunch more code means you get a couple of additional risk points. The citizen coder may be really well intentioned, but a) they may not know what they’re actually doing with that code, and b) which we know with ChatGPT and also with Copilot – it will give you wrong answers. And it will give them to you with confidence and persuasiveness.

Confident wrongness can lead to code that doesn’t do what you want it to do, or incomplete code.

THQ:

And suddenly, it feels like we’re ten minutes in to a Michael Crichton movie.

KB:

The thing is, it used to be that I would copy and paste code. A good engineer can copy and paste code, but I had to know what I was doing with these different code blocks.

Now, anyone can do that, but without the requirement of knowing what they’re doing. What that means is that you can get code that’s prone to errors. Code that’s rickety or clunky. And, which is the ultimate point, code that can be either intentionally or accidentally malicious.

Can ChatGPT or GitHub Copilot, and others as they come online, actually be trained to introduce vulnerabilities?

And, again, if you’re a non-expert coder, whether that’s a weekend coder or someone who has absolutely no knowledge of what code is supposed to look like, you’re not going to know the difference. So that starts to create new sets of risks.

A matter of trust.

THQ:

How do we regain trust in the code and the apps we run, without trying to impossibly put a genie back in its bottle?

KB:

How indeed. The competitive advantage in many industries these days is driven by code, and the developers who write them.

What we need to do and – what businesses haven’t been good at doing until now – is authenticate the code that machines are running. And that’s an opportunity, because we have all the techniques we need to do this.

In sophisticated organizations, where you’ve got sophisticated development teams, this is something they’ve been doing, authenticating code. So, for instance, we’re talking about signing code as it goes through their pipelines, or even, these days, signing containers before they’re run. This is something that they know, these sophisticated developers with their organized teams.

But someone now who’s gone and gotten a PowerShell script, and was able to run it, and maybe it can change a function on a load balancer or some type of data science analytics service – that’s completely new. So that’s a whole new set of risk. And we need then to make sure that all code is authenticated when it runs in the business, whether that is someone at their desktop, whether that’s someone who uses and runs code maybe trying to change Salesforce.com, we need to make sure that it’s always authenticated. And that it’s signed, and then of course, that it has an identity when it talks to whatever other service it’s made to interact with.

The authentication factor.

That is something different security teams might have focused on, in terms of some of the most high-risk applications, some of the really sophisticated development teams. Maybe they have, maybe they haven’t. But now we’ve got a much bigger threat surface, and a much bigger opportunity for attack.

That’s why AI generative code-writing could be a risk. It’s something that, when I’ve talked to security professionals and analysts, isn’t immediately apparent, because when they think of generative AI, and they think about writing code, they think about it in terms of it helping me as a seasoned developer move faster.

Which, by the way, it absolutely does. Generative AI isn’t going to put professional coders out of business, but it’s going to make them go faster, and make them more efficient.

But the potential danger lies in what it can do for all of those others, who aren’t professional developers. It gives them the opportunity to be dangerous.

THQ:

You mean, in the old days, ignorance of coding meant you couldn’t code. And now, ignorance of coding means you can code by proxy – without necessarily having a clue about what you’re doing, only in the end result you want to achieve? That’s the downside of digital democracy?

Have you heard the good news?

KB:

Pretty much. There’s some good news, though.

THQ:

That’s a relief.

KB:

The good news is that this is something that’s now built into desktops, laptops, servers, cloud-native Kubernetes. We’ve been building in the capability to authenticate code. But it’s not always turned on by default. So, for example, when you install a new application to Windows, by default, the operating system is going to authenticate the code. But by default, your Windows desktop or server is not going to say “That PowerShell script that’s running? Who built it? And are we authenticating who built it? Do we want to allow that code to run?”

The same is true with containers. Containers don’t, by default, come enabled for authentication as they come on. Who actually built them? Have they been changed? The good news is that it’s something you can turn on – but it’s not on by default, and it really needs to be.

On by default.

That’s where there needs to be much more awareness – we need to get to a point where authentication everywhere happens by default.

Once authenticating code by default is the norm, your downside of digital democracy reduces significantly.

In the United States, the Biden administration has produced its five points of cybersecurity strategy, which is great advance – and one of the focuses of that strategy is to drive accountability for code to software developers, not just the people or businesses that run the software. That’s a trend I think that we’re only going to see play out more.

THQ:

Shutting down the downside? OK then – we guess we’ll let democracy live.

For now…

 

While we had him in the chair, we asked Kevin some other important security questions of the day. In our next article with him, we’ll tackle cloud-native authentication and identity strategies.