A Simple Evaluation of ChatGPT’s Programming Skills

Joyfully
2 min readJun 5, 2023

--

I had conversations with many programmers and asked them about their knowledge of ChatGPT’s programming skills. Surprisingly, not many senior programmers are familiar with it. One of them mentioned that it is still a prototype and hadn’t tried it at all. Another shared that they had tried it twice, and found it interesting. A PhD holder told me that ChatGPT is like a summary of StackOverflow. However, my own experiments revealed something far beyond that.

When it comes to software, there are three fundamental skills. I experimented with ChatGPT in all three of them, which I will explain below.

  1. Writing Code

Yes, ChatGPT is able to write code.

You just describe what you want and it in the matter of milliseconds, provide you the code.

Which programmer has such a speed record in the history? And it is able to write it in almost any language. Interestingly ChatGPT knows all famous libraries and plugins.

This skill is perhaps the easiest. Many claim that ChatGPT has read all the pages of StackOverflow to acquire such expertise. If you are a typical programmer and a regular user of StackOverflow, you already know the time-consuming nature of searching through questions and comments. And sometimes, StackOverflow fails to provide an answer.

2. Understanding Code

Understanding code may be seen as the other direction of creating code.

Please explain to me what code X is about.

And surprisingly ChatGPT is able to do it. It provides an explanation of the code in natural language. It is indeed a Wow!

3. Changing Code

Now, understanding code may seem like a trivial task if we simply go line by line and provide comments. However, ChatGPT is capable of modifying the given code based on a description. This skill means it possesses an understanding of each part of the program, including variables and commands. For example, ChatGPT was able to add new variables and determine how the output should be calculated based on a description. As a joke, I even asked if it could SMS the output to my mother. Interestingly, it modified the code to send the SMS using external libraries. It simply asked me to manually subscribe to the service before running the code.

The conclusion is truly remarkable. Hardly anyone could have imagined this happening in 2023!

--

--

Joyfully

Joyfully writes about human psychology and neuroscience, as well as their application in coaching and making positive impacts..