7 min read
I don’t expect this to be too detailed, but I did feel like writing down some thoughts about generative AI as it exists today and how it is being used throughout the world.
I use relative dates which are not at all meant to be a strict timeline of events. Don’t @ me if I get a year or timespan wrong. This is merely to paint a picture of the AI landscape.
Regardless of your thoughts on gen AI, it’s hard to admit that it hasn’t seen a huge boom in progress and improvements over the past few years. I remember when I first hear of GPT-2 and how it was a funny little thing that a research group was working on. It wasn’t really that useful and gave responses that were just barely reasonable in some areas. I remember one example from a Sebastian Lague coding challenge video where he used it to try to explain something about “why rocks float”. It gave a technically reasonable answer, if you ignore reality “rocks float because they are lighter than air”. I mean, if that were true, it would be technically correct. So at this point, it was a cute tool to be silly with.
Fast forward to last year and ChatGPT was really taking off all over the place. The AI boom was in full swing and companies were starting to take more and more interest in it. Development in AI skyrocketed and we had GPT-3 and early GPT-4 which could do very reasonable impressions of human speech. Things, somehow, have continued to accelerate and improve to the point where AI art is impressive and deep fakes are scarily accurate.
Yes, I do use gen AI for some things in my day to day life, and even at work. It’s the kind of thing that really is more of a tool like intellisense giving me autocomplete suggestions or helping me write large swaths of boilerplate code for mundane tasks. I find it to be just good enough to handle the simple tasks and to generate example code that I pick apart, put back together, and of course error check it. Funny enough, I’ve learned a lot about stuff I used to have no knowledge about: like jekyll and ruby. I’ll be the first to admit that I know just enough to be dangerous, but isn’t that how we all start out? I’m certainly not here claiming that I’m an expert and that I vibe coded by way through my job and this site.
Now, do I advocate that students use gen AI to pass all their software dev classes? Fuck, no. Let’s be honest, you’re not going to pass with the crap that AI will give you if you ask it for something with even a few specific requirements. It seems to struggle to incorporate multiple ideas into one solution: there is a problem juggling and balancing requirements and concepts. The code it gives can often be very inefficient, but still get the job done. So maybe you can take it at face value for a very rough prototype, but please never put anything AI into production.
Despite leveraging GPT for generating code now and then, or helping me understand some obscure API documentation with half-baked examples, I do have to speak out against it in some areas: I already mentioned students using it briefly above. Really, my argument boils down to the need to learn and internalize concepts when learning that first programming language. With the exception of a few languages out there, everything has the basic concepts of iteration and branching logic. Not being able to truly understand why is the biggest downfall of all the students I used to tutor in college. The other biggest downfall was not being able to interpret requirements into a design.
AI has a hard time understanding requirements from the perspective of really drilling down to what is needed. Clients never actually know what they really want, they just have an idea of what they don’t want and a best guess at what might be helpful. While this isn’t as much of a problem in academic courses, there are definitely really good professors that are going to give requirements like a business client for a capstone project, for example.
Being able to take the ask of “I want to see the monthly recurring revenue for this account” and turn it into a proper problem statement and understand what all is needed to get that number is something so specific to the underlying nature of the data, account structure, sales team methodology, contracting process, and more. Good luck getting an AI to help with that without spending 4 hours adding all the required context.
I’m not entirely sure if why is answered enough these days. Comments in code, for example, need to tell someone why a decision was made to do it this way. I don’t need you to tell me this is a loop over the items in a player’s inventory, I can see that easily. But why are you doing it this way? Is it a threading issue, are there issues with the inventory services that need workarounds for now? The questions can go on and on.
Understanding the why behind a coding problem is crucial to forming that gut feeling. “Why did you use a for loop here?” is something I would ask my students all the time and I would usually get something back like “because that’s what the professor did”. Once we worked out why that loop was used, they were able to understand it’s purpose better. Honestly, I don’t care if you need to look up the syntax for something to get it written correctly. I care more about you knowing that you need to do something and why it is a good solution. I’m sure AI can understand simple scenarios and explain why the code is what it is, but it’s still not thinking.
At the end of the day, these large language models, no matter how advanced, seem to still lack the capability for reasoning and the ability to think. My favorite thing to point out is that GPT and other LLMs don’t handle counterfactuals very well and they will almost never admit it doesn’t know the answer. While the counterfactual part is getting better with web searches providing multiple viewpoints and the tool being able to aggregate these into a cohesive pro/con list, for example, I don’t think it’s where it needs to be yet to be trusted.
I think we’ve all had an LLM spit out something completely inaccurate: like hallucinating libraries or functions. I think this is because these models, again, do not think. They are still just statistical models, albeit very very fancy ones that can do a lot more than they used to.
All in all, I think AI is a cool toy to play with, but I think the world needs to pump the brakes a bit. There’s a bubble out there and it will burst at some point. Until then, let’s just all use it responsibly and not give our AI access to production environments.
Comments