At this stage it's almost de rigueur to have a personal website. Especially in academia. So while it was probably time that I also jumped in on the action, I really wanted to avoid just directly working upon a templte from either Hugo or Jekyll. So instead I decided to try and make the website creation process a bit of a project, where I'd hopefully be able to learn something through getting my hands dirty (or as dirty as ones hands can get writing code anyway), possibly through working with an unfamiliar programming language. However, I do work in AI, and chatGPT is the new hotness, which inspired a bit of a challenge: could I possibly write a website without doing any coding at all, just using chatGPT?
This isn't the first time someone has attempted this. Lee Rob recently created a one page landing page using chatGPT using React, where even the content itself was a product of chatGPT. That this was clearly possible motivated me to want to try and up the ante a little bit.
Rather than a single page, I wanted to build a full academic portfolio website. While writing the content itself would be impersonal, I wanted all the scaffolding to be written by chatGPT. So I set myself the following rules:
The aim of this was to really explore the boundaries of what chatGPT could achieve, particularly from a coding perspective. As such, I picked a programming language that I didn't currently know anything about - in this case, React - and attempted to only use my intuition and my knowledge of other programming languages, as a guide to how to approach solving the problem of hitting the design brief I'd set for myself.
Speaking of design, in the context of the rules regarding the use of chatGPT the next step involved sketching out a very high level design document, that I wanted chatGPT to fill in the blanks for. The broad aims were for a website within which most of the content was included within a single page, that would be able to display
To keep things consistent, it also needed to maintain a shared header and footer bar across all pages and sub-pages.
Beyond this, I wanted to the design to have an overall style that followed the brief of:
But there's one important caveat here - none of the content for the page was to be written by chatGPT. Not a single sentence. Except maybe this one. After all, if one of the goals of the exercise was to create an academic website that showed a bit of personality, well a personality as provided by a large language model is no real personality at all
In a word: badly.
I outlined to chatGPT what I was looking for, in terms of the aesthetic of the page. I started by trying to outline the basics of an index page, which was fine. However attempting to extend the code beyond that point lead to spaghetti code, failure, and a touch of pulling my own hair out (though I'm bald, so realistcally, no difference there).
So, undeterred by how quickly my approach had failed, I tried again with a fresh start. That also went badly. Third time though? Yep, that was a fail too. ChatGPT and I each sent over 130 messages and it went absolutely nowhere.
Because nobody has ever said "fourth time's the charm!" I took a slightly different tack for my next attempt. Rather than trying to get chatGPT to write a website completely from scratch, the plan was instead to modify an excellent React template by Tim Baker using chatGPT, with all my design elements and modifications to be constructed as extensions of the original template. In doing so I was able to provide a scaffold to the code that would keep things on track. Initially I was skeptical about if such an approach would work, because now chatGPT would not just need to be internally consistent - a task that it had struggled with in earlier attempts - but it would need to be consistent with an already established style. This, surprisingly, turned out to be quite a successful approach, as chatGPT was seemingly better able to deal with small chunks of code, rather than full files. However, while it may have been more consistent, it still took 755 messages from me to chatGPT to get to the point where I had achieved my major design goals.
LaTeX parsing was the source of much teeth gnashing - when I could finally convince it to compile, it produced a range of head scratching rendering bugs. The level of difficulty associated with this problem was heightened because there were no errors and nothing in the logs that I could use to inform chatGPT of the error, so all I could do was describe the visual bugs I was experiencing, which primarily involved math-font being printed multiple times on the screen. Concerningly, frustratingly, and totally in character for chatGPT, no matter how I prompted a question, chatGPT proceeded to loop through a set of potential fixes (of which none worked) in a frustratingly confident manner.
Similar frustrations appeared when trying to implement other technical changes, especially around markdown formatting. While such issues aren't outside the realms of expectations, the resultant errors were clearly far enough from chatGPT's test set that resolving them took a considerable amount of what was essentially a fruitless chatGPT guided guess and check. In the end, I did have to resort to googling to resolve the issue, which lead to an answer far quicker than working through chatGPT, however even with full knowledge of the solution there was no way to lead chatGPT to a solution, and I ended up having to code in those particular parts myself.
Another pain point that didn't take a huge amount of time to resolve, but was particularly revealing was my pursuit of an animated, text based intro for my lead in page. GreenSock is a popular approach for such work, and after talking with chatGPT it seemed like the best framework to pursue. However, crucially, GreenSock has both paid and public variants, and chatGPT was unable to parse the difference between the two, or to correctly interpret error codes relating to my not possessing the paid version. Once again, the only way to resolve this was to track down the issues myself. In the end, no matter how I prompted chatGPT to try and avoid GreenSock functions that are only available in the paid version, it never was able to produce functional code. To try and hit my desired outcome, I ended up drawing heavily from Bionic Julia's write up on text rendering - but even still, I had to prompt chatGPT with large chunks of her code before it was able to create a variant that worked for me. This is definitely one of those times where just doing the work myself would have been significantly faster. The final effect on the the home page was also augmented by CSS styling from Sarah L. Fossheim .
So I still have a bit of a wishlist of thigns that I'd like to achieve, but haven't quite been able to make work with chatGPT yet.
Will these get fixed at some stage? When you read this they may well have already been fixed. But as of the time of writing it's time to put my foot in the sand and say that I've reached the goals that I set with this experiment, and that it's time to move onto other things for a bit. I'm sure chatGPT will be happy to not hear from me for a few days as well.
However, given that I plan to fix these, it's important to also consider if I'll fix these problems with chatGPT, or if I'll code them up myself? Given the deceptively long time frames involved with successfully coding even little features with chatGPT took, I think I would be likely to just code them by hand myself, but I'm also not averse to reaching out to my friendly local neighbourhood large language model for a little bit of help too, if the need arises. But that said, I think it's more likely that google will be my first port of call.
...maybe don't attempt to construct a website using chatGPT. What I had hoped was going to be a weekend project ended up taking way more time than I would care to admit.
But it was an interesting way to expose myself to a new language. Obviously without doing any real coding it was just learning by osmosis, which is not a great way to learn to code, but I still believe that the very project oriented, self-directed nature of the learning has helped me pick up enough pieces that I could solve React problems independently. Of course there's the whole Dick Cheney "unknown unknowns" thing, in terms of that I don't know what common problems chatGPT has potentially been paving over for me, without my realising that they were pain points.
Besides my misgivings, chatGPT performed admirably at translating specific error messages, and was often able to provide useful explanations and plausible solutions. Deploying the solutions may not have always worked, but the fact that it was able to translate debugging code is both interesting and powerful, and speaks to the strengths of large language models. Perhaps more interesting though is the behaviour of chatGPT when there is not a clear, easy pathway to a solution. Working with it very quickly starts to give rise to a feeling of solution whack-a-mole, where it repeatedly, and confidently produces answers that are manifestly wrong. In the worst cases, this ends up leading to what feels like an infinite loop of incorrect solutions, where chatGPT will iterate through a list of options before returning bck ot the start. The size of these loops is surprisingly small, suggesting a limit to the ability of chatGPT to generalise from its training set.
Another significant concern with programming with chatGPT is its lack of self-consistency. Functions and references can vary, and without care this can lead to issues, and introduce a burden for integrating chatGPT into programming workflows. And it's also just a pain.
I think it's important here to take note of the ethical concerns in taking the product of peoples labour - specifically, their code - aggregating it into a large language model, and then spitting out results. Yes, the code that chatGPT was trained on was publicly available, but in most cases that code was released for people to either a) promote themselves or b) to contribute to a view of the internet that was free, open, and participatory. This content was then mined by a company that now has a multi-billion dollar valuation, and that valuation is driven by its ability to produce written content, including code, in a similar fashion to a human. While at this current point of time the public view of large language models is primarily one of awe, I believe that it's important to acknowledge that these systems are effectively fancy autocompletes, where the materials it uses to inform its decisions was scraped from the internet without the consent of those who originally created the works. This will be all the more interesting in the future, if openAI moves chatGPT to a paid access model.
I amm planning to write more about some of the ethical considerations of large models (including large language models and approaches like Stable Diffusion), which I'll link to here when I am done. It could go here, but I don't want this write up to end up being longer than the Lord of the Rings.
Well, you can. But that's not the question here. In all honesty, no, the current variant of chatGPT is not ready for this. My process of constructing this website was long, slow, and oftentimes painful, and I can only imagine the levels of torture involved if I hadn't been a literate programmer. That said, it must be noted that my experience is rooted in the time in which I performed this experiment - OpenAI have been hiring programmers to help it flesh out chatGPT's ability to provide relevant code responses, so this appraisal of working with chatGPT may change very quickly.
But while you probably shouldn't attempt to write a website with chatGPT if your goal is to create a website, for me it was an illuminating example of just what chatGPT can do, where it struggles, and what I'd like to see out of future large language models. It's not often that you get to be really experimental and playful with something that so many people are excited about, and that has so rapidly moved the needle in terms of what we consider as possible with AI.