FIAn AI blog or what?
AI tools are all the rage right now. I’m doing my doctoral research on small language models and agents—essentially, edge AI. It helps me with writing this blog to some extent, even though I don't spend time pondering Claude or ChatGPT in my research.
Since my research work is still ongoing, I’m not going to write about that today. Instead, I’ll share a bit of my own experiences related to Claude. It is certainly more familiar to most people than fine-tuning, distillation, quantization, or other optimization methods. I’ve been using Claude since last summer. The journey has included successes and also, let's say, iteration. Iteration—there has been plenty of that.
In a rush, or doing it with AI?
I let out a long sigh when I realize that it’s time to work AI-style again. Are we in a rush, or are we doing it with AI? Or rather, are we in a rush, or do you have time to "iterate" for a couple of days? AI = Always Iterating. Well, not always. Only sometimes. Meaning, whenever I use it.
All autumn long, I’ve been whipping up various tools for different needs with the help of AI—and every time "real quick like." I’ve put together a stem separator for music stuff, a video stream pattern recognition tool for a "fly... human paper" display at a trade fair stand, a data-crunching app for investment optimization (read: promoting the irresponsible squandering of money), a simultaneous speech transcription app for my hearing-impaired mother’s Android phone, demos for various purposes, and many other workflow-boosting needs. A couple of other projects have also landed on my desk, but let’s leave those out to avoid repetition.
The Claude experience so far
I started out using the free version and built a stem separator for my music hobby needs. The first version came together quickly and was almost finished, but then I ran out of tokens and a whole new batch of cropping issues popped up every day. Every now and then, the thought crossed my mind that "you’d already be there if you walked," wondering if actual coding could really be this painful. But it would be; I remember the "Hello World" in Symbian all too painfully well. Enough of that, back to prompting.
Finally, after a couple of weeks of iteration loops, I got my first app working on my own machine. Hurrah! But 50 new issues followed immediately when the app was installed on another computer. Then I discovered helper tools that gave Claude project memory. Right after that, I bought a paid subscription and gained access to its more powerful language models. Now things started to rock. I got all the compatibility issues sorted out in a couple of days and gained the courage to try other, slightly more important applications.
As I mentioned earlier, in a relatively short time I built various mobile apps, client demos, and applications for Windows and macOS—and pretty much all of them crossed the finish line insanely fast. Time is money. Also when using AI. I’ll come back to that at the end. First, let’s hallucinate.
Hallucination
The term "hallucination" is stubbornly attached to AI. This is because AI makes a fair share of mistakes. Fortunately, it’s not quite Uuno Turhapuro’s shorthand writing level (Uuno Turhapuro in the Army), but it’s clear that you can’t trust any output as-is, even though the answer is always presented with a "cross my heart and hope to die" attitude.
It is paramount to remember to check the answers given by AI, or the applications it develops, that you publish. The same applies to decision-making, especially in critical situations, but fundamentally in every single situation!
So, we can use AI to conveniently get new tools, but we do need to watch out a bit for hallucinations. It can surprise you even in small matters—situations you’d hardly mess up yourself. Below is an example of such a "surprise face-plant."
Case example:
A writer of a scientific article recently made the mistake of finally adding sources using AI after a long, multi-phase writing process to speed up the finalization. The AI recorded the authors' names in the source documents like this (note: names made up): P. Writer $\rightarrow$ the AI expanded the middle/first initial with a self-invented first name and wrote down "Peter Writer." The author's name should have been something like "Paris Writer," or preferably the original: P. Writer.
The change made by the AI went unnoticed, and the article received a rejection notice in peer review mentioning "hallusinated sources." Apparently, it was a "lovely moment" to read that feedback. So everything always needs to be checked. Absolutely everything, because you, as the publisher of the output, are responsible for what you have produced with the help of AI. That is why software architects, experienced developers, and testers are still needed to review the AI-written code of a published application as well.
Claudeamus Igitur: "let us rejoice"
From my experiences so far, I’ve made some guidelines for myself as notes: always assess your own need before doing anything, then plan—perhaps with AI's assistance—the way you can implement the most suitable solution. That way, prompting goes better and there will be less iteration. It's really not that much more complicated.
If everything has to run in the customer’s private cloud on-prem in a bunker, figure out one way or another how to get a sufficiently powerful edge solution into your own environment that can operate right there in your environment without an internet connection. Might require a bit of initial capital and a trip to the "hardware store."
If, on the other hand, you can anonymize your own data and after that it’s relatively indifferent where that data spreads and whether it’s used for AI training, it’s totally fine to use public frontier models and you don’t even need to buy an expensive enterprise license.
Does it cost money?
Depends entirely on your environment and needs. Get in touch, and let’s talk!
The cost structure can often be optimized to be very low, and the initial investment isn't heavy either. There are many factors affecting costs, and their accuracy will become clear as soon as we know the details of the solution need.
If, on the other hand, you are already developing apps yourselves using, say, Claude or Cursor and you are happy with your situation, you probably don’t need much help with them, but hopefully this blog was entertaining and relatable.
What's next?
Edge AI is on the research desk. I’ll likely write about that next.
So: Run on coffee! And whatever other idioms are thrown around at the end of a blog.
The author is a Leading Consultant at AtoZ and an AI doctoral researcher, Tommi Tiitinen, MSc, currently brewing some coffee.
.png)