As the owner of multiple content websites, I use ChatGPT every day for multiple tasks, including — but not limited to — content creation. ChatGPT is always there for me, whether crafting texts or discussing my business goals with me.
But as with all powerful tools, ChatGPT and similar Large Language Models (LLMs) have their limitations. I have stumbled upon them many times during my time working with AI. If you rely on ChatGPT in your business without understanding its limitations, that’s a recipe for disaster.
Here are some common mistakes that you could be making if you think ChatGPT thinks like a human:
1. Neglecting to fact-check AI output
I use AI tools like ChatGPT to create informational web content, which has proven to be an effective strategy for boosting my publishing business. The articles produced by AI writing tools utilizing models like GPT-4 are typically well-written and helpful. Of course, using these tools is also infinitely more cost-effective than hiring writers.
However, while the AI-generated articles provide a great starting point, they are rarely robust enough to be published as-is without human oversight. It is crucial to thoroughly fact-check the content, as AI tools can get details wrong, especially more nuanced facts that fall outside general knowledge. Make a point to check dates, locations, numbers and any claim that seems very specific. In many cases, the claims are unsubstantiated and need to be taken out of your article.
2. Using the generic ChatGPT style
Left to its own devices without any guidance or customization, ChatGPT tends to use a particular writing style. This default style is typically authoritative in tone, yet also dull, lifeless and formal-sounding. It sometimes reminds me of high school essay writing. This would not be an effective style choice for crafting engaging, compelling web content that connects with readers.
When utilizing ChatGPT or other AI writing assistants, prompting the model for your particular style is important. One helpful technique is to provide the AI with a few samples of your own writing, then ask it to analyze your style and implement similar stylistic elements into the new text it generates.
3. Failing to guide the AI in a structured, step-by-step manner
While ChatGPT is capable of generating coherent text, its output quality suffers greatly when prompted to produce long-form content all in one go. A far better approach is to break down the writing process into stages:
-
Discuss the topic, goals and target audience with ChatGPT to help set the stage.
-
Ask the AI to craft an outline based on your discussion. Assess the outline, and make sure it covers the topic properly.
-
Prompt ChatGPT to write individual sections one at a time, offering additional guidance and examples as required.
-
Ask it to suggest improvements for its own work to refine and polish the wording further.
-
Thoroughly edit and refine the full draft as needed.
Guiding ChatGPT in a structured, step-by-step manner with regular human feedback tends to yield much higher quality writing. This approach is far superior to simply prompting the AI to produce a full piece in one shot and leaving it to its own devices for long stretches of uninterrupted text generation.
In the web publishing industry, we have tools that create quality content based on a similar method with many pre-determined prompts and a built-in back-and-forth process. You can achieve the same by prompting ChatGPT to do the same.
4. Using LLMs for tasks outside language processing
LLMs like ChatGPT excel at language processing and generation tasks. They converse with us in the same way another human would. It’s easy to assume they can do other things that humans do — like counting, for example.
ChatGPT confidently informed me that the paragraph above contained 42 words. Go ahead and count. It’s easy for you to do as a human. You’ll see right away that the correct number is 37.
When prompted to generate a numbered list of all the words in the paragraph, ChatGPT struggled badly, either apologizing that it could not get the count right or actually fabricating nonexistent words to reach the incorrect word count it had provided.
Additional areas I’ve found ChatGPT has considerable difficulty with are solving simple anagram word puzzles or even reliably reversing a string of characters.
There is a solid rationale behind these weaknesses — ChatGPT was trained to mimic conversational human responses, which it does amazingly well. However, it was not designed for tasks like arithmetic, word games or manual data manipulation. Being aware of exactly when to rely on its language strengths versus utilizing other, more specialized systems is key.
5. Believing the AI’s self-assessment of capabilities
When needing to determine if ChatGPT or a similar language AI can handle a particular task well, avoid directly asking the model itself. ChatGPT does not have accurate insight into the full extent of its strengths and limitations.
For example, when I inquired whether ChatGPT could count words accurately, it confidently assured me it could handle such a simple math task. But as the earlier example illustrates, it failed at word counting multiple times. To assess an LLM’s true capabilities, real-world testing is far more informative than taking the AI’s word on what it can or cannot do.
The future is here, but tread carefully
ChatGPT and similar AI tools represent an incredible step forward that can amplify our capabilities if used judiciously. But these are not human equivalents — merely brilliant mimics lacking complete self-awareness.
By understanding their limitations, prompting creatively, guiding systematically, minding the task suitability and verifying through hands-on testing, we can maximize value while avoiding potential pitfalls.
Source: Entrepreneur