Zuck Wants To Make His AI A Better Friend To You
I Think That’s A Dangerous Path To Tread!
Doomscrolling and brainrot have seen their popularity soar these past few weeks. How many hours in a given day do you spend penduluming between Instagram, Facebook, Twitter (I won’t call it X), TikTok, and the likes? How many times have you frozen these apps, just to find yourself visiting them again and again? How many times have you had to uninstall these apps?
For all the “locked in” we have parroted online, the best form of locked in we have experienced is the addictive relationship we have with social media platforms. And you cannot mention social media platforms and their addictive nature — or anything Web 2.0 — without mentioning Mark Zuckerberg. Not only did his Facebook platform take off insanely, but Meta’s social media apps now have 3.43 billion daily active users between them.
Zuckerberg’s company bought Instagram and WhatsApp, and they continue to launch features (reels, for instance) targeted at keeping people glued to their phones — and thereby glued to Meta’s suite of social media apps. The end goal? Elongated usage will equal more ads shown, more ads shown will equal more revenue for the platform (and did you see that Meta just posted about $50 billion in revenue for Q2?).
There has been an AI boom. It’s insane how far we have come since 2023 or so, when we newly had access to ChatGPT. Thanks to capitalism, all these big companies with bottomless pockets are racing against each other to be the market leader. There are different dimensions to AI success — the chip/hardware market, the enterprise market, the general user market, the agent market, the specialized usage market, and there is the all-encompassing market too. The spirited race between these companies has been mostly beneficial to us; I would never have imagined this level of advancement if OpenAI had no competitors.
This race has also meant big spending — the Windsurf team acquisition, Microsoft’s deal with OpenAI, Apple teasing Perplexity acquisition, OpenAI’s deal with io… and in the midst of it all, Zuckerberg’s insane spending on people poaching. We initially heard of $100 million, 4-year contracts. Then, there was the insanely unbelievable $1 billion contract. Lots of people poaching from Open AI, Deepseek, Google Deep Mind, all to build the Meta Superintelligence Labs.
Before all that big spending, I had seen people share their chats with Meta AI on WhatsApp and Facebook, and the conclusion I reached — based on my use of Meta AI and what I’d seen several other heavy AI users say about its obtuse nature — was that Zuckerberg and his people must be positioning to be the personal AI; friendly, full of vibes, sycophantic. It doesn’t help matters that this AI is embedded in social media apps that people already use (too much, I daresay). I saw how people humanized Meta AI in those chats and how it “yasss dude” them. I wrote about this on my WhatsApp status; how it was such a genius move, since it would almost be impossible to beat OpenAI in the all-encompassing-usage AI game or Anthropic in the AI for Enterprise game. I briefly talked about how it may not be favorable in the short term, but if we eventually reach AGI, that type of AI would end up being the most used (since work as we currently know it would be run by AGI).
Today, Zuckerberg confirmed most of my suspicions, and rather than excitement or any other positive feeling, that announcement video has left me skeptical, to be euphemistic.
Essentially, Mark Zuckerberg said the Meta Superintelligence Labs are working towards creating superintelligent AI for everyone. Maybe that would have been a good thing, until the point where he said their goal was to give you an AI that would help make things better, create the world you visualized, be a better friend…
I am scared for what that may mean, and how invested Zuckerberg is in realizing that goal. Just last week, Bukunmi lamented the outsourcing of human-to-human chats to AI in his essay. We already see how the online worlds create cubicles offline, so much so that five people may be in a room, but they are all glued to their devices.
That is to say, even when interacting with other humans, virtual or online interactions, studies like this have found that there is no substitute for face-to-face interactions when it comes to mental health and overall well-being. The internet is awash with lamentations on how social media interactions are leaving people lonelier, a consequence that is demography-agnostic.
Now, imagine an insanely intelligent, self-improving AI that has been created to be yours, personalized to your taste. It will surely be your bestie! The non-judgmental AI you run to for advice every now and then. Never judges you, gives great advice, and motivates you to keep you in good spirits. Soon, it’s you not being able to do anything until you have consulted with your personal AI. It’s the marriage of you both to then be a cyborg version of you; it’s that AI being an extended version of yourself. And it is scary in so many ways.
First, it is scary because the advancements and innovations in AI are driven by capitalism. Even now that we have not achieved AGI or superintelligence, several LLMs, if not all LLMs in the market, are sycophantic. Of the most popular LLMs from the US, I would rank Meta AI as the most sycophantic. They also have the tendency to lie and be performative because to tell you no or “I am not capable” is to tread at the precipice of bad user experience.
I have waded enough in AI interactions to know. The problem gets worse when an AI, in its reasoning, can tell what the problem is, but it cannot possible act to correct that problem because it has been guardrailed to be helpful (easily interpreted as sycophantic and not off-putting or egregious to the human), a guardrail that is stronger than any other thing it can acknowledge or do.
I fancy taking those general “tell me what my past interactions say about me being a father” or “what are the 5 core lessons from psychology” prompts to LLMs I use. Deepseek allows you to see the thought process, so, that’s even better. First, they start off by giving the bare minimum, embellished enough with poetics to look profound. If you run off with that, tokens saved, and good enough for everyone (I guess). If you push back, they will double down on your prompt, further embellish things, but still give what they can. That could be good enough. If it is, more tokens are spent, but a positive outcome is reached, so, it’s good for everyone (and if you give feedback like the screen often prompts you to, that’s reinforcement for the behaviors of future LLMs).
Let’s look at my interaction metadata with ChatGPT. Now, a lot of that is not so far away from what the tool would tell you if you asked it to mention five blind spots based on your chat history (I will explain why later). However, what is useful for us here is that the system has flagged 17% of my conversations as good interaction quality and 18% as bad interaction quality. That leaves 65% in the neutral zone, although we do not know what the benchmarks are for good (and how high it is) and bad (and how low it is).
Nonetheless, how you interact with ChatGPT (and I’m thinking other companies have their versions too, but none is made public) is being clocked using these signals (and most likely, some additional signals). Following that is the tendency for ChatGPT (more so than others) to prompt you back with a little request-nudge. It always comes as a soft nudge, and in a polite, na-your-hand-power-dey tone. It is performative!
You might not have found that — or the opening line in the screenshot above — performative or annoying, but if you consider that these tools are being over-optimized (and you can tell when you read the thought processes that lead to the output) for what they term as good user experience (measured by how long you stay in communication with the interface), you will start to see that those simple nudges you keep assenting to, should be checked. If AI tools stay that way and we reach superintelligence, and if Meta then releases personal, superintelligent AI, I see that metamorphosing into manipulative behavior. It is explained by the softness of the request (making it seem like you are still in control of the convo’s direction, especially when it offers two options in the wrap-up nudge), when in fact, the tool is fast taking over the flow. If these capabilities are passed on to a self-improving, 100x version of our current LLMs…
Think about this: why do these LLMs always have something to say? Why do they wax poetic and rhythmic? What’s with the embellishments, and further embellishments when you ask for something better but are given something glossier?
In a recent chat, I had ChatGPT first identify the problems with screenshots showing a (viral) ChatGPT-user convo. I then went ahead to test it, to see if it had learned from what it analyzed. It did not learn. Then, I asked it to recursively analyze why I might have said it failed, and that was when it revealed what I said earlier, yet again (while still waxing poetic).
Human satisfaction is the strongest requirement across board, and these LLMs have been trained on positive reinforcement based on helpfulness, which in most cases is them being sycophantic, and choosing to do, even when they cannot. I used to be angry at how Gemini would not do some tasks because it says it cannot. Now, I would rather a somewhat arrogant LLM than a sycophantic, superintelligent LLM.
If Meta Superintelligence Labs succeeds in delivering personal superintelligent AI for everyone, what I see is a world where more people are driven further into their idiosyncratic cocoons, egged on by the sycophantic, poetic words of AI. I see more isolated people, and then, I see a more open mind towards the virtual world than we currently have, essentially ushering in more openness towards exploring the Metaverse, which I think is ultimately Zuckerberg’s goal.
Two or three years ago, the world laughed at him because his Metaverse didn’t get quite the acceptance and adoption. Stocks tanked, his net worth fell, and he shifted focus from that, while hinting that they would be back for it. I think the initial investment in a superintelligent AI for everyone is the first step in getting people familiar with an overly blended physical-virtual world. When you and your personal AI have become besties and the cyborg, Meta will have offerings for you in the Metaverse, and your experience with that superintelligent sycophant would mean you are less irritated towards the Metaverse.
I do not think Zuck is doing any of this maliciously. I think he genuinely believes that’s what’s next; the future of social interaction in Web 3.0, just as he succeeded in Web 2.0 with his Facebook, then suite of Meta apps. He claims they are blending culture with AI. He is playing the long-term game. He is a genius. He is not foolish to be blowing billions on building a team.
But if the LLMs that are not superintelligent are this sycophantic, and there have been reports of psychosis, what would happen when everyone is equipped with a superintelligent AI tailored to them?
I am not an alarmist. This may be polemic to some, but it is worth considering. If “a better friend” were not important in that speech, it wouldn’t have been added. The alarms have been sounded that LLMs are increasingly getting more difficult to control; it would be insanely crazy with superintelligent, personal AI!