Mystery of Q-Star | The AI which threatens Humanity | Explained by Dhruv Rathee
Exactly one year ago, ChatGPT was launched to the public, and since then, it has created a profound impact on the world. People found it difficult to believe that an Artificial Intelligence software could be this powerful. The company that developed it, OpenAI, and its CEO Sam Altman, have become symbols of the ongoing AI revolution. However, for the last few days, strange events took place in the company. All of a sudden, the Board of Directors of the company Sam Altman was fired! When the other employees found out about this, they caused an uproar. Many people threatened to resign.
It could manipulate humans however it wanted, and
mathematically, it could predict the future through the mass surveillance. Predicting the future didn't mean that it
could provide a 10-year or 15-year outlook, instead, in any particular
situation, it could predict the future outcome for the next day or the next
week. This AI excelled not just in 2-3
specific tasks; it outperformed humans in nearly all tasks it undertook. This form of Artificial Intelligence is known
as AGI, or Artificial General Intelligence.
AGI, a computer system that can do any job or any task that a human does but only better. -Ilya Sutskever
The famous AI software’s present today like the Language
Learning Models like ChatGPT, or generative AI tools like MidJourney, are
classified into the category of Weak AI.
Because these software’s are considered to be weak. They excel in specific tasks, they execute
the tasks for which they were trained better than humans, but they specialise
in only those tasks. If a robust AI like
AGI exists it would be able to perform various tasks better than humans. But at this point in time, Strong AIs do not
exist. OpenAI was established as a
non-profit in 2015. It had a singular
mission: to develop an Artificial
General Intelligence or AGI, for the benefit of humanity. The company's website clearly outlines its
mission. "To ensure that Artificial
General Intelligence benefits all of humanity." Prominent tech entrepreneurs came together to
create this company, including Sam Altman and Elon Musk.
There were 10 co-founding members of OpenAI in total. The
current chief scientist, Ilya Sutskever, and the President of OpenAI, Greg
Brockman were also included. Remember
these names because they play a crucial role in this saga. All co-founders collectively pledged $1
billion to OpenAI. In 2019, four years
after its incorporation, Sam Altman assumed the role of CEO. Four years after that, on November 17, 2023, the
board of directors unexpectedly fired Sam.
In case you don't know, friends, there is often a Board of Directors in
large corporations, in most cases, they hold the authority to appoint or
dismiss the CEO. As per the rules, it is
normal for the Board to have this authority. In this case, OpenAI's Board comprised six
members, I have already named 3 of them.
Sam, Ilya, Greg, and three independent executives: Adam, the CEO of Quora, Tasha McCauley, a
Tech Entrepreneur, and Helen Toner. Decisions
are typically made through majority voting, and in this scenario, Sam wouldn't
fire himself obviously. So only five
directors remain. Apart from firing Sam,
the board of directors dismissed Greg too.
This suggests that Greg likely did not vote against Sam. With only four individuals remaining, namely
Ilya, Adam, Tasha, and Helen, these four informed Sam of their decision to fire
him via a Google Meet call. The public
was not provided with much information about the reason behind this. The board vaguely mentioned issues with Sam's
communication and hinted that he may have been concealing information. After their termination, neither Sam nor Greg
issued any statements explaining the circumstances. Sam expressed his disappointment on Twitter. This was a shocking decision in the tech
world. The abrupt firing of such a
powerful and influential CEO by four members of the Board. What was the undisclosed reasons behind this? As mentioned earlier, OpenAI was initially
established as a non-profit, something that is very unique and important.
Because the other popular companies in Silicon Valley are
all for-profit. Facebook, Meta, Google,
Microsoft, and Apple, develops products and sells their services to earn money. They operate to make profits. But OpenAI's primary objective was to develop
an AGI for the benefit of humanity, this was more of a research facility. It was outlined in its charter that the
company's duty is towards humanity. Neither
towards the investors, nor towards its employees. However, this non-profit model was
short-lived. In 2019, the year Sam
Altman assumed the role of CEO, OpenAI introduced a for-profit subsidiary
company named OpenAI Global LLC. This
subsidiary operates on a capped profits model, The profit earning would be
capped with a limit. They limited
returns to 100 times the initial investment.
Any investors investing in this for-profit company would get 100 times
the return on their investment at most. Any
excess profits earned by this company would be directed back to the non-profit
parent company. In 2019, OpenAI's
for-profit subsidiary secured its first funding from Microsoft.
A monumental $1 billion investment from Microsoft. Over the following four years, as the subsidiary
gained widespread recognition, it successfully attracted a total of $13 billion
in investments. Presently, Microsoft is
reported to hold a 49% stake in OpenAI's for-profit arm. Before delving into Microsoft's involvement, it's
crucial to understand that while establishing a for-profit subsidiary, it was
asserted that the primary control of this subsidiary would remain with the
non-profit main company, OpenAI. However,
these conflicting arguments raise a major question: how should the balance between for-profit and
non-profit activities be struck? Furthermore,
once AGI is developed, how much of it should be commercialized and how much
should be designated as non-profit? If
profit is prioritised in all aspects, it would have an adverse effect on the
world. As exemplified by Facebook, whose
algorithms prioritized profit over users' well-being, whether it involves
mental health issues, addiction among users, spreading hate speech on the
platform, whether it causes riots in the world.
"Facebook, in India, has been selective in curbing hate speech, misinformation
and inflammatory posts, this is according to leaked documents obtained by the
Associated Press Some people are worried that AI would do the same. Before Sam was fired, some insiders revealed
that researchers working at OpenAI wrote a letter to the Board of OpenAI. This letter expressed concerns about Q*, the
AI they are developing, that could be a significant step toward achieving AGI.
But they were concerned about the potential of Q-Star. The exact capabilities of Q-Star are known
only to those researchers and employees within the company, but conceptually, Q-Learning
is an AI concept that falls within the realm of reinforcement learning. it is
named after a Q-value function, This function is denoted as this [Q(S,A)] where S represents the state and A signifies the action. Q* is a function that encapsulates the most
optimal point. To illustrate, consider a
game of chess. Imagine your chess piece
occupying a specific square on the board its position would be termed state S. The subsequent move you intend to make in
chess would be referred to as action A. Its
Q-value function would predict all possible scenarios.
All the potential outcomes for every possible move you could
make. Following a thorough analysis, the
function identifies the best possible move based on your current position. This best and most optimal move derived by
the Q-value function, is termed Q-star in mathematical language. This would be the best move for you to play in
any given situation in this game of chess. I used the chess board as an
example. But the same can be done with anything in the world. Picture driving on a highway where AI can
predict the speeds and driving styles of surrounding cars. By analysing these factors, it can forecast
how these cars might move in the next few seconds based on which it can provide
you with the most effective driving instruction.
Similarly, envision Q* AI analysing every possible scenario
before an election, predicting potential outcomes of the election. If OpenAI's mysterious Q* possesses such
capabilities, it has the potential to make significant predictions about the
future. the things in the Mission
Impossible film, might just be possible.
Predicting human thought patterns.
analysing all possible decisions in any given moment, accurately
predicting how a certain person might make decisions in specific situations. From business dealings to political moves, this
Q* AI has the potential to influence everything. While humans can do this to some extent, however,
AGI can surpass human capabilities since humans have their biases and are
emotional. We make decisions based on
our emotions. But the AI would make its
predictions solely based on mathematics.
In a chess game, for instance, after evaluating all possible moves, this
AI can provide the most optimal move. This
AI would be able to give you the most accurate predictions. For now, ChatGPT excels at writing and
language translation by predicting the next word.
Non-profit supporters, on the other hand, express concerns
that a profit-driven approach could compromise the original mission and pose
significant risks to humanity. A few
days before this controversy began, Microsoft's President Brad Smith said this
at a conference on 10th November. "Which
would you have more confidence in? Getting
your technology from a non-profit, or a for-profit company that is entirely controlled by one human
being?" Here, he was indirectly
talking about Mark Zuckerberg. But the
intricate balance between both sides in OpenAI, was deteriorating for some
time. In February 2023, ChatGPT+ was introduced
as the first paid version, followed by the launch of an API Application
Programming Interface on March 1st, allowing other companies to integrate
ChatGPT into their systems. On March
14th, GPT-4 was unveiled. Employees
noted a shift towards hyper- commercialization in the preceding months, creating
a divide between the two factions. After
ChatGPT's release, the path to revenue and profits were evident.
They couldn't continue being an idealistic research lab. They had customers and had to serve those
customers. In October 2023, OpenAI
introduced its advanced image generator, DALL-E3, integrating it with the paid
versions of ChatGPT. Subsequently, on
November 6th, the company hosts its first developer conference,
"Welcome to our first-ever OpenAI Dev Day. Today, we got about 2 million developers building on our APIs for a wide variety of use cases, doing amazing stuff..."
During this conference, Sam delivers a presentation reminiscent
of those from Apple or Google, announcing the capability to create custom-built
models of ChatGPT, they were referred to as the GPTs. This shift expands from a singular ChatGPT to
numerous customized GPTs, with the increasing commercialization, on one side
were Sam and President Greg advocating for and encouraging commercialization, while
on the other side, Chief Scientist Ilya and others were feeling uncomfortable
due to this. Ilya was driven by a strong
commitment to AI safety. At one point,
he had told his employees, that he was worried about AGI systems treating
humans similarly to how humans treat animals today. According to Ilya, AGI is not a distant
prospect; and that we will see AGI in action in the near future, so we need to
be prepared for it.
In July OpenAI announced the formation of a super-alignment team dedicated
to AI safety techniques, led by Ilya. The
company allocates 20% of its computer chips exclusively for this purpose, emphasizing
safety and AI alignment. By
August-September, a clear dichotomy emerged between two distinct factions
within OpenAI. They were working towards
opposite directions. Sam was focusing on
upcoming launches and the next big thing talking about GPT-5, while Ilya
concentrated on enhancing AI safety within the company, outlining necessary
precautions. While Sam pursued raising
billions in investment for accelerated development, Consequently, the other
four members of the Board of Directions in OpenAI, were leaning towards Ilya's
conservative approach. Upon receiving the letter about the development of the
powerful Q*, the board recognized the potential benefits of removing the
for-profit section from the company. Following
Sam's dismissal, the board swiftly appoints Meera Murati as the interim CEO on
November 17th. However, by the morning
of November 19th, a stir ensued among the company's employees, with a
significant majority supporting Sam.
Then came the biggest twist in this story. Ilya signed the letter and posted this tweet. Ilya realized that the company's survival is
crucial for implementing safety precautions.
As a result, three co-founders, Sam, Ilya, and Greg, unite. However, the three independent board members
remain steadfast. Adding to the
complexity, the new CEO, Emmett, also threatens to resign, demanding clarity on
the reasons behind Sam's termination. In
the aftermath of these events, it became evident that there was no alternative.
Consequently, on November 21st, Sam was
reinstated as the CEO of OpenAI. At the
end, the three independent directors were powerless against the CEO,
co-founders, and employees of the company.
After Sam was reinstated, 2 out of these 3 board members were fired. Leaving only Adam D'Angelo. Both board members who left were women, and
two new members joined the board: Brett
Taylor, former co-CEO of Salesforce, and Larry Summers, former Secretary of the
Treasury. The new board's initial task
is to appoint a larger board with nine members. Sam posted a tweet, stating, "I
love OpenAI, and everything I've done in the past few days has been in service of keeping this team and
its mission together.
When I decided to join Microsoft..., it was clear that was
the best path for me... (but) I'm
looking forward to returning to OpenAI with
the new Board and with Satya's support..." From Satya Nadella's perspective, in this
situation, regardless of the outcome, Microsoft stood to benefit. even if OpenAI couldn't survive as a company,
a new department created in Microsoft would have absorbed almost all the
employees, CEO, and co-founders of OpenAI.
They could've continued with their projects without issues. But now that OpenAI has survived this
turmoil, they have an established partnership with Microsoft, and Microsoft
would continue benefitting. Satya
Nadella tweeted: "We are encouraged
by the changes to the OpenAI board. We
believe this is a first essential step on a path to more stable, well-informed,
and effective governance. It is expected
that Satya Nadella might join OpenAI's board.
While the company is currently secure, the question remains: will OpenAI lean more towards a for-profit
approach or maintain its non-profit values?
And how would it impact the AGI development? Only time can provide the answers. One certainty is that AI isn't going
anywhere. Artificial Intelligence has become
an integral part of our world, and the sooner you adapt to using it, the easier
it would be for you to stay ahead in this rapidly changing world.