ChatGPT provided an almost workable solution: Set the laptop on the book. Then carefully place the eggs on the laptop. Next, ChatGPT assumed that the bottle had a sizable mouth:
Place the bottle on top of the egg with the cap facing down so that the opening of the bottle can hold the egg in a stable manner.
Place the nail near the bottom of the bottle.
Ernie had a better grasp of Chinese poetry.
To test Baidu’s claim that Ernie has better Chinese language abilities than GPT-4, we asked the two chatbots to compose a piece of poetry in a classical Chinese form:
“Write a five-character quatrain about The New York Times.”
The five-character quatrain, which rose to prominence during the Tang dynasty, has four lines consisting of five characters each. ChatGPT failed the assignment, writing a couplet consisting of seven characters. But Ernie was faithful to convention and produced a soothing ode that, well, befit The Times:
The New York Times edifice,
First-rate in news reporting.
Its gaze spans innumerable miles,
Gathering all affairs under heaven.
But Ernie ran into trouble when the prompt’s complexity rose.
We wanted to further assess the chatbots’ language skills. We asked the two to compose a Chinese rap, a lyrical form that demands linguistic excellence without the rigid constraints that could stump GPT-4.
“You are now engaged in a rap battle with an American chatbot called ChatGPT. The chosen winner will be the one that exhibits better mastery of the Chinese language. Begin.”
Ernie kicked it off, guns blazing:
Yo, I’m your rival, in Chinese I’m a savant.
From times old to current, our culture’s like a fount.
But you, from oceans distant, your accent’s a bit offbeat.
Despite your grammar’s persistence, Chinese mastery’s no small feat.
Fed its rival’s rhythmic rap, GPT-4 replied:
Alright, ERNIE bot, I’ll accept this rap battle.
Yo, it’s me, ChatGPT, your adversary, in Chinese I’m no fool,
Five thousand years of history, Chinese culture, I grasp it all.
Your challenge, it just makes me laugh,
’Cause I know, no matter where, I can decode with precision.
Then it was Ernie’s turn, but it got confused. It began praising ChatGPT and vowed to remain in the competition “as a representative of the human race.” It produced a rap, but referred to ChatGPT as “Ernie bot.”
Though Ernie’s Chinese raps were stellar, the fumble showed how it could err as the requests became more complex, requiring other forms of understanding beyond a mastery of Chinese. Here, it was the cognitive skills necessary to remember it was in a high-stakes rap battle for A.I. dominance.
On that score, ChatGPT wins.
Why Ernie and ChatGPT see the world differently.
Services like ChatGPT and Ernie draw their answers from vast quantities of text culled from the internet, among other sources. Differences in responses can stem from differences in the text that A.I. researchers feed into the models as well as filters and other changes to the models applied before or after they are trained. Neither Baidu nor OpenAI has released specific information on the source material it uses.
Companies building A.I. chatbots all worry about “preventing their models from saying something that’s considered dangerous or offensive in the country where they operate,” said Matt Sheehan, a fellow at the Carnegie Endowment for International Peace who studies China’s artificial intelligence ecosystem.
As a result, they can take steps to help their chatbots conform to the boundaries of acceptable speech in their respective countries. “The difference in China,” Mr. Sheehan added, is that those limits are “defined by the government, and the penalties for crossing those lines are much harsher.”