The internet has always been a place where you could construct a better image of yourself, as long as you were careful to curate what you put out there. I’m like most people in that I rarely write about my insecurities and failings online. I certainly don’t post about anything I’m ashamed of. But a recent article in Wired makes the point that new generative AI capabilities make our public image more subject to manipulation, not only by us but by others. (Thanks for pointing out the article, Perry Kinkaide.)
I was curious to find out the difference in the responses one would get from OpenAI’s ChatGPT vs Microsoft’s Bing version, so I decided to test them both out with a subject I know well: me. The prompt was simply “Tell me about Tema Frank”.
The OpenAI result was certainly the one that seemed the most comprehensive and compelling, but it was riddled with factual errors. I’ve noted them below.
Bing’s version gives you three choices: Creative, Balanced or Precise. I tried all three. They were shorter, and none made me sound as impressive as the OpenAI version but they were more accurate, if incomplete.
I’ve noted before that I worry about OpenAI’s persuasive but inaccurate responses. Despite their claims to be improving the accuracy, my bio check a couple of days ago suggests they haven’t succeeded.
Microsoft seems to have taken the approach that they’d rather under-report than make stuff up. I also like that they cite their sources, which OpenAI still doesn’t do. (At least not in the publicly available version.) Those limitations, though, also mean that it is not giving you anything like the kind of depth that proponents of these new generative technologies are so excited about.
It seems to me that we are still a very long way from the excited “this will make the world so much better” claims of the proponents becoming real. But the risks of deception, intentional or accidental, have arrived.
OpenAI’s ChatGPT Response to Prompt: Tell me about Tema Frank
My corrections to the OpenAI ChatGPT bio are in square brackets. Six of the 11 sentences (55%) contained false information.
Tema Frank is a Canadian author, speaker, and customer experience strategist. She is known for her work in the field of customer experience and has authored two books on the subject. [I have authored 2 books, but only one of them was on customer experience.]
Frank began her career in the tech industry in the 1980s, working for companies such as Microsoft and Compaq. [Nope. Never worked for either of those companies.] In 2001, she founded Web Mystery Shoppers Inc., a company that provides website usability and customer service testing services to businesses.
Frank has also been a speaker and consultant on customer experience, helping companies to improve their customer service, increase customer loyalty, and drive business growth. She has spoken at conferences and events around the world, including the World Customer Service Forum in Brazil and the Customer Experience Management Conference in the UK. [I have spoken internationally, including far-flung places like South Africa and Kuwait, but not at either of those events. Didn’t even attend them, so I have no idea where it got that idea from.]
In addition to her work in customer experience, Frank is an advocate for accessibility and inclusion in the digital world. She has written and spoken extensively on the subject, and is a founding member of the Accessibility Advisory Committee for the City of Edmonton. [I have never been on that committee. Does it even exist?]
Frank has authored two books on customer experience: “Canada’s Best Employers for Women: A guide for job hunters, employees and employers” [not about customer experience] and “PeopleShock: The Path to Profits When Customers Rule”. She is also a regular contributor to various publications, including Forbes and Customer Experience Magazine. [I’ve been quoted in Forbes, but I don’t think I ever wrote for them. I did write a regular column for User Experience Magazine, but I don’t recall having written for Customer Experience Magazine. If I did, it isn’t showing up in their search.]
Overall, Tema Frank is a respected figure in the field of customer experience, known for her expertise, insights, and thought leadership on the subject.
ChatGPT (OpenAI), result provided April 8, 2023
Bing’s ChatGPT equivalent on Balanced Mode
Tema Frank is a customer experience and digital marketing pioneer, who founded the world’s first omnichannel customer service testing company in 2001, with the creation of Web Mystery Shoppers International1. She has worked in customer experience, marketing, and usability testing for over three decades2. She is also the author of best-seller, PeopleShock: The Path to Profits When Customers Rule3.
These results came mostly from sources I’d provided data to, so they are accurate. But it’s not much of a bio.
Here are my comments on the sources it referenced. (And kudos to them for at least citing their sources!)
1. allamericanspeakers.com/celebritytalentbios/Tema+Frank/405705
[Interesting, since I don’t think I ever registered with them (though maybe I did). Nice that they think I’m “celebrity talent” though!]
2. https://www.wbresearch.com/events-b2bmarketingwbresearchcom/blog/tema-frank-explores-digital-marketing-customer-experience
[I spoke at some of their events, and the article they’ve linked to is an interview they did with me.]
3. https://customerthink.com/author/temafrank/
[I wrote several articles for them, and this is from a bio I submitted to them.]
Bing GPT on Creative Mode
Tema Frank is a historical fiction and business author, a speaker, a customer experience expert, and a global traveler1. She has written two bestsellers, including “PeopleShock: The Path to Profits When Customers Rule” and “Canada’s Best Employers for Women”. She also hosts the Frank Reactions podcast on customer experience2. [Dated, but I DID host a podcast from 2012 till 2018.]
She founded the world’s first omnichannel customer service testing company in 2001, called Web Mystery Shoppers International, which used a panel of 75,000 website and customer service testers worldwide3. She has spoken and taught in various countries, such as US, Canada, France, Mexico, Peru, and South Africa2.
She is also a University of Alberta Senator and serves on various boards of directors, such as BusinessLink, Litfest, and Ethically Aligned AI2. She enjoys traveling to different places and writing about her experiences on her website1.
I liked that it includes my more current activities — i.e. that I’m writing historical fiction. Logical that the first place the search engine would go would be my own website. Oddly, OpenAI’s bot didn’t even look there, apparently. I also liked that it referenced my two bestselling books, but didn’t assume they were both about customer experience. I’m no longer on some of the boards mentioned in the 3rd paragraph, but at least I did, in fact, serve on those boards.
My comments on the sources referenced:
1. temafrank.com [My website, so it should be accurate!]
2. My linkedin bio [So again, it is accurate.]
3. AllAmerican Speakers bio. [See comments Bing Balanced Mode source 1.]
Bing GPT on Precise Mode
Tema Frank is a bestselling author, international speaker, and senior customer experience and marketing strategist based in the Greater Edmonton Metropolitan Area 1. She is also a historical fiction and business author, speaker, customer experience expert, and global traveler 2. In 2001, she founded Web Mystery Shoppers International, the world’s first omnichannel customer service testing company 3.
As with their Balanced Mode, this one is not very comprehensive, but at least it is factual. Nothing to add about the sources. (Sources: 1. My LinkedIn bio 2. My website 3. AllAmerican Speakers)
How Will GPT Search Responses Change Behaviour?
One thing that really scares me is that, because OpenAI’s model provides more extensive answers, people will gravitate to it. I’ve even found myself doing that: I start with Bing (mainly because about 3/4 of the time I try to use OpenAI it won’t connect), but since those results feel too skimpy I do tend to want to go check OpenAI. But since OpenAI isn’t providing sources, few users will bother to verify the accuracy of its results. In fact, you wouldn’t even know which claims to verify. Realistically, who’s going to spend the time fact-checking every phrase? (Most people wouldn’t bother even if it did link to its sources, but at least those who cared could.)
If you haven’t tried yet, I’d urge you to check out what OpenAI’s ChatGPT and Bing AI have to say about you. Please share their responses in the comments section (scroll down to find it), and add your notes about what was true and what wasn’t.
Thanks for the feedback. You’d think a “super-intelligence” would be able to distinguish between Tema and Tonia! (Even a basic computer does that: I remember when I was learning programming in the late 1970s and got a million errors because — based on my typewriter days — I used zeros and Os interchangeably and a lower-case L and the number one!)
I’d actually wondered about joining such a board after my 3 months in London confined to a mobility scooter. Talk about eye-opening! I think all urban planners should have to spend time in a wheelchair or mobility scooter. (You might find this article I wrote from that period of interest https://temafrank.com/the-catch-22-of-access-for-people-with-disabilities/ )
Interesting, and somewhat frightening, Tema. As to your question about the Edmonton Accessibility Advisory Committee, perhaps Chat GPT is confusing you with Tonia. See link: https://www.edmonton.ca/city_government/city_organization/board-members