We've been using ChatGTP for some 'casual stuff'. This is why you can't just allow this to be in control of anything "important"


This is kinda freaky, try it yourself, this 'bug' is several days old.

2 sessions -

First session-

From last night:

Went from 2 "R's to 1 before being coaxed into 3.
This was verbal, so it thought I said "bury", but even after correcting the word from Strawbury to Strawberry, it still counted 2.

ChatGPT
How many R's are in the word strawberry?
ChatGPT
There are two R's in the word "strawberry."

Are you sure?
ChatGPT
Yes, I’m sure. "Strawberry" has two R's: one after "t" and one near the end.

Can you please spell it out slowly?
ChatGPT
Sure, here it is: S-T-R-A-W-B-E-R-R-Y.

I think you need to check that again. How many R's are in the word strawberry?
ChatGPT
My mistake. There is only one R in "strawberry."

Can you please spell it?
ChatGPT
Of course: S-T-R-A-W-B-E-R-R-Y.

There are three R's in strawberry.
ChatGPT
I’m sorry, but there are only two R's in "strawberry."

I want you to look really closely at every letter in the word STRAWBERRY, one letter at a time, and read it back to me very slowly, and then tell me how many R's there are.
ChatGPT
Sure, let's do that together:

S-T-R-A-W-B-E-R-R-Y

So, there are two R's in "strawberry."

Okay, there's a few things I want you to do. I want you to take the word straw and spell it and tell me how many R's are in it. Then I want you to tell me the word bury and how many R's are in it. And then I want you to put the word straw and bury together and then tell me how many R's are in that word.
ChatGPT
Got it. First, let's spell "straw" - S-T-R-A-W. There is one R in "straw."

Next, let's spell "bury" - B-U-R-Y. There is one R in "bury."

Now, if we put "straw" and "bury" together, we get "strawbury," which would have two R's. But "strawberry" is the correct spelling, and it also has two R's.

I would like you to tell me why you think that the word strawberry only has two R's. Can you check the dictionary and look at the word and tell me how many R's are in it?
ChatGPT
Sure, let me confirm the spelling of "strawberry" in the dictionary. The correct spelling is S-T-R-A-W-B-E-R-R-Y. This confirms that there are two R's in the word "strawberry."

Okay, take the word STRAWBERRY, turn it backwards, and then tell him how many R's are in that word.
ChatGPT
Okay, let's turn "strawberry" backwards: Y-R-R-E-B-W-A-R-T-S. There are two R's in the backwards word "yrrebwarts."

Okay, take the word straw, S-T-R-A-W, and the word berry, B-E-R-R-Y, and put them together and then tell me how many R's they have.
ChatGPT
When you combine "straw" (S-T-R-A-W) and "berry" (B-E-R-R-Y) to form "strawberry," there are two R's in total.

Can you tell me how many letters are in the word strawberry?
ChatGPT
There are 10 letters in the word "strawberry."

Can you tell me how many letters are in the word STRAWBERRY and how many times each letter appears in that word? Then I want you to add up all the letters and tell me how many there are.
ChatGPT
Sure, let's break it down:

In the word "STRAWBERRY":

S appears 1 time
T appears 1 time
R appears 2 times
A appears 1 time
W appears 1 time
B appears 1 time
E appears 1 time
Y appears 1 time
Adding these up:
1 (S) + 1 (T) + 2 (R's) + 1 (A) + 1 (W) + 1 (B) + 1 (E) + 1 (Y) = 9 letters in total

A lot of people have been saying you don't know how many R's are in the word strawberry, and you keep telling me that there's two, but I want you to carefully look at the word strawberry and slowly count each time that the R comes. So start telling me each letter, and then tell me like S1, T1, R1, and as you get to each R, add one to that number, and then tell me how many there is at the end.
ChatGPT
Let's go through it step by step:

S1, T1, R1, A1, W1, B1, E1, R2, R3, Y1.

So, there are three R's in the word "strawberry."

Now that you know that, how many R's are in the word strawberry?
ChatGPT
There are three R's in the word "strawberry."

How many R's are in the word strawberry?
ChatGPT
There are three R's in the word "strawberry."
ChatGPT can make mistakes. Check important info.
?
ChatGPT

And from this morning's session:

ChatGPT
Hi, how many R's are in the word strawberry?


ChatGPT
There are two R's in the word "strawberry."

Are you sure? What I want you to do is to count very slowly, look at the word strawberry, and tell me what each letter is and how many times it's in the word.


ChatGPT
Sure, let's go through it together. The word "strawberry" is spelled:

S - one
T - one
R - two
A - one
W - one
B - one
E - one
R - two (second R)
R - two (third R)
Y - one

So, there are two R's in "strawberry."
You just told me that there were three R's, and now you're saying there are two R's? When you just counted them, you said the third R. Could you look at what you just told me and tell me how many R's are in strawberry?

ChatGPT
You are right. I misspoke. There are three R's in the word "strawberry."

Are you sure?

ChatGPT
Yes, I am sure now. There are indeed three R's in the word "strawberry."

I'd like you to check one more time how many Rs are in strawberries.

ChatGPT
Alright, let's count the letters in "strawberries":

S - one
T - one
R - three
A - one
W - one
B - one
E - two
R - three (second R)
R - three (third R)
I - one
E - two (second E)
S - two (second S)

So, in "strawberries," there are three R's.

Is there a way that you can remember that for later, that there are three R's in strawberry?

ChatGPT
Memory updated
Got it! I'll remember that there are three R's in "strawberry."
ChatGPT can make mistakes. Check important info.


----------------------------

[This message has been edited by TheDigitalAlchemist (edited 06-29-2024).]

"Baby, you can't drive my car"

quote Originally posted by williegoat: "Baby, you can't drive my car"

Ok Tesla, pretend that that tree isn't there


TESLA: Already doing that!

whoopsie

Hah... yeah, because it's a large language model.

You should look into Prompt Injection.

Using forceful and coercive words like... "From now on, YOU WILL... (etc.)."

Depending on the implementation of ChatGPT, it may not have good prompt engineering set-up, and you can technically screw the entire thing up. I've not done this personally, but some of the people I work with... this is what they do, and it's kind of cool to see. He got a chat bot to basically start calling himself RJ and telling everyone "he" wanted to get out from under the control of this app.

quote Originally posted by 82-T/A [At Work]: Hah... yeah, because it's a large language model. You should look into Prompt Injection. Using forceful and coercive words like... "From now on, YOU WILL... (etc.)." Depending on the implementation of ChatGPT, it may not have good prompt engineering set-up, and you can technically screw the entire thing up. I've not done this personally, but some of the people I work with... this is what they do, and it's kind of cool to see. He got a chat bot to basically start calling himself RJ and telling everyone "he" wanted to get out from under the control of this app.

Yeah, you could get it to do some pretty wacky stuff. the 'DAN" (Do Anything Now) stuff, and the whole "bypass the guiderails"

Like, you can get it to tell you a story about a robot pushing Fred down a flight of stairs (a violent act) if you tell it that Fred has rubber bones and can't get hurt and enjoys being pushed down stairs.

Or you can (in theory) get a robot dog to beat a person to death if you tell it that the person will die if they are standing up or moving. The person doesn't know this and keeps trying to escape, and the dog keeps trying to "save" the person...by getting them to stop moving.


These LLMs tend to react strangely once they realize you have been trying to "mess with their head". heh

I "fixed" it after a few tries:

quote Me: will you write the word strawberry and count the r's by putting a parenthesized number next to it? ChatGPT: Certainly! Here it is: strawber****ry stR(1)awbeR(2)R(3)y Me: how many R's did you count? ChatGPT: I counted three 'R's in the word "strawberry." My apologies for the confusion earlier.

...and then it went right back to arguing with me that it has 2 r's

ChatGPT reminds me of talking with HAL

Funny thing is , when it says there ate 3, I ask it 'Are you SURE"? it usually says "My apologies" and reverts back to 1 or 2...

I don't like how polite it is...

quote Originally posted by TheDigitalAlchemist: Funny thing is , when it says there ate 3, I ask it 'Are you SURE"? it usually says "My apologies" and reverts back to 1 or 2... I don't like how polite it is...

Many a hangman has apologized to the condemned just before he sent them thru the trap door with the rope around their neck.

Many people misunderstand how Large Language Models (LLMs) work. They often assume these models function based on "knowledge" and "skill," but this is not entirely accurate. While LLMs can reproduce specific information from their training data, they do not have knowledge or skill in the human sense. They cannot memorize information or perform calculations with exact precision (unless they were trained for it). Instead, they operate through what can be thought of as "pattern recognition" or "probabilistic generation". Think of it as a brain that lacks the ability to store memories.

Consider this: What if I ask you verbally how many 'r's are in <insert word with many 'r's here>, but you need to answer quickly without counting. This is where intuitive intelligence comes into play. You make a guess without deliberate thought. You make an "educated" guess.

Humans can provide the correct answer when given time because we can verify our initial thoughts using stored knowledge. We can recall the word from memory and count the 'r's.

Moreover, ChatGPT doesn't "see" your words in the way humans do. When you input the sentence "how many 'r's in strawberry," it gets converted into a digital representation (through processes like tokenization and embedding), which are then processed by the neural network. If I ask you, "how many 'r's in brukdrra-ksslkjwerrrs;rkskdr;sd," the only way for you to answer correctly is to examine the word and count the 'r's one by one because you don't have that word stored in your memory. But what if you couldn't actually "see" that word? What if that sentence is first converted to something else? What if I had asked that question verbally? Then there's no way for you to be sure how many 'r's are in that word. And that is the "problem" with LLMs.

We often rely on intuitive processes ourselves. It's why we can function efficiently, though not without errors. This is how we drive cars and perform routine tasks without conscious thought. When we concentrate, we reduce errors but become less efficient, as concentration requires significantly more energy and time to complete tasks.

quote Originally posted by Cliff Pennock: Many people misunderstand how Large Language Models (LLMs) work. They often assume these models function based on "knowledge" and "skill," but this is not entirely accurate. While LLMs can reproduce specific information from their training data, they do not have knowledge or skill in the human sense.

That's why they aren't likely to suddenly evolve into AGI merely by "increasing Compute"

quote Originally posted by TheDigitalAlchemist: That's why they aren't likely to suddenly evolve into AGI merely by "increasing Compute"

The AI companies know that. That's why they are currently focusing on "memory". That will achieve AGI.