sc-openai-c2-L4-vid5_1.srt

1
00:00:04,400 --> 00:00:04,766
In this

2
00:00:04,766 --> 00:00:09,733
section, we'll focus on tasks to process
inputs, either tasks that take the input

3
00:00:09,966 --> 00:00:12,966
and generate a useful output, often
through a series of steps.

4
00:00:13,666 --> 00:00:16,900
It is sometimes important for the model
to reason in detail about a problem

5
00:00:17,100 --> 00:00:19,200
before answering a specific question.

6
00:00:19,200 --> 00:00:22,066
And if you took our previous course chart
prompt engineering

7
00:00:22,066 --> 00:00:24,900
for developers, you will have seen
a number of examples of this.

8
00:00:25,500 --> 00:00:26,666
Sometimes a model

9
00:00:26,666 --> 00:00:29,833
might make reasoning errors
by rushing to an incorrect conclusion.

10
00:00:30,066 --> 00:00:33,333
So we can reframe the query
to request a series of relevant

11
00:00:33,333 --> 00:00:36,433
reasoning steps
before the model provides a final answer

12
00:00:36,666 --> 00:00:39,733
so that it can think longer
and more methodically about the problem.

13
00:00:40,300 --> 00:00:43,566
And in general, we call the strategy
of asking the model to reason

14
00:00:43,566 --> 00:00:46,500
about a problem and steps
chain of thought reasoning.

15
00:00:46,700 --> 00:00:48,900
For some applications,
the reasoning process

16
00:00:48,900 --> 00:00:52,766
that our model uses to arrive at a final
answer would be inappropriate to share

17
00:00:52,766 --> 00:00:53,833
with the user.

18
00:00:53,833 --> 00:00:57,900
For example, in tutoring applications,
we may want to encourage students

19
00:00:57,900 --> 00:00:59,533
to work on their own answers.

20
00:00:59,533 --> 00:01:00,900
But a model's reasoning process

21
00:01:00,900 --> 00:01:03,933
about the student's solution
could reveal the answer to the student's

22
00:01:04,633 --> 00:01:07,633
inner monologue as a tactic
that can be used to mitigate this.

23
00:01:08,066 --> 00:01:11,933
And this is just a fancy way of saying hiding the model's reasoning from the user.

24
00:01:13,466 --> 00:01:14,000
The idea of

25
00:01:14,000 --> 00:01:16,966
in a monologue is to instruct the model
to put parts of the output

26
00:01:17,100 --> 00:01:17,933
that are meant to be hidden

27
00:01:17,933 --> 00:01:21,633
from the user into a structured format
that makes passing them easy.

28
00:01:22,200 --> 00:01:25,600
Then, before presenting the output
to the user, the output is passed

29
00:01:25,600 --> 00:01:27,633
and only part of
the output is made visible.

30
00:01:28,900 --> 00:01:31,866
So remember the classification problem
from a previous video.

31
00:01:32,066 --> 00:01:33,600
When we ask the model to classify

32
00:01:33,600 --> 00:01:37,233
a customer query
into a primary and secondary category.

33
00:01:37,633 --> 00:01:41,366
And based on that classification, we might
want to take different instructions.

34
00:01:41,533 --> 00:01:45,566
Imagine the customer query had been
classified into the product information

35
00:01:45,800 --> 00:01:49,400
category in the next instructions
will want to include information

36
00:01:49,400 --> 00:01:51,366
about the products we have available.

37
00:01:51,366 --> 00:01:54,733
And so in this case,
the classification would have been primary

38
00:01:55,033 --> 00:01:58,266
general inquiry,
secondary product information.

39
00:01:58,700 --> 00:02:01,166
And so let's dive into an
example starting from there.

40
00:02:02,133 --> 00:02:05,833
So let's start with our usual set up.

41
00:02:08,600 --> 00:02:12,300
So for this in a monologue example,

42
00:02:12,866 --> 00:02:16,366
we'll start with our same
two limiters that we've been using.

43
00:02:17,066 --> 00:02:21,133
And now let's
go through our system message.

44
00:02:22,800 --> 00:02:25,733
And so what we're doing here
is asking the model to reason

45
00:02:25,733 --> 00:02:28,500
about the answer
before coming to its conclusion.

46
00:02:30,233 --> 00:02:34,166
So the instruction is follow these steps
to answer the customer queries.

47
00:02:34,433 --> 00:02:38,466
The customer query will be delimited
with four hashtags or delimiter.

48
00:02:39,100 --> 00:02:41,300
So now we've split this up into steps.

49
00:02:41,933 --> 00:02:44,533
So the first step is to decide
whether the user

50
00:02:44,533 --> 00:02:47,566
is asking a question
about a specific product or products.

51
00:02:48,066 --> 00:02:50,133
And a product category doesn't count.

52
00:02:50,600 --> 00:02:54,200
Step two So if the user is asking
about specific product,

53
00:02:54,466 --> 00:02:56,866
identify whether the products
are in the following list.

54
00:02:57,300 --> 00:03:00,033
And now we've included
a list of available products.

55
00:03:00,300 --> 00:03:02,400
So here we have five available products

56
00:03:03,133 --> 00:03:07,033
that all varieties of laptops
and these are all made up

57
00:03:08,333 --> 00:03:09,400
products.

58
00:03:09,400 --> 00:03:13,366
They were actually generated by GBG for

59
00:03:14,000 --> 00:03:15,266
step three.

60
00:03:16,200 --> 00:03:18,400
If the message contains
products in the list

61
00:03:18,400 --> 00:03:22,200
above, list any assumptions that the user
is making in that message.

62
00:03:22,300 --> 00:03:25,700
For example, that Laptop X is bigger

63
00:03:25,700 --> 00:03:29,600
than a laptop Y or that Laptop
Z has a two year warranty.

64
00:03:29,600 --> 00:03:32,233
For example.

65
00:03:33,066 --> 00:03:36,066
Step four is
if the user made any assumptions.

66
00:03:36,066 --> 00:03:39,266
Figure out whether the assumption is true
based on your product information.

67
00:03:39,600 --> 00:03:42,800
And step five is first politely correct.

68
00:03:42,800 --> 00:03:45,066
The customer's incorrect assumptions,
if applicable,

69
00:03:45,633 --> 00:03:48,700
only mentioned or reference products
in the list of five available products,

70
00:03:49,066 --> 00:03:51,300
as these are the only five products
that the store sells.

71
00:03:51,633 --> 00:03:54,266
And answer the customer
in a friendly tone.

72
00:03:54,266 --> 00:03:58,333
And this kind of very pedantic
instructions are probably unnecessary

73
00:03:58,333 --> 00:04:00,600
for a more advanced
language model like Djibouti four.

74
00:04:01,600 --> 00:04:02,300
And then

75
00:04:02,300 --> 00:04:04,833
ask the model to use a following format.

76
00:04:05,200 --> 00:04:06,800
So step one.

77
00:04:06,800 --> 00:04:11,333
It's a limit to its reasoning,
step to the limits of reasoning and so on.

78
00:04:11,733 --> 00:04:15,066
And using the delimiter will mean
that it will be easier for us later

79
00:04:15,066 --> 00:04:19,200
to get just this response to the customer
and kind of cut off

80
00:04:19,200 --> 00:04:21,733
everything before.

81
00:04:22,966 --> 00:04:24,866
So now let's try an example.

82
00:04:24,866 --> 00:04:26,500
Use a message.

83
00:04:27,033 --> 00:04:29,866
So our message is

84
00:04:29,866 --> 00:04:33,966
by how much is a blue wave Chromebook
more expensive than the tech pro desktop?

85
00:04:34,733 --> 00:04:38,533
So let's take a look at these
two products.

86
00:04:39,166 --> 00:04:43,300
The blue wave Chromebook is 249 99

87
00:04:43,900 --> 00:04:48,000
and the tech pro
desktop is actually 999 99.

88
00:04:48,266 --> 00:04:50,366
This is not actually true.

89
00:04:50,366 --> 00:04:56,233
And so let's see how the model handles
this user request.

90
00:04:56,466 --> 00:05:00,233
So we'll format into our messages array
and we'll get a response

91
00:05:08,633 --> 00:05:11,233
and then we'll print it.

92
00:05:16,133 --> 00:05:18,500
And so what we're hoping for
is that the model takes all of these

93
00:05:18,500 --> 00:05:22,900
different steps and realizes that the user
has made an incorrect assumption

94
00:05:23,300 --> 00:05:25,633
and then follows the final step

95
00:05:25,900 --> 00:05:29,533
to politely correct that user.

96
00:05:29,666 --> 00:05:33,133
And so within this one prompt,
we've actually maintained

97
00:05:33,466 --> 00:05:36,633
a number of different complex states
that the system could be in.

98
00:05:36,633 --> 00:05:37,733
So, you know, at

99
00:05:37,733 --> 00:05:38,600
any given point,

100
00:05:38,600 --> 00:05:40,766
that could be a different output
from the previous step

101
00:05:41,033 --> 00:05:42,833
and we would want to do
something different.

102
00:05:42,833 --> 00:05:46,300
For example,
if the user hadn't made any assumptions in

103
00:05:46,500 --> 00:05:50,100
in step three, then in step four,
we wouldn't actually have any output.

104
00:05:50,100 --> 00:05:53,233
So this is a pretty complicated
instruction for the model.

105
00:05:53,233 --> 00:05:55,433
So let's see if it did it right.

106
00:05:55,433 --> 00:05:59,500
So step one, the user is asking a question
about specific products,

107
00:06:00,133 --> 00:06:03,000
the asking about the price difference
between these two products.

108
00:06:03,533 --> 00:06:06,600
The user assumes that the blue wave
Chromebook is more expensive

109
00:06:06,600 --> 00:06:10,200
than the textbook pro
and this assumption is actually incorrect

110
00:06:11,066 --> 00:06:13,833
reasoning through taking longer
to think about the problem

111
00:06:14,100 --> 00:06:16,733
in the same way
that human would also take some time

112
00:06:16,733 --> 00:06:19,800
to reason
about an answer to any given question.

113
00:06:20,100 --> 00:06:22,933
The model performs better
if it also has time to think.

114
00:06:23,533 --> 00:06:26,966
And so the final response to the user is
the blue wave

115
00:06:26,966 --> 00:06:30,766
Chromebook is actually less expensive
than the tech, but pro the textbook pro

116
00:06:30,766 --> 00:06:34,633
desktop costs
999 99 while the blue wave Chromebook

117
00:06:34,633 --> 00:06:38,500
costs 249 99.

118
00:06:38,500 --> 00:06:41,800
And so let's see
another example of a user message.

119
00:06:42,633 --> 00:06:44,866
And also at this point,
feel free to pause the video

120
00:06:44,866 --> 00:06:49,433
and try your own messages.

121
00:06:49,433 --> 00:06:51,266
So let's format this user message.

122
00:06:51,266 --> 00:06:53,633
So the question is, do you sell TVs?

123
00:06:53,633 --> 00:06:58,200
And if you remember in our product list,
we've only listed two different computers.

124
00:06:58,200 --> 00:07:01,633
So let's see what the model says.

125
00:07:02,133 --> 00:07:06,166
So in this case, step one,
the user is asking if the store sells TVs,

126
00:07:06,466 --> 00:07:08,633
but TVs are not listed
in the available products.

127
00:07:08,633 --> 00:07:12,400
So as you can see, the model
then skips to the response to use a step

128
00:07:12,400 --> 00:07:17,200
because it realizes that the intermediary
steps are not actually necessary.

129
00:07:17,800 --> 00:07:20,966
I will say that we did ask for the output
in this specific format.

130
00:07:20,966 --> 00:07:24,400
So technically the model hasn't
exactly followed our request.

131
00:07:24,866 --> 00:07:27,066
Again, more advanced
models will be better at doing that.

132
00:07:27,800 --> 00:07:30,900
And so in this case,
our response to the user is, I'm sorry,

133
00:07:30,900 --> 00:07:37,666
but we do not sell TVs at the store
and then it missed the available products.

134
00:07:37,666 --> 00:07:40,233
So again, feel free
to try some of your own responses.

135
00:07:40,866 --> 00:07:44,900
And so now we only really want
this part of the response.

136
00:07:44,933 --> 00:07:47,466
We wouldn't want to show
the earlier parts to the user.

137
00:07:47,766 --> 00:07:50,266
So what we can do is actually just

138
00:07:51,066 --> 00:07:53,500
cut the string at the last occurrence

139
00:07:53,500 --> 00:07:57,233
of this delimiter token
or string of four hash tags

140
00:07:57,366 --> 00:08:02,066
and then only print the final
part of the model output.

141
00:08:03,433 --> 00:08:04,000
So let's write

142
00:08:04,000 --> 00:08:06,433
some code to get
only the final part of this string.

143
00:08:07,766 --> 00:08:10,166
So we're going to use a try accept block

144
00:08:10,666 --> 00:08:13,900
to gracefully handle errors
in case the model has some kind of

145
00:08:13,900 --> 00:08:18,300
unpredictable output and doesn't
actually use these these characters.

146
00:08:18,966 --> 00:08:22,700
And so we're going to say
our final response is the response,

147
00:08:23,433 --> 00:08:26,366
and then we're going to split the string
at the delimiter string.

148
00:08:26,866 --> 00:08:29,633
And because we want the final comments,
we just want to get the last

149
00:08:30,266 --> 00:08:32,700
item in the output list,

150
00:08:32,700 --> 00:08:35,033
and then we're going to strip
any whitespace

151
00:08:36,000 --> 00:08:38,766
because as you can see, there
might be whitespace after the characters

152
00:08:40,433 --> 00:08:43,500
and we're going to catch any errors

153
00:08:44,733 --> 00:08:50,100
and have a fallback response,

154
00:08:50,100 --> 00:08:55,500
which is sorry, I'm having

155
00:08:57,733 --> 00:08:59,700
I'm having trouble right now.

156
00:09:01,500 --> 00:09:02,900
Please

157
00:09:03,266 --> 00:09:06,000
try asking another question

158
00:09:08,300 --> 00:09:09,300
and then let's print

159
00:09:09,300 --> 00:09:21,966
our final response.

160
00:09:21,966 --> 00:09:26,766
And so as you can see, we just cut
the string to get this final output.

161
00:09:26,766 --> 00:09:28,400
And so this is what we would show
to the user

162
00:09:28,400 --> 00:09:30,433
if we were building this
into an application.

163
00:09:32,233 --> 00:09:34,233
And overall, I just want to call out

164
00:09:34,233 --> 00:09:38,266
this prompt might be slightly convoluted
for this task.

165
00:09:38,266 --> 00:09:40,966
You might not actually need
all of these intermediate steps.

166
00:09:41,366 --> 00:09:45,100
And so why don't you try and see
if you can find an easier way to do

167
00:09:45,100 --> 00:09:47,433
the same task in your in prompt

168
00:09:49,866 --> 00:09:51,900
and in general,
finding the optimal trade off

169
00:09:51,900 --> 00:09:54,600
in prompt complexity requires
some experimentation.

170
00:09:55,033 --> 00:09:57,533
So definitely good
to try a number of different prompts

171
00:09:57,533 --> 00:09:59,766
before deciding to use one.

172
00:09:59,766 --> 00:10:02,400
And in the next video
we'll learn another strategy to handle

173
00:10:02,400 --> 00:10:07,466
complex tasks by splitting these complex
tasks into a series of simpler subtotal

174
00:10:07,800 --> 00:10:10,500
rather than trying to do
the whole task in one prompt.