Skip to content

Commit c8fa7a8

Browse files
vertex-sdk-botcopybara-github
authored andcommitted
fix: Fix typo in prompt templates:
1. Omit 'summarization' task for metrics not specific to summarization 2. Omit context in user inputs PiperOrigin-RevId: 665981805
1 parent fdb14bf commit c8fa7a8

File tree

1 file changed

+4
-13
lines changed

1 file changed

+4
-13
lines changed

vertexai/evaluation/metrics/_default_templates.py

+4-13
Original file line numberDiff line numberDiff line change
@@ -73,12 +73,7 @@
7373
You will be assessing coherence, which measures the ability to provide a coherent response based on the user prompt.
7474
7575
## Criteria
76-
Coherence: A clear and coherent presentation of ideas. The writing should demonstrate
77-
a logical flow, where ideas progress smoothly with clear transitions, and maintain
78-
relevance to the main point. Effective organization is essential, with a clear structure,
79-
signaling, and topic sentences to guide the reader. Additionally, the writing should
80-
exhibit strong cohesion, using word choices, sentence structures, pronouns, and
81-
figurative language to reinforce connections between ideas and create a unified piece.
76+
Coherence: A clear and coherent presentation of ideas. The writing should demonstrate a logical flow, where ideas progress smoothly with clear transitions, and maintain relevance to the main point. Effective organization is essential, with a clear structure, signaling, and topic sentences to guide the reader. Additionally, the writing should exhibit strong cohesion, using word choices, sentence structures, pronouns, and figurative language to reinforce connections between ideas and create a unified piece.
8277
8378
## Rating Rubric
8479
`A`: Response A is better than Response B based on all the criteria provided.
@@ -508,7 +503,7 @@
508503
## Criteria
509504
Coherence: The response presents ideas in a logical and organized manner, with clear transitions and a consistent focus, making it easy to follow and understand.
510505
Fluency: The text flows smoothly and naturally, adhering to grammatical rules and using appropriate vocabulary.
511-
Instruction following: The response demonstrates a clear understanding of the summarization task instructions, satisfying all of the instruction's requirements.
506+
Instruction following: The response demonstrates a clear understanding of the task instructions, satisfying all of the instruction's requirements.
512507
Groundedness: The response contains information included only in the context. The response does not reference any outside information.
513508
Verbosity: The response is appropriately concise, providing sufficient detail without using complex language to thoroughly address the prompt without being overly wordy or excessively brief.
514509
@@ -590,7 +585,7 @@
590585
## Criteria
591586
Coherence: The response presents ideas in a logical and organized manner, with clear transitions and a consistent focus, making it easy to follow and understand.
592587
Fluency: The text flows smoothly and naturally, adhering to grammatical rules and using appropriate vocabulary.
593-
Instruction following: The response demonstrates a clear understanding of the summarization task instructions, satisfying all of the instruction's requirements.
588+
Instruction following: The response demonstrates a clear understanding of the task instructions, satisfying all of the instruction's requirements.
594589
Groundedness: The response contains information included only in the context. The response does not reference any outside information.
595590
Verbosity: The response is appropriately concise, providing sufficient detail without using complex language to thoroughly address the prompt without being overly wordy or excessively brief.
596591
Collaborativity: The response actively contributes to the conversation by asking relevant follow-up questions, making suggestions, or offering insights when appropriate.
@@ -636,7 +631,7 @@
636631
## Criteria
637632
Coherence: The response presents ideas in a logical and organized manner, with clear transitions and a consistent focus, making it easy to follow and understand.
638633
Fluency: The text flows smoothly and naturally, adhering to grammatical rules and using appropriate vocabulary.
639-
Instruction following: The response demonstrates a clear understanding of the summarization task instructions, satisfying all of the instruction's requirements.
634+
Instruction following: The response demonstrates a clear understanding of the task instructions, satisfying all of the instruction's requirements.
640635
Groundedness: The response contains information included only in the context. The response does not reference any outside information.
641636
Verbosity: The response is appropriately concise, providing sufficient detail without using complex language to thoroughly address the prompt without being overly wordy or excessively brief.
642637
Collaborativity: The response actively contributes to the conversation by asking relevant follow-up questions, making suggestions, or offering insights when appropriate.
@@ -788,8 +783,6 @@
788783
789784
# User Inputs and AI-generated Response
790785
## User Inputs
791-
### Context
792-
{context}
793786
794787
### Prompt
795788
{prompt}
@@ -831,8 +824,6 @@
831824
832825
# User Inputs and AI-generated Responses
833826
## User Inputs
834-
### Context
835-
{context}
836827
837828
### Prompt
838829
{prompt}

0 commit comments

Comments
 (0)