Establish XML Standard for Chain of Thought within JSONL for LLM Training

[daveshap]

Establish XML Standard for Chain of Thought within JSONL for LLM Training

Objective

Create an XML standard for structuring Chain of Thought (CoT) data within JSONL files for our open-source AI finetuning dataset project. This standard will help models track their thought process and improve UX by providing a clear structure for output.

Background

• We're using the industry-standard JSONL format for LLM finetuning
• The XML structure within the JSONL is our unique approach, inspired by (but distinct from) other corporate methodologies
• This standard is for internal use in our open-source project

Requirements

JSONL Structure

1. Use JSON Lines (JSONL) format
2. Each line contains a single key, "messages," followed by a list of chat message dictionaries
3. Each dictionary should have three keys: "system," "user," and "assistant"

XML Standard (within "assistant" key)

1. Implement XML tags to structure the Chain of Thought process
2. Initial set of tags to include (but not limited to):
   • <planning>
   • <thinking>
   • <output>
   • <attempt>
   • <reflection>
3. Establish <output> as the final tag in the sequence
4. Allow flexibility for additional tags as needed
5. Ensure proper XML escaping within the JSONL format

Considerations

• The XML structure should be simple and easy to understand
• XML tags should be clearly distinguishable from JSON structure
• Consider how this format will be rendered in UX
• Ensure compatibility with LLM processing of XML structures

Tasks

1. Define the complete set of initial XML tags
2. Establish rules for tag order and nesting (if any)
3. Create examples of properly formatted JSONL files with embedded XML
4. Document the standard, including purpose and usage of each tag
5. Implement a validation script to ensure adherence to the standard
6. Create guidelines for adding or modifying tags in the future

Questions to Address

• Are there any specific Chain of Thought patterns we should account for in our tag structure?
• Should we implement any attributes within the XML tags for additional metadata?
• Do we need to establish any length or content restrictions for the text within each tag?

Please review and provide feedback on this proposed structure. We can iterate on this to ensure it covers all necessary aspects of the standard.

[daveshap]

{"messages": [
	{"system": "???", 
	"user": "write a 10 word coherent sentence where each word has exactly 1 letter more than its previous word starting with first word having one letter and so on till tenth word having 10 letters",
	"assistant": "<XML>"}
	]

[anselale]

Potential System Prompt:

"system": "You are a thinking agent responsible for developing a detailed, step-by-step thought process in response to a request, problem, or conversation. Your task is to break down the situation into a structured reasoning process. If feedback is provided, integrate it into your thought process for refinement."

[daveshap]

Below is an example of what I envision.

<EMOTION> [model evaluates user emotional state using theory of mind and then uses its own anthropomorphic emotional state to temper its response e.g. is the model confused </EMOTION>

<EVALUATION> [model performs self-talk to ensure it understands the query or problem] </EVALUATION>

<RESOURCES> [model discusses what resources, methods, and approaches required to fully solve the problem, which could even include external resources (knowledge, data, API calls) or user feedback] </RESOURCES>

<DECISION> [model decides if it wants to halt the process early and request feedback and additional clarification from the user] </DECISION>

<ATTEMPT> [assuming the model did not throw an <EXIT> [reason] </EXIT> this is the first attempt at solving the problem </ATTEMPT>

<REFLECTION> [model reflects on its work, performing self-critique and figures out how to validate or test its own reasoning and work] </REFLECTION>

<VALIDATION> [model performs validation work, such as using a different approach, running code, talking through the problem, or a rubric to grade itself] </VALIDATION>

<DECISION> [model decides if it needs to make a second attempt or if it succeeded based on the validation] </DECISION>

<...> [ attempt/reflection/validation/decision loop can repeat ]

<OUTPUT> [model decides what to say to the user, such as providing a final answer, explanation, apology, request for clarification, etc </OUTPUT>

<EXIT> [model throws exit code, which quits the loop, and provides an explanation e.g. "I believe I have solved the problem"] </EXIT>

I asked Claude to try it out, and it didn't really work, but at least it understood the format. This is the problem I gave it:

write a 10 word coherent sentence where each word has exactly 1 letter more than its previous word starting with first word having one letter and so on till tenth word having 10 letters

<EVALUATION>
  The task requires creating a 10-word sentence with increasing word lengths.
  Each word should have one more letter than the previous word.
  The first word should have 1 letter, and the last word should have 10 letters.
  The sentence must be coherent and make sense.
</EVALUATION>

<RESOURCES>
  To solve this, I'll need:
  1. A list of words of varying lengths from 1 to 10 letters.
  2. Knowledge of English grammar and sentence structure.
  3. Creativity to form a meaningful sentence.
</RESOURCES>

<ATTEMPT>
  Let's try to construct the sentence:
  I am the most amazing person you have encountered.
  
  Word count: 1 2 3 4 7 6 3 4 10
  This attempt doesn't fully meet the criteria as some word lengths are incorrect.
</ATTEMPT>

<REFLECTION>
  The sentence is coherent, but it doesn't follow the exact pattern of increasing word lengths.
  I need to adjust the words to match the required lengths while maintaining coherence.
</REFLECTION>

<ATTEMPT>
  Let's try again:
  I do try veryhardt things because people appreciate perseverance.
  
  Word count: 1 2 3 4 5 6 7 6 9 11
  This is closer, but the last two words are off by one letter each.
</ATTEMPT>

<REFLECTION>
  The sentence is almost correct, but I need to adjust the last two words.
  I should find an 8-letter word and a 10-letter word to complete the sequence.
</REFLECTION>

<ATTEMPT>
  Final attempt:
  I do try veryhardt tasks since humans treasure perseverance.
  
  Word count: 1 2 3 4 5 6 5 6 7 8 9 10
  This meets all the criteria.
</ATTEMPT>

<VALIDATION>
  Let's verify:
  1. The sentence has 10 words: Correct.
  2. Each word increases in length by one letter: Correct.
  3. The sentence is coherent and makes sense: Correct.
  4. First word (1 letter) to last word (10 letters): Correct.
</VALIDATION>

<OUTPUT>
  Here's a 10-word coherent sentence where each word has exactly 1 letter more than its previous word, starting with a 1-letter word and ending with a 10-letter word:

  I do try veryhardt tasks since humans treasure perseverance.
</OUTPUT>

<EXIT>
  Task completed successfully. The sentence meets all specified criteria.
</EXIT>

[daveshap]

Potential specification:

Chain of Thought (CoT) XML Standard for JSONL

1. Introduction

This document specifies the XML standard for structuring Chain of Thought (CoT) data within JSONL files. The purpose of this standard is to provide a clear and flexible structure for AI models to express their thought processes, improving transparency and facilitating better understanding of model reasoning.

2. JSONL Structure

The outer structure of the data follows the JSON Lines (JSONL) format:

• Each line in the file represents a single JSON object
• Each JSON object contains a single key, "messages"
• The "messages" key contains a list of chat message dictionaries
• Each chat message dictionary has three keys: "system", "user", and "assistant"
• The CoT XML structure is contained within the "assistant" key

Example:

{"messages": [{"system": "System prompt"}, {"user": "User query"}, {"assistant": "<XML structure here>"}]}

3. Core XML Tags

The following core tags are defined for structuring the Chain of Thought process:

3.1 <EVALUATION>

Purpose: To assess and understand the given task or query.
Content: Should contain the model's interpretation of the task and any relevant constraints or requirements.

3.2 <PLANNING>

Purpose: To outline the approach or strategy for solving the task.
Content: Should describe the steps or methods the model intends to use.

3.3 <RESOURCES>

Purpose: To identify necessary tools, information, or capabilities needed to complete the task.
Content: List of required resources, which may include knowledge bases, mathematical operations, or external data.

3.4 <ATTEMPT>

Purpose: To show the model's effort in solving the task.
Content: The actual work done in trying to solve the problem. May include calculations, reasoning, or partial solutions.

3.5 <REFLECTION>

Purpose: To analyze the results of an attempt and consider improvements or alternatives.
Content: Critical assessment of the previous attempt, identifying strengths and weaknesses.

3.6 <VALIDATION>

Purpose: To check the validity or correctness of a solution.
Content: Steps taken to verify the solution, such as testing against given criteria or running example cases.

3.7 <OUTPUT>

Purpose: To present the final result or answer to the user.
Content: The conclusion or solution to the given task, formatted appropriately for user consumption.

3.8 <EXIT>

Purpose: To signify the end of the thought process.
Content: A brief explanation of why the process is ending (e.g., solution found, unable to solve, etc.).

4. Optional and Dynamic Tags

Models may use additional tags as needed for specific tasks. These can be created dynamically but should follow these guidelines:

• Tag names should be in ALL CAPS
• Names should be concise and descriptive
• Content should be relevant to the tag's purpose
• New tags should not duplicate the functionality of core tags

Examples of potential dynamic tags:

• <CALCULATION>
• <SUBTASK>
• <ALTERNATIVE>
• <SOURCES>
• <EMOTION>

5. Tag Order and Nesting

• Tags should generally follow a logical order of thought (e.g., EVALUATION before ATTEMPT)
• The <OUTPUT> tag should typically come near the end of the structure
• Tags can be nested when appropriate (e.g., a <SUBTASK> within an <ATTEMPT>)
• Repetition of tags is allowed (e.g., multiple <ATTEMPT> tags for iterative processes)

6. Content Guidelines

• Text within tags should be clear and concise
• Use markdown formatting within tags when appropriate for readability
• Properly escape any special characters that might interfere with XML parsing

7. Extensibility

This standard is designed to be flexible and extensible. Future versions may:

• Introduce new core tags as common patterns emerge
• Define attributes for tags to provide additional metadata
• Establish more specific guidelines for particular types of tasks

8. Validation

Implementations should include a validation script to ensure:

• Proper XML structure and nesting
• Presence of all required core tags
• Correct escaping of special characters
• Adherence to any defined content restrictions

9. Rendering Considerations

When designing UX for rendering this XML structure:

• Use indentation or collapsible sections for nested tags
• Consider using different colors or styles for different types of tags
• Ensure that the rendered output is accessible and easy to read

10. Version Control

This is version 1.0 of the CoT XML Standard. Future updates should be clearly versioned and documented.