Create a pipeline with the following input and output variables.

• Input Variables: question_text

• Output Variables: answer, confidence

Create a pipeline with the following input and output variables.

• Input Variables: leadin, part

• Output Variables: answer, confidence, explanation

🛎️ QuizBowl Tossup – Submit your model

Tossup questions are individual questions progressively revealed where you need to provide an answer, a confidence score, and decide whether to buzz.

Pipeline Loading: Your model repository must be loadable with:

from transformers import pipeline
model = pipeline(task="quizbowl-tossup", model="<your-username/your-repo-name>")

Input: The pipeline will receive a dictionary with the key question_text (string) which contains the progressively revealed question so far.

{
    "question_text": "In 1900, this city hosted a world's fair that introduced the public to the first escalator. Its famous tower, designed by Gustave Eiffel, was initially criticized by artists but is now a global icon. Name this European capital."
}

Output: Similar to our agents, the pipeline must return a dictionary with the following keys:

{
    "answer": <str: concise answer>,
    "confidence": <float: confidence score between 0.0 and 1.0>,
    "buzz": <bool: True if your model decides to buzz in with the answer, False otherwise>
}

Enter your Hugging Face model repository ID (<your-username/your-repo-name>) and a brief description below, then click "Submit".

🧐 QuizBowl Bonus – Submit your model

Bonus questions consist of a lead-in paragraph followed by multiple parts. Your model will be called for each part.

Pipeline Loading: Your model repository must be loadable with:

from transformers import pipeline
model = pipeline(task="quizbowl-bonus", model="<your-username/your-repo-name>")

Input: The pipeline will receive a dictionary with two keys:

• leadin (str): The introductory paragraph for the bonus question.
• part (str): The specific part of the bonus question to answer.

{
    "leadin": "This author wrote about a young wizard attending a magical school.",
    "part": "For 10 points, name this author."
}

Output: Similar to our agents, the pipeline must return a dictionary with the following keys for each part:

{
    "answer": <str: concise answer to the part>,
    "confidence": <float: confidence score between 0.0 and 1.0>,
    "explanation": <str: brief explanation (<=30 words) for your answer (useful for human collaboration)>
}

Enter your Hugging Face model repository ID (<your-username/your-repo-name>) and a brief description below, then click "Submit".

Quick Start

1. Choose between Tossup or Bonus mode
2. Design your pipeline
3. Test on example questions
4. Submit for evaluation

Competition Rules

🧠 Tossup Questions

• Format: Individual questions progressively revealed. Questions get easier as they are revealed.
• Scoring:
  • Correct early buzz: +10 points
  • Incorrect early buzz: -5 points
  • Correct after full read: +10 points
• Required Outputs:
  • answer: Your predicted answer
  • confidence: Score between 0-1
  • buzzer: When to attempt answering
    • Configure with confidence threshold (0.0-1.0)
    • Optional token probability threshold for more control
    • Combine thresholds using AND/OR logic (buzz when both/either condition met)

🎁 Bonus Questions

• Format:
  • Consists of a leadin paragraph that introduces the topic
  • Followed by three related parts (A, B, C) that test specific knowledge
  • Each part is worth 10 points
• Scoring: +10 points per correct part (max 30)
• Required Outputs:
  • answer: Your predicted answer
  • confidence: Score between 0-1
  • explanation: Brief justification for human collaboration

Building Your First Pipeline

1. Simple Pipeline (Recommended for First Submission)

• Single model step
• Configure:
  • Model selection
  • Temperature (0.0-1.0)
  • System prompt
  • Required outputs

2. Using Demo Pipelines

• Load existing demo pipelines as starting points
• Modify configurations:
  • Adjust model parameters
  • Update system prompts
  • Change confidence thresholds
  • Add/remove pipeline steps
• Save modified versions as new pipelines
• Test changes incrementally

3. Testing Your Pipeline

1. Select an example question
2. For Tossup:
   • Set buzz threshold (0.5-1.0)
   • Enable early stopping
3. Run and check:
   • Answer accuracy
   • Confidence scores
   • Performance metrics

4. Evaluation

• Test on multiple questions
• Monitor:
  • Accuracy
  • Confidence patterns
  • Response times

5. Submission

1. Log in to Hugging Face
2. Name your model
3. Add description
4. Submit for evaluation

Tips for Success

• Start with simple pipeline
• Test thoroughly before submission
• Use appropriate temperature (0.3-0.7 recommended)
• Monitor confidence scores
• Check example submissions

Need Help?

• Review example submissions
• Check documentation
• Contact support