PIQA stands for Physical Interaction: Question Answering. It is a dataset of 16,000 multiple-choice questions about everyday physical tasks, requiring the model to choose the most appropriate action in different scenarios. The questions are generated by an adversarial filtering method that ensures that simple heuristics or word associations are not enough to solve them

PIQA is a benchmark for testing the commonsense reasoning ability of large language models (LLMs), such as BERT, GPT-3, and RoBERTa. It can be used to evaluate how well these models can understand natural language and perform physical tasks in various domains, such as cooking, cleaning, gardening, etc. PIQA can also be used to encourage the development of more robust and generalizable models that can handle diverse and challenging scenarios

• Examples: Here are some examples of questions from the PIQA dataset, taken from different domains:
  • How to quickly soften butter for baking?
    • Place the stick of butter inside a freezer bag and beat it with a wooden rolling pin
    • Place the stick of butter atop a freezer bag and beat it with a wooden rolling pin
    • The correct answer is A, as it is the most effective way to soften butter without melting it.
  • How to make a hole in a wall?
    • Use a drill and a drill bit that matches the size of the hole you want
    • Use a hammer and a nail that matches the size of the hole you want
    • The correct answer is A, as it is the most precise and efficient way to make a hole in a wall.
  • How to remove a splinter from your finger?
    • Use a pair of tweezers to gently pull out the splinter
    • Use a pair of scissors to cut out the splinter
    • The correct answer is A, as it is the safest and least painful way to remove a splinter.

• • The original paper by Bisk et al. that introduces the dataset and the adversarial filtering method.
  • The GitHub repository by Bisk that contains the dataset, the code for adversarial filtering, and the models for PIQA.
  • The Hugging Face website that provides the dataset download link and the documentation for loading and using the dataset.
  • The Papers With Code website that lists the papers and code related to the PIQA dataset and task.
  • The PyPI website that provides the piqa package, which is a collection of PyTorch metrics for image quality assessment.
  • WinoGrande