Language Modeling

Showing 1-1 of 1 results
Social Interaction QA; Social IQA

Social Interaction QA; Social IQA

Stefan
Open Dataset
Creators: Sap, Maarten; Rashkin, Hannah; Chen, Derek; Le Bras, Ronan; Choi, Yejin
Publication Date: 2019
Creators: Sap, Maarten; Rashkin, Hannah; Chen, Derek; Le Bras, Ronan; Choi, Yejin

Social Interaction QA, a new question-answering benchmark for testing social commonsense intelligence. Contrary to many prior benchmarks that focus on physical or taxonomic knowledge, Social IQa focuses on reasoning about people’s actions and their social implications. For example, given an action like “Jesse saw a concert” and a question like “Why did Jesse do this?”, humans can easily infer that Jesse wanted “to see their favorite performer” or “to enjoy the music”, and not “to see what’s happening inside” or “to see if it works”. The actions in Social IQa span a wide variety of social situations, and answer candidates contain both human-curated answers and adversarially-filtered machine-generated candidates. Social IQa contains over 37,000 QA pairs for evaluating models’ abilities to reason about the social implications of everyday events and situations. The dataset is relatively small, with a size of about 0.01 GB, and is available in JSON format.

The structure of the dataset consists of a set of question-answer pairs, where each entry contains:

  • A context describing a social situation.
  • A question that requires reasoning about the context.
  • Three answer choices (one correct, two incorrect).
  • A label indicating the correct answer.

Sign In

New to BERD@NFDI? Register

Forgot password?

Register

Reset Password

Please enter your username or email address, you will receive a link to create a new password via email.