Resources by Stefan

Civilian Complaints Against New York City Police Officers

Civilian Complaints Against New York City Police Officers

Stefan
Open Dataset
Creators: ProPublica
Publication Date: 2020
Creators: ProPublica

This free download is a database of more than 12,000 civilian complaints filed against New York City police officers. After New York state repealed the statute that kept police disciplinary records secret, known as 50-a, ProPublica filed a records request with New York City’s Civilian Complaint Review Board, which investigates complaints by the public about NYPD officers. The board provided us with records about closed cases for every police officer still on the force as of late June 2020 who had at least one substantiated allegation against them. The records span decades, from September 1985 to January 2020. Each entry includes specifics such as the nature of the allegation (e.g., use of force, abuse of authority), the outcome of the investigation, and any disciplinary actions taken. The dataset provides information on the officers involved, including their rank and assignment at the time of the complaint. Entries contain timestamps and locations of the alleged incidents, facilitating analyses of patterns over time and across different areas.​​

Structurally, the dataset contains the following information: ​

  • Complaint ID: A unique identifier for each complaint.​

  • Date and Time: When the incident allegedly occurred.​

  • Location: Where the incident took place.​

  • Officer Details: Information about the officer(s) involved, such as badge number, rank, and assignment.​

  • Allegation Details: Type of misconduct reported (e.g., excessive force, discourtesy).​

  • Investigation Outcome: Findings of the investigation, including whether the allegation was substantiated, unsubstantiated, exonerated, or unfounded.​

  • Disciplinary Action: Any penalties or corrective actions imposed following the investigation.

Facebook Social Connectedness Index

Facebook Social Connectedness Index

Stefan
Open Dataset
Creators: Meta
Publication Date: 2021
Creators: Meta

We use an anonymized snapshot of all active Facebook users and their friendship networks to measure the intensity of connectedness between locations. The Social Connectedness Index (SCI) is a measure of the social connectedness between different geographies. Specifically, it measures the relative probability that two individuals across two locations are friends with each other on Facebook. Each entry represents a pair of locations, detailing the strength of social connectedness between them. By doing so, the SCI provides a measure of the relative probability that two individuals from different locations are Facebook friends, offering insights into social ties across regions. The dataset has a a size of 3,9 kB and reflects a specific snapshot in time, with the latest available data from October 2021. The dataset is organized into multiple sub-datasets, each detailing social connectedness at different geographic levels:​

  1. Country-Country Pairs:

    • user_loc: ISO2 code of the first country.​

    • fr_loc: ISO2 code of the second country.​

    • scaled_sci: Scaled Social Connectedness Index between the two countries.​

  2. US County-Country Pairs:

    • user_loc: 5-digit FIPS code of the U.S. county.​

    • fr_loc: ISO2 code of the country.​

    • scaled_sci: Scaled Social Connectedness Index between the U.S. county and the country.

​

Fact Check Data

Fact Check Data

Stefan
Open Dataset
Creators: Data Commons
Publication Date: 2019
Creators: Data Commons

This is a data feed of ClaimReview markups created via the Google Fact Check Markup Tool and the new ClaimReview Read/Write API. The data in the feed also follows the schema.org ClaimReview standard, namely the same schema as the data in the historical research dataset. It compiles structured metadata from fact-checking articles and serves as a valuable resource for researchers and developers aiming to analyze and combat misinformation. Each entry follows the ClaimReview schema, providing standardized fields such as the claim reviewed, the author, the date of publication, and the URL to the original fact-checking article. It aggregates fact-checks from multiple reputable organizations, offering a comprehensive view of fact-checking efforts across various domains​. In total, the dataset has a size of 73,4 kB.

Amazon Brand and Exclusives

Amazon Brand and Exclusives

Stefan
Open Dataset
Creators: Jeffries, Adrianne; Yin, Leon
Publication Date: 2021
Creators: Jeffries, Adrianne; Yin, Leon
My co-author Adrianne Jeffries and I found Amazon gave its own branded products an advantage over better-rated competitors in search results. This repository contains code to reproduce the findings featured in our story “Amazon Puts Its Own ‘Brands’ First Above Better-Rated Products” and “When Amazon Takes the Buy Box, it Doesn’t Give it up” from our series Amazon’s Advantage. Each product in this dataset is identified by its unique Amazon Standard Identification Number (ASIN), facilitating precise tracking and analysis. They are categorized based on their association with Amazon, distinguishing between Amazon-owned brands, exclusive partnerships, and proprietary electronics. Data is derived from extensive web scraping of Amazon’s product listings, ensuring a comprehensive and up-to-date collection. The data collection occurred primarily in early 2021, with search results gathered in January 2021 and product pages in February 2021. In total, the dataset comprises 137,428 products, each represented by a unique ASIN and has a size of 221,0 kB. It is organized into several sub-datasets, each serving a specific analytical purpose:​

  1. Amazon Private Label Products: Contains detailed information on 137,428 products identified as Amazon brands, exclusives, or proprietary electronics.​

  2. Search Results: Includes parsed search result pages from top and generic searches, totaling 187,534 product positions.​

  3. Product Pages: Comprises parsed product pages corresponding to the search results, encompassing 157,405 product pages.​

  4. Training Set: Provides metadata used to train and evaluate machine learning models, with feature engineering conducted in associated Jupyter notebooks.

  5. Trademarks: Contains a dataset of trademarked brands registered by Amazon, collected from USPTO.gov and Amazon.

​

CVE List

CVE List

Stefan
Open Dataset
Creators: CVE
Publication Date: 2024
Creators: CVE

The mission of the CVE® Program is to identify, define, and catalog publicly disclosed cybersecurity vulnerabilities identified from 1999 through June 25, 2024, providing a historical perspective on the evolution of cybersecurity threats over a span of 25 years. Its primary purpose is to standardize the identification of vulnerabilities across various platforms and security tools, facilitating consistent and efficient communication within the cybersecurity community. As of June 25, 2024, the CVE List comprises 269,759 records with a size of 36,3 kB.

Each CVE entry includes several key components:​

  • CVE Identifier (CVE ID): A unique alphanumeric code assigned to each vulnerability, following the format “CVE-YYYY-NNNN,” where “YYYY” denotes the year of identification, and “NNNN” is a sequential number.​

  • Description: A brief summary outlining the nature of the vulnerability, including affected software or hardware, potential impacts, and any known exploits.​

  • References: Links to external resources such as security advisories, vendor bulletins, or detailed analyses that provide additional context or mitigation information.

IMINTREG data

IMINTREG data

Stefan
Open Dataset
Creators: Zuber, Christina Isabel
Publication Date: 2011
Creators: Zuber, Christina Isabel

IMINTREG offers data on immigrant integration policies of Italian regions, Spanish autonomous communities and German Länder. You can download the original regional integration laws, as well as measures of in- vs. exclusiveness of regional integration policies based on my coding of the laws here. The dataset has a size of 84,2 kB and entails the following aspects:

  • Regional Integration Laws: The dataset includes the original texts of regional integration laws from the specified regions, allowing for direct analysis of legislative content.​

  • Inclusiveness Measures: It provides coded measures assessing the inclusiveness of these policies, facilitating comparative studies across different regions.​

  • Comparative Scope: By covering multiple countries with varying approaches to integration, the dataset offers a unique opportunity to study the effects of regional policies within different national contexts.​

Snap Political Ads Library

Snap Political Ads Library

Stefan
Open Dataset
Creators: Snapchat
Publication Date: 2024
Creators: Snapchat

The Political and Advocacy Ads Library is an important step in our efforts to increase the level of transparency around political and issue advertising on. The dataset includes advertisements from 2018 through 2025 with a total size of 538,8 kB. It offers detailed insights into political advertising on Snapchat, including:​

  • Ad Content: Access to the creative content of each advertisement through unique URLs.​

  • Financial Information: Details on the amount spent on each ad campaign, specified in the local currency.​

  • Impressions: Data on the number of times each ad was viewed by Snapchat users.​

  • Targeting Criteria: Information on the demographic and geographic targeting parameters used in each campaign.

These features enable researchers, policymakers, and the public to analyze the reach, expenditure, and strategies of political advertisers on Snapchat.

The greatest hip-hop songs off all time

The greatest hip-hop songs off all time

Stefan
Open Dataset
Creators: BBC
Publication Date: 2019
Creators: BBC

BBC Music polled over 1001 critics in 15 countries to find the best hip-hop song ever. This repo contains poll data, originally published by BBC Music, as well as code for transforming the data, adding cover artwork, and publishing charts via Datawrapper. The poll data was extracted from this article on bbc.com: The greatest hip-hop songs of all time – who voted. It covered over 300 hip-hop songs, each representing an individual observation. The songs listed span from the late 1970s to 2019, covering the evolution of hip-hop over approximately four decades. In total, the dataset has a size of 60,8 kB. Structurally, it is built upon the following key variables:​

  • Rank: The position of the song in the overall ranking.​

  • Title: The name of the song.​

  • Artist: The performing artist(s) of the song.​

  • Year: The release year of the song.​

  • Total Points: The cumulative points the song received based on the poll’s scoring system.​

  • Number of Votes: The count of critics who included the song in their top five lists.

Sign In

New to BERD@NFDI? Register

Forgot password?

Register

Reset Password

Please enter your username or email address, you will receive a link to create a new password via email.