Tumblr GIF Description Dataset
Animated GIF Description - Sequence to Sequence - Video Captioning
Yahoo Research & University of Rochester

Overview


The Tumblr GIF (TGIF) dataset contains 100K animated GIFs and 120K sentences describing visual content of the animated GIFs. The animated GIFs have been collected from Tumblr, from randomly selected posts published between May and June of 2015. We provide the URLs of animated GIFs in this release. The sentences are collected via crowdsourcing, with a carefully designed annotation interface that ensures high quality dataset. We provide one sentence per animated GIF for the training and validation splits, and three sentences per GIF for the test split. The dataset shall be used to evaluate animated GIF/video description techniques.

Basic Statistics

Splits Train Validation Test Overall
# Animated GIFs 80,000 10,708 11,360 102,068
# Frames (Total/Average) 3,258,373/40.76 431,359/40.30 453,718/39.96 4,143,450/40.62
# Shots (Total/Average) 204,553/2.56 26,559/2.48 27,933/2.46 259,338/2.54
Duration (Total/Average) 81h/3.66s 11h/3.60s 12h/3.65s 103h/3.65s
# Sentences (Total/Average) 80,000/1 10,708/1 34,101/3 125,782/1.23
# Tokens (Total/Median) 911,593/11 10,831/11 34,101/11 1,418,555/11
# Unique Tokens 10,685 4,755 7,083 12,228
Average Term Frequency 85.32 25.72 54.31 116.00

Download


Citation

@InProceedings{tgif-cvpr2016,
  author = {Li, Yuncheng and Song, Yale and Cao, Liangliang and Tetreault, Joel and Goldberg, Larry and Jaimes, Alejandro and Luo, Jiebo},
  title = "{TGIF: A New Dataset and Benchmark on Animated GIF Description}",
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2016}
}

Data Examples


A well-dressed young guy with gelled red hair glides across a room and scans it with his eyes.

A well-dressed young guy with gelled red hair glides across a room and scans it with his eyes.

a woman in a car is singing.

a woman in a car is singing.

a person is waving and smiling while looking to the right.

a person is waving and smiling while looking to the right.

two boys sit and smile, and one of them laughs while the other gazes at him.

two boys sit and smile, and one of them laughs while the other gazes at him.

a man with a microphone is drinking a beverage.

a man with a microphone is drinking a beverage.

a man wearing a suit smiles at something in the distance.

a man wearing a suit smiles at something in the distance.

People


Yuncheng Li avatar

Yuncheng Li

University of Rochester

Yale Song avatar

Yale Song

Yahoo! Research

Liangliang Cao avatar

Liangliang Cao

Yahoo! Research

Joel Tetreault avatar

Joel Tetreault

Yahoo! Research

Jiebo Luo avatar

Jiebo Luo

University of Rochester