Introduction¶
The TMDb data provides details of 10,000 movies. The data include details like casts, revenue, budget, popularity etc. This data can be analysed to find interesting pattern between movies.
We can use this data to try to answer following questions:
- What is the yearly revenue change?
- Which genres are most popular from year to year?
- What kinds of properties are associated with movies that have high revenues?
- Who are top 15 highest grossing directors?
- Who are top 15 highest grossing actors?
In [1]:
# import necessary libraries
%matplotlib inline
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import seaborn as sns
In [2]:
# Load TMDb data and print out a few lines. Perform operations to inspect data
# types and look for instances of missing or possibly errant data.
tmdb_movies = pd.read_csv('tmdb-movies.csv')
tmdb_movies.head()
Out[2]:
In [3]:
tmdb_movies.describe()
Out[3]:
Data Cleaning¶
As evident from the data, it seems we have cast of the movie as string separated by | symbol. This needs to be converted into a suitable type in order to consume it properly later.
In [4]:
# Pandas read empty string value as nan, make it empty string
tmdb_movies.cast.fillna('', inplace=True)
tmdb_movies.genres.fillna('', inplace=True)
tmdb_movies.director.fillna('', inplace=True)
tmdb_movies.production_companies.fillna('', inplace=True)
In [5]:
def string_to_array(data):
"""
This function returns given splitss the data by separator `|` and returns the result as array
"""
return data.split('|')
Convert cast, genres, director and production_companies columns to array¶
In [6]:
tmdb_movies.cast = tmdb_movies.cast.apply(string_to_array)
In [7]:
tmdb_movies.genres = tmdb_movies.genres.apply(string_to_array)
In [8]:
tmdb_movies.director = tmdb_movies.director.apply(string_to_array)
In [9]:
tmdb_movies.production_companies = tmdb_movies.production_companies.apply(string_to_array)
It's evident from observations below that there is no clear trend in change in mean revenue over years.
- Mean revenue from year to year is quite unstable. This can be attributed to number of movies and number of movies having high or low revenue
- The gap between budget and revenue have widened after 2000. This can be attributed to circulation of movies worldwide compared to earlier days.
- There seems to be a correlation between gross budget and gross revenue over years.
- When log of revenue_adj is plotted against log of budget_adj, we can see a clear correlation between revenue of a movie against the budget
In [10]:
def yearly_growth(mean_revenue):
return mean_revenue - mean_revenue.shift(1).fillna(0)
In [11]:
# Show change in mean revenue over years, considering only movies for which we have revenue data
movies_with_budget = tmdb_movies[tmdb_movies.budget_adj > 0]
movies_with_revenue = movies_with_budget[movies_with_budget.revenue_adj > 0]
revenues_over_years = movies_with_revenue.groupby('release_year').sum()
revenues_over_years.apply(yearly_growth)['revenue'].plot()
revenues_over_years[['budget_adj', 'revenue_adj']].plot()
Out[11]:
In [12]:
def log(data):
return np.log(data)
In [13]:
movies_with_revenue[['budget_adj', 'revenue_adj']].apply(log) \
.sort_values(by='budget_adj').set_index('budget_adj')['revenue_adj'].plot(figsize=(20,6))
Out[13]:
Research Question 2: Which genres are most popular from year to year?¶
Since popularity column indicates all time popularity of the movie, it might not be the right metric to measure popularity over years. We can measure popularty of a movie based on average vote. I think a movie is popular if vote_average >= 7.
On analyzing the popular movies since 1960(check illustrations below), following onservations can be made:
- Almost all popular movies have
Dramagenre - Over years
Comedy,ActionandAdventuregot popular. - In recent years,
Documentry,ActionandAnimationmovies got more popularity.
In [14]:
def popular_movies(movies):
return movies[movies['vote_average']>=7]
In [15]:
def group_by_genre(data):
"""
This function takes a Data Frame having and returns a dictionary having
release_year as key and value a dictionary having key as movie's genre
and value as frequency of the genre that year.
"""
genres_by_year = {}
for (year, position), genres in data.items():
for genre in genres:
if year in genres_by_year:
if genre in genres_by_year[year]:
genres_by_year[year][genre] += 1
else:
genres_by_year[year][genre] = 1
else:
genres_by_year[year] = {}
genres_by_year[year][genre] = 1
return genres_by_year
In [16]:
def plot(genres_by_year):
"""
This function iterates over each row of Data Frame and prints rows
having release_year divisible by 5 to avoid plotting too many graphs.
"""
for year, genres in genres_by_year.items():
if year%5 == 0:
pd.DataFrame(grouped_genres[year], index=[year]).plot(kind='bar', figsize=(20, 6))
In [17]:
# Group movies by genre for each year and try to find the correlations
# of genres over years.
grouped_genres = group_by_genre(tmdb_movies.groupby('release_year').apply(popular_movies).genres)
plot(grouped_genres)
Research Question 3: What kinds of properties are associated with movies that have high revenues?¶
We can consider those movies with at least 1 billion revenue and see what are common properties among them.
Considering this criteria and based on illustrations below, we can make following observations about highest grossing movies:
AdventureandActionare most common genres among these movies followed byScience Fiction,FantasyandFamily.- Most of the movies have more than 7 average vote, some movies have less than 7 but that is because of less number of total votes. This means highest grossing movies are popular as well.
Steven SpielbergandPeter Jacksonare directors who have highest muber of movies having at least 1 billion revenue.- Most of the directors have only one movies having at least a billion revenue, hence there seems to be no corelation between highest grossing movies and directors.
- Most of the cast have one movie having at least a billion revenue.
Warner Bros.,Walt Disney,Fox FilmandUniversal pictureseems to have figured out the secret of highest grossing movies. They have highest number of at least a billion revenue movies. This does not mean all their movies have pretty high revenue.
In [18]:
highest_grossing_movies = tmdb_movies[tmdb_movies['revenue_adj'] >= 1000000000]\
.sort_values(by='revenue_adj', ascending=False)
highest_grossing_movies.head()
Out[18]:
Find common genres in highest grossing movies¶
In [19]:
def count_frequency(data):
frequency_count = {}
for items in data:
for item in items:
if item in frequency_count:
frequency_count[item] += 1
else:
frequency_count[item] = 1
return frequency_count
In [20]:
highest_grossing_genres = count_frequency(highest_grossing_movies.genres)
print(highest_grossing_genres)
pd.DataFrame(highest_grossing_genres, index=['Genres']).plot(kind='bar', figsize=(20, 8))
Out[20]:
Popularity of highest grossing movies¶
In [21]:
highest_grossing_movies.vote_average.hist()
Out[21]:
Directors of highest grossing movies¶
In [22]:
def list_to_dict(data, label):
"""
This function creates returns statistics and indices for a data frame
from a list having label and value.
"""
statistics = {label: []}
index = []
for item in data:
statistics[label].append(item[1])
index.append(item[0])
return statistics, index
In [23]:
import operator
high_grossing_dirs = count_frequency(highest_grossing_movies.director)
revenues, indexes = list_to_dict(sorted(high_grossing_dirs.items(), key=operator.itemgetter(1), reverse=True)[:20], 'revenue')
pd.DataFrame(revenues, index=indexes).plot(kind='bar', figsize=(20, 5))
Out[23]:
Cast of highest grossing movies¶
In [24]:
high_grossing_cast = count_frequency(highest_grossing_movies.cast)
revenues, index = list_to_dict(sorted(high_grossing_cast.items(), key=operator.itemgetter(1), reverse=True)[:30], 'number of movies')
pd.DataFrame(revenues, index=index).plot(kind='bar', figsize=(20, 5))
Out[24]:
Production companies of highest grossing movies¶
In [25]:
high_grossing_prod_comps = count_frequency(highest_grossing_movies.production_companies)
revenues, index = list_to_dict(sorted(high_grossing_prod_comps.items(), key=operator.itemgetter(1), reverse=True)[:30]\
, 'number of movies')
pd.DataFrame(revenues, index=index).plot(kind='bar', figsize=(20, 5))
Out[25]:
Highest grossing budget¶
Research Question 4: Who are top 15 highest grossing directors?¶
We can see the top 30 highest grossing directors in bar chart below.
It seems Steven Spielberg surpasses other directors in gross revenue.
In [26]:
def grossing(movies, by):
"""
This function returns the movies' revenues over key passed as `by` value in argument.
"""
revenues = {}
for id, movie in movies.iterrows():
for key in movie[by]:
if key in revenues:
revenues[key].append(movie.revenue_adj)
else:
revenues[key] = [movie.revenue_adj]
return revenues
In [27]:
def gross_revenue(data):
"""
This functions computes the sum of values of the dictionary and
return a new dictionary with same key but cummulative value.
"""
gross = {}
for key, revenues in data.items():
gross[key] = np.sum(revenues)
return gross
In [28]:
gross_by_dirs = grossing(movies=movies_with_revenue, by='director')
director_gross_revenue = gross_revenue(gross_by_dirs)
top_15_directors = sorted(director_gross_revenue.items(), key=operator.itemgetter(1), reverse=True)[:15]
revenues, index = list_to_dict(top_15_directors, 'director')
pd.DataFrame(data=revenues, index=index).plot(kind='bar', figsize=(15, 9))
Out[28]:
Research Question 5: Who are top 15 highest grossing actors?¶
We can find the top 30 actors based on gross revenue as shown in subsequent sections below.
As we can see Harison Ford tops the chart with highest grossing.
In [29]:
gross_by_actors = grossing(movies=tmdb_movies, by='cast')
actors_gross_revenue = gross_revenue(gross_by_actors)
top_15_actors = sorted(actors_gross_revenue.items(), key=operator.itemgetter(1), reverse=True)[:15]
revenues, indexes = list_to_dict(top_15_actors, 'actors')
pd.DataFrame(data=revenues, index=indexes).plot(kind='bar', figsize=(15, 9))
Out[29]:
Conclusions¶
We tried to answer quite a few questions using TMDb data whch are summarised below:
- The gap between gross budget and revenue have increased over years.
Dramagenre is common traits among popular movies withComedy,Action,AdventureandAnimationgetting popular in recent years.- There seems to be correlation between highest grossing movies and
Adventure&Actiongenres. Also Highest grossing movies are correlated with high popularity. - There also seems to be correlation between a movie's budget and it's revenue. A movie having high budget seems to have a high revenue.