Using Python libraries like requests and BeautifulSoup, I am attempting to scrape the tables from the following Wikipedia page: https://en.wikipedia.org/wiki/Mobile_country_code. While I am able to retrieve all the data within the tables, my goal now is to include an additional column labeled Country, populated with the names of the tables. You can view an example of the Wikipedia table (above) and the desired table layout (below) here.
The code snippet provided below enables me to fetch all the data without incorporating the Country column:
import requests
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np
wiki = requests.get('https://en.wikipedia.org/wiki/Mobile_country_code')
soup = BeautifulSoup(wiki.content, 'html.parser')
# Extract all the tables
tables = soup.find_all('table',class_="wikitable")
# Extracting the column names
column_names = [item.get_text() for item in tables[0].find_all('th')]
# Extracting the content
contents = [item.get_text() for item in tables[0].find_all('td')]
# Placing all the content into a list
values=[]
for table in tables:
for item in table.select('td'):
temp = item.get_text()
values.append(temp)
# As there are 7 columns, determining the number of rows and reshaping the table
len(values)/7 # Returns 2452 rows
# Modifying the shape of the table
data = np.reshape(values,(2452,7))
# Creating a dataframe to store all the data
df = pd.DataFrame(data = data, columns=header_list)