Currently, I am utilizing Selenium with Python to crawl the drop-down menu of this particular page. By employing the find_elements_by_css_selector function, I have successfully obtained all the data from the second drop-down menu.
However, when attempting to print both the group number and its corresponding model name using model.get.attribute('href'), I encountered an issue where the Korean characters in the model name appeared broken as shown below: 5%EC%8B%9C%EB%A6%AC%EC%A6%88 (which represents 5-series in BMW).
While I understand that 'model.text' provides only the model name, my requirement also includes retrieving the group name for each model simultaneously.
This necessitated me to use model.get.attribute('href') instead of model.text.
I would appreciate any assistance to help resolve this matter.
Below you will find a snippet of my code:
enter code here#!/usr/bin/env python
#-*- coding: utf-8 -*-
import re
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from bs4 import BeautifulSoup
from time import sleep
link = 'http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I'
driver = webdriver.PhantomJS()
driver.set_window_size(1920, 1080)
driver.get(link)
sleep(.75)
s = BeautifulSoup(driver.page_source, "html.parser", from_encoding='utf-8')
makers = ['아우디', 'BMW', '벤츠']
for maker in makers:
# open manufacturer layer
next_elem = driver.find_element_by_xpath('//a[@title="제조사 선택"]')
next_elem.click()
next_elem = driver.find_element_by_link_text(maker)
print(maker)
next_elem.click()
print("clicked maker")
sleep(.75)
# open model layer
next_elem = driver.find_element_by_xpath('//a[@title="모델 선택"]')
next_elem.click()
# select model
models = driver.find_elements_by_css_selector("#layer_group ul.list li a")
for model in models:
print (model.get_attribute('href'))