how to export selenium results to csv file

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

how to export selenium results to csv file


The code below uses selenium to scrape some data from a website and prints the results to the screen. Would anyone know how I could save this data to csv file?  I tried to openpyxl but couldn't figure it out.  Thanks

from time import sleep from selenium import webdriver from import By from import WebDriverWait from import expected_conditions as EC from selenium.common.exceptions import TimeoutException url = '' driver = webdriver.Firefox() driver.get(url) try: selector = '' condition = EC.visibility_of_element_located((By.CSS_SELECTOR, selector)) matches = WebDriverWait(driver, 10).until(condition) matches = int(matches.text.split()[0]) except (TimeoutException, Exception): print ('Problem finding matches, setting default...') matches = 4895 # Set default # The page loads 150 rows at a time; divide matches by # 150 to determine the number of times we need to scroll; # add 5 extra scrolls just to be sure num_loops = int(matches / 150 + 5) for _ in range(num_loops): driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") sleep(2) # Pause briefly to allow loading time # will give a list of all tickers tickers = driver.find_elements_by_css_selector('') # will give a list of all EMPs emps = driver.find_elements_by_xpath('//tbody/tr/td[10]') # will give a list of all sectors sectors = driver.find_elements_by_xpath('//tbody/tr/td[11]') for index in range(len(tickers)): print("Row " + tickers[index].text + " " + emps[index].text + " " + sectors[index].text + " ")

You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit
For more options, visit