how to export selenium results to csv file

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

how to export selenium results to csv file

jk000ru

The code below uses selenium to scrape some data from a website and prints the results to the screen. Would anyone know how I could save this data to csv file?  I tried to openpyxl but couldn't figure it out.  Thanks


from time import sleep from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.common.exceptions import TimeoutException url = 'http://www.tradingview.com/screener' driver = webdriver.Firefox() driver.get(url) try: selector = '.js-field-total.tv-screener-table__field-value--total' condition = EC.visibility_of_element_located((By.CSS_SELECTOR, selector)) matches = WebDriverWait(driver, 10).until(condition) matches = int(matches.text.split()[0]) except (TimeoutException, Exception): print ('Problem finding matches, setting default...') matches = 4895 # Set default # The page loads 150 rows at a time; divide matches by # 150 to determine the number of times we need to scroll; # add 5 extra scrolls just to be sure num_loops = int(matches / 150 + 5) for _ in range(num_loops): driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") sleep(2) # Pause briefly to allow loading time # will give a list of all tickers tickers = driver.find_elements_by_css_selector('a.tv-screener__symbol') # will give a list of all EMPs emps = driver.find_elements_by_xpath('//tbody/tr/td[10]') # will give a list of all sectors sectors = driver.find_elements_by_xpath('//tbody/tr/td[11]') for index in range(len(tickers)): print("Row " + tickers[index].text + " " + emps[index].text + " " + sectors[index].text + " ")

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/da78cdac-ebff-4c73-bc1a-30dba7948bde%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.