How can I iterate through a list of items and extract a specific part using Selenium and Python

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

How can I iterate through a list of items and extract a specific part using Selenium and Python

developernina2
Please someone help me. I have been working on this for more than a week with no luck!

From this web page "https://meshb.nlm.nih.gov/treeView", I want to iterate through each node of the tree and if I see the word "Cardiovascular..." in their items, I want to create a dictionary that lists the top level node along with all of cardiovascular associated items. For example, in the above page you can see that if you expand "Anatomy [A]", you will see cardiovascular. Now, I want this part along with whatever included in cardiovascular if you expand it. I have attached 2 screenshots of a part of html.  Here is my Python code:
from selenium import webdriver
from selenium import webdriver
chrome_path=r"G:\My Drive\chrome_driver\chromedriver_win32\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)
driver.get('https://meshb.nlm.nih.gov/treeView')
sidebar = driver.find_element_by_xpath("/html/body/div[2]/div")
i=1
for i in range(16):  # since I have 16 div(s)
   sidebar.find_elements_by_xpath("/html/body/div[2]/div/div[i]")       
   element = driver.find_element_by_xpath("/html/body/div[2]/div").find_element_by_xpath("/html/body/div[2]/div/div[i]").find_element_by_xpath("//*[@class='ng-scope']/span")
   element.click()

But I keep getting this error:no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[2]/div/div[i]"}
Any help is appreciated!

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/a1a9cf5e-589f-4d10-ad69-14463e295ba3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How can I iterate through a list of items and extract a specific part using Selenium and Python

developernina2
I  updated my code with this line: element = driver.find_element_by_xpath("/html/body/div[2]/div").find_element_by_xpath("/html/body/div[2]/div/div[" + str(i) + "]").find_element_by_xpath("/html/body/div[2]/div/div[" + str(i) + "]/div/span")
When I print it, it shows me the first level nodes like Anatomy,.... But it does not open all plus buttons!
On Friday, June 22, 2018 at 11:19:58 AM UTC-5, [hidden email] wrote:
Please someone help me. I have been working on this for more than a week with no luck!

<a href="https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG" style="margin-left:1em;margin-right:1em" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG&#39;;return true;">

<a href="https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG" style="margin-left:1em;margin-right:1em" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG&#39;;return true;">

<a href="https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG" style="margin-left:1em;margin-right:1em" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG&#39;;return true;">

From this web page "<a href="https://meshb.nlm.nih.gov/treeView" rel="nofollow" style="font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;color:rgb(0,89,153)" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView", I want to iterate through each node of the tree and if I see the word "Cardiovascular..." in their items, I want to create a dictionary that lists the top level node along with all of cardiovascular associated items. For example, in the above page you can see that if you expand "Anatomy [A]", you will see cardiovascular. Now, I want this part along with whatever included in cardiovascular if you expand it. I have attached 2 screenshots of a part of html.  Here is my Python code:
from selenium import webdriver
from selenium import webdriver
chrome_path=r"G:\My Drive\chrome_driver\chromedriver_win32\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)
driver.get('<a href="https://meshb.nlm.nih.gov/treeView" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView')
sidebar = driver.find_element_by_xpath("/html/body/div[2]/div")
i=1
for i in range(16):  # since I have 16 div(s)
   sidebar.find_elements_by_xpath("/html/body/div[2]/div/div[i]")       
   element = driver.find_element_by_xpath("/html/body/div[2]/div").find_element_by_xpath("/html/body/div[2]/div/div[i]").find_element_by_xpath("//*[@class='ng-scope']/span")
   element.click()

But I keep getting this error:no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[2]/div/div[i]"}
Any help is appreciated!

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/934c61e5-d423-4bce-9262-c7bc8b693dc2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How can I iterate through a list of items and extract a specific part using Selenium and Python

Selenium007
In reply to this post by developernina2
Did you try findElements? This returns the list of Web Elements and you can iterate through the list.
Also, relative xpath might be better than dynamic xpath

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/edf5e83a-9a30-4256-8359-db8ef7dfba50%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How can I iterate through a list of items and extract a specific part using Selenium and Python

developernina2
I did. Here is the updated code with the help of a friend. But the problem is that it says the <i> is not clickable and when I changed it to span only, it got stuck on the first node! Any help?

while True:
elements = driver.find_elements_by_xpath("//a[@class='ng-scope']/following-sibling::span[@ng-if = 'node.HasChildren']/i")
if (len(elements) > 0):
elements[0].click() # click on the first in the list
driver.implicitly_wait(10) # wait until list will be updated
continue
break
spans = driver.find_elements_by_xpath("//span[contains(., 'Cardiovascular')]/parent::*/parent::*/descendant-or-self::node()/a/span")
for span in spans:
print (span.text)



On Friday, June 22, 2018 at 12:34:57 PM UTC-5, Selenium007 wrote:
Did you try findElements? This returns the list of Web Elements and you can iterate through the list.
Also, relative xpath might be better than dynamic xpath

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/3e2e97bd-0c75-426c-8fa5-054dccd45530%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How can I iterate through a list of items and extract a specific part using Selenium and Python

Selenium007
How about "//a[@class="ng-scope"]/following-sibling::span[@ng-if = 'node.HasChildren']/i[contains(@class, "fa fa-plus-circle")]" since i has plus and minus based on if its expanded

On Friday, June 22, 2018 at 2:37:14 PM UTC-4, [hidden email] wrote:
I did. Here is the updated code with the help of a friend. But the problem is that it says the <i> is not clickable and when I changed it to span only, it got stuck on the first node! Any help?

while True:
elements = driver.find_elements_by_xpath("//a[@class='ng-scope']/following-sibling::span[@ng-if = 'node.HasChildren']/i")
if (len(elements) > 0):
elements[0].click() # click on the first in the list
driver.implicitly_wait(10) # wait until list will be updated
continue
break
spans = driver.find_elements_by_xpath("//span[contains(., 'Cardiovascular')]/parent::*/parent::*/descendant-or-self::node()/a/span")
for span in spans:
print (span.text)



On Friday, June 22, 2018 at 12:34:57 PM UTC-5, Selenium007 wrote:
Did you try findElements? This returns the list of Web Elements and you can iterate through the list.
Also, relative xpath might be better than dynamic xpath

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/85be52cd-b336-4c2b-9a15-e76f9188932e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How can I iterate through a list of items and extract a specific part using Selenium and Python

developernina2
Thanks for your comment, but , still does not work!

On Friday, June 22, 2018 at 2:50:18 PM UTC-5, Selenium007 wrote:
How about "//a[@class="ng-scope"]/following-sibling::span[@ng-if = 'node.HasChildren']/i[contains(@class, "fa fa-plus-circle")]" since i has plus and minus based on if its expanded

On Friday, June 22, 2018 at 2:37:14 PM UTC-4, [hidden email] wrote:
I did. Here is the updated code with the help of a friend. But the problem is that it says the <i> is not clickable and when I changed it to span only, it got stuck on the first node! Any help?

while True:
elements = driver.find_elements_by_xpath("//a[@class='ng-scope']/following-sibling::span[@ng-if = 'node.HasChildren']/i")
if (len(elements) > 0):
elements[0].click() # click on the first in the list
driver.implicitly_wait(10) # wait until list will be updated
continue
break
spans = driver.find_elements_by_xpath("//span[contains(., 'Cardiovascular')]/parent::*/parent::*/descendant-or-self::node()/a/span")
for span in spans:
print (span.text)



On Friday, June 22, 2018 at 12:34:57 PM UTC-5, Selenium007 wrote:
Did you try findElements? This returns the list of Web Elements and you can iterate through the list.
Also, relative xpath might be better than dynamic xpath

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/f7f3c7d5-175f-4494-8d9e-73620e5c088e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How can I iterate through a list of items and extract a specific part using Selenium and Python

Shiva Prasad Adirala
In reply to this post by developernina2
Hi Mate,
As per my understanding - I wrote snippet for your problem

1. Get List of Node's
2. Expand Each Node One After other
3. Get List Of Sub-Nodes
4. Compare each sub-node and expand sub-node if text==Cardiovascular
5. Get List of Sub-Sub-Node text's (Need to store in list)

from selenium import webdriver
from selenium import webdriver
from selenium.webdriver.common.by import By
import time

chrome_path=r"E:\Selenium\drivers\Chrome64\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)

#navigate to website
driver.get('https://meshb.nlm.nih.gov/treeView')

# Get list of nodes
main_list = driver.find_elements_by_xpath("//div[@ng-repeat='node in trees']/div/span")

# Iterate for list of nodes
for i in range(1, len(main_list)+1):

    # Expand each node
    node_expand_icon = driver.find_element_by_xpath("(//div[@ng-repeat='node in trees']/div/span)[" + str(i) + "]")
    node_expand_icon.click();
    # Wait for 5 seconds
    time.sleep(5)

    # Get list of sub-nodes
    sub_nodes_list=driver.find_elements_by_xpath("(//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li")
    
    # Iterate for list of sub-nodes    
    for j in range(1, len(sub_nodes_list)+1):
        sub_node_text=driver.find_element_by_xpath("((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "]").text

        # If sub-node contains "Cardiovascular" then expand
        if "Cardiovascular" in sub_node_text:
            sub_node_expand_icon=driver.find_element_by_xpath("((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li/span)[" + str(j) + "]")
            sub_node_expand_icon.click();
            # Wait for 5 seconds
            time.sleep(5)

            # Get sub-sub-node lists
            sub_sub_nodes_list=driver.find_elements_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li")

            # Iterate for list of sub-sub-nodes            
            for k in range(1, len(sub_sub_nodes_list)+1):

                sub_sub_node_text=driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]").text
                print    sub_sub_node_text

I have tested from my end.Please let me know if you need any help on this

On Friday, June 22, 2018 at 9:49:58 PM UTC+5:30, [hidden email] wrote:
Please someone help me. I have been working on this for more than a week with no luck!

<a href="https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG" style="margin-left:1em;margin-right:1em" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG&#39;;return true;">

<a href="https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG" style="margin-left:1em;margin-right:1em" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG&#39;;return true;">

<a href="https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG" style="margin-left:1em;margin-right:1em" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG&#39;;return true;">

From this web page "<a href="https://meshb.nlm.nih.gov/treeView" rel="nofollow" style="font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;color:rgb(0,89,153)" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView", I want to iterate through each node of the tree and if I see the word "Cardiovascular..." in their items, I want to create a dictionary that lists the top level node along with all of cardiovascular associated items. For example, in the above page you can see that if you expand "Anatomy [A]", you will see cardiovascular. Now, I want this part along with whatever included in cardiovascular if you expand it. I have attached 2 screenshots of a part of html.  Here is my Python code:
from selenium import webdriver
from selenium import webdriver
chrome_path=r"G:\My Drive\chrome_driver\chromedriver_win32\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)
driver.get('<a href="https://meshb.nlm.nih.gov/treeView" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView')
sidebar = driver.find_element_by_xpath("/html/body/div[2]/div")
i=1
for i in range(16):  # since I have 16 div(s)
   sidebar.find_elements_by_xpath("/html/body/div[2]/div/div[i]")       
   element = driver.find_element_by_xpath("/html/body/div[2]/div").find_element_by_xpath("/html/body/div[2]/div/div[i]").find_element_by_xpath("//*[@class='ng-scope']/span")
   element.click()

But I keep getting this error:no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[2]/div/div[i]"}
Any help is appreciated!

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/c5790ed9-e4b6-4445-9fc8-1cd265100cce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How can I iterate through a list of items and extract a specific part using Selenium and Python

developernina2

Hi Shiva, thanks for taking your time answering my question.  However, the code does not do my intended goal. Please see the attached picture, As you can see in the picture, if "Cardiovascular" is found, I want that item to be expanded and if it has more buttons to expand, I want all of them to be expanded as well. But, using your code, this does not happen. Also, the print function does not work either. I want for example,  have Cardiovascular Disease as the header and all of  the items that falls under it.  If any of these nodes has a sublist, I want them as well...

On Saturday, June 23, 2018 at 1:29:51 PM UTC-5, Shiva Prasad Adirala wrote:
Hi Mate,
As per my understanding - I wrote snippet for your problem

1. Get List of Node's
2. Expand Each Node One After other
3. Get List Of Sub-Nodes
4. Compare each sub-node and expand sub-node if text==Cardiovascular
5. Get List of Sub-Sub-Node text's (Need to store in list)

from selenium import webdriver
from selenium import webdriver
from <a href="http://selenium.webdriver.common.by" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fselenium.webdriver.common.by\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFJ-xP5N2YmZKzh4vsfqt50siCLXQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fselenium.webdriver.common.by\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFJ-xP5N2YmZKzh4vsfqt50siCLXQ&#39;;return true;">selenium.webdriver.common.by import By
import time

chrome_path=r"E:\Selenium\drivers\Chrome64\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)

#navigate to website
driver.get('<a href="https://meshb.nlm.nih.gov/treeView" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView')

# Get list of nodes
main_list = driver.find_elements_by_xpath("//div[@ng-repeat='node in trees']/div/span")

# Iterate for list of nodes
for i in range(1, len(main_list)+1):

    # Expand each node
    node_expand_icon = driver.find_element_by_xpath("(//div[@ng-repeat='node in trees']/div/span)[" + str(i) + "]")
    node_expand_icon.click();
    # Wait for 5 seconds
    time.sleep(5)

    # Get list of sub-nodes
    sub_nodes_list=driver.find_elements_by_xpath("(//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li")
    
    # Iterate for list of sub-nodes    
    for j in range(1, len(sub_nodes_list)+1):
        sub_node_text=driver.find_element_by_xpath("((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "]").text

        # If sub-node contains "Cardiovascular" then expand
        if "Cardiovascular" in sub_node_text:
            sub_node_expand_icon=driver.find_element_by_xpath("((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li/span)[" + str(j) + "]")
            sub_node_expand_icon.click();
            # Wait for 5 seconds
            time.sleep(5)

            # Get sub-sub-node lists
            sub_sub_nodes_list=driver.find_elements_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li")

            # Iterate for list of sub-sub-nodes            
            for k in range(1, len(sub_sub_nodes_list)+1):

                sub_sub_node_text=driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]").text
                print    sub_sub_node_text

I have tested from my end.Please let me know if you need any help on this

On Friday, June 22, 2018 at 9:49:58 PM UTC+5:30, [hidden email] wrote:
Please someone help me. I have been working on this for more than a week with no luck!

<a href="https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG" style="margin-left:1em;margin-right:1em" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG&#39;;return true;">

<a href="https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG" style="margin-left:1em;margin-right:1em" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG&#39;;return true;">

<a href="https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG" style="margin-left:1em;margin-right:1em" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG&#39;;return true;">

From this web page "<a href="https://meshb.nlm.nih.gov/treeView" rel="nofollow" style="font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;color:rgb(0,89,153)" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView", I want to iterate through each node of the tree and if I see the word "Cardiovascular..." in their items, I want to create a dictionary that lists the top level node along with all of cardiovascular associated items. For example, in the above page you can see that if you expand "Anatomy [A]", you will see cardiovascular. Now, I want this part along with whatever included in cardiovascular if you expand it. I have attached 2 screenshots of a part of html.  Here is my Python code:
from selenium import webdriver
from selenium import webdriver
chrome_path=r"G:\My Drive\chrome_driver\chromedriver_win32\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)
driver.get('<a href="https://meshb.nlm.nih.gov/treeView" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView')
sidebar = driver.find_element_by_xpath("/html/body/div[2]/div")
i=1
for i in range(16):  # since I have 16 div(s)
   sidebar.find_elements_by_xpath("/html/body/div[2]/div/div[i]")       
   element = driver.find_element_by_xpath("/html/body/div[2]/div").find_element_by_xpath("/html/body/div[2]/div/div[i]").find_element_by_xpath("//*[@class='ng-scope']/span")
   element.click()

But I keep getting this error:no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[2]/div/div[i]"}
Any help is appreciated!

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/dc7d8238-d85b-4a4e-ba19-c17d7725895e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How can I iterate through a list of items and extract a specific part using Selenium and Python

Shiva Prasad Adirala
Dude,
Following snippet is similar to your requirement, Do some modifications in loop to get to meet requirement. Following snippet will give you the idea for expand each element. Hope you get the code and you develop code based on this.

This approach is not suggested, since it takes lot of time to get result. Will keep post u on this

from selenium import webdriver
from selenium import webdriver
from selenium.webdriver.common.by import By
import time

chrome_path=r"E:\Selenium\drivers\Chrome64\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)

#navigate to website
driver.get('https://meshb.nlm.nih.gov/treeView')

# Get list of nodes
main_list = driver.find_elements_by_xpath("//div[@ng-repeat='node in trees']/div/span")

# Iterate for list of nodes
for i in range(1, len(main_list)+1):

    # Expand each node
    node_expand_icon = driver.find_element_by_xpath("(//div[@ng-repeat='node in trees']/div/span)[" + str(i) + "]")
    node_expand_icon.click();
    # Wait for 5 seconds
    time.sleep(2)

    # Get list of sub-nodes
    sub_nodes_list=driver.find_elements_by_xpath("(//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li")
    
    # Iterate for list of sub-nodes    
    for j in range(1, len(sub_nodes_list)+1):
        sub_node_text=driver.find_element_by_xpath("((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "]").text

        # If sub-node contains "Cardiovascular" then expand
        if "Cardiovascular" in sub_node_text:
            sub_node_expand_icon=driver.find_element_by_xpath("((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li/span)[" + str(j) + "]")
            sub_node_expand_icon.click();
            # Wait for 5 seconds
            time.sleep(2)

            # Get sub-sub-node lists
            sub_sub_nodes_list=driver.find_elements_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li")

            # Iterate for list of sub-sub-nodes            
            for k in range(1, len(sub_sub_nodes_list)+1):

                child_sub_node=driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]")

                # Expand if it has child element's
                try:
                    has_child_element = driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/span")
                    has_child_element.click();
                    # Wait for 5 seconds
                    time.sleep(2)
                    
                    print child_sub_node.text
                    
                    # Get Super Sub node
                    super_sub_node_list=driver.find_elements_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/ul/li")

                    # Iterate for list of sub-sub-nodes            
                    for l in range(1, len(super_sub_node_list)+1):

                        super_child_sub_node=driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/ul/li[" + str(l) + "]")

                        # Expand super childs if
                        try:
                            has_super_child_element = driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/ul/li[" + str(l) + "]/span")
                            has_super_child_element.click();
                            # Wait for 5 seconds
                            time.sleep(2)

                            print super_child_sub_node.text

                            super_super_sub_node=driver.find_elements_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/ul/li")

                            # Iterate for list of sub-sub-nodes            
                            for m in range(1, len(super_super_sub_node)+1):

                                super_super_child_sub_node=driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/ul/li[" + str(l) + "]/ul/li[" + str(m) + "]")

                                # Expand super childs if
                                try:
                                    has_super_child_element = driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/ul/li[" + str(l) + "]/ul/li[" + str(m) + "]/span")
                                    has_super_child_element.click();
                                    # Wait for 5 seconds
                                    time.sleep(2)

                                    print super_super_child_sub_node.text

                                    a_list=driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/ul/li[" + str(l) + "]/ul/li[" + str(m) + "]/ul/li")

                                    # Iterate for list of sub-sub-nodes            
                                    for n in range(1, len(a_list)+1):

                                        super_super_super_child_sub_node=driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]/ul/li[" + str(l) + "]/ul/li[" + str(m) + "]/ul/li[" + str(n) + "]")
                                        print super_super_super_child_sub_node.text
                                       
                                except Exception:
                                    print super_super_child_sub_node.text

                        except Exception:
                            print super_child_sub_node.text

                except Exception:
                    #print Exception
                    print child_sub_node.text

                #print    child_sub_node.text



On Monday, June 25, 2018 at 1:53:12 AM UTC+5:30, [hidden email] wrote:

<a href="https://lh3.googleusercontent.com/-YnIoIBGd0MA/Wy_77SZsN7I/AAAAAAAAAAo/BMh4rhtzWfktWQbaN9GLzhuy1yyfa6SfgCLcBGAs/s1600/sele.PNG" style="margin-left:1em;margin-right:1em" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-YnIoIBGd0MA/Wy_77SZsN7I/AAAAAAAAAAo/BMh4rhtzWfktWQbaN9GLzhuy1yyfa6SfgCLcBGAs/s1600/sele.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-YnIoIBGd0MA/Wy_77SZsN7I/AAAAAAAAAAo/BMh4rhtzWfktWQbaN9GLzhuy1yyfa6SfgCLcBGAs/s1600/sele.PNG&#39;;return true;">

Hi Shiva, thanks for taking your time answering my question.  However, the code does not do my intended goal. Please see the attached picture, As you can see in the picture, if "Cardiovascular" is found, I want that item to be expanded and if it has more buttons to expand, I want all of them to be expanded as well. But, using your code, this does not happen. Also, the print function does not work either. I want for example,  have Cardiovascular Disease as the header and all of  the items that falls under it.  If any of these nodes has a sublist, I want them as well...

On Saturday, June 23, 2018 at 1:29:51 PM UTC-5, Shiva Prasad Adirala wrote:
Hi Mate,
As per my understanding - I wrote snippet for your problem

1. Get List of Node's
2. Expand Each Node One After other
3. Get List Of Sub-Nodes
4. Compare each sub-node and expand sub-node if text==Cardiovascular
5. Get List of Sub-Sub-Node text's (Need to store in list)

from selenium import webdriver
from selenium import webdriver
from <a href="http://selenium.webdriver.common.by" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fselenium.webdriver.common.by\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFJ-xP5N2YmZKzh4vsfqt50siCLXQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fselenium.webdriver.common.by\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFJ-xP5N2YmZKzh4vsfqt50siCLXQ&#39;;return true;">selenium.webdriver.common.by import By
import time

chrome_path=r"E:\Selenium\drivers\Chrome64\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)

#navigate to website
driver.get('<a href="https://meshb.nlm.nih.gov/treeView" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView')

# Get list of nodes
main_list = driver.find_elements_by_xpath("//div[@ng-repeat='node in trees']/div/span")

# Iterate for list of nodes
for i in range(1, len(main_list)+1):

    # Expand each node
    node_expand_icon = driver.find_element_by_xpath("(//div[@ng-repeat='node in trees']/div/span)[" + str(i) + "]")
    node_expand_icon.click();
    # Wait for 5 seconds
    time.sleep(5)

    # Get list of sub-nodes
    sub_nodes_list=driver.find_elements_by_xpath("(//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li")
    
    # Iterate for list of sub-nodes    
    for j in range(1, len(sub_nodes_list)+1):
        sub_node_text=driver.find_element_by_xpath("((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "]").text

        # If sub-node contains "Cardiovascular" then expand
        if "Cardiovascular" in sub_node_text:
            sub_node_expand_icon=driver.find_element_by_xpath("((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li/span)[" + str(j) + "]")
            sub_node_expand_icon.click();
            # Wait for 5 seconds
            time.sleep(5)

            # Get sub-sub-node lists
            sub_sub_nodes_list=driver.find_elements_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li")

            # Iterate for list of sub-sub-nodes            
            for k in range(1, len(sub_sub_nodes_list)+1):

                sub_sub_node_text=driver.find_element_by_xpath("(((//div[@ng-repeat='node in trees']/div)[" + str(i) + "]/ul/li)[" + str(j) + "])/ul/li[" + str(k) + "]").text
                print    sub_sub_node_text

I have tested from my end.Please let me know if you need any help on this

On Friday, June 22, 2018 at 9:49:58 PM UTC+5:30, [hidden email] wrote:
Please someone help me. I have been working on this for more than a week with no luck!

<a href="https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG" style="margin-left:1em;margin-right:1em" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-8hO9tBlWMtg/Wy0LrvV3EsI/AAAAAAAAAAM/TkoplTljUDIcvDOjEcWdtPtxrpk0cSzSACLcBGAs/s1600/stack1.PNG&#39;;return true;">

<a href="https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG" style="margin-left:1em;margin-right:1em" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-GaemJcRQEjA/Wy0LuTuupHI/AAAAAAAAAAQ/40QwfjSejeo5EOKIENtZ2PRmVRu2DNzCQCLcBGAs/s1600/stack3.PNG&#39;;return true;">

<a href="https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG" style="margin-left:1em;margin-right:1em" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG&#39;;return true;" onclick="this.href=&#39;https://lh3.googleusercontent.com/-jEiJOQvoexk/Wy0L1BdGIJI/AAAAAAAAAAU/B6UK4Xf-_kcSJW5fHPDQ18A8cvWAbGOmACLcBGAs/s1600/stack.PNG&#39;;return true;">

From this web page "<a href="https://meshb.nlm.nih.gov/treeView" rel="nofollow" style="font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;color:rgb(0,89,153)" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView", I want to iterate through each node of the tree and if I see the word "Cardiovascular..." in their items, I want to create a dictionary that lists the top level node along with all of cardiovascular associated items. For example, in the above page you can see that if you expand "Anatomy [A]", you will see cardiovascular. Now, I want this part along with whatever included in cardiovascular if you expand it. I have attached 2 screenshots of a part of html.  Here is my Python code:
from selenium import webdriver
from selenium import webdriver
chrome_path=r"G:\My Drive\chrome_driver\chromedriver_win32\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)
driver.get('<a href="https://meshb.nlm.nih.gov/treeView" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fmeshb.nlm.nih.gov%2FtreeView\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGqySLAiT-J4QOkW4mFZ4RCX2VrnQ&#39;;return true;">https://meshb.nlm.nih.gov/treeView')
sidebar = driver.find_element_by_xpath("/html/body/div[2]/div")
i=1
for i in range(16):  # since I have 16 div(s)
   sidebar.find_elements_by_xpath("/html/body/div[2]/div/div[i]")       
   element = driver.find_element_by_xpath("/html/body/div[2]/div").find_element_by_xpath("/html/body/div[2]/div/div[i]").find_element_by_xpath("//*[@class='ng-scope']/span")
   element.click()

But I keep getting this error:no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[2]/div/div[i]"}
Any help is appreciated!

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7c9f7bb8-65f0-474a-91b2-d5d0e3293b11%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.