Avoid getting CAPTCHA's on a bot-sensitive website

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Avoid getting CAPTCHA's on a bot-sensitive website

Jack Zhang

I'm trying to scrape a login-only, bot-sensitive website. After logging in, when I perform a simple selenium function like driver.find_element_by_id('button').click(), the website displays a message along the lines of We think you are a bot. Please complete the CAPTCHA below to continue.

Is there any way for me to make selenium more human-like so I don't trigger CAPTCHA's?

I am using the ChromeDriver


--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/f5ee84b4-a4a5-4204-b12b-0f781105f0ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Avoid getting CAPTCHA's on a bot-sensitive website

Joe Ward
Well it's not wrong, is it? You are technically a bot in the situation you describe. Many websites introduce CAPTCHAs to avoid this type of scraping. If you are running some sort of automated test I would recommend that you take the advice in this link https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha-v2-what-should-i-do

On 25 May 2018 at 20:30, Jack Zhang <[hidden email]> wrote:

I'm trying to scrape a login-only, bot-sensitive website. After logging in, when I perform a simple selenium function like driver.find_element_by_id('button').click(), the website displays a message along the lines of We think you are a bot. Please complete the CAPTCHA below to continue.

Is there any way for me to make selenium more human-like so I don't trigger CAPTCHA's?

I am using the ChromeDriver


--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/f5ee84b4-a4a5-4204-b12b-0f781105f0ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAJPyTs02AKk5xMZ-mif00VXiGD1uD-hYvqd-uv_P%2BDXjkzsFTQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Avoid getting CAPTCHA's on a bot-sensitive website

Naruto Lee
​i think u need ​be modify to ChromeDriver features. however  selenium have a features. u can enter 'navigator.webdriver' on Chrome Console , u will see that return 'true', in fact. chrome be without this field. u can use 'Object.defineProperty()' to be falsify that and it should be return 'undefined'.

By: Naruto Lee

2018-05-27 21:46 GMT+08:00 Joe Ward <[hidden email]>:
Well it's not wrong, is it? You are technically a bot in the situation you describe. Many websites introduce CAPTCHAs to avoid this type of scraping. If you are running some sort of automated test I would recommend that you take the advice in this link https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha-v2-what-should-i-do. 

On 25 May 2018 at 20:30, Jack Zhang <[hidden email]> wrote:

I'm trying to scrape a login-only, bot-sensitive website. After logging in, when I perform a simple selenium function like driver.find_element_by_id('button').click(), the website displays a message along the lines of We think you are a bot. Please complete the CAPTCHA below to continue.

Is there any way for me to make selenium more human-like so I don't trigger CAPTCHA's?

I am using the ChromeDriver


--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/f5ee84b4-a4a5-4204-b12b-0f781105f0ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAJPyTs02AKk5xMZ-mif00VXiGD1uD-hYvqd-uv_P%2BDXjkzsFTQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAFiTbOsrxDTRSMHYQ_XoX%2BWrGcEiJXrfKd0eiog9zw%2Bxh%2BTbuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.