Selenium Integration
Selenium integration with Incogniton enables browser automation using custom profiles. This combination allows for web scraping, testing, and data extraction while maintaining consistent browser fingerprints. Learn how to set up and use Selenium with Incogniton profiles below.
Official Incogniton SDKs are currently under development and are expected to be available in the coming weeks. In the meantime, this guide provides a reliable solution for leveraging Selenium with Incogniton to automate browser tasks.
Prerequisites
- Python: Version 3.8 or higher recommended
- Selenium: Latest package via
pip install selenium
- Requests: Install via
pip install requests
How to Set up Selenium with Incogniton (Python)
Follow these steps to set up Selenium with Incogniton for browser automation:
Install Dependencies
Run the following command in your Python environment:
pip install selenium requests
Launch Browser Profile (GET)
Use a GET request to start the browser profile and receive the remote endpoint URL:
from selenium import webdriver
import requests
import ast
PROFILE_ID = 'your-profile-id' # Replace with your actual profile ID
API_URL = f"http://127.0.0.1:35000/automation/launch/python/{PROFILE_ID}"
response = requests.get(API_URL)
response.raise_for_status()
payload = response.json()
remote_url = payload['url']
data_dict = ast.literal_eval(payload['dataDict'])
# Connect to the remote webdriver
options = webdriver.ChromeOptions()
# Add any desired ChromeOptions here, e.g.:
# options.add_argument('--disable-notifications')
with webdriver.Remote(command_executor=remote_url, options=options) as driver:
driver.get('https://example.com')
print(driver.title)
Launch Browser Profile with custom settings (POST)
To take things a step further, you can utilize a POST request to include custom launch arguments:
from selenium import webdriver
import requests
import ast
PROFILE_ID = 'your-profile-id'
API_URL = 'http://127.0.0.1:35000/automation/launch/python/'
# Optional: Stop any running profile
try:
stop_url = f"http://127.0.0.1:35000/profile/stop/{PROFILE_ID}"
requests.get(stop_url)
except requests.RequestException:
print('Could not stop existing profile')
# Launch with custom arguments
payload = {
'profileID': PROFILE_ID,
'customArgs': '--disable-notifications --start-maximized'
}
response = requests.post(API_URL, json=payload)
response.raise_for_status()
data = response.json()
remote_url = data['url']
# Connect and use
options = webdriver.ChromeOptions()
with webdriver.Remote(command_executor=remote_url, options=options) as driver:
driver.get('https://example.com')
print(driver.current_url)
Key Features
- Headless or Headed Mode: Control headless behavior via ChromeOptions.
- Device Emulation: Use Selenium capabilities to emulate mobile devices.
- Network Interception: Leverage Chrome DevTools Protocol via Selenium.
Troubleshooting
- Increase Selenium timeouts for slow-loading pages using
driver.set_page_load_timeout(60)
anddriver.implicitly_wait(10)
- If permission issues arise, launch Incogniton with
--no-sandbox
or run as administrator. - Wrap automation code in try/except to catch and log exceptions.