Puppeteer Integration

Puppeteer integration with Incogniton enables browser automation using custom profiles. This combination allows for web scraping, testing, and data extraction while maintaining consistent browser fingerprints. Learn how to set up and use Puppeteer with Incogniton profiles below.

Prerequisites

  • Node.js: Version 14.x or higher recommended
  • Puppeteer-core: Install via npm or yarn

How to Setup Puppeteer with Incogniton

Follow these steps to set up Puppeteer with Incogniton for browser automation:

1

Install Puppeteer-core

Run the following command in your Node.js project. Unlike regular Puppeteer, puppeteer-core doesn't download Chromium, making it ideal for use with Incogniton's browser profiles.

npm install puppeteer-core
2

Usage (JavaScript)

Here's an example of using Puppeteer to scrape a page:

import puppeteer from 'puppeteer-core'

const main = async () => {
  try {
    const profileId = 'your-profile-id' // Replace with your actual profile ID
    const launchUrl = `http://localhost:35000/automation/launch/puppeteer`
    const requestBody = {
      profileID: profileId,
    }

    const response = await fetch(launchUrl, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
      },
      body: JSON.stringify(requestBody),
    })

    const data = await response.json()
    const { puppeteerUrl } = data

    console.log('The Incogniton browser is launching...')
    await new Promise((resolve) => setTimeout(resolve, 30000)) // Wait for browser to launch

    const browser = await puppeteer.connect({
      browserURL: puppeteerUrl,
      acceptInsecureCerts: true,
    })

    const page = await browser.newPage()
    await page.goto('https://example.com', { waitUntil: 'networkidle2' })

    const text = await page.$eval('#sample-id', (el) => el.textContent)
    console.log(`Extracted text: ${text}`)

    await browser.close()
    console.log('Browser session closed successfully.')
  } catch (error) {
    console.error('Error during scraping session ->', error)
  }
}

main()

Key Features

  • Headless Mode: Run Chromium in the background for efficient automation.
  • Rich APIs: Take screenshots, emulate devices, or intercept network requests.
  • Fast Integration: Seamlessly interact with any web page, even those with dynamic content.

Troubleshooting

  • Use try/catch for robust error handling in async code
  • Increase the timeout value to resolve timeout errors when navigating to a page
  • Use the --no-sandbox flag if Puppeteer is not launching correctly due to permission issues
  • Run in non-headless mode or spoof the user-agent string to bypass blocked network requests or CAPTCHAs

Was this page helpful?