How to spawn browser singleton for reusable PagePool? about rod

[github-actions]

Please fix the format of your markdown:

63:62 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1]

_{generated by check-issue}

[ysmood]

Have you tried this?

rod/examples_test.go

Lines 533 to 574 in 46baf3a

	// We can use PagePool to concurrently control and reuse pages.
	func ExamplePage_pool() {
	browser := rod.New().MustConnect()
	defer browser.MustClose()
	// We create a pool that will hold at most 3 pages which means the max concurrency is 3
	pool := rod.NewPagePool(3)
	// Create a page if needed
	create := func() *rod.Page {
	// We use MustIncognito to isolate pages with each other
	return browser.MustIncognito().MustPage()
	}
	yourJob := func() {
	page := pool.Get(create)
	defer pool.Put(page)
	page.MustNavigate("http://example.com").MustWaitLoad()
	fmt.Println(page.MustInfo().Title)
	}
	// Run jobs concurrently
	wg := sync.WaitGroup{}
	for range "...." {
	wg.Add(1)
	go func() {
	defer wg.Done()
	yourJob()
	}()
	}
	wg.Wait()
	// cleanup pool
	pool.Cleanup(func(p *rod.Page) { p.MustClose() })
	// Output:
	// Example Domain
	// Example Domain
	// Example Domain
	// Example Domain
	}

Also the browser pool works fine to me:

rod/examples_test.go

Lines 599 to 637 in 167ecc0

	// We can use [rod.BrowserPool] to concurrently control and reuse browsers.
	func ExampleBrowser_pool() {
	// Create a new browser pool with a limit of 3
	pool := rod.NewBrowserPool(3)
	// Create a function that returns a new browser instance
	create := func() *rod.Browser {
	browser := rod.New().MustConnect()
	return browser
	}
	// Use the browser instances in separate goroutines
	var wg sync.WaitGroup
	for i := 0; i < 3; i++ {
	wg.Add(1)
	go func() {
	defer wg.Done()
	// Get a browser instance from the pool
	browser := pool.Get(create)
	// Put the instance back to the pool after we're done,
	// so the instance can be reused by other goroutines.
	defer pool.Put(browser)
	// Use the browser instance
	page := browser.MustPage("https://www.google.com")
	fmt.Println(page.MustInfo().Title)
	}()
	}
	// Wait for all the goroutines to finish
	wg.Wait()
	// Cleanup the pool by closing all the browser instances
	pool.Cleanup(func(p *rod.Browser) {
	p.MustClose()
	})
	}

[lukeed]

Yea tried both. They both work because the pool/browser is created in the same function lifetime. I’m trying to have a singleton that multiple top level functions refer to over time

[ysmood]

have a singleton that multiple top level functions refer to over time

Sorry, I'm not sure I understand what you mean. If you just need a global variable, then it's just a code design issue, not related to rod, such as use a closure:

var globalPool = func() func() (*rod.Page, func()) {
	pool := rod.NewPagePool(3)

	create := func() *rod.Page {
		browser := rod.New().MustConnect()
		return browser.MustIncognito().MustPage()
	}

	return func() (*rod.Page, func()) {
		page := pool.Get(create)
		return page, func() {
			pool.Put(page)
		}
	}
}()

func main() {
	for range "...." {
		go func() {
			page, release := globalPool()
			defer release()

			page.MustNavigate("http://example.com")
		}()
	}
}

[lukeed]

I think my problem was that I was using defer l.Cleanup(), which I now see waits until the browser(s) it launched are closed.

Thank you for your snippet. I now have this code running & working; however, I'd still like to make it so that a browser is reused and the pool is creating Page from the same Browser struct (instead of creating new browsers every time, like in your code). The reason for this is that I have expensive logic running to figure out which browser to use & that only needs to be run once

package main

import (
	"fmt"
	"sync"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/launcher"
)

var globalPool = func() func() (*rod.Page, func()) {
	pool := rod.NewPagePool(3)

	fmt.Println("attempting...")

	create := func() *rod.Page {
		l := launcher.New().Headless(true)
		if path, exists := launcher.LookPath(); exists {
			fmt.Println("~> found local browser")
			l.Bin(path)
		}

		// defer l.Cleanup()
		browser := rod.New().ControlURL(l.MustLaunch()).MustConnect()
		return browser.MustIncognito().MustPage()
	}

	return func() (*rod.Page, func()) {
		fmt.Println("before create")
		page := pool.Get(create)
		fmt.Println("found page")
		return page, func() {
			fmt.Println("> released")
			pool.Put(page)
		}
	}
}()

func main() {
	var wg sync.WaitGroup

	for v := range []int{1, 2, 3} {
		fmt.Println("inside loop")
		wg.Add(1)
		go func() {
			defer wg.Done()
			fmt.Println("inside goroutine")
			page, release := globalPool()
			defer release()

			fmt.Println(">> eval", string(v))
			page.MustEval("function(x) { console.log('~>', x) }", v)
		}()
	}

	wg.Wait()
}

[lukeed]

I got it working now, thank you :) It just came down to using l.Cleanup(), which I should be calling manually elsewhere.