Interacting with Shadow DOM - The Playwright way

You might already know that Playwright comes with all the bells and whistles which make it easy to use without the need to spend time to do set up or writing custom functions. This is true for handling elements in the shadow DOM, but only for open shadow DOM. Playwright struggles when dealing with texts enclosed in ‘user-agent’ or ‘closed’ shadow DOM trees. In this article, let me walk you through how I handled ‘open’ and ‘user-agent’ shadow roots.

Let’s first understand what a Shadow DOM is.

Shadow DOM encapsulates web components, isolating them from the rest of the DOM tree. Envision the main DOM (Document Object Model) as a big tree that contains all the elements and content of a webpage, like paragraphs, images, buttons, and more. Now, think of the Shadow DOM as a smaller, hidden tree that can be placed inside this big tree. This smaller tree (the shadow tree) has its own elements and styles, completely separate from the main DOM tree. This isolation helps keep these components distinct from the rest of the DOM. This way, any styling, behavior, or structure applied to the main DOM tree won't affect the components inside the Shadow DOM, allowing for custom styling. Another benefit is encapsulation, enabling developers to modify components independently without conflicts. 

Shadow DOMs are incorporated into the main DOM tree using an element called the ‘shadow host.’ Developers can incorporate child ‘shadow hosts’ within this, thereby forming a shadow tree.

According to MDN: 

Shadow DOM allows hidden DOM trees to be attached to elements in the regular DOM tree — this shadow DOM tree starts with a shadow root, underneath which you can attach any element, in the same way as the normal DOM. 

• Shadow host: The regular DOM node that the shadow DOM is attached to.
• Shadow tree: The DOM tree inside the shadow DOM.
• Shadow boundary: the place where the shadow DOM ends, and the regular DOM begins.
• Shadow root: The root node of the shadow tree.

A shadow tree is attached to the host using either ‘open’ or ‘closed’ mode. When the developer attaches an element (or shadow tree) using ‘open’ mode, it is possible to modify the element/content using JavaScript, whereas the ‘closed’ mode makes it really difficult to deal with the element.

While inspecting html of some of the web applications, you would have come across ‘shadow-root(user-agent)’. This is shadow DOM provided by the browser itself for elements such as <input>, <select> & <video>.

Note: If you want to see shadow-root in your Chrome browser, then open the Developer console (CMD+SHIFT+C), go to settings (F1) and select the ‘Show user agent shadow DOM’ checkbox under Elements section.

How Playwright Interacts with Shadow Trees

There are various locator strategies you can apply for interacting with any regular web element in Playwright, but you must use CSS selectors to traverse into shadow-root.

Open Shadow-root

Playwright can easily traverse an ‘open’ shadow-root like any regular web element but remember XPath won’t work.

Consider the below example: 

From the HTML, I want to get the label for the button. The below line of code will get you the text value.

const label = await page.locator('#shadow-host-for-open #mybutton').textContent();

Note: If you try to inspect the DOM using the above CSS locator in the Elements tab of browser, you won’t be able to because the browser can only take you to the host element (i.e., <div id="shadow-host-for-open"></div>) and not into the shadow-root. But int Console tab you can identify the parent element using document.queryselector() and then traverse to shadow-root child nodes like this

document.querySelector("#shadow-host-for-open ").shadowRoot.querySelector('#mybutton'')

User-agent & Closed Shadow-root for Ext.js Apps

That was easy, but how do you get the value from a ‘user-agent’ or ‘closed’ shadow tree element?

There is no direct option for Playwright to interact with elements inside of ‘user-agent’ or ‘closed’ shadow root. But if your web application is built using Ext.js, then you are in luck. We can leverage Ext.js ‘component’ feature to write a direct CSS locator and interact with the element in the ‘user-agent’ tree.

Let’s check the below web component and its HTML:

Our intention is to get the value of the input textbox, which is ‘Enter name’.

If we try to validate the input using a direct CSS (#shadow-host-for-open input) either in the Elements tab or in the Console tab, you won’t get any result.

But when you use page.evaluate(), Ext.component(), and document.querySelector(), the magic happens. See below how we do this in Playwright.

const Ext: any = undefined;
const myLocator = "#shadow-host-for-open input"
const placeholderValue = await this.page.evaluate(({ myLocator }) => Ext.Component.from(document.querySelector(myLocator)).getPlaceholder(), { myLocator });
console.log('Placeholder value found as ', placeholderValue);

Note: I didn’t get a chance to verify the above approach for ‘closed’ shadow-root as the application under test built on Ext.js had only ‘user-agent’ shadow-root.

User-agent & Closed Shadow-root for Non-Ext.js Apps

A direct approach to interacting with the user-agent and closed shadow-root for web apps built on different frameworks seems unavailable.

However, you can try this workaround: First, locate and click on a web element positioned just before your target element within the shadow DOM. Then, use the ‘Tab’ key to navigate to the target element.

In the example below, our intention is to enter a secret key and then copy the same value to ensure that it has been entered correctly. As the Key field is enclosed in a closed shadow-root, first, we will click on the Name field and then use the Tab key to move to the target field.

Let’s see how we can do it in Playwright. 

            await this.page.locator("#inputForName").click();
            await this.page.keyboard.press('Tab');
            await this.page.waitForTimeout(1000);
            await this.page.keyboard.type('Superman');
            await this.page.waitForTimeout(1000);
            await this.page.keyboard.press('Meta+A')
            await this.page.keyboard.press('Meta+c');
            const copiedText = await this.page.evaluate(async () => {
                return await navigator.clipboard.readText();
            });
            console.log("Copied Text is ", copiedText);

Note: Meta is used in Mac, and if you are doing the scripting in Windows/Linux, then you can use the Control key instead of Meta.

I hope this article has made handling Shadow DOM elements in your test automation easier. If you have any questions or feedback, please leave a comment below