How to look for new DOM elements with a certain class inside a container?

[Hash6232]

I want to observe changes on a web page listing volumes for a certain novel but the output result when I send a notification shows trigger text from all the volume containers in the page instead of just new ones. New DOM elements are added over time and inside these elements other unrelated changes I am not interested in happen but for now this is beside the point.

My use case is the following:

1. Look for changes in the list of volumes on a web page
2. Trigger change when a new .novel element is added to the page
3. Return text content of the h2 > a element as output

Fetch method is basic (plaintext). Filter is .novel[id]:not(:has(~ .novel)) AKA last .novel element with any id that doesn't have .novel siblings beneath. Trigger is /Volume \d+/ and only Added lines and Only trigger when unique lines appear are checked. The content of the notification body is {{triggered_text}}.

The HTML looks like this:

/* Series of nested obfuscated <div> elements*/
  <div class=".. novel" id="volume-1">
    /* Series of nested obfuscated <div> elements*/
      <h2>
        <a href="#volume-3">Volume 3</a>
        ..
      </h2>
  </div>
  <div class=".. novel" id="volume-2"> .. </div>
  <div class=".. novel" id="volume-3"> .. </div>

but the notification test returns

Volume 1
Volume 2
Volume 3

[dgtlmoon]

ChatGPT answer..

To achieve your goal of observing changes and triggering notifications when a new .novel element is added to the page, specifically extracting the text content of the h2 > a element, you can use an XPath 2.0+ expression to identify the last .novel element without any subsequent .novel siblings. This will help you isolate newly added elements.

Here's the XPath expression to achieve this:

//div[contains(@class, 'novel') and not(following-sibling::div[contains(@class, 'novel')])]/h2/a/text()

Explanation:

• //div[contains(@class, 'novel')]: Selects all div elements that have a class containing the word "novel".
• not(following-sibling::div[contains(@class, 'novel')]): Filters these div elements to select only those that do not have any following sibling div elements with a class containing the word "novel". This ensures you get only the last .novel element on the page.
• /h2/a/text(): After selecting the appropriate div, it looks inside for an h2 element with an a element inside it and retrieves the text content of the a element.

Usage in Your Context:

• Filter: .novel[id]:not(:has(~ .novel)) — This filter ensures that you target the last .novel element that does not have any .novel siblings beneath it.
• Trigger: /Volume \d+/ — The trigger pattern you have looks for the specific text pattern "Volume X" (where X is a number).
• Output: {{triggered_text}} — This would return the text content from the h2 > a element of the last .novel element.

This XPath expression will help you monitor the latest volume additions and ensure that your notifications are triggered only for new .novel elements.

[Hash6232]

Thanks for the sample. Although it didn't work in my case, it directed me toward the right direction:

//*[contains(@class, 'novel') and starts-with(@id, 'volume')][last()]

I removed the trigger line and used the Extract text field to return line-by-line the info I needed to extract:

/Volume (\d)/
/Digital\n.+(\d{4}-\d{2}-\d{2})/

In the end I also had to enable javascrpit parsing because some info like dates weren't static like the rest of the markup.

Finally I used jinja to turn the raw output into a neatly formatted markdown notification:

# Notification title
**{{watch_title}}**

# Notification body
{% set lines = current_snapshot.split('\n') %}*Volume {{ lines[0] }}* releasing on *{{ lines[1] }}*

Something that greatly helped me debugging was removing all the trigger/extract rules, emptying the history for the page I wanted to debug and let changedetection do a first raw check. The preview page showcased the text content of the DOM element I filtered along with any possible line breaks and special characters that might make regex parsing difficult