malformed attribute selector about soupsieve

[brandonscholet]

Also, I just wanted to say thank you for what you do.

[facelessuser]

I don't have just a simple regex to tell if an entire complex selector is good. In general, I'm basically tokenizing the selectors.

To better explain, I start at the beginning and I look for allowed things identifiers (tags), classes (identifiers preceded by .), pseudo-class names (identifiers preceded by :). If a pseudo class is followed by (, and it is recognized as a pseudo class that takes functional options, then we switch context and parse according to the rules of that pseudo-class. If we see a [, we switch context to handle an attribute.

I'm mainly giving a general overview, the exact details may be a little different, but you get the idea. There is no simple magic regex that can tell if the entire selector is valid, we just take it apart, piece by piece, making sure each piece makes sense in context of what came before, and if it doesn't, we halt by throwing an error.

[brandonscholet]

That's fair. Thank you for your time!

You should check out my tool wappybird. It's built on wapppalyzer. I use it to scan very large ranges for penetration tests to search for technologies.

[facelessuser]

@brandonscholet One thing you could do is compile the selector to test it in your own validation function.

>>> import soupsieve as sv
>>> def is_valid_selector(sel):
...     try:
...         sv.compile(sel)
...     except (sv.SelectorSyntaxError, NotImplementedError):
...         return False
...     return True
...
>>> is_valid_selector('> this')
False
>>> is_valid_selector('tag.class[attr=value]')
True

[brandonscholet]

You are a gentlemen and a scholar. It's quick and easy to add to the guy's script. and adds no time to the larger amount of things to parse through without adding to execution time.

[facelessuser]

Yeah, we cache recently compiled patterns, so subsequent compiles should take much less time.

[facelessuser]

Just a note for anyone else checking this issue out. Soupsieve also offers its own API. It was designed this way as we were not sure when we originally implemented this if BeautifulSoup would accept us as the official CSS selector implementation.

We can build a query before using it, and use the query directly by passing in the BeautifulSoup tag.

>>> from bs4 import BeautifulSoup
>>> import soupsieve as sv
>>> soup = BeautifulSoup('<div><span>text</span></div>', 'html.parser')
>>> query = sv.compile('div > span')
>>> query.select(soup)
[<span>text</span>]

This allows you control of the selecting process. You can catch bad selectors and then choose to continue or not continue with the query without having to issue a recompile. The API also allows performing matches with match and finding the closest parent with closest, etc.

One of these days, I may see if BeautifulSop can add closest and match into their API as well, but they are always usable through our API.