List element inside a table is lost about trafilatura

[mikhainin]

Sure, I just filed #534

[mikhainin]

I was able to fix it this way:

diff --git a/trafilatura/core.py b/trafilatura/core.py
index 63699a4..1970c25 100644
--- a/trafilatura/core.py
+++ b/trafilatura/core.py
@@ -397,7 +397,7 @@ def handle_table(table_elem, potential_tags, options):
                     # add child element to processed_element
                     if processed_subchild is not None:
                         subchildelem = SubElement(newchildelem, processed_subchild.tag)
-                        subchildelem.text, subchildelem.tail = processed_subchild.text, processed_subchild.tail
+                        subchildelem.text, subchildelem.tail = ''.join(processed_subchild.itertext()), processed_subchild.tail
                     child.tag = 'done'
             # add to tree
             if newchildelem.text or len(newchildelem) > 0:

But not sure if this is the correct solution

[adbar]

@mikhainin Thank you for reporting the bug and the solution, could you please draft a PR with your solution? If the tests pass I would integrate it.

[adbar]

Note: the issue is now fixed if recall option is on.

[alroythalus]

Try it for spotify https://www.spotify.com/in-en/legal/privacy-policy/
The lists in the tables arnt being captured yet
@adbar @mikhainin