Issue #51 added support for the Bloom filter optimization, but it only applied the filter on the top-level group. For a deep select operation, it should be checking it at all levels and should not traverse paths where the filter cannot possibly succeed.

The change isn't complete trivial, because you need to carry the Selector with you throughout the entire descent. It's probably easiest to tackle this when #49 is worked on, since that will require a restructuring of PathCreator anyway.

Hi. I am using antixml 0.3. I am having trouble with modifying the tree using Zippers.

Here is a gist for a successful experiment:

https://gist.github.com/1871638

And a failed experiment where things go awry and the replacement is erroneous:

https://gist.github.com/1871648

Note, in the failed experiment the original xml was modified so the 'replace' element with an id of 'bar' has an ancestor 'replace' element.

It could just be me, but I could not find a mailing list to dig deeper.

Thanks for your help.

I recently needed to do some XML AST conversions and found the scala.xml wholly unusable... so I wrote a clean minimal XML AST and encoders / decoders, with a backend for the scala.xml parser (should be easy to write one for any other DOM parser)

https://gitlab.com/fommil/scalaz-deriving/blob/master/examples/xmlformat/src/main/scala/xmlformat/XNode.scala

Currently, if a node is transformed to many nodes via flatMap, then merges with conflicting children only occur on the first node. The other nodes are left unchanged and are not presented to the merge strategy.

While I can certainly see cases where that is desirable, the choice of first node is arbitrary and the merge strategy has no control over this.

Issue #63 has a fix for this (along with some other changes). Essentially, merge strategies are presented with the entire list of replacements rather than just the first. This also allows strategies to implement custom behavior for conflicted deletions, which are presented to the strategy using an empty list.

In the case of the default merge strategy, I chose to apply the conflict-merge to every node produced by the flatmap rather than just the first. That seems like a less arbitrary default.

Suppose you have

val xml : Elem = <foo><bar/></foo>.convert

and I want to modify the document to <foo><bar><baz/></bar></foo>.

val bar_zipper : Zipper[Elem] = xml \ "bar"
val new_bar_zipper : Zipper[Elem] = bar_zipper.updated(0, bar_zipper.head addChild <baz/>.convert)
val new_xml = new_bar_zipper.unselect apply 0

The problem is that unlike most of the other Zipper methods, Zipper[T].unselect returns a Zipper[Node] rather than Zipper[T]. Thus new_xml has the general type Node rather than Elem, even though the original zipper was of type Zipper[Elem]. If you want to use Elem-specific functions on the result, you have to cast it:

val new_xml_with_extra_child : Elem = new_xml.asInstanceOf[Elem] addChild <blah/>.convert

If Zipper.flatMap is used to replace one node with many, then Zipper.unselect does not preserve the order of the many. This is because a Map[Int,Set[Int]] is used to correlate elements from the original group to elements in the modified group, and Set makes no ordering guarantees.

For Zipper as it stands, I think the elements of that set will always form a contiguous range, so a Set might actually be overkill here. I'm going to try replacing it with a Range or similar and see if that holds up. Assuming that works, do you envision ever needing the more general correlation provided by Set?

scala> val tree = <top><a /></top>.convert
tree: com.codecommit.antixml.Elem = <top><a/></top>

scala> val expand = (tree \ 'a) flatMap {
     |   case e:Elem =>
     |   for(indx <- 0 until 10) yield e.copy(name=e.name+indx)  
     | }
expand: com.codecommit.antixml.Zipper[com.codecommit.antixml.Elem] = <a0/><a1/><a2/><a3/><a4/><a5/><a6/><a7/><a8/><a9/>

scala> val newTree = expand.unselect
newTree: com.codecommit.antixml.Zipper[com.codecommit.antixml.Node] = <top><a0/><a5/><a1/><a6/><a9/><a2/><a7/><a3/><a8/><a4/></top>

I know Anti-XML is not an XHTML library, but if you want to be able to use it for XHTML (as I did want to), you should know that while <script src="foo"></script> is valid XHTML, <script src="foo"/> isn't. And now the browser HATES me.

scala.xml.XML seems to go to the other extreme, always expanding empty tags to non-empty tags. I guess I could use Anti-XML just for queries and manipulation XML, but use the xml.NodeSeq's .toString to write it to disk, but then I'd either need to keep both representations around, or have some way to convert back from antixml.Node to xml.Node.

Any suggestions?

Thanks for a nice little library!

P.S. The only valid empty tags in XHTML 1.0 Strict are: <base/>``<meta/>``<link/>``<hr/>``<br/>``<param/>``<img/>``<area/>``<input/>``<col/>

the "anti" here should be "convert"
https://github.com/djspiewak/anti-xml/blob/master/src/main/scala/com/codecommit/antixml/package.scala#L51

here attrs=Map("first" -> "yes") should be attrs=Attributes(...)
http://anti-xml.org/zippers.html

Hi. I am using antixml 0.3. I am having trouble with modifying the tree using Zippers.

Using original xml:

and an injectable element:

I can modify the tree using:

val replace = (original \ 'replace) filter (item => item.attrs("id").contains("bar"))
replace.updated(0, insert).unselect

However if I swap the origin xml to:

and run the same search and replace code things go awry.

It could just be me, but I could not find a mailing list to dig deeper.

Thanks for your help.

Right now, deep-select is essentially equivalent to a flatMap recursive deep-select concatenated with the results of a shallow-select. This is very convenient as a definition, but it's probably not the most intuitive ordering. It's basically a breadth-first traversal, which is very different from the order in which you would encounter XML nodes in a lexical context. This natural lexical ordering works out to be depth-first. To mirror these (more intuitive) semantics, the deep-select operator should return items in a depth-first order.

We need a version of deep-select that short circuits the tree traversal when it finds a match. Thus, if there is a selector that matches some node as well as one or more of its children, the resulting Group will contain only the parent node (rather than also containing the children). The major advantage to such an operation would be guaranteed unambiguity in the unselect operation. This short-circuit deep-select method should be named \\! (sound off in the comments if you have a better name!)

Hi Daniel,
I have downloaded the latest version of anti-xml and try to compile with sbt but it hangs. I had the same with a previous version.
C:\Users\Dave\.ivy2\cache\org.scala-lang\scala-compiler\jars\scala-compiler-2.9.0.jar.part
stops/ hangs at 960 kB
System: Windows 7 Home Premium

I am also wondering why downloading if it is possible to copy directly from C:\scala-2.8.1.final\lib or C:\scala-2.9.0.final\lib directory since I have the scala 2.8.1.final and 2.9.0.final distributions locally?
Maybe searchpath variable in project properties file?

Greetings,
Dave

Output

C:\scala-2.9.0.final\djspiewak-anti-xml-2e196e7>java -Xmx512M -jar "c:\sbt\sbt-l
aunch-0.7.5.jar"
Getting Scala 2.7.7 ...
:: retrieving :: org.scala-tools.sbt#boot-scala
        confs: [default]
        2 artifacts copied, 0 already retrieved (9911kB/374ms)
Getting org.scala-tools.sbt sbt_2.7.7 0.7.5 ...
:: retrieving :: org.scala-tools.sbt#boot-app
        confs: [default]
        16 artifacts copied, 0 already retrieved (4271kB/951ms)
[info] Recompiling plugin definition...
[info]    Source analysis: 1 new/modified, 0 indirectly invalidated, 0 removed.
[info]
[info] Updating plugins...
[info] :: retrieving :: plugin-definition#plugin-definition_2.7.7 [sync]
[info]  confs: [compile, runtime, test, provided, system, optional, sources, jav
adoc]
[info]  3 artifacts copied, 0 already retrieved (174kB/187ms)
[success] Plugins updated successfully.
[info]
[info]  Extracted source plugin .\lib_managed\scala_2.7.7\sbt-eclipsify-0.7.0.ja
r ...
[info]  Extracted source plugin .\lib_managed\scala_2.7.7\sbt-scct-for-2.8-0.1-S
NAPSHOT.jar ...
[info] Recompiling plugin...
[info]    Source analysis: 8 new/modified, 0 indirectly invalidated, 0 removed.
[info] Recompiling project definition...
[info]    Source analysis: 1 new/modified, 0 indirectly invalidated, 0 removed.
Getting Scala 2.9.0 ...
downloading http://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.9.0/sc
ala-compiler-2.9.0.jar ...

Even though XML itself doesn't guarantee order preservation during attribute parsing, we should probably still try to achieve it ourselves within the Attributes data structure. Harry H makes a pretty compelling argument here w.r.t. string comparison (in testing) as well as just the principle of least surprise. Fundamentally, there's no way we can correct the issue if an underlying parser implementation decides to reorder attributes, but we can at least preserve order through our own code.

Spec:

forAll { pairs: List[(QName, String)] =>
  Attributes(pairs: _*).toList mustEqual pairs
}

Hi there,

I don't think I see any facility for pretty-printing, is that correct? Would you be interested in a patch to support it?
(Yeah, I know, mostly pretty-printing is annoying, but it has its uses...)

Currently, it is possible to do something like this following:

Elem(None, "<", Attributes(), Group())

Clearly, this Elem name doesn't make any sense. We should run all identifiers through a regexp to ensure this case is rejected at runtime.

I've been trying to use anti-xml to parse and generate some XML documents that use some elements from the atom namespace 1. My code uses our own private namespace, but in the documents it is bound to the default namespace. An example XML looks like this:

<profile xmlns="http://.." xmlns:atom="http://www.w3.org/2005/Atom">
  <atom:link id=".." href=".."/>
</profile>

I've run into two issues with this.

The first is that the conversions drop the namespace entirely (see 2). I've fixed this so that it's more in line with the existing API.

The second issue I see now is that the entire API is oriented around the "prefix" parts instead of the namespace part. When I'm converting the XML to my Profile object I don't care about the prefix, I just want the elements named "link" inside the Atom namespace. Right now I'll have to do this by hand.

I would like to adjust the API so that it's more in line with java.xml.QName and W3's definition, see 3 and 4. The "prefix" is not a part of the qualified name of an element and really not very interesting when it comes to matching on objects.

I'm hoping to be able to write something like this:

def atomLinks(e: Elem) = (e / (Namespaces.atom -> "link"))

Does this make sense? I really like anti-xml and we're using it for most of our XML stuff now but this came up as an issue the other day and I don't see a way to fix it without changing anti-xml.

I've implemented my ideas under my own repository, 5. I'm not entirely satisfied with the current solution but it shows what I want to achieve. All the existing tests passes and I've added some more too.

Hi,

  def syncElems(e1: Elem, e2: Elem): Elem = {
    val fullAttrs = e1.attrs.filterNot(_._2.isEmpty)
    val newAttrs = e2.attrs.filterNot(
      at => fullAttrs.keys exists (at._1 == _)).filter(
      at => e1.attrs.keys exists (at._1 == _))
    e1.copy(attrs = fullAttrs ++: newAttrs)
  }

throws a CastClassException while

  def syncElems(e1: Elem, e2: Elem): Elem = {
    val fullAttrs = e1.attrs.filterNot(_._2.isEmpty)
    e1.copy(attrs = fullAttrs ++: e2.attrs.filterNot(
      at => fullAttrs.keys exists (at._1 == _)).filter(
      at => e1.attrs.keys exists (at._1 == _)))
  }

works fine.

Thanks,

G.N

We crossbuilders would love to see an anti-xml release for Scala 2.9.0-1 and Scala 2.9.1. 2.9.0 is binary compatible, but depending on a different crossbuild is messy and can lead to classpath duplicates as more crossbuilds are released.

Is 0.3 imminent? If not, could we do a retro release of 0.2 for these Scala versions? I'm willing to submit a patch for either case.

The TODO list is quite outdated by now, and should probably be revised.

Also, I'm looking for a new issue to tackle, so having an updated list would definitely help me choosing.

Thanks.

I am assuming Zippers are supposed to behave like a normal collection. Let me know if this is not the case.

steps

1. run this from $ sbt console:

   import com.codecommit.antixml._
   val foo = <parent><a/><b/><c/><d/><e/></parent>.convert
   val notA = foo \ * filter {
     case Elem(p, name, a, s, c) => name != "a"
     case _ => false }
   notA.unselect
   val notAOrB = foo \ * filter {
     case Elem(p, name, a, s, c) => name != "a"
     case _ => false } filter {
     case Elem(p, name, a, s, c) => name != "b"
     case _ => false }
   notAOrB.unselect
   

2. this produces the following outputs:

   foo: com.codecommit.antixml.Elem = <parent><a/><b/><c/><d/><e/></parent>
   notA: com.codecommit.antixml.Zipper[com.codecommit.antixml.Node] = <b/><c/><d/><e/>
   res0: com.codecommit.antixml.Zipper[com.codecommit.antixml.Node] = <parent><b/><c/><d/><e/></parent>
   notAOrB: com.codecommit.antixml.Zipper[com.codecommit.antixml.Node] = <c/><d/><e/>
   res1: com.codecommit.antixml.Zipper[com.codecommit.antixml.Node] = <parent><a/><c/><d/><e/></parent>
   

problem

1. The final res1 includes <a/> I filtered in the earlier filter.

expectations

1. Filtering can be chained multiple times:

   res1: com.codecommit.antixml.Zipper[com.codecommit.antixml.Node] = <parent><c/><d/><e/></parent>
   

note

This is the implementation of filter:

    override def filter(f: A => Boolean): Zipper[A] = collect {
      case e if f(e) => e
    }

collect is equally short, so the problem is likely in the flatMap.
I have an impl for some more context-aware Zipper operations, but it's not working because of this.

At the moment, unselection onDeepZipper relies on hashing of parent nodes lists, this eventually boils down to computing the hash code of the entire XML tree, which gets really slow when dealing with large trees.

One workaround for this would be to compute the hash using the locations of the parent nodes and ignoring the contents, similar to what I've done in the Path class in 93f92cf.

A more elegant solution, to both problems, is to cache the hash code of Elem instances upon creation. e.g. adding something like: override val hashCode: Int = scala.runtime.ScalaRunTime._hashCode(this) to Elem (not sure whether this should be transient or not).

Are there any problems to do so?

It would make sense to work together with the Scala Integrated Query people to make sure Anti-XML can benefit from their work and provide fully typed XML via WSDL or schema files to users.

It would be very bad if we would need the third XML replacement in 2 years ...

Further information:

Type providers in F#: http://msdn.microsoft.com/en-us/library/hh156509%28v=VS.110%29.aspx with some blog post what people can do with it: http://tomasp.net/blog/goto-loosely-structured-data.aspx

http://stackoverflow.com/questions/7429999/are-there-plans-to-support-type-providers-for-scalas-siq-scalaintegratedquery/7431606#7431606

When I run the +publish or the +publish-local tasks, I must run them twice in order to get it to work. On the first run, the result is always the following:

[info] Generating API documentation for main sources...
[info]  delivering ivy file to /Users/daniel/Development/Scala/anti-xml/target/scala-2.9.0.1/ivy-0.3-SNAPSHOT.xml
[info] Compiling 20 Scala sources to /Users/daniel/Development/Scala/anti-xml/target/scala-2.9.0.1/classes...
[error] source file '/Users/daniel/Development/Scala/anti-xml/target/scala-2.9.0.1/src_managed/main/CompatTraversable.scala' could not be found
[info] No documentation generated with unsucessful compiler run
[error] one error found
[error] source file '/Users/daniel/Development/Scala/anti-xml/target/scala-2.9.0.1/src_managed/main/CompatTraversable.scala' could not be found
[error] one error found
[error] {file:/Users/daniel/Development/Scala/anti-xml/}default-46d821/doc: Scaladoc generation failed
[error] {file:/Users/daniel/Development/Scala/anti-xml/}default-46d821/compile: Compilation failed
[error] Total time: 3 s, completed Jul 24, 2011 8:18:23 AM

Running a second time works just fine.

Using anti-xml 0.2 with scala 2.9.1.

val xmlString = new Scanner(this.getClass.getClassLoader.getResourceAsStream(thefilestring), "UTF-8").useDelimiter("\A").next
val xml = XML.fromString(xmlString)

The above throws StackOverflowException on certain files like the one linked to below.

java.lang.StackOverflowError
at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4466)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3694)
at java.util.regex.Pattern$Branch.match(Pattern.java:4502)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4466)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3694)
at java.util.regex.Pattern$Branch.match(Pattern.java:4502)
...

See this link for an example file that triggers this exception.

http://pastebin.com/5CmM8SY3

Consider the following spec:

forAll { (left: String, right: String) =>
  Group(Text(left), Text(right)) mustEqual Group(Text(left + right))
}

This is a very commonly-hit gotcha in scala.xml.

Right now, the NodeMergeStrategy is a member of DeepZipper. This really should be an implicit parameter on unselect with the default instance (as defined in the companion object for NodeMergeStrategy) as BasicNodeMergeStrategy. I think that's the default, anyway…

The only hang-up with just doing this straight away is that NodeMergeStrategy is a type alias for a function. We can't take this as an implicit parameter, since it pollutes the implicit dispatch space. Given that it is a function of arity-2, I don't think it will be a problem, but I'd rather not take chances. In any case, we need a companion object in order to handle lookup properly, and type aliases don't have that.

We need to redefine NodeMergeStrategy as a trait with an apply method. Basically, isomorphic to the function it represents, but without actually being a nominal function type. Then, we can define the appropriate companion object and stick BasicNodeMergeStrategy into there. We then add an implicit parameter to unselect, pass the data down the call chain and we're golden!

I'll take care of this in a few days if no one else gets to it first. I think this is really the only thing remaining to be done before we land zipper-replacement in master. (well, that and a rename of DeepZipper to Zipper, but that's trivial)

Are there plans to release anti-xml for Scala 2.10? I see that there's no.arktekk:anti-xml_2.10:0.5.1 in Maven Central, but that doesn't look like an official anti-xml release. Or is it?

We need a single-level select operation on Selectable that functions exactly like collect except that it returns a DeepZipper rather than merely a Group. The method for this operation should be named select.

Invoking utility methods (e.g. map, flatMap, etc) on something of type Group will return something of type Zipper. This isn't too bad at first glance, since Zipper <: Group. However, the context of this resulting Zipper is invalid, and thus operations performed on that Zipper will have some very strange results. This can be seen with the following trivial example:

val g: Group[Node] = ...
g map identity map identity        // => Group(), regardless of what g is

This is closely related to a second issue, which is that Zipper instances with empty contexts (e.g. resulting from Group#toZipper) do not define a working flatMap. See: issue #13.

I get a StackOverflowError when loading either spending.xml or discogs_20110201_labels.xml from the test/resources folder.

The following line in the Text case class seems to be the culprit:

if (CharRegex.unapplySeq(text).isEmpty) 
    throw new IllegalArgumentException("Illegal character in text '" + text + "'")

Couldn't figure out what character breaks it.

I think 2.9.1 broke something in the "initialCommands" functionality of SBT's console task. When I enter "console", I see the initial commands followed by a scala> prompt and I can then enter a single line. The REPL hangs after that. The problem goes away if I switch back to 2.9.0-1 or if I remove the initialCommands setting from build.sbt

I'll log a related issue to SBT; I'm guessing the issue is is in the interpreter itself, but I'm not familiar with how SBT interacts with it.

I did find one interpreter issue that might be related, but I haven't been able to get the suggested workaround to work in SBT:

https://issues.scala-lang.org/browse/SI-4945

Here are the latest test-perf results from my machine running on the zipper-replacement branch:

[info] -- System Information --
[info] Heap: 2039MB
[info] Java: Apple Inc. 1.6.0_26
[info] OS: Mac OS X 10.7.1 x86_64
[info] 
[info] -- Memory Usage (7 MB) --
[info] anti-xml:  53413432
[info] scala.xml: 26514864
[info] javax.xml: 49023256
[info] 
[info] -- Execution Time --
[info] Loading a 7 MB XML file
[info]  + anti-xml: min: 418 ms, max: 438 ms, average: 425 ms
[info]  + anti-xml StAX: min: 415 ms, max: 438 ms, average: 425 ms
[info]  + scala.xml: min: 241 ms, max: 255 ms, average: 248 ms
[info]  + javax.xml: min: 124 ms, max: 138 ms, average: 129 ms
[info] Shallow selection in a 7 MB tree
[info]  + anti-xml: min: 145618 ms, max: 148546 ms, average: 146193 ms
[info]  + scala.xml: min: 19 ms, max: 24 ms, average: 21 ms
[info] Deep selection in a 7 MB tree
[info]  + anti-xml: min: 1451 ms, max: 1518 ms, average: 1474 ms
[info]  + scala.xml: min: 270 ms, max: 285 ms, average: 277 ms
[info]  + javax.xml: min: 15 ms, max: 18 ms, average: 17 ms

Take special note of that shallow selection time! Needless to say, we need to get that down a bit before we cut a release where selection is entirely based on DeepZipper.

I'm grabbing the XHTML 1.0 from this page:

http://en.wikipedia.org/wiki/Adams_State_College

As one big string blob. This is in Scala 2.9.1. on Windows 7, JRE 1.6. When i try to perform:

var xml = XML.fromString(body)

It's throwing me a StackOverflowException, a short segment of the stack trace looks like:

at java.util.regex.Pattern$GroupTail.match(Pattern.java:4227)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4078)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3345)
at java.util.regex.Pattern$Branch.match(Pattern.java:4114)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4168)
at java.util.regex.Pattern$Loop.match(Pattern.java:4295)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4227)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4078)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3345)
at java.util.regex.Pattern$Branch.match(Pattern.java:4114)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4168)
at java.util.regex.Pattern$Loop.match(Pattern.java:4295)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4227)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4078)

The rest of the stack trace looks about the same. This is a pretty typical, if slightly large, XHTML page (~450kb), and it should not be cause the XML parser to fail.

When you have a namespace asigned to the root node default prefix like this.

<root xmlns="urn:foo:bar">
<child/>
</root>

This translates into the Anti-xml tree

Elem(Some(""), "root", Attributes(), Map(("" -> "urn:foo:bar"), Group(Elem(None, "child", Attributes(), Map(("" -> "urn:foo:bar"), Group.empty))

This is incorrect. The prefix in the child node should also be Some("")

The code below produces java.lang.RuntimeException when entered into the scala console:

import com.codecommit.antixml._
((<html><head><title></title></head><body /></html>).anti \\ "body").unselect

When a Zipper is created with an empty context (e.g. via Group#toZipper), the resulting Zipper does not define a working flatMap. If this were just a result of toZipper, then it would be excusable. Unfortunately, this arises when actually using Zipper normally:

val g: Group[Node] = ...
(g \ "foo").unselect flatMap { e => Some(e) }         // => Group(), regardless of what g is

This appears to be a product of the fact that flatMap builds its result by mapping over the zipper context. Since that context is empty after unselect (in this case), the result will always be Group().

I think we should devise some target benchmark for zipper unselection (deep, shallow, or whatever).

Otherwise, uber horrible performance like the one before the fix in issue #55 may creep in without us noticing.

Consider the following:

CDATA("]]>").toString

This will produce:

<![CDATA[]]>]]>

Naturally, this isn't going to fly. We should be checking for the magic control sequence in the constructor for CDATA and raising an exception if found:

CDATA("]]>") must throwAn[IllegalArgumentException]

Find due to @bartschuller.

Any chance of a retro-release of 0.2 for Scala 2.8.2? Specs is already published. Still need a 2.8.2 ScalaCheck to do it The Right Way™.