scala-uri
is a small Scala library that helps you work with URIs. It has the following features:
- A DSL for building URIs
- A parser to parse URIs from Strings.
- Can be used outside a servlet environment as it has zero dependencies on the servlet spec or existing web frameworks.
- Ability to replace and remove query string parameters
- Support for custom encoding such as encoding spaces as pluses
- Support for protocol relative urls
- Support for user information e.g.
ftp://user:[email protected]
- Support for matrix parameters
To include it in your SBT project from maven central:
"com.netaporter" %% "scala-uri" % "0.4.10"
Note: This library works best when using Scala 2.11.2+
. Due a bug in olders versions of Scala, this library can result in StackOverflowException
s for very large URLs when using versions of Scala olders than 2.11.2
. More details
import com.netaporter.uri.dsl._
val uri = "http://theon.github.com/scala-uri" ? ("p1" -> "one") & ("p2" -> 2) & ("p3" -> true)
uri.toString //This is: http://theon.github.com/scala-uri?p1=one&p2=2&p3=true
val uri2 = "http://theon.github.com/scala-uri" ? ("param1" -> Some("1")) & ("param2" -> None)
uri2.toString //This is: http://theon.github.com/scala-uri?param1=1
To add query string parameters, use either the ?
or &
method and pass a Tuple2
as an argument. The first value in the Tuple is a name of the query string parameter, the second is the value. If a parameter value is an Option
, it will only be rendered provided it is not None
.
import com.netaporter.uri.dsl._
val p = ("key", true) :: ("key2", false) :: Nil
val uri = "http://example.com".addParams(p)
uri.toString //This is: http://example.com/?key=true&key2=false
import com.netaporter.uri.dsl._
val uri = "http://theon.github.com" / "scala-uri"
uri.toString //This is: http://theon.github.com/scala-uri
To add path segments, use the /
method
To set the fragment, use the `#`
method:
import com.netaporter.uri.dsl._
val uri = "http://theon.github.com/scala-uri" `#` "fragments"
uri.toString //This is: http://theon.github.com/scala-uri#fragments
Provided you have the import com.netaporter.uri.dsl._
, Strings will be implicitly parsed into Uri
instances:
import com.netaporter.uri.dsl._
val uri: Uri = "http://theon.github.com/scala-uri?param1=1¶m2=2"
However, if you prefer, you can call Uri.parse()
explicitly:
import com.netaporter.uri.Uri.parse
val uri = parse("http://theon.github.com/scala-uri?param1=1¶m2=2")
The mapQuery
method will transform the Query String of a URI by applying the specified Function to each Query String Parameter
val uri = "/scala-uri" ? ("p1" -> "one") & ("p2" -> 2) & ("p3" -> true)
//Results in /scala-uri?p1_map=one_map&p2_map=2_map&p3_map=true_map
uri.mapQuery {
case (n, v) => (n + "_map", v + "_map")
}
uri.mapQuery(_.swap) //Results in /scala-uri?one=p1&2=p2&true=p3
The mapQueryNames
and mapQueryValues
provide a more convenient way to transform just Query Parameter names or values
val uri = "/scala-uri" ? ("p1" -> "one") & ("p2" -> 2) & ("p3" -> true)
uri.mapQueryNames(_.toUpperCase) //Results in /scala-uri?P1_map=one&P2=2&P3=true
uri.mapQueryValues(_.replace("true", "false")) //Results in /scala-uri?p1=one&p2=2&p3=false
The filterQuery
method will remove any Query String Parameters for which the provided Function returns false
val uri = "/scala-uri" ? ("p1" -> "one") & ("p2" -> 2) & ("p3" -> true)
//Results in /scala-uri?p2=2
uri.filterQuery {
case (n, v) => n.contains("2") && v.contains("2")
}
uri.filterQuery(_._2 == "one") //Results in /scala-uri?p1=one
The filterQueryNames
and filterQueryValues
provide a more convenient way to filter just by Query Parameter name or value
val uri = "/scala-uri" ? ("p1" -> "one") & ("p2" -> 2) & ("p3" -> true)
uri.filterQueryNames(_ > "p1") //Results in /scala-uri?p2=2&p3=true
uri.filterQueryValues(_.length == 1) //Results in /scala-uri?p2=2
By Default, scala-uri
will URL percent encode paths and query string parameters. To prevent this, you can call the uri.toStringRaw
method:
import com.netaporter.uri.dsl._
val uri = "http://example.com/path with space" ? ("param" -> "üri")
uri.toString //This is: http://example.com/path%20with%20space?param=%C3%BCri
uri.toStringRaw //This is: http://example.com/path with space?param=üri
The characters that scala-uri
will percent encode by default can be found here. You can modify which characters are percent encoded like so:
Only percent encode the hash character:
import com.netaporter.uri.encoding._
implicit val config = UriConfig(encoder = percentEncode('#'))
Percent encode all the default chars, except the plus character:
import com.netaporter.uri.encoding._
implicit val config = UriConfig(encoder = percentEncode -- '+')
Encode all the default chars, and also encode the letters a and b:
import com.netaporter.uri.encoding._
implicit val config = UriConfig(encoder = percentEncode ++ ('a', 'b'))
The default behaviour with scala uri, is to encode spaces as %20
, however if you instead wish them to be encoded as the +
symbol, then simply add the following implicit val
to your code:
import com.netaporter.uri.dsl._
import com.netaporter.uri.encoding._
implicit val config = UriConfig(encoder = percentEncode + spaceAsPlus)
val uri: Uri = "http://theon.github.com/uri with space"
uri.toString //This is http://theon.github.com/uri+with+space
If you would like to do some custom encoding for specific characters, you can use the encodeCharAs
encoder.
import com.netaporter.uri.dsl._
import com.netaporter.uri.encoding._
implicit val config = UriConfig(encoder = percentEncode + encodeCharAs(' ', "_"))
val uri: Uri = "http://theon.github.com/uri with space"
uri.toString //This is http://theon.github.com/uri_with_space
By Default, scala-uri
will URL percent decode paths and query string parameters during parsing:
import com.netaporter.uri.dsl._
val uri: Uri = "http://example.com/i-have-%25been%25-percent-encoded"
uri.toString //This is: http://example.com/i-have-%25been%25-percent-encoded
uri.toStringRaw //This is: http://example.com/i-have-%been%-percent-encoded
To prevent this, you can bring the following implicit into scope:
import com.netaporter.uri.dsl._
implicit val c = UriConfig(decoder = NoopDecoder)
val uri: Uri = "http://example.com/i-havent-%been%-percent-encoded"
uri.toString //This is: http://example.com/i-havent-%25been%25-percent-encoded
uri.toStringRaw //This is: http://example.com/i-havent-%been%-percent-encoded
If you wish to replace all existing query string parameters with a given name, you can use the uri.replaceParams()
method:
import com.netaporter.uri.dsl._
val uri = "http://example.com/path" ? ("param" -> "1")
val newUri = uri.replaceParams("param", "2")
newUri.toString //This is: http://example.com/path?param=2
If you wish to remove all existing query string parameters with a given name, you can use the uri.removeParams()
method:
import com.netaporter.uri.dsl._
val uri = "http://example.com/path" ? ("param" -> "1") & ("param2" -> "2")
val newUri = uri.removeParams("param")
newUri.toString //This is: http://example.com/path?param2=2
To get the query string parameters as a Map[String,Seq[String]]
you can do the following:
import com.netaporter.uri.Uri
val uri = Uri.parse("http://example.com/path?a=b&a=c&d=e")
uri.query.paramMap //This is: Map("a" -> Seq("b", "c"), "d" -> Seq("e"))
scala-uri
supports user information (username and password) encoded in URLs.
Parsing URLs with user information:
val uri = "http://user:[email protected]"
uri.user //This is Some("user")
uri.password //This is Some("pass")
Modifying user information:
import com.netaporter.uri.dsl._
val mailto = "mailto://[email protected]"
mailto.withUser("jack") //URL is now [email protected]
import com.netaporter.uri.dsl._
val uri = "http://user:[email protected]"
uri.withPassword("secret") //URL is now http://user:[email protected]
Note: that using clear text passwords in URLs is ill advised
Protocol Relative URLs are supported in scala-uri
. A Uri
object with a protocol of None
, but a host of Some(x)
will be considered a protocol relative URL.
import com.netaporter.uri.dsl._
val uri: Uri = "//example.com/path"
uri.scheme //This is: None
uri.host //This is: Some("example.com")
Matrix Parameters are supported in scala-uri
. Support is enabled
using aUriConfig
with matrixParams = true
like so:
import com.netaporter.uri.dsl._
implicit val config = UriConfig(matrixParams = true)
val uri = "http://example.com/path;paramOne=value;paramTwo=value2/pathTwo;paramThree=value3"
//Get parameters at the end of the path
uri.matrixParams //This is Vector("paramThree" -> "value3")
//Add parameters to end of path
val uri2 = uri.addMatrixParam("paramFour", "value4")
uri2.toString //This is http://example.com/path;paramOne=value;paramTwo=value2/pathTwo;paramThree=value3;paramFour=value4
//Get parameters for mid path segment
uri.pathPart("pathTwo").params //This is Vector("paramOne" -> "value", "paramTwo" -> "value2")
//Add parameters for mid path segment
val uri3 = uri.addMatrixParam("pathTwo", "paramFour", "value4")
By default scala-uri
uses UTF-8
charset encoding:
val uri = "http://theon.github.com/uris-in-scala.html" ? ("chinese" -> "网址")
uri.toString //This is http://theon.github.com/uris-in-scala.html?chinese=%E7%BD%91%E5%9D%80
This can be changed like so:
implicit val conf = UriConfig(charset = "GB2312")
val uri = "http://theon.github.com/uris-in-scala.html" ? ("chinese" -> "网址")
uri.toString //This is http://theon.github.com/uris-in-scala.html?chinese=%CD%F8%D6%B7
scala-uri
0.4.x
is currently built with support for scala 2.10.x
and 2.11.x
For 2.9.x
support use scala-uri
0.3.x
Release builds are available in maven central. For SBT users just add the following dependency:
"com.netaporter" %% "scala-uri" % "0.4.10"
For maven users you should use (for 2.11.x):
<dependency>
<groupId>com.netaporter</groupId>
<artifactId>scala-uri_2.11</artifactId>
<version>0.4.10</version>
</dependency>
For the latest snapshot builds, add the Sonatype OSS repo to your SBT build configuration:
resolvers += "Sonatype OSS" at "https://oss.sonatype.org/content/repositories/snapshots"
Add the following dependency:
"com.netaporter" %% "scala-uri" % "0.4.11-SNAPSHOT"
Contributions to scala-uri
are always welcome. Good ways to contribute include:
- Raising bugs and feature requests
- Fixing bugs and developing new features (I will attempt to merge in pull requests ASAP)
- Improving the performance of
scala-uri
. See the Performance Tests project for details of how to run thescala-uri
performance benchmarks.
The unit tests can be run from the sbt console by running the test
command! Checking the unit tests all pass before sending pull requests will be much appreciated.
Generate code coverage reports from the sbt console by running the scct:test
command. The HTML reports should be generated at target/scala-2.10/coverage-report/index.html
. Ideally pull requests shouldn't significantly decrease code coverage, but it's not the end of the world if they do. Contributions with no tests are better than no contributions :)
For the scala-uri
performance tests head to the scala-uri-benchmarks github project
- Package changes / import changes
- All code moved from
com.github.theon
package tocom.netaporter
package scala-uri
has been organised into the following packages:encoding
,decoding
,config
anddsl
. You will need to update import statments.- Name changes
PermissiveDecoder
renamed toPermissivePercentDecoder
QueryString
andMatrixParams
constructor argumentparameters
shortened toparams
Uri.parseUri
renamed toUri.parse
protocol
constructor arg inUri
renamed toscheme
Querystring
renamed toQueryString
- Query String constructor argument
parameters
changed type fromMap[String, List[String]]
toSeq[(String,String)]
Uri
constructor argumentpathParts
changed type fromList
toVector
Uri
method to add query string parameters renamed fromparams
toaddParams
. Same withmatrixParams
->addMatrixParams
PercentEncoderDefaults
object renamed toPercentEncoder
companion object.- Copy methods
user
/password
/port
/host
/scheme
now all prefixed withwith
, e.g.withHost
- New
UriConfig
case class used to specify encoders, decoders and charset to be used. See examples in Custom encoding, URL Percent Decoding and Character Sets
scala-uri
is open source software released under the Apache 2 License.