comcast / ip4s

Defines immutable, safe data structures for describing IP addresses, multicast joins, socket addresses and similar IP & network related data types

ip4s: IP Addresses for Scala, Scala.js & Scala Native

This project defines immutable, safe data structures for describing IP addresses, multicast joins, socket addresses and similar IP & network related data types.

There are two defining characteristics of this project that make it different from other similar projects:

• all data types are immutable and every function/method is referentially transparent (e.g., no accidental DNS lookups by calling InetAddress.getByName(...))
• published for Scala, Scala.js and Scala Native

See the guide and ScalaDoc for more details.

Getting Binaries

This library is published on Maven Central under group id com.comcast and artifact id ip4s-core_${scalaBinaryVersion}. Add the following to your SBT build:

libraryDependencies += "com.comcast" %% "ip4s-core" % "version"

Interop

As of 1.4, ip4s depends on cats and provides type class instances directly in data type companion objects. For Scalaz support, we recommend shims.

Copyright and License

This project is made available under the Apache License, Version 2.0. Copyright information can be found in NOTICE.

Code of Conduct

See the Code of Conduct.

ip4s's Issues

Suggestion: CIDR could have a method that returns the number of IPs

For example:

val cidr = ip"192.168.1.0" / 24
cidr.totalIps // returns 256

I didn't fully confirm if this is the correct implementation:

def totalIps: BigInt = {
  import scala.math._
  val maxPrefixBits: Double = address.fold(v4 => 32.0, v6 => 128.0)
  val ips = pow(2d, maxPrefixBits - prefixBits)
  BigDecimal(ips).setScale(0, BigDecimal.RoundingMode.HALF_UP).toBigInt
}

I can make a PR if it helps.

Enum for version of IP

Suggestion - implement something like:

enum IpVersion {
  case V4, V6
}

class IpAddress {
  def version: IpVersion
}

class Ipv4Address {
  override def version: IpVersion = IpVersion.V4
}

class Ipv6Address {
  override def version: IpVersion = IpVersion.V6
}

Use cases:

1. Allows having configurations/settings per IP version:

val configs: Map[IpVersion, Long] = ???
val subnetId = configs.get(IpVersion.V4)

2. On a list of IpAddresses/Cidrs easily group them by version:

val cidrs: List[Cidr[IpAddress]] = ???
val perVersion: Map[IpVersion, List[Cidr[IpAddress]]] = cidrs.groupBy(_.address.version)

Cidr.fromString allows invalid CIDR ranges

invalid masks are accepted in Cidr.fromString:

scala> Cidr.fromString("0.0.0.0/99")
val res0: Option[com.comcast.ip4s.Cidr[com.comcast.ip4s.IpAddress]] = Some(0.0.0.0/32)

Receipts: https://datatracker.ietf.org/doc/html/rfc4632#section-3.1

In CIDR notation, a prefix is shown as a 4-octet
quantity, just like a traditional IPv4 address or network number,
followed by the "/" (slash) character, followed by a decimal value
between 0 and 32 that describes the number of significant bits.

tangentially also #137 which I do believe is really a bug (but breaking changes are bad).

IpAddress should implement scala.Ordered

In the JVM ecosystem, the interface java.lang.Comparable is implemented by types that have a canonical ordering; whereas java.util.Comparator is used for those that could have many sortings.

In the scala ecosystem scala.Ordered and scala.Ordering implement, and take the place of these interfaces.

IpAddresses have a canonical ordering and thus they should implement scala.Ordered, which will also allow them to implement java.lang.Comparable, so they can inter-op with Java and Scala code based on these interfaces.

Support for IPv4-mapped IPv6 addresses

Using IPv4-mapped IPv6 addresses causes runtime exceptions

Actual Behavior

ip"::ffff:f:f" returns an IPv6 address and throws an exception when invoking toInetAddress

ip"::ffff:f:f".toInetAddress // Exception in thread "main" java.lang.ClassCastException

Expected Behavior

I expect ip"::ffff:f:f" to return an IPv4 address like InetAddress.getByName does

ip"::ffff:f:f".toInetAddress // /0.15.0.15

Add toSsm/toAsm on MulticastJoin

I propose to add the following to MulticastJoin

def toAsm: Option[AnySourceMulticastJoin[A]]
def toSsm: Option[SourceSpecificMulticastJoin[A]]

to make it easier working with these types. I am constantly narrowing from MJ to SSM in my code and while fold makes it easy, it's very verbose.

IDNTest flake

Jotting this down. Politely ping @isomarcte, in case it's obvious 😅

==> X com.comcast.ip4s.IDNTest.support any hostname 0.66s munit.FailException: Failing seed: HEdDQ_KfMvIxSQ7qNNGUDD_mi2f8zUiF6S-3_PUZjcF=
You can reproduce this failure by adding the following override to your suite:

  override def scalaCheckInitialSeed = "HEdDQ_KfMvIxSQ7qNNGUDD_mi2f8zUiF6S-3_PUZjcF="

Exception raised on property evaluation.
> ARG_0: 1m.3--N---D0-l9-1----oU-3--q-p--7Q--lm-P-d--FS--Wn--hg-TL-yRS----m.xn--n3-gIg4eid-5-------U.xP6N6P-r--C-Y3Or-vw---W-q5N-----I.lW--P----U------7-qSPC-C-F1As--xgdAr-B-N01n----J---c--w-xo-WJ8x
> Exception: java.lang.IllegalArgumentException: Reached end of input in incomplete decoding state. This is not a validly encoded Bootstring string.

Hostname type

As discussed on Reddit.

A Hostname type that encapsulates a Hostname, including validation (as you suggested, RFC1123/RFC952 could be neat) and possibly some utilities like a localhost object.

As you said, including a DNS resolution mechanism exclusively for the JVM would be neat, but the requirement for a dependency on an Effect type would muddy things somewhat.

I'm thinking adding a simple shim typeclass that can be used would largely resolve this issue. Another way to handle it would be to provide the DNS resolution entirely in a separate sub-project, where the Effect type will already be fixed (e.g. ip4s-cats or ip4s-scalaz, etc.)

Ultimately, DNS resolution might be outside the scope of this project; it's network activity, so where do you stop? Why not add an open method and a Socket type etc? Before you know it, this would start mutating in to a full-blown network library, which I imagine is something you'd prefer to avoid.

ip4s -> ip4s-core in docs

Looking at https://repo1.maven.org/maven2/com/comcast/, ip4s was renamed to ip4s-core as of version 1.1.0, so install would look more like

"com.comcast"               %% "ip4s-core"                      % version,

Scala 3.0 should be used to publish Scala 3 artifacts

Scala 3.1.0 is not forward binary compatible with Scala 3.0. According to this recommendation https://scala-lang.org/blog/2021/10/21/scala-3.1.0-released.html#compatibility-notice libraries should be published with latest Scala 3.0 version (at least if any Scala 3.1 features aren't used).

Support unix socket addresses, and possibly arbitrary other address types (?)

This is almost definitely a breaking change, so just something to keep in mind for the future.

Seems like there are a plethora of socket types, which have different kinds of addressing. e.g. AF_UNIX, AF_INET, AF_INET6. Don't know if any of the others are important.

https://man7.org/linux/man-pages/man2/socket.2.html

Currently in FS2 we have an awkward SocketGroup and UnixSockets, when (in theory) they can actually be sharing the same machinery. Also, it's possible to use UDP with unix sockets, which the current FS2 encoding also does not support.

java.net.InetAddress -> com.comcast.ip4s.IpAddress

Hi -

Thanks for this library. Does such a function exist, java.net.InetAddress -> com.comcast.ip4s.IpAddress?

I did not see one:

$ggrep -r InetAddress . | ggrep -v test | ggrep -v docs | ggrep -v README
./cats/jvm/src/main/scala/com/comcast/ip4s/HostnameResolver.scala:import java.net.{InetAddress, UnknownHostException}
./cats/jvm/src/main/scala/com/comcast/ip4s/HostnameResolver.scala:        val addr = InetAddress.getByName(hostname.toString)
./cats/jvm/src/main/scala/com/comcast/ip4s/HostnameResolver.scala:        val addrs = InetAddress.getAllByName(hostname.toString)
./jvm/src/main/tut/guide.md:When compiling for the JVM, the various IP address classes have a `toInetAddress` method which returns a `java.net.InetAddress`, allowing easy integration with libraries that use `InetAddress`.
./jvm/src/main/tut/guide.md:val homeIA = ip"127.0.0.1".toInetAddress
./jvm/src/main/tut/guide.md:val home4IA = ipv4"127.0.0.1".toInetAddress
./jvm/src/main/tut/guide.md:val home6IA = ipv6"::1".toInetAddress
./jvm/src/main/scala/com/comcast/ip4s/IpAddressPlatform.scala:import java.net.{InetAddress, Inet4Address, Inet6Address}
./jvm/src/main/scala/com/comcast/ip4s/IpAddressPlatform.scala:  /** Converts this address to a `java.net.InetAddress`. Note this method only exists on the JVM. */
./jvm/src/main/scala/com/comcast/ip4s/IpAddressPlatform.scala:  def toInetAddress: InetAddress
./jvm/src/main/scala/com/comcast/ip4s/IpAddressPlatform.scala:  override def toInetAddress: Inet4Address =
./jvm/src/main/scala/com/comcast/ip4s/IpAddressPlatform.scala:    InetAddress.getByAddress(bytes).asInstanceOf[Inet4Address]
./jvm/src/main/scala/com/comcast/ip4s/IpAddressPlatform.scala:  override def toInetAddress: Inet6Address =
./jvm/src/main/scala/com/comcast/ip4s/IpAddressPlatform.scala:    InetAddress.getByAddress(bytes).asInstanceOf[Inet6Address]
./jvm/src/main/scala/com/comcast/ip4s/SocketAddressPlatform.scala:  def toInetSocketAddress: InetSocketAddress = new InetSocketAddress(ip.toInetAddress, port.value)
./shared/src/main/scala/com/comcast/ip4s/IpAddress.scala:  * If using `IpAddress` on the JVM, you can call `toInetAddress` to convert the address to a `java.net.InetAddress`, for use

Thanks!

mdoc dependency

mdoc is being included as a dependency in the final pom. Not sure if it is intentional, but its inclusion makes a fat jar almost 3 times bigger.

Pure Scala IDNA implementation

The forthcoming Scala Native cross-build is currently using icu4c for IDNA. Although this works, it introduces various build complexities (see scala-native/scala-native#2778) and requires downstreams to have icu4c installed if they (indirectly) call IDNA related methods.

This motivates a pure Scala IDNA implementation.

1. Does it make sense to do it in ip4s, or an external dependency (hosted where)?
   There is plenty of prior art to mimick. They all seem to use some form of source generator.

   • https://github.com/servo/rust-url/tree/master/idna
   • https://github.com/golang/net/tree/master/idna
   • https://github.com/kjd/idna
   • https://github.com/hexonet/idna-uts46

2. Should we swap out the JVM and JS implementations as well?

   • Critically, the current JS implementation is broken/wrong, because it uses punycode.js without nameprep.

     The RFC specifically says:

     Implementations of IDNA MUST fully implement Nameprep and Punycode; neither Nameprep nor Punycode
     are optional.

     https://www.ietf.org/rfc/rfc3490.txt

     See also mathiasbynens/punycode.js#40.

     In theory we could shell out to an npm package but dealing with npm from Scala.js is terrible.
     https://www.npmjs.com/package/idna-uts46

   • Meanwhile, the JVM implementation is using JDK APIs that are based on IDNA 2003 which IIUC is deprecated in favor of IDNA 2008. We could add a dependency to icu4j.

Thoughts? Since nobody is complaining about the existing broken/deprecated implementations, do we care enough to do this right?

apply returning Option?

One thing I find a bit surprising in the API is apply(String) on the companion objects returning Option. I usually call that parse or fromString and save apply for things that return the companion class.

Breaking changes are bad, so it may not be worth adjusting. Just some feedback as major releases are considered.

Enjoying it very much overall!

Cidr range are a bit messy

damning snippet:

val thirtyinvalid = ipv4"127.168.0.1" / 30
// val thirtyinvalid: com.comcast.ip4s.Cidr[com.comcast.ip4s.Ipv4Address] = 127.168.0.1/30
val thirtyvalid = ipv4"127.168.0.0" / 30
// val thirtyvalid: com.comcast.ip4s.Cidr[com.comcast.ip4s.Ipv4Address] = 127.168.0.0/30
val validIps = (thirtyvalid.prefix.toLong to thirtyvalid.last.toLong).map(Ipv4Address.fromLong)
// val validIps: IndexedSeq[com.comcast.ip4s.Ipv4Address] = Vector(127.168.0.0, 127.168.0.1, 127.168.0.2, 127.168.0.3)
validIps.forall(thirtyinvalid.contains)
// val res1: Boolean = true
validIps.forall(thirtyvalid.contains)
// val res2: Boolean = true
thirtyvalid == thirtyinvalid
// val res3: Boolean = false

This is messy: I would expect two CIDR ranges to be equal iff they represent the same addresses, but that doesn't hold: equality is defined on the base address, but when the base address has bits that are masked, these ranges are different even though they represent the same addresses.

Also note that (ipv4"127.168.0.1" / 30).toString == "127.168.0.1/30" which is not a valid CIDR range.

How to fix it, avoiding breaking compatibility much as possible?

I suggest making sure address == prefix when constructing a CIDR range. This may break the not entirely unreasonable, if buggy, expectation of address == Cidr(address, 16).address to hold for any address, or Cidr(address1, 16) == Cidr(address2, 16) to hold if and only if address1 == address2. This isn't good behaviour, but it's possible people rely on this in the wild.

Another option is to keep taking different addresses, but redefine equality to use prefix rather than address. A big problem of that is that Cidr(address1, 16) == Cidr(address2, 16) would now hold if they represent the same addresses, but now Cidr(address1, 16).address != Cidr(address2, 16).address and maybe even worse, Cidr(address1, 16).productElements != Cidr(address2, 16).productElements

Some class/method are not available with Scala.js

This is a bug report .

• ip4s 1.4.0
• Scala 2.13.3
• Scala.js 1.1.1
• OpenJDK Runtime Environment (build 11.0.7+10-post-Ubuntu-3ubuntu1)
• Ubuntu 20.04 LTS

When Cidr.fromString(str) is used, Scala.js failed to fastOptJS/fullOptJS due to:

[error] Referring to non-existent class com.comcast.ip4s.Cidr$
...
[error] Referring to non-existent method com.comcast.ip4s.Cidr$.fromString(java.lang.String)scala.Option
...

Same error happens when the apply method Cidr(ip"10.123.45.67", 8) is used.

Note

I cloned this repository and run testKitJS/test, which includes test for Cidr.fromString.
The tests finished successfully on my end.
So I wonder this might be publishing issue ... ?

Feature Request: IsPrivate

Hi ip4s team,

I'm casting around for a Scala/Java library that can parse ipv4 and 6 addresses and tell me whether they're 'internal' i.e. come from a private subnet. I don't think this is possible with ip4s at the moment but it would be a great feature.

Use case: Some endpoints in my application I want to be restricted to internal services only, which can be using a mix of ipv4 and 6. Or an external caller on ipv6 can try and access the endpoint and needs to be rejected.

Ciris Codecs

It'd be great if there were ciris codecs available as part of this project (unless I'm missing something that makes ciris work automatically with these classes).

I'd consider contributing a ip4s-ciris module - would there be interest in adopting it to this project?

Need more lenient SourceSpecificMulticastJoin

SourceSpecificMulticastJoin correctly enforces that group IP is in the range 232/8 as defined by RFC 4607 ( https://datatracker.ietf.org/doc/html/rfc4607). However, in practice, values outside this range are often used. MulticastJoin is a sealed class so it cannot be extended outside of this library.

I propose we add a UnvalidatedSourceSpecificMulticastJoin or provide some way to bypass the range validation SourceSpecificMulticastJoin.

Improve CIDR validation for a given mask

For a given mask, if the IP address has bits set to the right, it should be invalid.

For example:
2.2.2.0/22 should be invalid because there are bits set to the right of the mask.

ip address bit pattern (2.2.2.0):
00000010 00000010 00000010 00000000

mask bit pattern (22):
11111111 11111111 11111100 00000000

Host.fromString parses a hostname with four or fewer characters into a Ipv6Address

I'm using Host.fromString in conjunction with ciris to validate configuration values that are passed to an application.
I then use the Host value to create a string that represents a JDBC URL for a database library. The JDBC URL is malformed when the user passes a hostname with four or fewer characters because the toString method of Ipv6Address adds two colons at the end of the hostname.

Example:

Host.fromString("db").toString  // -> "Some(db::)"

I'm definitely not an expert in all involved RFCs but this seems like a bug to me. According to RFC4291, Section 2.2 the string in the example should not be considered an Ipv6Address because it is missing the :: that indicates compressed leading or trailing zeros. From skimming the code it seems to me that the underlying issue is probably rooted in Ipv6Address.fromString.

scala: v2.13.6
ip4s: v3.0.3

Convert build to GitHub actions

See scodec-bits for an example.

Alternative version of Host.fromString that returns error message?

I recently had a problem where Host.fromString failed due to an underscore character (which seems reasonable). Could you provide an alternate version, eg. Host.fromStringEither which explains why host parsing failed?