chitralverma / scala-polars

Polars for Scala & Java projects!

Hi there 👋

scala-polars's Issues

top_k descending flag reversed

//> using dep "org.polars::scala-polars:0.1.0-SNAPSHOT"

import org.polars.scala.polars.api.{DataFrame, Series}

val sdf = DataFrame.fromSeries(Series.ofInt("i32_col", Array[Int](1, 2, 3, 4, 5, 6, 7, 8, 9, 10)))

val tk = sdf.top_k(4, "i32_col", descending = true, nullLast = true, maintainOrder = false)
tk.show()

val sk = sdf.sort("i32_col", descending = true, nullLast = true, maintainOrder = false).head(4)
sk.show()

$ scala-cli topkbug.sc
Compiling project (Scala 3.4.1, JVM (21))
Compiled project (Scala 3.4.1, JVM (21))
shape: (4, 1)
┌─────────┐
│ i32_col │
│ ---     │
│ i32     │
╞═════════╡
│ 1       │
│ 2       │
│ 3       │
│ 4       │
└─────────┘
shape: (4, 1)
┌─────────┐
│ i32_col │
│ ---     │
│ i32     │
╞═════════╡
│ 10      │
│ 9       │
│ 8       │
│ 7       │
└─────────┘

I've been digging through the code and I don't see the problem offhand. Will keep looking.

Maven artifacts missing

The maven dependency coordinates given in https://github.com/chitralverma/scala-polars/packages/1795604:

<dependency>
  <groupId>org.polars</groupId>
  <artifactId>scala-polars_2.12</artifactId>
  <version>0.1.0-SNAPSHOT</version>
</dependency>

don't seem to be available. Searching https://mvnrepository.com/search?q=polars doesn't yield anything either.

Reading parquet from s3

Hi, Thank you on this awesome work @chitralverma !
I am trying to read from s3 in scala . I can see that writing is pretty simple as you added utility function to pass options in write_utils.rs

write.options(
        Map(
          "aws_default_region" -> "us‑east‑2",
          "aws_access_key_id" -> "ABC",
          "aws_secret_access_key" -> "XYZ"
        )

I am trying to do the same reading into a df, passing a s3 path to Polars.scan obviously throws

val df = Polars.parquet.scan("s3://test1/myfile.zstd.parquet")

Configuration 'aws' must be provided in order to use s3 cloud urls."))

Is there something I need tweak in the config to be able to read directly from s3 ? Maybe we need a read_utils.rs . I would be able to contribute, if you have some guidance :)

Add .rows function to DataFrame

Wondering if there is any consideration for adding a .rows function to the org.polars.scala.polars.api.DataFrame implementation.

https://pola-rs.github.io/polars/py-polars/html/reference/dataframe/api/polars.DataFrame.rows.html