mattpap / iscala Goto Github PK

Is there any hope of an update to scala 2.11.1?
I played with updating the library dependencies, but after resolving everything, the compiler errors were too much for me to dive into with just a cursory glance.
I'm happy to help if you have pieces that I could work on.

best

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\Mike>cd "\Program Files (x86)\scala"

C:\Program Files (x86)\scala>java -jar lib/IScala.jar
connect ipython with --existing C:\Program Files (x86)\scala\profile-4104.json
Exception in thread "IScala" java.io.IOException: Access is denied
at java.io.WinNTFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(Unknown Source)
at scalax.io.support.FileUtils$$anonfun$scalax$io$support$FileUtils$$preOpen$1.apply(FileUti
ls.scala:66)
at scalax.io.support.FileUtils$$anonfun$scalax$io$support$FileUtils$$preOpen$1.apply(FileUti
ls.scala:58)
at scala.collection.immutable.List.foreach(List.scala:318)
at scalax.io.support.FileUtils$.scalax$io$support$FileUtils$$preOpen(FileUtils.scala:58)
at scalax.io.support.FileUtils$$anonfun$openOutputStream$1.apply(FileUtils.scala:19)
at scalax.io.support.FileUtils$$anonfun$openOutputStream$1.apply(FileUtils.scala:18)
at scalax.io.managed.OutputStreamResource.scalax$io$managed$OutputStreamResource$$nResource$
1(OutputStreamResource.scala:38)
at scalax.io.managed.OutputStreamResource$$anonfun$writableByteChannel$1.apply(OutputStreamR
esource.scala:40)
at scalax.io.managed.OutputStreamResource$$anonfun$writableByteChannel$1.apply(OutputStreamR
esource.scala:40)
at scalax.io.managed.WritableByteChannelResource.scalax$io$managed$WritableByteChannelResour
ce$$nResource$2(WritableByteChannelResource.scala:35)
at scalax.io.managed.WritableByteChannelResource$$anonfun$writer$1.apply(WritableByteChannel
Resource.scala:37)
at scalax.io.managed.WritableByteChannelResource$$anonfun$writer$1.apply(WritableByteChannel
Resource.scala:37)
at scalax.io.managed.WriterResource.open(WriterResource.scala:17)
at scalax.io.Resource$$anonfun$1.apply(Resource.scala:170)
at scalax.io.Resource$$anonfun$1.apply(Resource.scala:170)
at scala.util.control.Exception$Catch$$anonfun$either$1.apply(Exception.scala:124)
at scala.util.control.Exception$Catch$$anonfun$either$1.apply(Exception.scala:124)
at scala.util.control.Exception$Catch.apply(Exception.scala:102)
at scala.util.control.Exception$Catch.either(Exception.scala:124)
at scalax.io.Resource$class.acquireFor(Resource.scala:169)
at scalax.io.managed.WriterResource.acquireFor(WriterResource.scala:8)
at resource.ManagedResourceOperations$class.acquireAndGet(ManagedResourceOperations.scala:25
)
at scalax.io.managed.WriterResource.acquireAndGet(WriterResource.scala:8)
at resource.ManagedResourceOperations$class.foreach(ManagedResourceOperations.scala:45)
at scalax.io.managed.WriterResource.foreach(WriterResource.scala:8)
at scalax.io.WriteChars$class.write(WriteChars.scala:57)
at scalax.io.managed.WriterResource.write(WriterResource.scala:8)
at scalax.io.Output$class.write(Output.scala:98)
at scalax.file.FileOps.write(FileOps.scala:78)
at org.refptr.iscala.IScala.(IScala.scala:37)
at org.refptr.iscala.IScala$$anon$1.run(IScala.scala:19)

C:\Program Files (x86)\scala>

Please mention in the documentation whether graphics/charting is supported or not.

This is a very important aspect of ipython notebook and possibly the primary reason why one would be interested by something similar, but backed by Scala.

Perhaps you can collaborate with the Scala Notebook project also.

If i try and execute a tab complete, I can sometimes get the below exception. I'd be okay with tab complete failing, but would like to avoid kernel crash.

Exception in thread "RequestsEventLoop" java.util.NoSuchElementException
at scala.collection.LinearSeqOptimized$class.last(LinearSeqOptimized.scala:134)
at scala.collection.immutable.List.last(List.scala:83)
at org.refptr.iscala.IScalaCompletion$literals$.simpleParse(Completion.scala:199)
at org.refptr.iscala.IScalaCompletion$literals$.follow(Completion.scala:202)
at org.refptr.iscala.CompletionAware$class.completionsFor(Completion.scala:29)
at org.refptr.iscala.IScalaCompletion$literals$.completionsFor(Completion.scala:198)
at org.refptr.iscala.IScalaCompletion$$anonfun$topLevelFor$1.apply(Completion.scala:229)
at org.refptr.iscala.IScalaCompletion$$anonfun$topLevelFor$1.apply(Completion.scala:229)
at scala.collection.immutable.List.flatMap(List.scala:327)
at org.refptr.iscala.IScalaCompletion.topLevelFor(Completion.scala:229)
at org.refptr.iscala.IScalaCompletion.completions(Completion.scala:232)
at org.refptr.iscala.CompleteHandler.apply(Handlers.scala:121)
at org.refptr.iscala.IScala$EventLoop.dispatch(IScala.scala:127)
at org.refptr.iscala.IScala$EventLoop$$anonfun$run$2.apply(IScala.scala:139)
at org.refptr.iscala.IScala$EventLoop$$anonfun$run$2.apply(IScala.scala:139)
at scala.Option.foreach(Option.scala:245)
at org.refptr.iscala.IScala$EventLoop.run(IScala.scala:139)

Thanks for this great tool!

I was wondering if there is a way to call commands like :cp, :load etc into the notebook.
currently I for :cp test.jar, I get:

error: illegal start of definition

Env:
OSX yosemite,
python: 2.7.6
Ipython: 3.1.0
java: 1.8.0_40-b27
scala: 2.11.6

Follow instructions here: https://github.com/mattpap/IScala to set it up
change --connection-file to --profile to make it work for ipython

fire up a notebook, try a simple fib function definition:

def fib(n:Int):Int = if (n <= 2) 1 else fib(n-1) + fib(n-2)

try to run it, got following error:

JSON deserialization error: JsResultException(errors:List((/user_variables,List(ValidationError(validate.error.missing-path,WrappedArray())))))

Any help will be appreciated.

Thanks

Please provide (and/or document) multi-user support.

This is a prerequisite for making a Scala Jupyter kernel available on SageMathCloud:
sagemathinc/cocalc#395 (comment)

Hi @mattpap,

I'm interesting in cleaning up the code a bit, by dropping many lines of code for PlayJson, Scala-macro and Formats and replace them with the much cleaner api of Json4s. The paradise plugin constantly mess with my IDE & there seems to be some hacking for compiler's caveat. Do you think this is possible or worth it?

Yours @tribbloid

Currently IScala notebook highlights python code.
It would be nice to have scala syntax highlighting instead :)

The first problem I ran into is that the procedure for installing alternative ipython kernels has changed since the README was written. I found the following procedure to work:

Create ~/.ipython/kernels/iscala/kernel.json and put it in:

{
  "argv": ["java", "-jar", "<whatever path>/lib/Iscala.jar", "--profile", "{connection_file}"],
  "display_name": "IScala",
  "language": "scala"
}

Run ipython with ipython console --kernel iscala or ipython notebook and then select IScala from the "New" dropdown box to create a new Scala Ipython notebook.

These two steps allow Ipython to connect to the IScala kernel, but there's some issue with the JSON communications between the IScala kernel and IPython that occurs in both the console and notebook interfaces. Here's the error for an attempt to run the console

ipython console --kernel iscala                                                                                                                 [ruby-2.0.0p481]
Welcome to Scala 2.10.2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_45)
IPython Console 3.1.0

In [1]: val r  = 0
JSON deserialization error: JsResultException(errors:List((/user_variables,List(ValidationError(validate.error.missing-path,WrappedArray())))))

Unfortunately I want to use IScala precisely to learn Scala, so I've no idea how to go about tracking down the problem, much less fixing it. Any help would be greatly appreciated. I've tried several other REPL+ interfaces to Scala, and this is the most natural so far, even if it seems quasi-dead.

BTW: if it turns out as I fear that this project is no longer being supported, scala-notebook worked out of the box for me, albeit has a clunky interface, and there's the Apache incubator Zeppelin project which looks very sweet (has visualization, built in spark support,...) but I have not tried. So there are some other options while we wait to see if IScala's still viable.

Currently, hitting gives an invalid list.

Hi all, I'm following the instructions in the README.md, but apparently something is not working properly:

[h4ml3t@redhatnode:~/IScala-0.1]$ ./bin/console 
[IPKernelApp] To connect another client to this kernel, use:
[IPKernelApp] --existing kernel-11557.json --profile scala
In [1]: val a = 0
  File "<ipython-input-1-e58f81840f45>", line 1
    val a = 0
        ^
SyntaxError: invalid syntax

I don't have any clue about why it is still an ipython console. What am I overlooking? If the problem is trivial I apologize for the disturb.
Cheers,
Luca

Your README states

To start IPython with IScala backend manually, issue:
ipython console --KernelManager.kernel_cmd='["java", "-jar", "lib/IScala.jar", "--connection-file", " {connection_file}", "--parent"]'

However:

java -jar ./lib/IScala.jar --connection-file

yields:

optsimple.UnrecognizedOptionException: 'connection-file' is not a recognized option

while this will work:

java -jar ./lib/IScala.jar --profile

Originally submitted in #7 by @lev112:

What is the right way to compile with scala 2.10?

I've tried to set scalaVersion := "2.10.4" in Build.scala, and it compiles,
but I see some strange behavior...

I try to import spark package, and in the console it works fine,
but in the notebook I get an error:
error: object spark is not a member of package org.apache

Is it a bug or did I do something wrong?
(in IScala 0.1 the same code works fine)

In [1]: import java.util.HashMap
Exception in thread "RequestsEventLoop" java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (table "history" has more than one primary key)
at org.sqlite.DB.newSQLException(DB.java:383)
at org.sqlite.DB.newSQLException(DB.java:387)
at org.sqlite.DB.throwex(DB.java:374)
at org.sqlite.NativeDB.prepare(Native Method)
at org.sqlite.DB.prepare(DB.java:123)
at org.sqlite.PrepStmt.(PrepStmt.java:42)
at org.sqlite.Conn.prepareStatement(Conn.java:404)
at org.sqlite.Conn.prepareStatement(Conn.java:399)
at scala.slick.jdbc.JdbcBackend$SessionDef$class.prepareStatement(JdbcBackend.scala:152)
at scala.slick.jdbc.JdbcBackend$BaseSession.prepareStatement(JdbcBackend.scala:389)
at scala.slick.jdbc.JdbcBackend$SessionDef$class.withPreparedStatement(JdbcBackend.scala:190)
at scala.slick.jdbc.JdbcBackend$BaseSession.withPreparedStatement(JdbcBackend.scala:389)
at scala.slick.driver.JdbcInvokerComponent$DDLInvoker$$anonfun$create$1$$anonfun$apply$mcV$sp$1.apply(JdbcInvokerComponent.scala:50)
at scala.slick.driver.JdbcInvokerComponent$DDLInvoker$$anonfun$create$1$$anonfun$apply$mcV$sp$1.apply(JdbcInvokerComponent.scala:49)
at scala.collection.Iterator$class.foreach(Iterator.scala:743)
at scala.collection.Iterator$JoinIterator.foreach(Iterator.scala:191)
at scala.slick.driver.JdbcInvokerComponent$DDLInvoker$$anonfun$create$1.apply$mcV$sp(JdbcInvokerComponent.scala:49)
at scala.slick.driver.JdbcInvokerComponent$DDLInvoker$$anonfun$create$1.apply(JdbcInvokerComponent.scala:49)
at scala.slick.driver.JdbcInvokerComponent$DDLInvoker$$anonfun$create$1.apply(JdbcInvokerComponent.scala:49)
at scala.slick.jdbc.JdbcBackend$BaseSession.withTransaction(JdbcBackend.scala:426)
at scala.slick.driver.JdbcInvokerComponent$DDLInvoker.create(JdbcInvokerComponent.scala:48)
at org.refptr.iscala.db.DB$$anonfun$db$1$$anonfun$apply$mcV$sp$1.apply(DB.scala:55)
at org.refptr.iscala.db.DB$$anonfun$db$1$$anonfun$apply$mcV$sp$1.apply(DB.scala:53)
at scala.collection.immutable.List.foreach(List.scala:381)
at org.refptr.iscala.db.DB$$anonfun$db$1.apply$mcV$sp(DB.scala:53)
at org.refptr.iscala.db.DB$$anonfun$db$1.apply(DB.scala:53)
at org.refptr.iscala.db.DB$$anonfun$db$1.apply(DB.scala:53)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)

Hi,

Is there a way to load up some code before launching the kernel? I want to define a DSL, so I need IScala to load some objects, functions, operators etc.. Using normal IPython, there is a startup folder where I can do this. Is this also possible in IScala?

Thanks for the IScala project!

to start the Scala kernel:
$java -jar IScala.jar
[info] connect ipython with --existing /home/user/IScala/kernel-27469.json
Welcome to Scala 2.11.2 (OpenJDK 64-Bit Server VM, Java 1.6.0_35)

to start Ipython with above kernel
$ipython --existing /home/user/IScala/kernel-27469.json console
It did not recognize scala syntax
then I tried
$ipython console --KernelManager.kernel_cmd='["java", "-jar", "/home/user/IScala/lib/IScala.jar", "--connection-file", "{/home/user/IScala/kernel-27469.json}", "--parent"]'
1/0
ZeroDivisionError: integer division or modulo by zero

IScala is already a very interesting tool! Thanks!

Is it possible to add a classpath to include any jars or class files? Perhaps this is possible via sbt but I do not know sbt very well. The idea is to be able to use the notebook for Scala classes which are under development.

I found your repository by clicking a link in the Jupyter documentation. Afaik IPython Notebook is obsolete and work is done on Jupyter now. Could you add clear working instructions for how to use your kernel in Jupyter?

What I mean is that the workflow to get it running on IPython doesn't work for Jupyter. For example there is no

jupyter profile create iscala

because jupyter doesn't know the profile command. So I tried using the IPython command and then making jupyter use the configuration that command creates using:

jupyter notebook --profile iscala

However, that command doesn't exist either. So I looked up the available commands and it turns out there is

jupyter --config="..."

This is what I used then. I specified the path to your IScala.jar in that config file created by IPython. However, when I want to create a new notebook it only shows the python3 kernel. I thought "Maybe it simply doesn't know that it isn't python, but Scala instead." and proceeded. Then it connects to the kernel when the notebook is shown. When it connects your kernel dies.

TLDR: I couldn't get your kernel to run with Jupyter. Can you please provide detailed instructions on how to achieve that?

Otherwise you will get ERROR: Kernel did not respond. See bin/2.10/console -m org.apache.spark::spark-core:0.9.1 on clean ~/.ivy2 as an example (may be down link dependent).

Setting up and running spark commands is working until I try to train a model with MLLIB naive Bayes. The training data consists of a small training data set of less than 800 instances of short text snippets and the code runs fine when running via spark-submit.

The spark context is set up this way:
val conf = new SparkConf()
.setMaster("local[6]")
.setAppName("CompanyChecker")
.set("spark.executor.memory", "5g")
.set("spark.driver.memory", "5g")
val sc = new SparkContext(conf)

and the command everything comes to an hold is
val modelNB = NaiveBayes.train(training, lambda = 1.0)

Exception in thread "Driver Heartbeater"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Driver Heartbeater"

Exception in thread "Executor task launch worker-0" Exception in thread "Executor task launch worker-3" Exception in thread "Executor task launch worker-2" Exception in thread "Executor task launch worker-1" java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space

The process is till running based on the CPU load being at about 60%, but I can't access localhost:4040 and I eventually have to kill the process.

any help would be highly appreciated.

Thanks,
Frank

I did a git clone to get the latest code and created IScala.jar using ./sbt, followed by 'compile' and 'assembly'. When I use my locally built IScala.jar with IPython, I get the following error even for simple scala commands. My environment is Mac OSX (one change I had to make was in your 'sbt' script where you had 'openssl md5 -r ...', but in Mac OSX, i had to change it to 'openssl md5 -hex ...').

If I instead use your latest 0.1 release IScala.jar (or even build locally using the 0.1 source.zip that you have provided at the release location), it works perfectly fine. I tried debugging by putting breakpoints etc in the Interpreter.scala, but I couldn't quite follow what was going on. Am I doing something wrong?

In [2]: val x = 1
scala.ScalaReflectionException: object $line10.$display.$iw not found.
scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:162)
scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:22)
org.refptr.iscala.Interpreter.runCode(Interpreter.scala:118)
org.refptr.iscala.Interpreter.display(Interpreter.scala:162)
org.refptr.iscala.Interpreter$$anonfun$10.apply(Interpreter.scala:206)
org.refptr.iscala.Interpreter$$anonfun$10.apply(Interpreter.scala:206)
org.refptr.iscala.Interpreter.withException(Interpreter.scala:97)
org.refptr.iscala.Interpreter.loadAndRunReq(Interpreter.scala:206)
org.refptr.iscala.Interpreter$$anonfun$interpret$1.apply(Interpreter.scala:236)
org.refptr.iscala.Interpreter$$anonfun$interpret$1.apply(Interpreter.scala:236)
org.refptr.iscala.Runner$Execution$$anonfun$1.apply$mcV$sp(Runner.scala:28)
org.refptr.iscala.IOUtil$$anon$2.run(Util.scala:21)
java.lang.Thread.run(Thread.java:744)

Thank you for starting this exciting project.

I was wondering if iScala can be used to produce output that is not restricted to raw text?

The iPython framework supports many various formats.

At the very least, can iScala we used to produce Markdown, Latex, and/or HTML text?

I want to use your awesome kernel with ipython 3.0.0-dev (current master).

I've created kernel-file iscala/kernel.json:

{
 "argv": ["java", "-jar", "/Users/romovpa/workspace/iscala/IScala-0.1/lib/IScala.jar", "--profile", "{connection_file}", "--parent"],
 "display_name": "IScala",
 "language": "scala"
}

And, woohoo, it works in console mode:

romovpa@95:~$ ipython console --kernel iscala
IPython Console 3.0.0-dev
Welcome to Scala 2.10.2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_05)

In [1]: (1 to 5).foreach { i => println(i); Thread.sleep(1000) }
1
2
3
4
5

But when I try to start notebook with IScala kernel, it fails with an exception:

romovpa@95:~$ ipython notebook
[I 18:01:21.367 NotebookApp] Using existing profile dir: u'/Users/romovpa/.ipython/profile_default'
...
[I 18:01:29.894 NotebookApp] Kernel started: ef9d90e1-b356-4495-b319-66fece8006f6
Exception in thread "main" joptsimple.UnrecognizedOptionException: 'profile-dir' is not a recognized option
    at joptsimple.OptionException.unrecognizedOption(OptionException.java:89)
    at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:493)
    at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
    at joptsimple.OptionParser.parse(OptionParser.java:433)
    at org.refptr.iscala.Options.<init>(Options.scala:12)
    at org.refptr.iscala.IScala$delayedInit$body.apply(IScala.scala:15)
    at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App$$anonfun$main$1.apply(App.scala:71)
    at scala.App$$anonfun$main$1.apply(App.scala:71)
    at scala.collection.immutable.List.foreach(List.scala:318)
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
    at scala.App$class.main(App.scala:71)
    at org.refptr.iscala.IScala$.main(IScala.scala:14)
    at org.refptr.iscala.IScala.main(IScala.scala)

Is it a big problem? What do we need to fix notebook mode?

Currently messages are sent to frontends when Results.Error occurs. Test case: @deprecated("msg", "since") def f () = 1; f().

Is the desired behavior of IScala to call a function immediately after it's defined?

In [1]: def f = 42
Out[1]: 42

compared with the scala REPL:

scala> def f = 42
f: Int

scala> f 
res2: Int = 42

If i leave a large seq as the last variable created in a cell, it can bring down my browser while trying to print.
Would you be able to put a cap on the size of the text blob sent to the browser for display?

thanks! I'm finding this really really useful

After downloading it is not able to find jarfile. I am running it in the right folder, right?

code at codes-MacBook-Pro.local  ~
$ cd Downloads/IScala-0.1
code at codes-MacBook-Pro.local  ~/Downloads/IScala-0.1
$ ls
LICENSE    README.md  bin/       lib/
code at codes-MacBook-Pro.local  ~/Downloads/IScala-0.1
$ bin/console
readlink: illegal option -- f
usage: readlink [-n] [file ...]
usage: dirname path
usage: dirname path
Error: Unable to access jarfile /lib/IScala.jar