Comments (5)
Not sure what the issue is, since it still works (when they're separate) I'm just going to close this out.
from spark-the-definitive-guide.
I appreciate it is not an issue for your book.
However, isn't it a little worrying that the combined case fails? I don't think it should. But I'm happy to be corrected.
And, of course, I'm happy to report it elsewhere if that is more appropriate (assuming it is a bug and not my lack of understanding).
Kind regards, Lawrence
from spark-the-definitive-guide.
It's probably something to do with the implicits and the REPL compiler, could you try running the first line alone (import ...) and then running the rest as one block?
from spark-the-definitive-guide.
Putting the import first doesn't solve the problem; however, it leads to a useful error message, namely:
<console>:18: error: Unable to find encoder for type stored in a Dataset. Primitive types (Int, ....
In this post the case class is moved out of the current scope and the problem is solved, so it seems that it is an implicit resolution issue.
I have tried this with the Ch3 example and it works.
So, in summary, declare the case class as a first snippet and then declare the import, the two vals and the filter/map/take as a second snippet and it is fine.
from spark-the-definitive-guide.
There we are. Glad we figured it out!
from spark-the-definitive-guide.
Related Issues (20)
- Spark SQL Help HOT 4
- not a member of org.apache.spark.sql.SparkSession HOT 7
- Chapter 3: java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext. HOT 1
- Error while connecting to Databases in DataBrick Community Edition on Cloud . HOT 4
- Code issue chapter2
- Code in Chapter 22 - Windows on Event Time
- Chapter 9: java.lang.ClassNotFoundException: org.sqllite.JDBC HOT 2
- Translation of the book,"Spark-The-Definitive-Guide", from English into Chinese
- Error while connecting to Databases in DataBrick Community Edition on Cloud HOT 4
- staticSchema error in chapter 3 in s HOT 1
- Chapter 6 - Working with JSON
- Chapter 7 - A small correction to sql code in sumDistinct
- Chapter 7 - A small correction in sql code in group query
- Not applicable - opened by mistake
- Ch9 - Writing to SQL database - SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: flight_info) HOT 1
- ch10 - sparksql- Inserting into tables query not working Cannot safely cast 'count': string to bigint
- Chapter 9: DataSources - Partitioning based on a sliding window section - python
- Multi-line chain of Dataframe transformations needs to be enclosed in parenthesis HOT 2
- ImportError: cannot import name '_centered' from 'scipy.signal.signaltools'
- code works under Spark 2.0 version
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spark-the-definitive-guide.