Comments (7)
Thanks for your input.
#89 is very high on the list of priorities, we've just finished some internal refactoring that allows us to think about updating. Your code would translate to something like:
cdm_nycflights13() %>%
zoom_to_tbl(flights) %>%
left_join(planes, select = c(engines)) %>%
mutate(fuel = distance * engines * !!fuel_per_mile_per_engine) %>%
select(-engines) %>%
update_tbl()
Or perhaps slicker:
cdm_nycflights13() %>%
zoom_to_tbl(flights) %>%
left_join(planes) %>%
mutate(fuel = distance * planes$engines * !!fuel_per_mile_per_engine) %>%
update_tbl()
Let's focus on mutating from different tables in this issue. It seems we need to do #89 first while keeping in mind your use case. Happy to take your feedback regarding the API for "simple" updates in #89.
from dm.
@courtiol: We now have joins for tables in the dm, you can use left_join()
and friends on a zoomed dm. I have already used it with success. The documentation is nowhere yet but you can look at dm:::left_join.zoomed_dm
to see the implementation and the extra parameters.
Our example looks like this:
library(dm)
#>
#> Attaching package: 'dm'
#> The following object is masked from 'package:stats':
#>
#> filter
fuel_per_mile_per_engine <- 2.5 # in gallons
zoomed <-
cdm_nycflights13() %>%
cdm_zoom_to_tbl(flights) %>%
left_join(planes, select = c(tailnum, engines)) %>%
mutate(fuel = distance * engines * !!fuel_per_mile_per_engine) %>%
select(-engines)
zoomed %>%
select(month, day, distance, fuel)
#> # A zoomed table of a dm: flights
#> # Filters for zoomed: FALSE
#> # A tibble: 336,776 x 4
#> month day distance fuel
#> <int> <int> <dbl> <dbl>
#> 1 1 1 1400 7000
#> 2 1 1 1416 7080
#> 3 1 1 1089 5445
#> 4 1 1 1576 7880
#> 5 1 1 762 3810
#> 6 1 1 719 3595
#> 7 1 1 1065 5325
#> 8 1 1 229 1145
#> 9 1 1 944 4720
#> 10 1 1 733 NA
#> # … with 336,766 more rows
# The original has 53 columns:
zoomed %>%
cdm_update_zoomed_tbl()
#> ── Table source ──────────────────────────────────────────────────────────────────────
#> src: <environment: R_GlobalEnv>
#> ── Data model ────────────────────────────────────────────────────────────────────────
#> Tables: `airlines`, `airports`, `flights`, `planes`, `weather`
#> Columns: 54
#> Primary keys: 3
#> Foreign keys: 3
#> ── Filters ───────────────────────────────────────────────────────────────────────────
#> None
zoomed %>%
cdm_insert_zoomed_tbl() %>%
cdm_draw()
#> New names:
#> * flights -> flights...3
#> * flights -> flights...6
Created on 2019-11-14 by the reprex package (v0.3.0)
Do you have further suggestions at this stage? (I'm aware you're not particularly fond of "zoom" but we can discuss this separately perhaps.)
from dm.
This is great that it works, but I would prefer a version without left_join
.
Something like:
zoomed <-
cdm_nycflights13() %>%
cdm_zoom_to_tbl(flights) %>%
mutate(fuel = distance * planes$engines * !!fuel_per_mile_per_engine)
As I understood it, the benefit of {dm}
was precisely that is allows for not having to merge tables, so I would love if it was possible to stick to this core idea.
from dm.
Thanks for your feedback. I'm not even sure how to approach implementation this syntax. We'll probably need to hijack the $
operator and do some advanced magic so that it works on databases and local data frames.
In this syntax it seems a left join is the only thing that makes sense -- in addition, as you mentioned, we need to ensure that the join doesn't duplicate rows.
Let's keep this issue open for now.
from dm.
About the $
: this is exactly why I had suggested to introduce a new operator in my first post above (engines %from% planes
), but perhaps you are right and overridding the $
is more natural.
from dm.
I think this now can be implemented as a macro:
- analyze the expression
- rename in related table (use column names like
...1
,...2
etc. to avoid collision) - zoom into target table
- join related table
- perform the mutate
- remove the helper columns
- update-zoom in target table
Would you like to contribute?
from dm.
Thanks @krlmlr,
I would love to say yes and try, but I promised myself I would not take on new things for a while so as to catch up with my ongoing work.
So if anyone else feel like helping, please feel free!!
If no one jumps on it and if there is no hurry I may revisit this later.
from dm.
Related Issues (20)
- `copy_dm_to()` issues on Oracle HOT 1
- Do we need `dm_cast()` ? HOT 1
- Silencing dm functions optionally HOT 1
- Make igraph dependency optional HOT 5
- Fix SQL Server CI/CD
- Examples in "Getting Started" vignette are broken HOT 2
- Action Required: Fix Renovate Configuration
- Error in example in "Cheatsheet" vignette : pull_tbl()
- dm incompatible with pkgdown in a package that imports dm
- Title in Documentation
- `as_dm()` accepts non-dataframe objects which can lead to unexpected errors
- Full list of problematic values from `dm_examine_constraints()` HOT 1
- Non existing remote
- How to modify keys of a dm from connection?
- Learning from SQL Server
- Disable failing DuckDB-related test-cases on Big-endian platforms (indices and sorting are broken in DuckDB upstream)
- Review starschemar
- Naming pattern not used in `dm_from_con()` when keys are learned successfully
- New `dm_add_rk()` (referencing key?) HOT 1
- Trouble understanding `check_key()`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dm.