Comments (2)
Unlike R, torch has support for scalar values (0d tensors). This differentation is required in some specific situations in torch.
Torch scalar tensors don't have dimensions, by definition, thus it's consistent that:
$size()
which returns a vector of dimensions size returns 0length vector.$dim()
which returns the number of dimensions, returns 0
It's also consistent witth length()
which is an equivalent to $numel()
, the number of elements in the tensor.
Since R doesn't really have scalar values IMO it's consistent that if 1 = c(1)
, torch_tensor(1) = torch_tensor(c(1))
even though I recognize this is debatable.
The default value of drop
is TRUE
, and torch can still drop one dimension, so we drop it. I'm up for discussing if this the best behavior. FWIW there are many other functions in torch that return scalar values, such as torch_max()
, torch_mean()
, etc.
Do you have an specific use case where returning scalar is causing problems to you?
from torch.
The use case is when using index in a multi-dimensional tensor without drop=FALSE. The dimension of the resulting tensor will depend on whether the index selects one or more elements.
I stumbled on this when implementing a seq2seq model with attention in R, which was inspired from pythong tutorial. In my implementation of the decoder (you can see the code here), I am using an index (b_index
) that selects sentences in the batch that have not yet terminated. When computing the loss, input should be a 2d tensor and target a 1d tensor:
loss <- nnf_nll_loss(input=state$prob$squeeze(2), target=target_padded[b_index,i]$view(length(b_index)),
reduction=loss_reduction) # ignore_index = padding
In the current implementation, state$prob
is a 3d tensor (b_index_size, 1, output_size) that contains the probability
distribution over target tokens for the current decoding step; target_padded
is a 2d tensor (batch_size, max_target_len) that contains the tokens for the target sentences; target_padded[b_index,i]
selects tokens for unterminated sentences for the current decoding step (i
goes from 1 to max_target_len). It is a 1d tensor because the second dimension as expected. The problem occurs when there is a single element in b_index, in which case the dimension drops to 0. It is not a huge issue because I can always reshape the vector with view for example (I cannot use drop=FALSE because this would prevent dropping to 1d for the 2nd dimension).
Actually, I am not sure what is the best behaviour. One should expect that a drop of dimension(s) when one selects a unique element in a multi-dimensional array. I guess the issue is whether to stop dropping dimension(s) at 1d array for consistency with R or if we should follow Python. What is weird is that x <- torch_tensor(1)
is 1d but x[1]
is 0d. The R interface to torch 0d tensors / arrays scalar is not clear (what is the standard way to create a 0d torch tensor from R?) and the problem is compounded by the fact that scalars and 0d tensors don't exist in R.
from torch.
Related Issues (20)
- A possible mistake in document "GET TECHNICAL" on the broadcasting
- Typo (?) in "get technical" document "Modules"
- Issue installing GPU-enabled torch HOT 13
- Issue installing Torch on M1 Mac HOT 1
- Sometimes running on device cuda and sometimes running on device cpu automatically, why? HOT 1
- RNN with packed sequences where enforce_sorted=TRUE gives an error.
- Failure to load Lantern error escapes tryCatch() function HOT 7
- torch_arange() behaviour depends on the dtype HOT 2
- torch_split() does not work HOT 2
- Parallel Dataloader failing when using num_workers > 0 HOT 1
- New release? HOT 1
- Cuda 11.6 deprecated HOT 2
- torch fails on new Mac M3 architecture HOT 13
- Release torch 0.13.0
- `torch_triu_indices` and `torch_tril_indices` return 0-based index
- first and second order derivative of gamma samples HOT 1
- Unable to load pre-trained alexnet weights HOT 2
- Can't download lantern from https://torch-cdn.mlverse.org when install torch 0.13.0 in win11 HOT 4
- Number of dimensions of a tensor increases when indexing a dimension of length 1 with a one-dimension logical vector HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torch.