Comments (5)
Yes. We will add a proper check and throw an error. Thank you!
from tt-metal.
import torch
import ttnn
device = ttnn.open_device(0)
a = torch.rand(1, 1, 2, 2048)
ttnn.enable_program_cache(device)
x = ttnn.from_torch(a, device=device, layout=ttnn.Layout.TILE, dtype=ttnn.DataType.BFLOAT16)
Now x
shape is [1, 1, 32, 2048]
- that 32 is a padded dimension.
I'd expect reshape
into [1, 2, 8, 256]
to throw as illegal in this case.
@arakhmati , wdyt, should we add more checks to reshape?
from tt-metal.
@ayerofieiev-tt is there an efficient way to work around this in TTNN? This reshape is one of the needed operation during my bringup of Gemma 2B on GGML. (I will need to eliminate this reshape for performance in the future. But now I'm focused on getting things running).
Thanks to your comment I figured out I can do this. But it's really slow,
if(dst->ne[0] % tt::constants::TILE_WIDTH != 0 || dst->ne[1] % tt::constants::TILE_HEIGHT != 0) {
// This path is SLOW.
tt::tt_metal::Tensor row_major_tensor = tt::tt_metal::untilize(t);
tt::tt_metal::Tensor reshaped = row_major_tensor.reshape(target_shape);
tt::tt_metal::Tensor ret = tt::tt_metal::tilize_with_zero_padding(reshaped);
*dst_meta = {
.tensor = std::make_shared<tt::tt_metal::Tensor>(std::move(ret)),
.ggtype = dst->type,
.bufctx = ((TensorWithMetadata*)dst->src[0]->extra)->bufctx
};
return;
}
IMO anything disallowed should throw. And footguns should be very hard to call and always has to be done explicitly.
from tt-metal.
@marty1885 this reshape is not currently supported in tile layout. You would need to write a custom operation for this case
from tt-metal.
I see. If that's the case can we make TTNN throw when a invalid reshape is issued?
from tt-metal.
Related Issues (20)
- External OP attributes include input tensors causing them to go over the allowed size of tracy messages
- Pack Untilize gives bad PCC when block_ct_dim < full_ct_dim HOT 1
- Fast Dispatch NOC Optimizations
- Clean up watcher stack reporting
- [Bug Report] Enviroment create by create_venv.sh fails to initialize PyTorch
- [Bug Report] Tensor.to(ttnn.TILE_LAYOUT) no longer handles non tile-aligned shape HOT 2
- unpacker race exposed when speeding up kernel launch
- make test_prefetcher and device.cpp use the semaphore indices returned by CreateSemaphore
- metal 1.0: profiler needs a clean API for user code
- metal 1.0: programming examples shouldn't use internal APIs or include internal headers
- Remove tt_lib in retinanet/demo , roberta, mamba/tests
- Runtime Error in llama3_1.8b model while porting on n300 HOT 1
- Add Non-blocking Circular Buffer APIs
- [Bug Report] Cannot import ttnn in python
- Enable double/triple buffering in All-Gather
- Investigate huge slowdowns on CI vm-58
- Sweeps: Add CI workflow for executing all sweep tests
- Save environment data and upload benchmark data even when CI jobs fail (single-card demos, t3k demos)
- Deterministic hang on CI for specific all_gather_datacopy test case
- Enable double/triple buffering in reduce scatter.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tt-metal.