Comments (20)
Hmm can you try "require cudnn" in a torch prompt (by command "th") and see if that works?
from im2markup.
For the first question, I think you need to put the dockerfile inside a folder, then inside this folder do docker build .
will generate the image.
For the second question, it allows using Lua inside docker container only.
from im2markup.
I have this as an answer, I m not very familiar with lua language
from im2markup.
sorry I meant entering "th" first,
then in the prompt, enter "require cudnn"
from im2markup.
thanks for the advice ! but it still doesn't work . It seems that cutorch is also not install . I tried "luarocks install cutorch " but I have failed building..!
from im2markup.
Oh I think it seems to be some issue with installing cudnn. Installing torch correctly might be hard, would you mind using docker? Here is a docker file that can be directly used: https://github.com/OpenNMT/OpenNMT/blob/master/Dockerfile
from im2markup.
Ok I didn't know about the existence of Docker, thanks for the tips it looks great!
I tried to build an image by copying the content of the dockerfile you sent me but I get "unable to prepare context: context must be a directory: /Users/teiferman27/Dockerfile"
When I tried to launch directly "docker build https://github.com/OpenNMT/OpenNMT/blob/master/Dockerfile#L6"
I have one more question if I succeed to launch this dockerfile. Then I can use the Lua language with file on my computer? or just inside the "container". Thanks you for the advice already !
from im2markup.
BTW, I think you might need to use nvidia-docker (https://github.com/NVIDIA/nvidia-docker) to support using GPUs inside docker container.
from im2markup.
Ok I succeeded to use docker build -t operating_lua .
After running all the afternoon to build the image, I then have tried to launch the command docker run operating_lua
but it is just opening and closing on the docker dashboard ..... You think docker doesn't support the container and I need to use nvidia-docker ? thanks for the respond
from im2markup.
I think it should be nvidia-docker run -it operating_lua /bin/bash
, but it might be better to directly check docker documentation.
from im2markup.
I wanted to install nvidia-docker but I needed to install NVIDIA driver first. but it seems that this step require Linux operating system and I am on MacOs .... but I was surprised because one good point for docker was that everyone can run it from every operating system !
Then I tried docker run -dit operating_lua
and I was able to open the container and read in it :
Moreover when I tried to see if the module 'cudnn' is in the system I get :
I am still confused about how to approach the big picture... Can I import files into the container? Do I really need nvidia-docker ?
Thank you again for your time @da03 .
from im2markup.
Hmm I suspect that your CUDA driver version might be too outdated (what's the output of nvcc --version
and nvidia-smi
?), which caused issues both for require cudnn
and for installing nvidia-docker
. There are actually CUDA drivers available for mac: https://www.nvidia.com/en-us/drivers/cuda/mac-driver-archive/. Fixing the driver version issue might solve all problems.
from im2markup.
On the link https://github.com/NVIDIA/nvidia-docker , they talk about Linux
nvcc --version
and nvidia-smi
are unkown for now but probably because I didn't install nvidia-docker yet ? I am going to install CUDA drivers and then nvidia-docker.
from im2markup.
Oh no, so it seems nvidia-docker
would not work on Mac... I have never used GPUs on Mac, but I think with a proper CUDA installation (https://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html), you should get both nvcc --version
and nvidia-smi
working.
from im2markup.
I can't install on my mac because Nvidia doesen't support mac system anymore.
I think my mac may be it is too old.
I check my graphics on the system information as shown in
https://www.quantstart.com/articles/Installing-Nvidia-CUDA-on-Mac-OSX-for-GPU-Based-Parallel-Computing/
But I don't have NVIDIA graphic card on my computer.
I think the incompatibility is also mentionned there
https://developer.nvidia.com/nvidia-cuda-toolkit-developer-tools-mac-hosts
I am surprised of this CUDA/NVIDIA requirement to use the container though.
from im2markup.
Oh that explains why: this code (or cudnn) only supports CUDA and cannot run on systems without GPUs. While this version (https://opennmt.net/OpenNMT-py/im2text.html, code can be found at https://github.com/OpenNMT/OpenNMT-py) supports CPU only training, doing so would be extremely slow without the parallelism provided by GPUs. Another way might be using cloud computes such as Amazon EC2 or Google GCE or Microsoft Azure, and rent a GPU instance.
from im2markup.
I manage to get another computer but the GPU is AMD Radeon and so I can't use the cudnn module ... I think it should be mentioned on the prerequisites since docker can't solve this hardware issue.
I was about to try CPU but I think that on the link you gave me (https://opennmt.net/OpenNMT-py/im2text.html) there is dependencies like torch vision and pytorch is required ( and so CUDA-enabled GPU are needed again no ? )
I try to follow the steps from https://opennmt.net/OpenNMT-py/im2text.html
but the command onmt_preprocess
is not found. There is a step I have missed ?
I will try to use cloud computes probably.
But just to be sure ( correct me if I am wrong) :
- OpenNMT-py project works with pytorch (https://github.com/OpenNMT/OpenNMT-py#requirements)
- this project works with torch ( https://github.com/harvardnlp/im2markup)
from im2markup.
Yes you are right that OpenNMT-py uses PyTorch and this project uses LuaTorch. PyTorch does not require GPUs (you can do CPU-only installation), but again, it might be extremely slow without using GPUs.
For the onmt_preprocess missing issue, have you installed OpenNMT-py following the instructions here? https://github.com/OpenNMT/OpenNMT-py
from im2markup.
I had issues with onmt commands because I use python environment using google collab ( You can activate GPU on the settings and it seems to be a good free solution )
Installing OpenNMT-py with pip instead of clonings the project worked for the onmt command.
Is Using Google-collab a good way to perform GPU calculations ? I am trying to train the model, but it takes a lot of times, do you now how much ?
from im2markup.
Yeah I think so! The only problem is that the runtime would be disconnected if it's idle for a certain period of time, and the instance would be freed so all progress would be lost. Therefore, you might want to connect to your google drive, and save progress (checkpoints) to your google drive.
from im2markup.
Related Issues (20)
- - HOT 1
- not working for below type of images (other than given by you). I think we need to put images in particular format HOT 8
- can anyone share the trained model file which is genralized on any type of image like mathpix HOT 3
- [Please Respond] Can you help me training the model for to recognize the out of given data image set HOT 1
- how to remove katex parser error HOT 1
- target vocab size HOT 5
- There is a bug in preprocess_latex.js HOT 3
- [regarding real dataset] Please respond HOT 18
- I am getting None with intermediate weights HOT 1
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe7 in position 2270: invalid continuation byte HOT 7
- How to make code show predicted mathematical expression in latex format HOT 1
- can you explain about value 'Accuracy'?
- why downsample by 2 in preprocess HOT 2
- Why using lua instead of python? HOT 1
- can you explain src\modeel\cnn.lua
- Getting low accuracy using customized images for test. HOT 2
- 'perl' and 'cat' is not recognized
- Can you provide a vocab dictionary?
- The python version of the dataset resource is not working
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from im2markup.