Comments (1)
cparisien
commented on August 18, 2024
Hi @notelainemade, thanks for the question!
NeMo Guardrails itself doesn't require a GPU to run, it's currently all in CPU. So you could definitely try it in a Jetson, it just won't use the GPU directly. All of the LLM calls are to external hosted services.
The one thing that might cause a problem on "slimmer" devices is that it uses SentenceTransformers to create an embedding index. Again, that's currently done in CPU. We're looking at other options for that embedding process, including using a local GPU if one is available.
By the way, we've now enabled GitHub Discussions as a good place to ask questions like this. I'm going to close the issue, but I'll convert it to a Discussion in case you have any other questions.
from nemo-guardrails.
Related Issues (20)
- Unable to install any version > 0.7.0 of nemoguardrails using pipenv install nemoguardrails HOT 1
- Error when using Flask, Gemini models(gemini-1.5-pro-001), and NemoGuardRails integrated with each other. HOT 4
- Error evaluating '$generation_options.rails.input': Attribute 'rails' does not exist in expression 'var_generation_options.rails.input' HOT 1
- passing in relevant_chunks as a message's context HOT 1
- Compatibility with LangChain's invoke() HOT 2
- Issue with streaming with Ollama HOT 1
- Internal error occurs whenever I pass a prompt to conversation chain with NeMo
- Issue with Overriding Prompt of SELF_CHECK_FACTS Task HOT 7
- Azure GPT models not working HOT 3
- Guardrails throwing error when input message has \n HOT 1
- Nemo Guardrails on Lambda FileNotFoundError with '.cache/embeddings' HOT 1
- TypeError: cannot pickle '_thread.RLock' object HOT 10
- Actions not being imported correctly HOT 4
- Issue with Integrating NeMo Guardrails with LangChain and LangServe for RAG Chatbot
- Error while execution generate_user_intent: Connection error. HOT 3
- .co file defination HOT 1
- No Response and No LLM calls were made. History: HOT 3
- output flow does not activate HOT 1
- Not blocking certain topics, and still ask LLM multiple times HOT 4
- How to get exception occurring within Nemoguardrails as output?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nemo-guardrails.