joshpoll / citoplasm Goto Github PK

We should explore ways to make CITOplasms robust to adversarial attacks.

There's a lot of room to explore better CITO prompts following prompt engineering best practices. One challenge of doing this in CITOplasm is the need to balance high quality prompts with generalizability across different functions that use CITO. So far, it seems like it might be possible to get away with a universal CITO prompt, but we'll see!

Prompt Engineering resources:

• https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api
• https://www.promptingguide.ai/

In addition to best practices, it's worth exploring different potential output formats. e.g. JSON, YAML, XML.

We should have documentation and possibly code that lets us interface with LangChain agents, chains, and tools.

LMQL is a prompt engineering language that could be a good target for optimizing CITOplasms. Inspired by PyTorch et al., one possible idea is to record a trace of a CITOplasm execution and try to fuse neighboring CITOplasm calls into a single prompt to trade off time and accuracy. It might also be possible to fine tune prompts to particular scenarios.

In this way, we can keep the user-facing API nice and declarative without sacrificing perf.

This will make it easier to produce more robust output by encapsulating the k-shot pattern.

This should probably be a list of records.

Each record should contain

• context(?)
• input
• thought
• output

Several of our functions already use an ad-hoc examples pattern so it would be fairly straightforward to push it into the cito API.

Agents can be modeled in several different ways. It's worth exploring these ways for the API.

e.g.

• https://arxiv.org/pdf/2212.01681.pdf
• https://danielmiessler.com/blog/spqa-ai-architecture-replace-existing-software/

Questions: Should agents be able to have their own contexts? Should agents be stateful?

The functions we have currently are way undertested. It would be very useful to have more tests!

Context requires thinking about state management more precisely. One option we've considered is using with blocks. e.g.

with LocalContext as context:
  ...

The advantage of making a Context primitive is that it lets us abstract over how to structure the context section.

Guardrails is a Python library that can help make our parsing more robust.