dmarx / bench-warmers

DigThatData's Public Brainstorming space

DigThatData's Bench Projects!

Come one! Come all! We've got something for everyone!

Wanna set up your own similar brainstorming space?

See template repository with setup instructions here: https://github.com/dmarx/workbench

FAQ

last_modified	title	est. idea maturity	tags
2024-04-14	Continous Graph Embedding	2332
2024-04-14	Continous Graph Embedding	2332
2024-04-10	sandbox to probe scaling laws for representation learning	1164
2024-03-26	jax support for tensorizer	274
2024-03-21	thinking fast and slow	622
2024-03-15	master document for LLM Finetune ideas	5361
2024-03-05	Predict learning trajectories conditioned on model embeddings	897
2024-03-02	memorization scaling law	2431
2024-03-02	Fit a LoRA paramerized by a VAE	76
2024-02-28	Dataset preprocessing for code-gen finetuning	513
2024-02-27	LoRA as a Topic Model	3495
2024-02-14	ELI5: history of the VAE	9866
2024-02-06	opt-in.ai	184
2024-01-27	open source guild UBI	645
2024-01-19	community traversal batch sampling	2903
2024-01-18	Population Convergence Training	622
2024-01-18	Split LoRA: hiding a third LoRA in the weights of two other LoRAs	388
2024-01-14	Diffusion Video Models for Climate Change Prediction	1005
2024-01-06	in-context RL with AlphaGo-esque self-play	253
2023-12-26	BiaB: Business in a Bottle	124
2023-12-01	use TTI generation exercises to teach visual vocabulary	218
2023-11-29	Van gogh lora	149
2023-11-29	publish pre-tensorized checkpoints	76
2023-10-19	dragdiffusion node for comfyui	67
2023-10-12	[ComfyUI Node] Depth "merge" with Boosting Monocular Depth	86
2023-09-27	publish a journal as a github repo	304
2023-09-24	"prior probing" for "safety"/nsfw scoring (model rating)	604
2023-09-19	investigating identity formation as a concrete example of grokking dynamics (+ NameBasis)	7169
2023-09-19	photo auto organizer	195
2023-08-23	(draft) vktrs design whitepaper	1134
2023-08-22	marble machine X x pytti-tools: video series building up an AI 4 creatives tool piece by piece	251
2023-08-22	Nonprofit to support FOSS devs	2000
2023-08-22	troll tanishq	76
2023-08-13	Ai art methods masterclass	295
2023-08-12	Course teaches intermediate python generative art topics by building a simple ai art library	5999
2023-08-11	stick-breaking prior for semi-correlated feature initialization	633
2023-08-11	Think Like a Neural Network: Transferring neural problem solving strategies for every day life	1005
2023-08-09	What is it like to be NASA?	7998
2023-08-09	git-tree	120
2023-08-09	Physics For ML	828
2023-08-09	auto green screen	176
2023-08-04	ethics of training on public data	129
2023-08-04	model sanitizer	226
2023-08-03	Tattoo Ideas	78
2023-08-02	Notes on differentiable permutations	1241
2023-08-01	simulating regional neuronal promotion/suppression in ANN activations	721
2023-07-31	font diffusion	117
2023-07-30	Population-based training + evolutionary strategies via locally correlated constraint	495
2023-07-29	insta-edit	425
2023-07-29	contrastive circuit sharpening	561
2023-07-29	weights, checkpoints, and .half()	53
2023-07-25	Social Media and Singularity Theory: Information geometry and society	5192
2023-07-25	Graph analytics on checkpoints	280
2023-07-25	Stego db	136
2023-07-25	Yaml-state	167
2023-07-25	Self-serve data lake	315
2023-07-25	Contrapositive odds-ratio for clip zero-shot classification AND GUIDANCE	200
2023-07-25	Klmc2 Edit exploration	256
2023-07-25	Stable paintbrush	269
2023-07-25	KLMC2 Prompt Entanglement Notebook	869
2023-07-25	standalone utility for adding latent encoding to exif	233
2023-07-25	checkpoint-to-LoRA	356
2023-07-25	denoising gradient descent	91
2023-07-22	my time at microsoft	1746
2023-07-19	klmc2 as video init	230
2023-07-19	Early Adoption: How to Contribute to Cutting Edge Research w/o needing a PhD	3431
2023-07-16	Palette of Priors	533
2023-07-14	Technical Constraints Considered Harmful.	177
2023-07-14	CLIP ReLoRA	141
2023-07-11	personalized anything	4159
2023-07-10	cycle refinement as mode finding (via fixed point iteration/annealing)	506
2023-07-07	attribution distillation/tuning	204
2023-07-06	Diffusion2GAN - GAN PEFT on diffusion generations	346
2023-06-28	AI bob ross youtube channel/twitter	191
2023-06-23	Seed differentiable vectorization with paths inferred from semantic segmentation	768
2023-06-15	animating on autopilot	2013
2023-06-15	animation autopilot	126
2023-06-03	Intro to Latents: The Geometry of Ideas	305
2023-05-30	generative outfill	116
2023-05-20	multimodal multiconditioning adaptor	105
2023-05-16	student-teacher model compression	319
2023-05-13	Sliding Tiling	859
2023-04-25	hypernetwork code embedding	889
2023-04-24	a1111 lessons learned	2586
2023-04-23	twin mixing bootstrap regularization	897
2023-04-14	LLM Helper	182
2023-04-14	we need a documentation embedding standard	2245
2023-04-11	Private Tutor	819
2023-04-07	prompt2startup	1512
2023-04-07	accelerated finetuning with "menu of marginals"	694
2023-04-07	warp klmc2.md	819
2023-04-07	Predictive Safety for First Responders	1237
2023-04-07	FOSS Touch Designer	797
2023-04-07	On "The Singularity"	475
2023-04-07	gh as blogging platform	520
2023-04-07	Fast Fourier Convolution FlexConv	297
2023-04-07	Pre-train and Release ViT-VQGAN and RQ-VAE checkpoints	633
2023-04-07	Bam: Alignment Solved.	2166
2023-04-07	INR CNN Filter Learning	526
2023-04-07	DINO diffusion	334
2023-04-07	sparse-latent-inpainting-interpolation.md	584
2023-04-07	AI-ify bond clown chase scene	379
2023-04-07	image to image video game movie	391
2023-04-07	focal finetuning	351
2023-04-07	Diffusion that respects transparency	246
2023-04-07	Denser CLIP representations	558
2023-04-07	dreambooth lite	421
2023-04-07	karlo + DIP	358
2023-04-07	minibatch bootstrapped gradients	550
2023-04-07	conversational agent to assist with narrative world building	285
2023-04-07	Auto-illustrator	344
2023-04-07	Scale-Agnostic VQGAN	387
2023-04-07	k-dimensional rotation as an AI art animation	357
2023-04-07	metatron: simple utility for connecting STT/TTS with apis	285
2023-04-07	Fingertrip tracking for hardware-free gloving practice	275
2023-04-07	test as you go programming	663
2023-04-07	alt gh account for blogging w githubb as publication platform	305
2023-04-07	data science project investigating "language acquisition" for TTI prompting	1319
2023-04-07	lego diffusion	188
2023-04-07	Brushstroke Diffusion	1408
2023-04-07	personalized guided meditation	255
2023-04-07	Relevance map	579
2023-04-07	github-pages blog + substack?	481
2023-04-07	i-like-that-shirt.com	510
2023-04-07	FlowFrames linux port	400
2023-04-04	auto-coderTROTFO: structured LLM colaboration	1223
2023-04-03	parallel diffusion	1209
2023-04-01	Paranoid Android	1666
2023-03-29	GPT assisted CI/CD	990
2023-03-28	P+ prompt inversion	405
2023-03-25	Quorum	379
2023-03-24	"why do you want to know?"-instruct-esque-fintetuning	1987
2023-03-23	On Openness in AI	517
2023-03-23	product thinking AI	14400
2023-03-23	mini RWKV port	778
2023-03-22	Make an NLP dataset out of the Stanford Encyclopedia of Philosophy	727
2023-03-21	Internal team of Open Source devs as mechanism to promote benefits of open source development practices on non-open codebase	3762
2023-03-15	klmc2 rotoscoping	926
2023-03-10	Git Utilities to simulate Github "Private Branch" functionality	1517
2023-03-10	chatgpt hotkeys	265
2023-03-09	Is "Idea Factory" a job title?	178
2023-03-03	executable readme	1287
2023-03-02	Pandas Wrangler	910
2023-02-25	[vktrs] USCSB -> SD	265
2023-02-23	look-ahead traversal sampling	1184
2023-02-23	Learned conditional semantic controls	781
2023-02-23	audioreactive tide	4302
2023-02-17	content helping people understand klmc2	414
2023-02-16	video edit model + klmc2 + cold-diffusion = text2video	1279
2023-02-16	video caption (chain of thought) to text2video	410
2023-02-05	contrastive learning as ontological intentionality triangulation	972
2023-01-27	Open Assistant	1017
2023-01-19	On Paperclip Problems	30222
2023-01-18	CLIP auto organizer	458
2023-01-15	The Shape Of Data	22991
2022-12-31	camera motion diffusion: few-shot finetuning using semantic checkpoint merging	838
2022-12-26	hyper-embedding	325
2022-12-21	Test-Driven Development as a programming language: prompting an entire software library.	804
2022-12-19	Automated Video Description for Vision Impaired	5070
2022-11-09	layer-wise update cadences	2139
2022-10-30	research and publication in the age of social media	2752
2022-10-24	multimae +text	479
2022-10-24	micro epoch training	733
2022-10-21	random batch + random average for massive asynchronous data-parallel training	3694
2022-10-21	diegetic classifier	381
2022-10-01	Expectation regularized diffusion process	952
2022-09-23	Whisper STT + timestamps	847
2022-09-23	subtitles to storyboard	1060
2022-09-09	Free to mint nfts with Github + PGP	2101
2022-09-08	docstring to CLI	238
2022-09-08	differentiable control flow parameterizations	713
2022-08-25	fully deterministic animation	278
2022-08-15	animation thing with shadow and 2scoops	118
2022-08-15	turn this into a template	186
2022-08-15	declarative infill	748
2022-08-15	Deep Image Prior with a UNet whose conv kernels are FlexConvs	123
2022-08-15	comment zapper.md	390
2022-08-15	registered datasets for stability compute	284
2022-08-15	Layer-wise Super Resolution	2118
2022-08-15	Inverse Heat Dissipation Model with FlexConvs	1001
2022-08-15	learning image segmentation from a colorization task	216
2022-08-15	flow field compositor	557
2022-08-15	NeRF-GAN / VQ-NeRF for procedural generation	314
2022-08-15	infinite canvas stylegan-xl	776
2022-08-15	design a 'file type' (or related spec/standard)	5288
2022-08-15	Grimoire: Prompting Toolbox	1371
2022-08-15	personalized VQVAE	519
2022-08-13	Deep DSL	4090
2022-08-07	GAUDI-XL	1175
2022-08-04	xpost reddit comments to twitter threads	387
2022-07-29	TTI cpu inference stack	382
2022-07-25	Design and train a model whose underlying representation was engineered specifically for text guided animation	404
2022-07-21	code2commit learning	466
2022-07-21	LAION++ Enriched Dataset	936
2022-07-21	multimodal model trained on comicbooks	402
2022-07-14	Tool for Converting Google Colab "Forms" UI annotations to a more environment agnostic framework	414
2022-07-14	conlang generator	267
2022-07-09	Train a SinNeRF on an AI-generated animation (e.g. PyTTI)	647
2022-07-09	CLIP with RWKV replacing QKV	159
2022-07-07	narrative coherence via 'video understanding' prior	185
2022-07-06	finetune a segmentation model on a medical classification task	476
2022-07-05	implicit CNN filters	329
2022-07-04	single video sparse quantized auto-encoder	556
2022-07-04	RETRO-ViT	1572

bench-warmers's Issues

trash/archival mechanism

automagically generate 'category' pages grouping similarly tagged items

see also #4

https://github.com/dmarx/bench-warmers/blob/main/multistage-unsupervised-deep-DSL-learning-from-prompts-data.md

Self-serve data lake

Plug-n-play tools that take advantage of personal data dumps, e.g. google export, netflix, etc.

• CLIP image search
• Personalized textual inversion vectors
• finetuned SD
• personalized stylegan-xl (portrait rfiting

Stego db

Use image file as storage, json -> eTxt

simpler in-document tagging

badges makes sense in generated content, but it's a PITA trying to remember the specific format for adding simple tags to documents. really, this should just be a yaml header or something like that. anyway, better metatags would be nice.

Course teaches basic python by building a simple ai art library

i think this might need content to get transferred over?

automatically move notes to a subfolder so users/visitors don't have to scroll forever to get to the ToC

automatically add metadata tags

author, created, modified, etc.

Figuring out SD forks for stable migration

Send an email
Take a survey
Don't just track every fork down like an epidemiologist. Invite them to report in.

add metadata field for rough indicator of level of development

could be something like number of characters, TOC length, etc. or even a visual thumbnail of the readme?

update scripts to reference configurable paths

• path/location(s) to look for new documents
• paths to put existing documents and check for updates
• for "modified date" inference, ignore file rename operations. ignore tag changes?

LivingDocument class

Easy way to carry information around, describes what this is for... could be the name of the project?

Yeah, I'll create a new repo to track a cookiecutter of benchwarmers or whatever

bibtex citation snippet

i should just get in the habit of adding this to all of my work. ditto adding a license block to the top of all code.

Van gogh lora

Van gogh concept is way overfitted to starry night.

add metadata field that generates a table column linking out to ongoing work

another really simple indicator of level-of-development could be something like number of characters, TOC length, etc. or even a visual thumbnail of the readme?

streamplot of tag prevalence over time

visualize how the broad themes i'm thinking about change and develop

new maturity metric: # commits on document

gh-pages frontend to facilitate idea navigation

• add or infer topical tags/vectors to articles
• build an interactive navigable graph to facilitate navigating ideas
• maybe there's a way we could incorporate arxiv articles in the graph?

Trigger to turn ideas in issues into living documents

Cvpr art demo submissions due march 10th

https://cvpr.thecvf.com/Conferences/2023/CallForDemos

Audioreactive tide/klmc2

To do: add ortgogonal friction reactivity

Stable paintbrush

Each mouse click is a diffusion step in the vicinity of the click.

Image is demonstrably different (even if only slightly) from what would've generated just from settings

Stable Diffusion SDK

Extend grpc client into SDK, build animation on top of that

Contrapositive odds-ratio for clip zero-shot classification AND GUIDANCE

Instead of argmin_theta x * \theta

Do argmin x* \theta / -x * theta

Or softmax or logodds or whatever

come up with a tag for `trash`

see file added to repo shortly before timestamp of this issue

Graph analytics on checkpoints

https://medium.com/rapids-ai/pytorch-rapids-rmm-maximize-the-memory-efficiency-of-your-workflows-f475107ba4d4

Klmc2 Edit exploration

Use klmc2 on instructpix2pix for better init image dreaming. Inspired by https://twitter.com/dmvaldman/status/1616575342969516034?t=kTGbBpV7SCq-6L9sOw1goA&s=19

auto-TOC for README

picturing this as a script that runs as a github action

1. script runs for any commit that isn't just a change to certain specific files
2. script walks through files in the project, collecting metadata from files that are tagged appropriately (e.g. *.md file that starts with an H1 annotation)
3. reformat metadata into links to files and write to a new file
4. insert this file contents into the README and commit (...in a fashion that won't trigger the gh action that launches this process)

Convert links to obsidian-compatible labels

1. update scripts that ignore specific commits to ignore front matter
   • non-trivial af. the current approach only works because labels are on a given line
   • here's something that could work, but would be a whole thing to implement
     1. get list of files modified
     2. identify line at which front matter ends
     3. only respect diffs below that line
2. copy labels into front matter
3. modify scripts that utilize/manipulate labels to parse front matter

add pages for tags

turn this repo into a template to encourage other people to publish their bench ideas

• attract interested collaborators
• join forces with people who've already developed similar ideas
• learn about research groups who are currently working on this already that you might be able to join
• alert visitors to research you might not be aware has already been performed
• you're not working on the thing already, what if you never get around to it? why not have other people working on stuff you wanna do instead of stuff you have no interest in, if given the option?

... I need to find a better way to pitch this

improved issue conversion

would be nice if we included the whole (non-automated) comment chain when migrating issues. shouldn't need to be a thing after a full migration has happened though, since as soon as an issue is created, it should get migrated.

...except, i never actually did add that workflow did i. hmm.

TODO: add a workflow that's triggered on issue creation (or maybe when a label is attached to an issue) for auto-migration.

Script that converts labeled issues ro articles

photo auto organizer

1. Segment anythung -> person detection
2. CLIP + DINO person embeddings
3. vector db, knn search

automate tag colorization

collect tags, unique-ify, draw colors, generate badge templates in dict, use dict to map tags back to standardized badges

Ai art methods masterclass

Gan
Vqgan
Diffusion
Img2img
Klmc2....

Whatever patrons want, really

Yaml-state

Persistent State manager w hierarchical yaml files a la vktrs + hydra.

mechanism for tagging commits that shouldn't be considered for updating TOC dates

i'd like to be able to make changes to documents or even across the whole project that don't impact the "last updated" dates that are auto-generated by commit history

add counts to tags

add a new mapping from tag to "tag name as it appears in the badge", so like an entry will show up as [![public_name(tag)](file_url(tag)) or somethign like that. the idea here is that I'd like e.g. the tag experimental to show up on a badge as 11-experimental or whatever to make it easy to visually distinguish tags that connect to a lot of related work.

could even use these counts or other similarity to develop a "relatedness" score to help dedupe related ideas and re-prioritize things i'm thinking about a lot.

sort *timestamp*, not just day

order is getting a little mangled

convert tags to `labels: ` format

all "badges" form tags should be replaced, and those commits should be ignored