Stitched together some Neovim autocmds with Niri window commands so that the app window is bound to the special "app" buffer position and size. One step closer to NeovimOS.

Thinking about reconfiguring my window manager setup to have an always-open terminal on the left. To serve as doc and navigation maybe.

I'm realizing part of the challenge of configuring is terminal + neovim is sort of its own window manager on its own - with splits and open fuzzy-findable buffers and everything - maybe there's benefits to having that be one always-present session, and it's actually browser and application windows that are more ephemeral.

An apriltag color-picker where x=red, y=green, rotation=blue. A freeform physical slider feels interesting

Also did a 3 card one - different tags all get IDs which makes me think of doing some sort of weird physical variable assignment thing.

Book-to-soundtrack experiment - a projector setup where I send the book image off to Gemini for recommendations and then play with Spotify. Meant to be kind of like putting on a record but for objects. (I clipped out the loading times.)

Trying out april tags. Thinking about how it could complement hand tracking and vision models. This is using https://github.com/arenaxr/apriltag-js-standalone. and I know what april tags are because of https://folk.computer.

Trying out a color picker to get a feel for tabletop UI. I can do it all with pointer fingers (rather than pinch) if I give myself a path to reach them without colliding with other things. Kind of like a maze.

Trying out hand-tracking triggered buttons and some simple LLM calls. There are definite limits imposed by hand-tracking and projector display - but they're kind of interesting ones.

Triggering buttons on overlap

Color change - was not working at first because of webcam reacting to light change, fixed by turning on the overhead light so it's more constant.

I got a projector and webcam set up over the tabletop. Experimenting with hand-tracking.

Tracking pinches

Tracking and projecting pointer fingers

Collecting some recordings of recent experiments. Taking photos of my books, using Gemini image edit generation to isolate those books and then removing their backgrounds, embedding generated summaries of those books and exploring them across various layouts.

Spines and covers

Tabletop + nano banana experiment where you draw over the top of the generated image and feed it back. Searching for a sweetspot of physical <-> digital interaction. (video sped up and edited)

A weird mc escher-ish one:

Working on a book site

Promising result from ranking my handwritten notes using embeddings and then prompting for connections to the top result. Need to prompt it into less clumsy language though.

Needs some cleanup, but having fun.

I got the basics of vector embedding retrieval working for my tabletop notes. Thinking about a more intensely minimal interface that really is image only.

Doing some experiments with my overhead camera setup and the work-in-progress sampler app. Not sure where it's headed but I do like moving between physical and digital arrangement.

Stretch vignette

I have this vague idea of using image blocks as 'light sources' that cast low-resolution light based off their content into the surrounding canvas. Haven't figured out the system yet though.