Trying out a color picker to get a feel for tabletop UI. I can do it all with pointer fingers (rather than pinch) if I give myself a path to reach them without colliding with other things. Kind of like a maze.
Book identifier
Trying out hand-tracking triggered buttons and some simple LLM calls. There are definite limits imposed by hand-tracking and projector display - but they're kind of interesting ones.
Projector buttons
Triggering buttons on overlap
Color change - was not working at first because of webcam reacting to light change, fixed by turning on the overhead light so it's more constant.
Projector experiments
I got a projector and webcam set up over the tabletop. Experimenting with hand-tracking.
Tracking pinches
Tracking and projecting pointer fingers
Book library experiments
Collecting some recordings of recent experiments. Taking photos of my books, using Gemini image edit generation to isolate those books and then removing their backgrounds, embedding generated summaries of those books and exploring them across various layouts.
Spines and covers
I was looking at a short school bus with three windows with three decals applied beneath. I think it was "Quality", "Bus", "Corp" or something like that. And I was looking to see how they had aligned the decals under the windows. Left-aligned on each one? Left, center, middle? read on scrawl
Tabletop and generative images experiment
Tabletop + nano banana experiment where you draw over the top of the generated image and feed it back. Searching for a sweetspot of physical <-> digital interaction. (video sped up and edited)
A weird mc escher-ish one:
New tabletop setup
Re-setup my workspace with a monitor in the tabletop and an overhead camera. Aiming to do more physical computing experiments.
The final arrangement
Stripped down the old setup
Matching and connecting
Promising result from ranking my handwritten notes using embeddings and then prompting for connections to the top result. Need to prompt it into less clumsy language though.
Project Scroll
Thinking about ways to combine this with some of my tabletop experiments. From Yancy Strickler.
