Through work, I've been playing with the Gemini model's ability to return bounding boxes for an image. I think I want to try and use it to make some sort of image (sketch or photo) to website generator. Maybe leaning into skeuommorphism of like a photo of a physical desktop that contains all the links to my projects as items on the desk.

A very sketchy proof of concept

Here's a brainstorm: I could do this for my homepage and make a new sketch each week. I would just want to ensure that it matched all the links. So the builder could be a list of links and a place to add an image. You get the boxes back from the model and you have something tgo check if all of the links got placed and maybe if all of the boxes are past a certain size threshold.

This also connects to being a bridge between the physical and the digital world, similar to some of the dynamicland stuff, kind of. I could have digital links attached to objects I move around in the real world. I could also maybe connect this to the thermal printer.

Oh! One thing you could do is combine it with a webcam, have pieces of paper (or objects) that serve as links. Then baically have the computer monitor those boxes for a... gesture? or just a finger entering the space? Or hmm maybe just a major change -- use that as a proxy for clicking. Then you could build a physical interface...

But what does that help? Something that could help give you clarity of focus. Reduce the options to just the task at hand.

Physicallizing digital objects. Making them less slippery.

That connects to a more tools for thought. Something that would let you make something more like a node graph. You could write out a concept then tie it to a physical token, and then you could rearrange those physical tokens, and on the digital side it would... what? Mirror the arrangement of the phsyical tokens? (Possible but necessarily that interesting?) What do I actually want to do? I want to be able to pack a complex idea into a movable container, but then I also want to be able to separate out pieces of that idea (that's why too rigid a structure doesn't work).

There's also the arrows angle, doing something when you draw arrows...