Projects for Google ATAP’s Soli Sandbox where I experimented with the Google Pixel 4’s Soli gesture sensor
Tile World Builder (source)
This project allows users to create a world using Everest Pipkin's city game tilset. It was built using p5.js and p5.speech(). Soli swipe events allow users to move the cursor location, and parsed speech from p5.speech() allows users to place custom tiles. However, at the time of making this, Soli Sandbox did not support microphone access. The way I got the version filmed to work was by using two devices: the Pixel 4 updated the cursor and my laptop's microphone updated the world. Both devices were networked via socket.io.
3D Construction Toy (source)
This project allows users to use Soli swipe events to describe various transformations on a cube in 3D space. The project was build using three.js and I learned many things about the THREE.Matrix4 object. I also initially copied some code from 2013 to handle collision detection. However, it was really buggy, so I wrote my own which works by calculating the axis-aligned bounding box (AABB) and detecting collisions. Since each cube must pass through 6 transformation filters, the program requires a lot of swipe gestures. In order to make the program easier to use and more toy-like, the number of cubes is limited to 4. Personally, I could not get the Soli tap event to work on my phone, which is why I only relied on swipe events. Perhaps using tap to drop could make the interaction less awkward.
Project (app, code) ← use arrow keys in place of swipe events
Breadcrumb Trails (source)
This project uses Soli swipe events to navigate a filepath and see the breadcrumbs. I initially wanted to pursue this idea using d3.js, but I was unable to configure it correctly, so this was made with p5. The data is from d3.js's examples (flare.json). The up and down swipes change the current branch to a sibling branch, whereas the left and right swipes change branch to either a parent or child branch respectively. I also wanted to connect this filepath navigation to p5.speech() to allow users to change the entries in the selected field. However, with my struggles to get p5.speech() to work for another Soli project, I decided to leave this project as it is.
Project (app, code) ← use arrow keys in place of swipe events