— Adrià Navarro @ ITP

My final project for Physical Computing is a series of toys that interact with the iPad screen to introduce you to a world of mirrors, illusions and surprise.

Out of the many objects I imagined and sketched, I only had the time to build a rough prototype of two of them, and a fully working demo of one. You can watch it working in the following video:

As you see, it’s sort of a magical box that needs to be placed on top of the iPad to reveal its content in form of a 3D hologram. You should imagine the final toy slick, dark and mysterious, not made with cardboard and duct tape.

I’m very excited about all the possibilites of this project and all the things I want to explore. I really want to push the project forward and, since it’s in in a quite early stage, I’m not sure about how should I document this, so I’ll try to summarize the parts that look more important to me.

Motivation

The iPad is an awesome device, but the way we interact with isn’t really satisfying in many cases, it’s cold and we get almost no physical feedback from it. I wanted to use the capabilities of its touchscreen to do something more physical and engaging. My initial idea was to go for something weird and unexpected, but ended up being more minimalistic.

Mike Knuepfel‘s final thesis was a big reference  for me, and also the few projects that are being developed in this area. Especially the interactive toys and cards by the french company Les Editions Volumiques, and some (fake) hologram projects like N-3D or the Holodesk, all based on the old pepper’s ghost effect.

So I basically wanted to take the concept of mirrors, holograms and illusions from this projects and make them more interactive, make the iPad know their position, react to them and provoke some amusement.

Design

The objects I’m working on are based on simple geometric shapes. I’d like to be able to associate each of these shapes to one concept or behavior, in order to create a context for the toys that could, eventually, interact among themselves when placed on the iPad.

I’m very interested in the mirror as a material, so the aesthetics I’m currently following are minimalistic, mostly black and mirror , although I’m not sure about that. I might try something more warm, like paper, but there are many constrains regarding the need of conductive materials.

That’s, in fact, one of my main worries. I really want the object to be mysterious, so I need to hide the copper tape somehow. Good news is that it works even if you cover it with paper, but will it do with something like vinyl? Another option could be building it from metal, but it’s always harder to work with.

I also need to find the right hinge or mechanism for the cap, since I want it to be in two positions only: closed or at 45º. This is necessary for the periscope effect.

Interaction

The capacitive screen of the iPad can detect up to 11 touch points which dont necessarily need to be human, so the way my objects interact with it is by extending your touch through conductive materials. The initial prototype uses copper tape, but any conductive material can potentially be used (the anti-static foam that protects integrated circuits is another popular one).

So I started making a debugging app to test if points were properly recognized. As you can see in the pictures at the end of the post, some materials are not working as I expected, like conductive ink, or copper tape when the mirror is so close to the surface (I suspect its coating is a little conductive).

Also the screen is supposed to detect electrical inputs, and it does it when testing with the battery if you apply some pressure. But when trying coin batteries it didn’t seem very powerful and I couldn’t rely on that for this time. It’s something I want to keep researching, though, because I’d really like to make the objects independent of human touch.

Graphics and code

Both the debugging app and the demo are built with the OpenFrameworks C++ library, that makes simpler to deploy graphic applications to iOS. Recognizing the points of touch and drawing shapes on the screen is simple; the two complex tasks the app performs are the ones you don’t notice when using it:

• Shape recognition: I didn’t expect the algorithm to detect the triangles determining the position and orientation of the object would be so hard to code, so I actually made a lot of assumptions for the demo: there’s only one object, individual touches (fingers) have no use… But it you want to prevent all the possible errors (that is, building a real app) it can be a nightmare.
• Hologram display: Since I was playing with the see-through mirror, I wanted the objects to be 3D and look like holograms. I load some models using the assimp library (which doesn’t compile in the last version of XCode, btw) and display them, and this is no problem in my first demo because the angle of the mirror is 45º, but with any other orientation I’d have to distort the image by doing some heavy maths. This is not moving the OpenGL camera as I firstly thought, it’s rather doing something like anamorphosis or projective texture mapping. It’s, in fact, the same that some artists do on the sidewalks asdfdsf

My future objectives are improving the recognition algorithm, using the code base to create something more fun or visually interesting and trying new possibilities, like using the built-in camera (if I can get an iPad 2).

Name

And last but not least: I need a name for this project! Help me find it!

.

dasfadsfds.

PComp midterm is finished, with more or less successful results. The assignment was to create a media controller and our ideas were all around the graphic space: video, real paint, projections…

We finally decided to recreate a digital graffiti with a background video. Some sort of light painting effect. We wanted independent objects (cans) that people could use to draw on the screen with natural movements and no cables.

We acomplished that with color tracking from a webcam and bluetooth communication between a laptop and an Arduino with a hooked accelerometer, all inside the can.

Our intention was to have many (at least 2) cans painting at the same time, but we didn’t have enough time to develop it, so in future posts I plan to show it working, also with some more technical and visual documentation.

For this assignment I’m going to comment a piece of public technology that may be very common to new yorkers, but that I’d never seen before: the check-out line in Whole Foods Market. After taking notes on that I spontaneously ran across another one which was quite funny: the self-checkout machine in Home Depot, but I’ll leave that for the second chapter.

Whole Foods line

This supermarket has an interesting system to distribute lines of people before checking out. First you have to choose a block of lines depending on the amount of items you’re carrying. Then you chose one of the five short lines and wait until you are the first, as always.

Once in the front, you have to look at a screen that, when a cashier is free, tells its number to the front person in one of the five lines, following a left-to right order.

The system is quite complex and not very affordable. It requires a lot of attention from the you, specially if you’re new: first you have to read a sign to know what line to choose, then you keep checking at the screen to figure out how it works, then you pay even more atention to avoid missing your turn (and reading the number, it only stays there for some seconds) and once you know the number of the register you have to read another sign to find out which way to take. Finally you locate your register, with a number and a glowing light. A lot of steps for task that usually requires only one.

However, it seems to work well. I’ve been told that sometimes people gets confused and messes everything, but during my observation everything was really smooth. Registers were always busy and the distribution was fine, probably faster than the traditional way, but I still didn’t get the point of the technology. Wouldn’t it be easier to have just one line, so only one person has to look at the screen instead of five?

And that made me think: they want me to look at the screen. I mean, probably the main reason is related to the space (I’d have to go to other Whole Foods to know that), but I’m sure the purpose of that system is differentiation. Having this short straight lines with the fancy screen and the nice glowing numbers seems to go well with the phylosophy of a supermarket that claims to have more quality than the regular ones. And they are probably right because, as Norman says, sometimes beautiful things just work better.

Step 1: choose the line

Step 2: watch the screen (the color of your line!)

Step 3: go to the assigned register

Here’s the video of our first real assignment for Physical Computing: the Stupid Pet Trick!

My project is a low-fi DJ set made with a PC fan for the scratching turntable, a hacked walkman to play cassette tapes at the desired speed, and an Arduino to map these two. There are also some LEDs to show the speed of the song in another way.

My initial idea was to have the fan acting as input and output at the same time. I intended to have it rotating all time at the speed of the music (output) and let the user slow it down or speed it up to control the song (input). To achieve that, I wanted to read the rpm of the fan through its third wire, as explained in this tutorial, but I didn’t success in it.

I really don’t know the reason. Maybe the fan was broken, maybe I was doing something wrong, but more probably it was because the so-called hall effect only appears at high rpm, not the ones you can produce manually. So I had to think of another trick: use the motor as a generator. This way I can measure the amount of current generated by the manual rotation of the fan. It’s not as accurate, but it’s enough for this project. But be careful when doing this, since you don’t want to plug more than 5V to an Arduino pin! I tested and the voltage generated was really small, but for higher amounts a voltage divider should be used.

Finally, I’d like to comment one of the most delicate parts of the project: the mapping, which is not linear at all. First, the reading from the fan had to be smoothed because it’s really unstable, but not too much to avoid delays between action and reaction. Second, writing the right voltage to the walkman motor became hard because it changed depending on many factors:

• The minimum voltage needed to start the motor is higher than the one needed to run it when it already started.
• The amount of current affects the motor even more than the voltage, so analogWrite(150) behaves very differently when the Arduino is plugged to the computer, to a 9V battery or to a power adapter, even you are providing always the same voltage.
• At a certain voltages, the motor wasn’t powerful enough and produced a very annoying sound.

Finding lost objects has been my concern for me since I was a child. For this reason I designed this little invention for this PComp assignment.

I’ve always been attracted by analog interfaces: levers, gears, buttons, needles, cranks… so I thought that giving a new use to a well-known object would be more interesting and would create a more familiar interface than designing another digital gadget. The stamp tool (I’m not sure if this is its correct name in English) worked perfectly with the analogy of marking our objects, so I combined it with the compass to guide us to the lost object. Everybody already knows how this two devices work, so the new interface becomes quite self-explanatory.

The tool used to put some kind of stickers on the objects that we want to be able to locate later. The voice is used to label the objects, and may be the most confusing part of the interface, but using a screen with text didn’t fit in my idea, since it would make it much less minimalistic and take away the analog charming thing. Finally, the brightness of the lights indicates us the proximity of the item.

Those are the sensors I found on my way from home to NYU:

- pictures coming soon-

• Phone
  • Accelerometers
  • Proximity sensor to disable the screen when you are talking
  • Capacitive touch sensor
• iPod
  • Touch sensors
  • Accelerometer
• Many switches (are they considered sensors?)
  • Fridge door
  • Telephone booth, to know if the telephone is hang down
  • Intercom buttons
• CO2  sensor in fire alarms
• Air coinditioning thermostats
• Nyu Bookstore
  • Bar code readers
  • Magnetic sensors to prevent robbery
• Subway
  • Metrocard readers
  • Doors that open when they find an obstacle
• Magnetic sensors to prevent robbery
• doors that open when they find an obstacle
• Proximity sensors in restrooms
• Street surveillance cameras

First assignment for Physical Computing: turning on an LED with a switch.

To make it a little more interesting, I imagined a book that had its own lighting, so you wouldn’t  need a lamp to read it at night. I did a fast prototype of it, which you can see on the following video.

Of course the led should be much more brighter and the switch much more solid.

According to Chris Crawford, interaction is a cyclic process where two actors alternatively listen, think and speak. In other words, it’s a purposeful exchange of information. Then, what’s physical interaction?

Two opposite arguments come to my mind: the first one is that saying physical interaction is a redundancy, since everything is physical. Everything involves and obeys the laws of physics, whether it’s a conversation, a visual interface or light. It’s the same as just saying interaction.

The second one relates more to the common use of language. If we say physical interaction it’s because this is a subset of the whole meaning of interaction. So let’s look at the definition of physical:

▶adjective

• 1 relating to the body as opposed to the mind.
  ■ involving bodily contact or activity: a physical relationship.

• 2 relating to things perceived through the senses as opposed to the mind; tangible or concrete.

• 3 relating to physics or the operation of natural forces generally

While the last definition supports my first argument, I think what we all understand for physical interaction relates more to the first and second definitions: the interaction that involves the body, and more specially the body contact than the other senses -since then we’d call it visual or audio interaction.

With this in mind, it looks obvious that Talk To Me is not about physical interaction, nor even about interaction. It’s an exhibition about communication, a bigger field that covers the other two. There are many works in MoMA that contain a great load of information and storytelling, specially the ones related to data visualization -like the famous Feltron Annual Report- or critical design -Phantom Limb-. They are great examples of the newest uses of digital technology but are definitely non interactive, since their communication is unidirectional. This is clear in the first statement of the exhibition: “Talk to Me explores the communication between people and things”.

The complete explaination on that comes with the sentence “The exhibition focuses on objects that involve a direct interaction, such as interfaces, information systems, visualization design, and communication devices, and on projects that establish an emotional, sensual, or intellectual connection with their users”. I’d prefer not to try to understand what exactly is direct interaction, but I would like to add that I found this focus to be too wide and to become the major flaw of Talk to Me, as also is in many exhibitions of its kind.

While I absolutely loved most of the individual works, I think the magnitude of the topic exhibited didn’t help in communicating me a curating intention -although the categorization panels were really useful- but rather disoriented me, ending with the seensation of having visited an exhibition about state of the art design and technology projects, with very little in common among them. Fortunately, Talk To Me is complemented with a journal where we can dig deeper into every piece and hopefully comprehend better the purposes of the curators.