Smart spaces and learning from Dynamicland

What is a smart space, and what does it let you do? Britelite Immersive has been creating immersive experiences for several years, and we’ve noticed an interesting trend in our work: the value of linking together multiple installations to create an entire environment that responds to users, a “smart space”. Taken together, they become the ultimate immersive experience.

A smart space is an environment equipped with sensors that let it understand what users visitors are doing, and ideally also has display and interaction surfaces that can respond intelligently.

In an ideal scenario, smart spaces let you reduce or eliminate friction from everyday tasks, enabling collaboration, allowing users to make their way around, find missing objects, make purchases without stopping at a cashier, and sometimes just having a lot of fun.

In this blog we’ll review some examples of smart spaces, what we’ve learned from them and how to succeed in building one for your organization.

Smart Spaces in use

We are fortunate to be located in a region where multiple different smart spaces are being tried out, and it has been very interesting to see their development. Here are some examples that we look to for inspiration.

Dynamicland

Dynamicland is an experimental space in downtown Oakland, where an entire building has been equipped with computer vision cameras and video projectors on most surfaces.

The goal is to build a space that enables collaboration in software development. Objects throughout the space are tagged with a system of colored dots that allows them to be precisely located. Projectors can then display information on these objects, regardless of where they are located. Objects can be connected just by placing them near each other.

For example, you might have a sheet of paper that is linked to a program that runs an animation. If you want to change the program, pick up a keyboard and point it at the paper, at which point you see the underlying program code projected on the sheet, and can edit it.

If you want to show your work to a colleague, pick up a little card (an “input”) and place it next to your work. You can then place an “output” on the wall, and point it at another sheet which acts as a screen.

The fundamental model behind Dynamicland is that of linking physical objects in the space to software code, and being able to assemble complex programs just by moving parts around the space. Everything is visible to all users, so collaboration is made easier.

The infrastructure behind the scenes is very interesting. While there are of course separate computers in the space for sensing and processing, a user need never know about them. As far as they are concerned, code is attached to the various physical objects, and they can be moved freely anywhere in the space.

The sensing and projection infrastructure relies on a large number of low-cost webcams and video projectors. The key thing is to achieve near total coverage of the space. (BTW you’ll notice we feature a lot of pictures of equipment up on the ceiling - you’ll see more of this…)

On of the most interesting things about Dynamicland is that the software system that runs it, can be edited using its own tools! This wall is actually the real codebase for Dynamicland, and you can potentially pick up any of the sheets and edit them.

The experience of being at Dynamicland can be magical, as you discover what is possible. Even things as simple as a user manual can become a delight: for example, a 3 ring binder that when opened will show you live examples directly on the page.

Cashierless retail (Amazon Go and Standard Cognition)

One of the most advanced and visible applications of smart spaces is in the trend towards cashierless stores. You can enter a store, check in with a mobile phone app, pick up your merchandise, and walk out.

The most well-known of these is Amazon Go, a retail experiment from the online commerce giant. The mobile app is tightly integrated with the rest of their systems: you log in using your Amazon ID, and can use any payment methods already associated with it.

In the store, a dense network of cameras and sensors keeps track of people in the store and what they pick up. I counted at least 26 cameras in the picture below! Can you see them?

In spite of all the sensors in space, the only interaction with users is basically letting them out of the space, and sending the receipt to their mobile phone when they leave the store.

It seems like there would be much potential to leverage the information collected to do more than just bill for purchases. Perhaps displays in the store could be used to recommend other purchases and help customers find what they are looking for. This would turn the store into more of a smart space.

Another competitor in this space is Standard Cognition, which uses a slightly different approach for sensing in the space, one that relies on fewer cameras.

These cashierless experiences can be delightful. Being able to just pick up merchandise, walk out without waiting in line and then have an accurate receipt turn up is a surprising experience.

Entertainment

A key area where smart space technology can be used to create magic is in entertainment venues. OneDome is an experiment, a first try at a new kind of entertainment venue that uses sensors and interactivity to create magical experiences.

The Unreal Garden, upstairs, uses Augmented Reality headsets to create a virtual gallery experience. LMNL downstairs uses a dense network of sensors and projectors to create a series of environments that respond playfully to visitor interactions.

As with Dynamicland and Amazon Go, their ceiling is full of stuff! There are projectors and depth cameras (in this case Kinect) throughout the space.

One thing that is missing at OneDome is an infrastructure connecting all the sensors and projectors, in order to create a single unified experience, a smart space. Each of the various installations operates independently.

However, the experience of being in a space that responds to your every move in surprising ways can be extremely rewarding.

Building Smart Spaces

At Britelite, we have been working on systems and tools that allow our projects to become smart spaces, and let our customers achieve things that would otherwise be impossible.

Integrated events

At the opening gala for the new Sutter CPMC hospital on Van Ness, Britelite provided 22 interactive installations, all of them activated by a smart wearable bracelet given to attendees.

These bracelets didn’t just activate specific installations: they were actually tracked by a network of ambient sensors that would identify a guest, and send their identifying information to an installation. Guests would be greeted by name, and wayfinding systems would direct them to areas that they had expressed interest in. In the future, we could actively manage congestion by directing guests to installations that don’t have any waiting time. (Or to bars with short lines!)

Guests at the event were delighted to be greeted by name by the wayfinding screens and interactive installations, and were also pleased by how the system had registered their interests and guided them to parts of the event that they would find appealing.

In addition, access to the entire set of interactions allowed very detailed analytics about the event to be produced almost in realtime, giving sponsors and producers accurate information about the success of the event.

Integrated installations

Britelite is increasingly focused on building large permanent installations. For Amgen, one of our clients, we have built several immersive spaces in their entrance lobbies. These include multiple installations, which are all tied together to create a coherent experience.

For example, groups of visitors can be greeted across all surfaces with a customer welcome message. Special messages can “take over” all the screens in the space for announcements and news.

As users interact with some of the interactive displays, the content on other screens in the space changes to match. A user at a touchscreen can easily “send” a video to a larger screen so that other visitors can see it. This is implemented by having a single integrated infrastructure system that manages communications amongst the installations.

What it takes to succeed in building smart spaces

Thinking of a corporate lobby, a marketing activation, a retail space (and many others) as a smart space opens up a world of possibilities. User interactions can be made more appealing and frictionless, and it is possible to use information gained this way to improve the quality of an organization’s relationship with customers.

By studying Dynamicland and other smart spaces, and studying the results from our own experiences, we’ve identified three factors that are critical to success in smart spaces:

• Sensing users and things: One of the foundations of a smart space is being able to accurately understand where people are in the space, and the things they interact with. We’ve worked with numerous different sensing systems, including video cameras, depth cameras, iBeacon, RFID and smart wearables. It is important to understand that these are ambient sensors, providing a sense of interaction in the entire space and not just a single installation. This is a key learning from Dynamicland.

• Compelling display and interaction: Once you know where a user is and what they are doing, it is important to support various ways of interacting with them. This can include video projection, touch screens, gesture detection, audio cues and physical responses. Moreover, these installations should be built in a way that they can easily react to external cues, and updated with new content. We have ample experience with many of the techniques used at experiences like OneDome.

• Infrastructure to link it all together: Finally, what transforms a collection of installations into a true smart space is linking everything together. Britelite has developed software infrastructure for managing deployment, messaging and content across multiple applications. This can also become the point of connection to other enterprise systems, such as payment, customer management, etc. This has become a robust platform for managing interactions at the scale of a smart space, once again much like the Dynamicland platform.

Here’s a diagram outlining these components. All of these are present in the examples described, including the Britelite projects.

A key factor that must be mentioned is issues around privacy and surveillance. The amount of information that can be gathered might make some people uncomfortable. It is important to be up-front about the presence of sensors in a space, to take steps to protect this information, and limit the scope to what is absolutely necessary (only during a certain time, or within a certain area).

If you are interested in building an immersive experience for your organization that goes beyond the demo, and actually creates a true smart space, please look at our portfolio of work, and contact us.