How Before Your Eyes Uses Better Cameras To Bring You To Tears On Mobile

Before Your Eyes is a game that doesn’t necessarily require, but is absolutely improved by, its blinking mechanic. The game uses a camera aimed at your face, and every time you blink, the story progresses–whether you want it to or not. Andrew King gave the game an 8/10 in GameSpot’s Before Your Eyes review, highlighting its emotional story and unique mechanic, but that unique mechanic also limited its reach and who could play it.

“There was a lot of virality and so many people talking about the game and watching the game,” game director Oliver Lewin said in an interview with GameSpot. “But there was an accessibility issue because it was exclusively on PC. There was the webcam element.”

To try and remedy that shortcoming and make Before Your Eyes available to as many players as possible, developer GoodbyeWorld Games (with the help of Netflix, its publisher Skybound, and mobile developer, BKOM) have brought the game to the platform with the largest audience that also happens to have a camera: mobile phones. We spoke with Lewin and creative director Graham Parkes about how this version of the game came to exist, whether it works better using the high-quality cameras on most mobile devices, and what they think of eye-tracking technology in virtual reality devices like the upcoming PlayStation VR 2.

GameSpot: How does the game functionally work on a mobile device? Do you hold it sideways? Do you play like you’re taking a selfie?

Oliver Lewin: To the last part of your question, one thing that we knew from the outset is we really wanted to have it be playable horizontally and vertically. So it could be like taking a selfie in that very comfortable way in your hand, or if you want more of that cinematic landscape-type picture, which is more similar to the PC version of the game, you can do that, too. That was a goal right from the outset and we’re really happy that we were able to do that because a lot of mobile games don’t do that, where you can just switch back and forth. That’s something that we wanted to take advantage of about phones.

And then, in terms of the core mechanic of the blink detection and using that to pass time in the game, that’s very much the same. Of course, phones have very high-quality front-facing cameras in them now. So that’s why we saw it as a perfect home for the game from very early on in development.

Graham Parkes: We always really wanted to come to mobile. We always felt like the use case was just so perfect because it’s the one place where face detection really does have a use value with iPhone filters and things like that. The face detection on just any iPhone is really sophisticated. We always just knew that it was such a natural fit.

I do think that from all the data that we’ve got back so far–and just from us with our own testing on our end as a team–that the blink detection on the phone is better by quite a bit than on the PC. So it’s just a really natural fit. It’s a really natural home for it just because that one-to-oneness of the blink is really felt in phones.

We were so concerned about things like, “Well, it’ll lose tracking if you’re moving your hand and stuff.” But again, that stuff has been so optimized for the phone that there really aren’t any issues with it. So in terms of just like a technical experience and a smooth experience, I think that the phone version might be the best way to play.

And it’s all the result of the improved cameras of mobile devices?

Oliver Lewin: Yeah, I think that’s one part of it where there is more consistency of high-quality cameras, whereas with PCs, there’s a huge variety of webcam hardware. There’s also an environmental factor of where you’re playing a mobile game. Your arm isn’t that long. You’re getting a pretty good image of your face. Whereas with PCs, there are so many environmental factors just because that webcam could be far away, and then maybe you like to be in a cave-type environment with no lights on in your room. That’s quite common and that leads to a less optimal experience. So yeah, I think it’s both the hardware and also environmental things that lead to the webcam experience or the blink detection experience on a phone being better.

So you’re allowing people to play in a dark room by themselves so no one can see them crying?

Oliver Lewin: Everybody should be able to play things that way, I think.

Graham Parkes: You do still need some light on your face. But I will say, it’s also the consistency of cameras. We partnered with a studio called BKOM on the mobile release. We’ve been overseeing it, but they’re real experts in mobile development and they’ve been really great. But one of the things that’s much easier is that there’s a lot less hardware you’re primarily making it for. You can predict most people are going to be playing it on an Android or an iPhone.

When we were developing it for PC, there were just so many different webcams. Some people have a really old webcam that has really bad definition, new webcams have different weird security issues. Just that amount of hardware we had to test for in QA (quality assurance) with the phone has been a much more predictable process and so it could be optimized a lot better, I think.

Oliver Lewin: The diversity of hardware and the issues that arise from that with regards to mobile devices–it’s not in terms of the camera quality, because camera quality has been quite good for a minute with phones. I think it’s moreso kind of the different shapes and sizes of phone screens that present challenges from a development standpoint.

You mentioned face tracking. Are you taking advantage of that?

Graham Parkes: The software can do all that stuff. Early on, we considered for a while making it detect if you’re smiling. Just the basic eye detection plugin that we use, you’ll see, can do a lot of sophisticated stuff. It can tell if you’re smiling or if you’re frowning. It can do things like nodding or shaking your head. Early on we considered doing that stuff because, “Oh, are we doing this face detection game?” But we ended up focusing on the blink and the metaphor of life flashing before your eyes and really focused on that. Adding those other things were going to start to feel like gimmicks and that we were adding them just to add them. Early on in development, we were like, “Okay, we’re going to leave all that on the table.” Obviously there is that point in the game where it detects if you’re holding your eyes closed and that for us felt like a really natural evolution of the mechanic. But other devs should be aware of this. We sometimes call it camera-as-controller, which is like the face detection software that has been developed and it’s really sophisticated and tracking blinks is only one thing that it can do. It can really do so much. Just a traditional webcam really can function as a totally new way to control games.

A lot of the reason that we made this game in the first place was our third lead, the creator of the game, realizing that the functionality was there and that developers really weren’t doing much with it. So we decided to just leave it to the blink, but I think there are a lot of other ways you can use the camera as a controller with where the tech is at this point.

For the parts where you needed to mouse around the screen on PC–is that now swiping on the touchscreen? Is there controller compatibility?

Oliver Lewin: There’s a touchscreen element of the gameplay, which is the camera, the actual player camera. When you’re looking around in-game, that’s something you’re kind of swiping around. We felt that was the most comfortable, natural way to play it, and I think it worked out really well.

The controller compatibility thing is interesting. I was just at a shop the other day and I saw one of those controllers that people use for mobile gaming. I’ve never used one. I think the team at BKOM has tested it. That would work… remapping the camera controlling and the touchscreen to the joystick. That’d be an interesting way to play but I haven’t done it yet, myself.

Graham Parkes: In really early versions of Before Your Eyes, we had player movement where you can move around these little scenes. It was a little closer to games that inspired us like Firewatch that have player movement. Very quickly, through testing, we realized that with the blink stuff, and thinking about tracking your eyes and blinking out of scenes, that allowing the player that amount of control, they just end up getting lost and then blink out of a scene. It was too much for the player to have to deal with and so we ended up locking off the camera in these scenes, just as a natural design evolution of, how do you design a game around blink?

But we’ve come to see that one of the unintended rewards of that is that it can adapt to other platforms very easily. Like in mobile, for a first-person mobile game, I always find one of the big bummers is 3D movement if you don’t have a controller. Having to do twin-stick movement with the little joysticks in the corner of the screen just feels so unnatural.

But because our screen is locked off, that little drag to look around thing? That’s so natural–so immediate. It doesn’t feel at all finicky. So, it was a very quick translation to mobile. We had great partners in BKOM, but I think we all found that we got lucky with this one because some games, when adapting to mobile, there are 100 issues that suddenly arise. But I think our game, because of the simplicity of the design and that focus on the face, the blink detection–it really just sort of naturally fits on the platform.

How did the partnership with Netflix come about?

Graham Parkes: That was really brought on through Skybound, our publisher. We always wanted to do a mobile version, but I think where the marketplace is right now with mobile gaming, more premium narrative titles just don’t sell, and so you kind of have to make a deal. Apple Arcade obviously came out as one of the places where you could do that. It was always this thing where it was our goal to get it onto phones. We really wanted to, but in terms of just going up on the App Store without some sort of a partnership, the business just didn’t make sense.

We just got really lucky that Skybound had this relationship with Netflix and Netflix loves the game and Netflix happened to be rolling out all these mobile titles. All the games they’re doing right now are more critically acclaimed, more narrative titles, and bringing the mobile versions to their platform. We were just so over the moon that it ended up happening The timing just happened that they were looking for this content just after we had released. It was really just a kind of a fortuitous thing because we always wanted to do the mobile version.

Balance isn’t the right word–that’s a term used in game development often–but how do you playtest and balance a sad game? “This part’s not sad enough? We have to turn up the sad on this part. We’ve got to turn it down here.” How does that work?

Graham Parkes: That’s a great question.

Oliver Lewin: I think we never set out to be, “We need to see X amount of emotions.” At least, I’m sure Graham did in a way when he was writing. But in terms of the QA process and play testing and getting feedback, I think that you listen to players and there’s a lot of gratitude to them for expressing what emotions they do experience.

But there would be things where we would hear them and be like, “Well, we know that’s not really what we want to be hearing people say,” like frustration would come up a lot. There are bad words that you don’t want to hear too much. You don’t want to hear people say they’re confused too much. You don’t want to hear people say they’re frustrated too much.

And so I think we started to weed out the things that didn’t quite feel right and then you start to get pleasantly surprised by, like, “Oh, we saw a play tester cry.” Because of the pandemic, we had to video record a lot of our play testers instead of getting actually to be able to see them in person. You can kind of see on the screen like, “Oh, okay. Someone started crying at X point in the game,” and you just sort of see that happening more often and it’s good to note and it’s always a reinforcing data point.

Graham Parkes: Yeah, it’s an interesting thing. I think that to make a story work, it’s a very structural thing. And early on, early drafts of the script were much longer and they had a lot more to it and we did actually spend a lot of time just with the script. That was really something that I think was unique about our process.

You know, this started as a student project. It went in this little 10-minute form and won the Student Award at GDC (Game Developers Conference) at the IGF (Independent Games Festival), and from there it kind of became this Kickstarter. So we had this long time where no one was full-time on it. We were all nights and weekends, where we were just exploring this mechanic, making little miniature versions of it that went to festivals.

But then, in 2018, we got some real investment in the game where we were able to go a little more full-time. We didn’t have a full team at that point. It really was me, Will, the creator, and Oliver Lewin. We were the only guys left, really.

So we were able to start with just the three of us, with all that experience of having built stuff and figured out design issues and figure out how to make a game around blinking. We kind of were able to just go, “Okay, we don’t have to hire anybody else. We’re just going to work on the story and the script I’m going to write.” So I would go write, I’d bring it to Will and Oliver. They would respond to what I was writing and we all sort of figured out what the story was.

And we had, I think, four or five months where really no one was building anything. We were just working on story. I think that’s something that you would see a lot more in film development. But it’s very rare in games because you often have a full-tilt team. So you’re building vertical slices and the writing’s catching up and everything’s happening at once.

I think we got really lucky, where we were able to iterate, and test and balance the emotions of the story just in the script phase. Where I’m writing stuff, giving it to a Oliver Lewin and Will, then we’re bringing in other readers, just friends. Our producer at RYOT, Jake Sally, would read early drafts.

So by the time that we hired up a full team, the sort of emotional element of it… I remember Jake saying, “Oh, it made me cry.” The second time he read it or something like that. We kind of got that opportunity to basically have the emotions of it almost pretested just in script form and I think that’s a real benefit that we had that a lot of teams don’t have….

Oliver Lewin: That’s making me remember too. I think it’s so interesting because I remember when it was just a script. I think, Graham, you were paying a lot of attention to not actually making it too sad because it’s a hard thing do a “sad story.” I think that actually there was more conversation about keeping it subtle in places and keeping it true to your own experiences and places, because a year later, when you start getting actors into the booth, and you’re putting music in there and you’re putting VFX in there, that adds a ton of emotion to it and it starts to get really sad. But for a long time, I think the focus was more so on keeping the subtlety there.

Graham Parkes: Well, once we figured out where the story was going to go, we were very cognizant of not wanting to do the Hallmark version of that and the weepy version of it that’s just trying to pry tears out of your eyes. I think it’s that thing of like, it’s always more emotional to watch an actor fighting back tears than sobbing. There’s something when there is a little level of restraint there.

So we were always trying things like really having fun in that happy childhood with the cats, and playing piano, and making art and falling in love. We really wanted to make all the parts that weren’t sad really, really feel happy and joyous and feel really, really pleasurable and sumptuous. So that, when you got to the sad part, you weren’t already exhausted.

Do you see yourself continuing to develop games with unique control schemes? Or do you think you will make something more traditional in the future?

Oliver Lewin: We do see ourselves continuing on with exploring the alternative cameras and controller type stuff. We’re still really excited about that. We feel like there’s a lot of meat left on the bone. I think we also consider ourselves capable of doing more traditional controller input, too, though. But right now, we’re still very excited about what we started with Before Your Eyes.

Graham Parkes: We’re huge game fans and it feels like there are so many amazing teams that are kind of honing the tradition. You look at Supergiant or something like that, and it just feels like, as a massive fan of that studio, they are honing a traditional gameplay format to such an incredible degree. Just the level of polish and just the level of refinement that’s going on in the space with a lot of games right now is so exciting. But what I see less of is that breadth of game, I feel like you saw that a little more even five years ago in the indie space, looking back at the Wii and the PS2, where there was just a little more of this energy of “Let’s try to use all parts of the buffalo and let’s try something really, really weird and experimental and see if it works.” I think we’re that spirit. It feels like we kind of lucked into a position where we’ve had a success that came from that place of being very experimental and it feels like that’s the kind of spot we can fill.

Again, because there’s so much incredible work going on right now, but I think that we’re maybe trying to be the team that’s willing to do the crazy idea no one else would think would work. So we want to keep with that. We want to keep being a little daredevil in what we try to do.

Oliver Lewin: I think there’s a good cocktail there too of something that’s very experimental and very emotional. I think, as emo game lovers, that’s what we get excited about. It’s something we don’t see as much of out there that feels like a really natural space for us to occupy.

What do you think of PlayStation VR 2’s eye-tracking tech?

Oliver Lewin: I think it’s so exciting. The moment I heard about it, a light bulb went off and I thought, “I need to talk to some of those Sony people.”

Graham Parkes: We’re big Sony fans. So when we heard, it was exciting news to us. Again, another really smart thing about [studio director and creator William Hellwarth], when he came up with this concept, was realizing that eye-tracking wasn’t going anywhere. This was eight years ago. But he already saw into the future with eye-tracking. It’s on phones, it’s on traditional webcams, and the future of VR will have eye-tracking. Even at that point, there were these conversations. Also always a little bit of a back-of-our-mind intention, in the way that we designed the game, was, “Oh, when that happens, this is going to be an amazing experience.” We actually did build something in VR for South by Southwest [Festival in Austin, Texas] in 2020. We were going to be bringing a 10-minute demo of the game to South By Southwest that we made for the Vive Pro Eye, which was their eye-tracking headset. We made this 10- to 15-minute version of Before Your Eyes. The eye-tracking within a headset is really perfect because the conditions are all there. We don’t have to deal with stuff like lighting position. All the stuff that makes eye detection hard on a computer, it’s easier on mobile, it’s even easier still on eye-tracking VR. And when we built it, it was so natural and so magical.

Obviously, though, that South By Southwest was canceled, so we ended up not getting to take it and show it and then we went down this other road and got Skybound as a publisher. We are really excited about VR and eye-tracking and we do have plans to develop in that space.

Oliver Lewin: I think like Graham said, there’s that environmental, fixed-variable thing, which is great to have the camera just always the same distance from the eyes, in terms of VR. There’s also just something exciting about it, which is that we are really enthusiastic about seeing more new and alternative input controls come into the space. So whenever we see, especially, some of the bigger companies and organizations getting into that, it’s exciting to us because ultimately we feel that expanding what can be a controller is really good for gaming at large.

The other big advantage of wearing a headset is no one can see you cry because your face is covered by the headset.

Oliver Lewin: Exactly, exactly. That’s why I wear sunglasses most days.

Graham Parkes: It was so funny. We were developing that VR version for South By Southwest and it was right at 2020. That South By Southwest canceling was one of the big canaries in the coal mine of COVID. But I remember the weeks before we were continuing to develop and kind of rushing to get it ready and COVID was starting to be talked about, and we were going, “Jesus, we’re going to be at this event and expecting people to put on a headset.” It was such a weird moment to be developing something that was going to be shown in a public setting. It was that moment where everyone was checking the news every day and we’re like, “Well, it’s not canceled yet. So we’ve got to keep developing.” And then we basically got a really great, polished thing ready and it felt like the next day, basically, we got the news.

At that time, and not until recently, building something for eye-tracking VR only made any sense if you were doing it in a festival environment because nobody had the Vive Pro Eye. It was very rare to have any headset with eyetracking. It was this really sad thing where we’re like, “Oh, we’re going to get to go to this festival and show off how cool this is.” Then the idea of festivals disappeared for a while, basically. Now suddenly, with PSVR 2 and I’m sure whatever Oculus is doing next, I think it’s going to not just be a thing that you can do at festivals. I think it’s going to be the future of consumer VR. They will all have eye-tracking, is the assumption.

Oliver Lewin: I’m laughing because I just remembered that when we were in that awkward phase of being like, “Are we going to cancel the South By Southwest thing?” But I’m quite happy that there’s a scene in the game where you’re in a doctor’s office and we were thinking it’ll be funny if the doctor said, “You know what? You’re sick. You shouldn’t have put on that VR helmet.” We didn’t do that. Some of our ideas are really stupid.

Before Your Eyes is available now on mobile devices via Netflix. For more on Netflix’s efforts in the mobile space, you can read GameSpot’s Poinpy review, as well our recently updated Into the Breach review, which covers all the recently added content and the mobile version.