James is a passionate and ambitious game designer with eight years of industry experience across AAA and indie games covering a variety of genres, including strategy, platformer, rhythm, and more. At 草莓视频在线, James brings his keen eye for design and collaborative spirit to several projects, many of which are still closely guarded secrets!

Tell us more about your role at 草莓视频在线

I鈥檓 a designer at 草莓视频在线, with a background in game design. My job can differ day-to-day, but generally, I鈥檓 involved with the conception, design, and implementation of certain aspects of the interactive experiences we make. I鈥檝e been involved with several different projects, but the one I can talk about, and am very proud to have had a hand in, is Metallica: Fuel. Fire. Fury. 鈥� a Fortnite concert we created for Epic Games, which you can read more about here!

What made you decide to pursue a career in this field?

Like many people who end up working in games and interactive media, I chose to pursue this career because I love playing games!

I have such vivid, cherished memories of sitting with my dad on the living room floor, playing Hogs of War together, laughing hysterically, and teaming up to try to beat the 鈥渃heating computer,鈥� which always managed to land every shot perfectly. Bathed in the dim light of a television that was so chunky, if it fell forward, it might have ended my career before it even began. Thank goodness my mum was always close by!

What piece of advice would you give to someone wanting to pursue your career?

Start making games now. They don鈥檛 have to be pretty, they don鈥檛 have to be complicated. Heck, they don鈥檛 even have to be good (and trust me, the first few definitely won鈥檛 be). But please, just start making games.

Developing games in your spare time is a great way to find out if you actually enjoy making them or if you just enjoy playing them. And if you find that you do enjoy making games, it鈥檒l help you figure out which parts of the development process you enjoy most, such as art, design, engineering, etc.

Also, if you鈥檙e reading this and the first thing that comes to mind is that it鈥檚 already too late for you, then you鈥檙e wrong! With all the free software, online tutorials, game development communities, and more, it has never been easier or more accessible to get into making games. So please, from one game dev to another potential future game dev; just start making games, and good luck!

A little birdie told us you鈥檝e self-funded, designed, and published two of your very own games. Tell us about that process, and something you learned while doing it.

I moved back in with my parents, sold my car, and then developed a game almost entirely on my own 鈥� just to prove to myself I had what it takes to be my own boss. Once that game was made, I set up my indie studio, including all the wonderful paperwork involved with tax, bank accounts, insurance, etc., and then published my game for free.

Not long after, I began conceptualising the next game, and soon after that, I began scouting for all the people I鈥檇 need to help me create it 鈥� artists, sound designers, musicians, etc. Where we didn鈥檛 have the budget, or if it was an area I felt at least semi-confident tackling, I filled in the gaps.

I spent my days making games, and my nights working in pubs. That then became my days working in restaurants, and my nights making games. Before I knew it, a couple of years had passed, and we鈥檇 made and published another game!

Shortly after, I ended up working full-time for another game studio, and because of the commitment required, I decided to close my own. Don't worry 鈥� I was the only employee. Everyone I鈥檇 worked with previously was a contractor, and their work was already complete. If they had been employees, it would have been a very different situation.

All in all, the entire experience taught me so much, but for the sake of time, I鈥檓 going to share what was perhaps the most important thing that I learned: you must take care of yourself. I was working two full-time jobs, one making games and one tending bars/waiting tables, and all that came of it was a bad case of burnout. So please, if you鈥檙e thinking about making games, put your health and your financial stability first, and remember, it鈥檚 not a race.

What鈥檚 your favourite thing to do when you鈥檙e not working?

This is going to sound so mushy, but, hand-on-heart, my absolute favourite thing to do when I鈥檓 not working is just spending time with my best friend, Katie 鈥� my wife. We could be doing something as exciting as white water rafting, or something as mundane as folding laundry, but as long as we鈥檙e together, we never stop laughing. Our favourite thing to do together is travel鈥nd play co-op games.

If you were on a game show, what would be your specialist subject?

Probably SpongeBob SquarePants quotes. I don鈥檛 even know how you could fashion that into a game show topic, but I know that if you could, I鈥檇 be high up on that leaderboard, without a doubt.

If you had unlimited resources and funding, what project or initiative would you launch?

I would love to start an Indie Game 草莓视频在线 Incubator: an organization that provides new and upcoming indie developers with resources, support, funding, etc.

So many of the best games I鈥檝e played over the past decade have been indie titles. Often titles that failed to garner funding from mainstream publishers because their work was seen as 鈥渢aking too many risks鈥�, ultimately culminating in them self-funding, crowdfunding, or, for the lucky few, pairing with a smaller publisher who was willing to take a chance on them.

Those are the people I want to see flourish and succeed, and if I can help them do that, I will. Because what the world needs now, more than ever, are more games that are just plain weird and unique. Here鈥檚 looking at you, .

How do you approach challenges and setbacks?

I take a step back.

For challenges, it allows me to break down a seemingly insurmountable task into smaller, more manageable, actionable steps to achieving the whole. Remember: any big, complicated thing that does a lot is just a combination of several little things that do one particular thing.

For setbacks, it allows me to frame the setback against the whole and see how serious it really is. More often than not, it鈥檚 far worse in my head than it actually is, helping me to dive right back in. And worst case scenario 鈥� that it is as bad as it seems 鈥� it helps me plan and prioritise the solution, taking into account any other work or features that the setback may affect.

At the end of the day, it鈥檚 important to remember that nothing鈥檚 as bad as it seems, and nothing鈥檚 impossible.

If you could invite three guests (dead or alive) to a dinner party, who would you invite?

Easy: Steve Buscemi, Will Ferrell, and Robin Williams.

Why? Because, beyond their numerous influential acting roles and beyond their incredibly interesting personal lives (which, don鈥檛 get me wrong, I absolutely want to dig into with them), they just seem like nice people.

A highlight of working at 草莓视频在线 is the dedicated time and space engineers receive for R&D and experimentation (read more about that here). This allows us to use our creativity and explore ideas in focused sessions outside our day-to-day work. The following is the journey of a recent R&D session of mine.

The idea

In recent months, you may have seen several articles and social media posts about "MCP servers" (a type of server allowing AI tools to access external systems) and connecting "AI assistants" to applications to automate tasks using Large Language Models (LLMs), which are powerful AI systems trained on vast amounts of data to predict and generate human-like text. Looking at these examples of integrations and the cool things people could do with them, I wanted to explore what integrating with an LLM could do for our platform, OKO, and its users in the future.

Why could this be useful?

One of our aims with OKO is to make 3D experiences available to everyone. This is especially true of the web version, which you can use by simply opening in a web browser. No installation needed!

We want to empower our users to create the cross-reality, connected spaces they imagine with easy-to-use tools, allowing them to express their creativity and share it with others. However, there will always be a learning curve when getting to grips with a new application. As simple as we try to make things, understanding the concepts behind the platform and how to use the tools takes time and can be daunting for some.

So, what if you could simply ask someone more experienced how to get started, or better yet, get them to work alongside you? Integrating an AI assistant into OKO would enable users to take their first steps, create a first draft of a space, or automate unwanted tasks. Beyond that, it creates pathways to improve accessibility for disabled or visually impaired users.

In addition, an AI assistant could also be useful when exploring a space. Imagine a virtual museum experience where an AI guide could take you on a personalised tour, answering your questions along the way. Applied responsibly, AI could unlock a whole new set of possibilities for those who wish to use it. The potential is there, and so begins our journey to a prototype! 

But first, let鈥檚 dive into what an MCP server is.

MCP and MCP servers

is an open-source standard created by Anthropic that enables AI assistants to connect with external data sources and tools in a standardised way. It offers a communication model that allows AI models to interact with various applications, data sources and services in a controlled manner.

The MCP architecture follows a client-server model where a client, for example, an AI assistant, can connect to one or more servers, exposing specific capabilities to the client. An MCP server is typically focused on a specific application or service, such as OKO.

In particular, an MCP server can expose a set of tools performing specific operations within an external system. These are the actions the AI assistant will be able to perform via the server 鈥� in our case, this will be creating or deleting objects inside OKO. 

A tool consists of three parts:

• A name and description giving the AI model some context about what the tool is for.

• A schema that unambiguously describes the data the tool accepts.

• A function with the code to carry out the operation.

Creating the prototype

This work was partly inspired by the efforts of the . We already use the PlayCanvas engine for the web version of OKO, so we're huge fans of their open-source tech and follow them closely.

To get started quickly with a prototype, I leveraged the infrastructure of PlayCanvas鈥� for our implementation. Remember, the goal was to get a prototype up and running, not to get bogged down in the details of getting the components to communicate.  

Happily, it turns out we can do something very similar. The PlayCanvas version consists of two main parts: the MCP server itself, which plugs into an LLM-powered desktop application (for example, Claude desktop), and a browser extension that runs commands within the editor via an API. These communicate over a WebSocket, effectively allowing Claude desktop to interact with the PlayCanvas editor.

For the OKO version, we can use the same architecture. We'll create:

• An OKO MCP server that Claude will connect to.

• A browser extension that will run commands using an OKO console API.

We don't currently have an OKO console API, so we'll need to create one to interface with the extension. The API is a set of commands we expose on the global window object in the browser that can be accessed within the browser JavaScript console. A console API will also be useful for advanced OKO users, which is another benefit of this work. 

OKO MCP server tools

As mentioned, an MCP server exposes a set of tools to the LLM-powered host application. We need to think about what tools would be useful for the prototype and could produce an interesting demo. 

An OKO space consists of entities and components, concepts that will be familiar to anyone who has worked with a game engine using an entity-component system. In short, an entity is an object that exists at a location in a space and is made of components that give the object an appearance and behaviours.

Entities and components are therefore key to allow the LLM to act within a space. To start with, we'll create the following tools:

• Query what entities and components are in the space.

• List what primitives are available to create in a space.

• Add a new entity to the space.

• Add a new model component to an entity.

Model components in OKO allow you to add a 3D model to an entity. In OKO, users can upload their own 3D models or search Sketchfab for suitable assets. For our prototype, we'll expose OKO's built-in primitive assets (sphere, cube, etc). This minimal set of tools should give us enough to prove the concept.

Creating an MCP server tool

To create a tool for our MCP server, we need to do a few things. Let's take creating a new entity in OKO as an example. An entity has the following properties:

• A name.

• A position, rotation, and scale in 3D space.

• A set of components belonging to the entity.

First, define the shape of the data we expect as a Zod schema. Zod () is a common schema validation library used for exactly this.

import { z } from "zod";

export const EntitySchema = z.object({
  name: z.string().optional().describe("The name of the entity."),
  position: Vec3Schema.describe(
    "The position of the entity in local space (x, y, z)."
),
  rotation: QuaternionSchema.describe(
    "The rotation of the entity in local space (qx, qy, qz, qw as a quaternion)."
  ),
  scale: Vec3Schema.describe(
    "The scale of the entity in local space (sx, sy, sz)."
  ),
  components: z
    .array(ComponentPropsTypesSchema)
    .describe("An array of components belonging to the entity")
});

The descriptions here are important as they will provide information to the LLM about what each property is for.

Secondly, we register a tool that uses our schema.

import { type McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { type WSS } from "../wss";
import { EntitySchema } from "./schemas/entity";

export const register = (server: McpServer, wss: WSS) => {
  server.tool(
    "create_entity",
    "Create a single entity",
    {
      entity: EntitySchema,
    },
    ({ entity }) => {
      return wss.call("entities:create_entity", entity);
    }
  );
};

Here, "create_entity" is our tool鈥檚 name, followed by a short description. This registers a tool that takes input data following our schema and creates a new entity in the space.

The last argument in the server.tool definition is a function that gets executed when the AI assistant invokes the tool. Here we are simply sending a command over a WebSocket with the entity creation data, following a custom protocol. On the other side, our browser extension will receive the command and invoke the appropriate call in OKO.

wsc.method("entities:create_entity", async (entityProps) => {
  console.log(`entities:create_entity`);

  const okoApi = window.okoConsoleApi;

  const entity = await okoApi.createEntity(entityProps);

  if (!entity) {
    return { error: "Failed to create entity" };
  }

  console.log(`Created entity ${entity.id}`);

  return { data: toJson(entity) };
});

This function will be called when the browser extension receives an "entities:create_entity" message. Here we retrieve the OKO console API from the global window object and call an associated function in the OKO console API. I mentioned earlier that we don't have one yet, so we'll also need to create that. Following a similar process, we can implement the remaining tools from our list.

Does it work?

To test our MCP server, we'll use Claude desktop. By using MCP, our prototype will be compatible with many AI assistants and tools, but Claude desktop is the most commonly used and easiest to set up.

Having first installed our browser extension, we open Claude desktop's config file (via File > Settings > Developer > Edit Config) and add our server.

{
  "mcpServers": {
    "oko": {
      "command": "cmd",
      "args": [
        "/c",
        "npx",
        "tsx",
        "C:\\path\\to\\mcp\\server.ts"
      ]
    }
  }
}

Success! We can see our server is running.

Let's try a few examples. First, we try with the prompt: "Create a 3x3 grid of cubes in OKO. Make them different colours".

It's working! You can see it鈥檚 adding each cube into the space one by one. An improvement here would be to implement a tool for batch-creating entities to speed up the process.

Let's try something a bit more ambitious: 鈥淐reate a level with platforms and ramps to jump on using primitive shapes. Add some decoration鈥�. Here鈥檚 what Claude created. We can even try it out!

Going even further: "Make a rendition of the Manhattan skyline using primitive shapes".

I suppose you can only go so far with primitive shapes, but you can see what Claude is trying to do here. Notice how it names all the objects in the space. These examples look quite basic, but we can start to see the possibilities. Not bad for a first go!

Learnings

With R&D and prototyping in general, it's important to focus on the actual creation rather than getting bogged down by time-consuming infrastructure setup or boilerplate work.

By learning from and adapting an existing project, we were able to reduce the setup and get straight into the interesting part: the OKO MCP server itself. Don't be afraid to lean on open-source and build on other projects to try out your idea!

That said, we did end up spending a chunk of time on creating a console API for OKO, which we didn't expect going into the project. In hindsight, perhaps there was a simpler way of achieving the same thing? That time spent also limited the number of tools we ended up creating for our prototype. This mini-project also shows the value of just jumping in sometimes. Do we understand all the ins and outs of MCP servers? No, but we have learnt a lot! By simply doing, we can get far, often further than we think.

What's next?

This made for an interesting prototype, but it remains a prototype. The setup is a little clunky with many moving parts, so we need to consider simplifying it and making it more user-friendly. It's not ideal to need a browser extension, for example, so we need to think of alternatives.

We've exposed only a very minimal subset of OKO's functionality in our MCP server. By exposing more tools, we believe we can exponentially expand possibilities and make it more compelling to use.

Finally, if we did proceed with this as a product feature, we'd need to consider integration. Ideally, we'd want an AI assistant directly inside the OKO UI. Imagine if we could directly chat with the AI assistant within OKO and ask it to create a first draft of a space, or help us figure out a problem we're having. This kind of direct, intelligent assistance would fundamentally transform how users build and explore, unlocking unparalleled creative freedom and efficiency for every user.

Mira Murphy is a management accountant at 草莓视频在线 UK. Originally from Sweden, Mira moved to the UK as a teenager and later earned a Bachelor鈥檚 degree in English Literature from the University of Hertfordshire. Her path to accounting was unconventional 鈥搘hile on maternity leave, she discovered a passion for finance through an entry-level accounting course. What started as a way to stay engaged quickly turned into a full qualification and a career!

Tell us more about your role at 草莓视频在线

I鈥檓 a management accountant on 草莓视频在线's UK side. Most of my time is spent in Excel, handling bills, preparing financial statements, creating reports, and making sure everyone gets paid at the end of the month!

What made you decide to pursue a career in this field?

It鈥檚 a bit of a funny story. Growing up, my passions were literature and creative writing, so a career in numbers never really crossed my mind. When my son was about four months old, I was bored and started looking for online courses. I found a basic accounting course, thinking it might help with admin jobs. I completed most of it while holding a sleeping baby, and by the end, I realized I wanted to go further and pursue a full qualification.

What three skills are essential for anyone in your role?

You don鈥檛 need to be a maths genius, but you should be comfortable with numbers and understand the basics behind Excel formulas. Confidentiality is also crucial because you handle sensitive information. Lastly, strong organizational skills are essential 鈥� there are many processes to follow, some of which are time-sensitive, and people tend to get upset if they don鈥檛 get paid!

What is the biggest lesson you鈥檝e learnt in your career?

If you are like me and have a tendency to tell yourself to keep quiet most of the time, don鈥檛! If you have ideas or opinions, speak up. Even if others are more senior or experienced, your perspective is valuable, and you might be surprised by how receptive are.

What鈥檚 your favourite thing to do when you鈥檙e not working? 

There are so many! I love spending time outside for a bit of "tree therapy," hanging out with my son, knitting, going to the gym (especially Zumba, even with two left feet), and occasionally diving into video games.

If you had unlimited resources and funding, what project or initiative would you launch?

If it were truly unlimited, I鈥檇 aim to solve global poverty. On a more practical level, I鈥檓 very conscious of how difficult it is for people with mental health problems to navigate systems like healthcare, work, benefits, and housing. I鈥檇 love to create an advocacy and advice service to help those people.

What song always puts you in a good mood?

I really love Leonard Cohen. It鈥檚 not exactly cheerful, but it鈥檚 so beautiful that I get completely lost in it.

How do you approach challenges and setbacks?

By panicking, generally! No, but seriously, I often find I overestimate the impact or severity of a challenge or setback, so I find it helpful to talk it through with someone, like a friend, who is not directly involved, to give me an outside perspective.

If you could have a superpower, what would it be?

My first instinct is healing powers, but the pressure of not being able to help everyone would be overwhelming. A more selfish choice? Teleportation 鈥� I hate the hassle of travelling, and public transport takes up way too much time.

Where is one place you want to travel to in the world?

I really want to go to Iceland. I鈥檇 like to go on a long excursion, riding Icelandic horses and bathing in the hot springs.

Over the years, I鈥檝e worked across various genres and platforms 鈥� from cozy exploration spaces to high-intensity combat arenas 鈥� and I鈥檝e developed a personal framework that helps me get started on any new level. I call it the SCOTT Method. Yes, it's named after me. At least it鈥檚 easy to remember. 

Level design shapes a player's journey through a space, balancing mechanics, pacing, and narrative in a way that feels intentional and reactive. It sits at the messy intersection of art, design, storytelling, and systems thinking. That鈥檚 why I created SCOTT: to bring some structure to an inherently chaotic and creative process.

It鈥檚 not a rigid checklist or a one-size-fits-all solution, but it helps me ask the right questions early on, stay focused on the player experience, and make better design decisions with the time and resources available. I鈥檝e found it especially useful when jumping between vastly different types of projects, which is something I get to do a lot of at 草莓视频在线 (one of the perks of working across such varied experiences).  

Let me walk you through what SCOTT stands for and how it鈥檚 shaped some of my recent work.

S  鈥�  Start somewhere

The blank page is always intimidating. It鈥檚 tempting to jump into the editor and start throwing geometry around, especially when you鈥檝e only got a Jira ticket and a deadline. But I鈥檝e learned it鈥檚 better to pause and zoom out.

Before I touch the editor, I try to answer a few key questions:

1. What鈥檚 the goal of this level?

2. What鈥檚 the player doing here?

3. What is the player meant to experience or feel? 

4. Who鈥檚 playing this, and on what platform?

By stopping and taking a second to answer the key questions, it gets you thinking early about potential communication with other departments and gets your creative wheels turning. During some R&D, I decided to apply the SCOTT Method to a custom project in the Unreal Editor for Fortnite (UEFN). Before I even started, I answered the above questions to define the project, which I named 鈥楢A Gun鈥�:

1. What鈥檚 the goal of this level?

   The player鈥檚 goal is to break through enemy defenses and destroy a nearby Anti-Air Gun, which serves as the final objective. It's a key tactical target in the mission, and completing this objective marks the end of the level.

2. What鈥檚 the player doing here?

   The player is navigating through a series of combat encounters, facing multiple enemy types with different tactics. They鈥檒l need to stay mobile, use cover wisely, and choose the right weapons to survive and progress. 

3. What鈥檚 the player meant to experience or feel? 

   The level empowers the player to feel heroic through skill-based progression. The player should feel challenged but capable, under pressure, yet able to overcome it through smart choices. The goal is to create a sense of excitement and achievement as the player pushes through tough encounters to progress. 

4. Who鈥檚 playing this, and on what platform?

   This is a solo experience designed for players of a third-person shooter using Fortnite's mechanics and style, likely on PC or console.

C  鈥�  Clarify your intent

Once I鈥檝e got a basic understanding of what the level needs to accomplish, I distill that into a single, clear statement. This becomes my North Star. If I can鈥檛 explain the level in one or two sentences, I鈥檓 probably not ready to build it yet.

For AA Gun, it was:
The player must break through enemy defences to destroy a nearby Anti-Air Gun, a critical target. The player will be challenged in multiple encounters with defenders who use different tactics against the player. The player should feel vulnerable going head-on and under pressure from enemies, but can overcome this using clever movement, appropriate cover, and smart weapon choices.

These statements help align every decision I make 鈥� whether it鈥檚 layout, pacing, art direction, or interactions 鈥� with the level鈥檚 core purpose.

O  鈥�  Oh god! Hasn鈥檛 someone already done this?

Every designer pulls from references, and there鈥檚 no shame in that. What matters is understanding why something resonates. I always try to gather a wide range of inspiration 鈥� games, movies, real-world locations 鈥� and break them down into useful components.

Relevant references can include:

• Anything with a similar narrative context, art style or environment type.

• Anything with a similar style of gameplay.

• Anything that evokes an emotion you are trying to replicate in your level design.

• Anything that has something cool happen in it that you鈥檇 like to include in your design!
  
  

For AA Gun, I took cues from:

• The gritty, industrial tone of Gears of War

• The pressure-cooker pacing of Modern Warfare

• The simple but engaging gameplay of Halo.  

I also keep a mental list of what not to do. Knowing what to avoid is just as useful as knowing what to chase.

T  鈥�  Try it

At some point, you鈥檝e got to stop planning and just build. Rapid prototyping is essential. I get basic geometry in the engine as quickly as possible so I can walk the space and feel it.

With AA Gun, I learned to prototype combat sequences first 鈥� using placeholder enemies and cover objects to dial in pacing and difficulty 鈥� before fleshing out the environment around them. I separated the combat into three very different arenas.

• The first needed strategic thinking, with a special enemy type, and lots of cover to provide choice.

• The second was a maze of shipping containers, containing powerful close-range enemies, little player control over their path, and high tension.

• The final section was a sniping encounter, where the player is under pressure from multiple enemies, but can still overpower them.

These were all inspired by various sources, but it was through building, testing, changing, and ultimately shaping them together within the level that they truly became my own.  

Trying things early, failing fast, and iterating often is the only way to uncover what actually works.

T  鈥�  Team up

Finally, level design doesn鈥檛 happen in a vacuum. The sooner I bring others into the process, the better the result.

Whether it鈥檚 grabbing a gameplay engineer to sanity-check an idea, asking another designer to run through a blockout, or watching someone playtest without giving them any context, it always surfaces blind spots I missed.

At 草莓视频在线, I鈥檓 lucky to work with a collaborative team that values early feedback. That kind of openness is essential, especially when building ambitious projects on tight timelines.

There were four key bits of feedback from my prototypes:

1. The player attacks and pushes through enemy positions, and gets to destroy the Anti-Air Gun, which meets the project objective.

2. The player is challenged across multiple encounters, but not to the desired extent. Enemies don鈥檛 feel threatening. 

3. The player is only under minor pressure if not using cover or skilful movement during encounters.

4. There are a few different weapons in the level, but not really enough to make the player think about their choices. Adding a bit more variety would help make weapon selection feel smarter and more rewarding.

Why it stuck

The SCOTT Method isn鈥檛 some revolutionary technique, it鈥檚 a simple framework that helps me keep my head straight during the early, often messy stages of level design. It encourages clarity, experimentation, and collaboration, which are all vital no matter what you鈥檙e building.

So if you鈥檙e ever staring at a blank editor window, unsure where to begin, maybe give SCOTT a try鈥� start somewhere.

These terms aren't interchangeable, and mixing them up can create confusion, especially when you're aiming to build something that genuinely bridges the digital and physical worlds. A static digital copy, a predictive simulation, and a responsive digital twin all serve different purposes, have different behaviors, and understanding the difference isn鈥檛 just a technical detail 鈥� it鈥檚 key to setting the right expectations, choosing the right tools, and building something that solves the problems you need it to.

In this article, I鈥檒l break it down simply: what each one actually is, where they鈥檙e useful, and why getting this right matters as we work toward a more connected, spatial web.

What鈥檚 a digital copy?

A digital copy is essentially a static replica 鈥� a one-off snapshot of a physical object or environment. Once created, it doesn鈥檛 update or interact with the real world. It鈥檚 perfect for things like showcasing a space, documenting how something looked at a certain point in time, or creating a visual for a pitch.

Think of it like the shopping mall directory map you see when you walk in the door: useful, but only accurate at the time it was printed. If shops move, close, or open, it quickly becomes out of date unless manually updated.

Or, imagine you鈥檙e creating a virtual tour of a museum. You鈥檇 take 3D scans of the exhibits and put them together into a polished virtual space. That virtual tour is a digital copy 鈥� it reflects the exhibits as they were, but unless you update the tour, it won鈥檛 change as new exhibits are added or old ones are moved around.

When you don鈥檛 need real-time data or live interaction, digital copies are very useful. They鈥檙e easier to produce, less complex, and serve a purpose in everything from marketing materials to archival documentation.

What鈥檚 a digital simulation? 

A digital simulation is all about behavior. Instead of just showing what something looks like, it also shows how it acts 鈥� under different conditions, over time, or in response to change. It鈥檚 not necessarily linked to the real world, but it helps you understand how something might perform.

Imagine you鈥檙e designing a new lighting system for a building. A digital copy could show where the fixtures are placed, but a simulation would let you test brightness levels, energy usage, and how natural light changes throughout the day. It鈥檚 like running a dress rehearsal for your physical system or environment, so you can spot issues, test ideas, and make improvements.

Or think about traffic planning. You could simulate how cars would flow through a new intersection design, try out different light timings, and even model rush hour conditions, all without ever pouring concrete.

Digital simulations are powerful tools for prototyping, testing, and decision-making. While they don鈥檛 always need real-world data, they can become even more valuable when connected to live inputs. And that鈥檚 where they start to blend into the world of digital twins.

What鈥檚 a digital twin?

A digital twin is a living, breathing version of whatever it鈥檚 mirroring. It鈥檚 always on, always updating, and always in sync with the real world. 

Whether it's a building, a machine, a city, or even an event, a digital twin monitors real-time data from sensors, IoT devices, and other inputs to reflect the state of its real-world counterpart.

With this data, you can not only allow digital and physical users to be truly copresent with one another, but also optimize performance, predict maintenance and make other improvements, whether it's reducing energy use in a building, managing crowd flow at a concert, improving production efficiency in a factory, or fine-tuning the operation of a machine. It鈥檚 all about making things work smarter, more efficiently, and be more accessible remotely in real-time.

Digital twins can be closed or open

Not all digital twins are created equal 鈥� and one of the biggest differences is whether they鈥檙e open or closed.

A closed digital twin locks you into a single ecosystem. It might work well 鈥� as long as everything else you use comes from the same provider. But the moment you need to connect to another system, tool, or platform you鈥檙e stuck doing extra work to make things fit, often with limited flexibility or expensive custom solutions.

On the other hand, an open digital twin is designed to integrate. It can exchange data across platforms, work with your existing tools, and evolve as your systems do 鈥� without forcing you into a specific stack.

At 草莓视频在线, we build interoperable digital twins that are open by default. Our tech is built from the ground up to integrate with all systems, with minimal effort. No walled gardens, just open, platform agnostic technology that will work with however you work best. This means teams can work with the tools they already use, avoid duplication, and adapt systems over time as needs evolve.

Why does this matter?

Choosing the right digital approach can have a direct impact on business efficiency, resilience, and growth. The table below compares a digital copy, a simulation, a closed digital twin, and an open digital twin. Each supports goals in different ways, from basic data access to system-wide optimization and collaboration.

Where our technology fits in

Our OKO platform and Connected Spaces Platform (CSP) provide the underlying infrastructure to support persistent, interoperable real-time digital twins that are accurate, interactive, collaborative, and scalable.

Here are a few ways this could play out:

1. Accelerating collaborative previs for production

   For live events and location-based entertainment, producers can use digital twins of real-world spaces to plan layouts, test different setups, and make collaborative decisions in real-time. This kind of previsualization makes productions faster and more efficient, without sacrificing creativity. Read more about how we used OKO to take over Times Square for a cross-reality music event.

2. Simulating & enhancing the customer journey

   By simulating customer interactions, a digital twin allows designers to visualize how customers navigate and engage with a space, and respond to stimuli. This helps identify pain points, optimize layout, and personalize experiences. A digital twin also enables testing and predicting outcomes before making changes in the physical space, ensuring a seamless and engaging customer journey.

3. Giving digital reach to the physical

   By creating a real-time digital twin of a venue, gallery, or installation, you can expand its reach far beyond its physical location. Remote users can interact with the space just as on-site visitors do, thanks to live integrations with spatial data, video, audio, and environmental feedback. It鈥檚 a new way of thinking about access and connection, especially for events with global audiences. We created the world鈥檚 largest digital twin for the World Expo in Dubai.

Where to start

This table breaks down a 鈥済ood, better, best鈥� progression from a basic closed digital twin to a fully interoperable one, giving a rough sense of the time, investment, and complexity involved.

To recap鈥�

A digital copy is like a snapshot; useful for storing or sharing information, like a 3D model or a scanned document. It鈥檚 simple and static. 

A digital simulation is a theoretical model built to explore how a system or process might behave under certain conditions. A simulation isn鈥檛 connected to a physical thing so it鈥檚 ideal for exploring what-if scenarios without real-world consequences.

While digital twins can run simulations too, they also incorporate live data and two-way interaction, making them broader in scope and more dynamic in application.

But when systems need to talk to each other 鈥� across departments, factories, or even companies 鈥� that鈥檚 where the open digital twin comes in. It connects beyond its own boundaries, enabling smarter decisions across entire networks, whether you're managing a supply chain, optimizing energy across a smart city, or integrating systems from multiple vendors.

Choosing the right one isn鈥檛 just about technology. It鈥檚 about what your business needs to see, do, and achieve.

Ian Palmer has been working in and around the video games industry for most of his professional life, working for (among others) EA, Codemasters, and Supermassive Games. A keen musician and bike enthusiast, he spends his spare time gigging, recording, or riding around the Alps.

Tell us more about your role at 草莓视频在线.

I鈥檓 the Director of Art for the UK studio, which basically means I鈥檓 responsible for anything art or animation-related! There鈥檚 a lot of management and project planning involved, but I still find time to 鈥済et the crayons out鈥� and do some proper work every now and again!

What attracted you to 草莓视频在线?

I love the diversity of projects as well as having something I can apply my games-related experiences to, which doesn鈥檛 involve making games! Also, I鈥檝e always been someone who enjoys working with like-minded people who know their stuff and 草莓视频在线 is full of people like that. I feel like I鈥檓 in my element and part of the family!

What made you decide to pursue a career in this field?

I鈥檝e been working in and around computer graphics since the mid-1990s, when I happened across some 3D modelling software while studying modelmaking at University. I鈥檇 always been arty and always been into computers (particularly during my school days when I was regarded as a proper nerd, 鈥榗ause I knew about computers and stuff!) Sufficed to say, I was hooked and never went back to the workshop again.

After graduating I worked as a technical illustrator before falling into a job in the games industry. And I guess that鈥檚 where things really took off. I joined as an artist and 20+ years later became a Senior Art Director, with some fun/interesting/challenging projects along the way. 

What鈥檚 your favorite thing to do when you鈥檙e not working? 

Music. If I hadn鈥檛 studied art, it鈥檚 likely I would have been in or around the music business in some capacity or another. I鈥檝e been playing in bands for 40 years now (!!!) and still find making music as rewarding now as it鈥檚 ever been. More recently I鈥檝e got back into recording, so I have a room at home decked out as a studio with all the gear in it. I also have a very understanding wife!

If you could go back and give your 18-year-old self some advice, what would you say?

Enjoy it鈥� it goes by too quickly.

How do you approach challenges and setbacks?

I think you just have to roll with the punches sometimes. I don鈥檛 think I鈥檝e ever worked on a project where everything has gone as initially planned. 鈥淣o plan survives first contact with the enemy鈥� tends to be my mantra, so I guess I鈥檓 always expecting everything from a hiccup to major course correction. The best advice is to stay positive and professional while adapting accordingly. And use what鈥檚 happened as a learning experience.

If you had unlimited resources and funding, what project or initiative would you launch?

Capture that big asteroid that鈥檚 worth mine it, and unlock resources that could transform our world. Imagine what we could achieve if scarcity and competition for wealth were no longer driving forces.

What鈥檚 your power anthem for when you need to get in the zone to get stuff done?

Something heavy. There鈥檚 something about double bass drum patterns and detuned guitars that get me fired up!

If you could wake up in the body of another person (just for one day) who would it be and why?

Probably someone who lives on a remote island in the middle of nowhere, has no connection to the outside world, hunts for food, and lives in a self-built hut with no TV, internet, or idea of what a completely crazy place the world is! And could I have a couple of days, please?

What would your tactic be for surviving a zombie apocalypse?

I鈥檝e seen , so probably my first priority would be to lose some weight!

鈥淭he Wizard of Oz鈥� may not be the first film shot in color, but many people remember it that way because of how director Victor Fleming cleverly used black-and-white film for the scenes set in Kansas.

Likewise, 鈥淭he Wizard of Oz鈥� may not be the first film to be reconceptualized with AI, but it may soon be known for that, too.

Permalink

]]>The AI magic behind Sphere鈥檚 upcoming 'The Wizard of Oz' experienceMilo BygraveTue, 08 Apr 2025 10:27:07 +0000/blog/designing-xr-games-and-experiences-for-long-term-engagement618131cf8cd8e779321e9666:6256e7c502af7b62119329a0:67e2b7de7c71e3193ac8750d

At 草莓视频在线, we鈥檙e fortunate to have a team that includes talent from the AAA games industry. Many of our developers, designers, and engineers have contributed to some of the most successful and immersive games, bringing with them a deep understanding of what makes interactive experiences engaging, dynamic, and rewarding.

Our team applies their experience in game mechanics, player psychology, and content longevity to ensure that our XR experiences are not just fun, but designed for long-term engagement (LTE). Long-term engagement isn't just about getting people to download and play a game 鈥� it鈥檚 about crafting experiences that keep them coming back. 

In this post, we鈥檒l explore the importance of LTE, how to measure it, and actionable strategies for designing engaging experiences. We鈥檒l also dive into how we used all of our knowledge of LTE to craft The Office World experience for Meta Horizon.

What is long-term engagement?

Long-term engagement refers to the ability of a game to retain players over an extended period. Players expect continuous innovation, seamless interactions, and evolving dynamic environments. This has the benefit of building a thriving community, strengthening brand loyalty, and ultimately lowering acquisition costs 鈥� keeping an existing player is far more cost-effective than acquiring a new one. Whether it's through compelling gameplay, social features, or consistent content updates, maintaining engagement separates successful games from those that don鈥檛 stand the test of time.

Key features of successful games: how and why they work

LTE can be implemented in various ways, but the same feature won鈥檛 deliver identical results across different genres and platforms. Factors like product strategy, development scope (indie, AA, AAA), and target audience should shape the approach.

For instance, a 40-minute dungeon with valuable rewards might work well on PC but be less effective in VR, where sessions average 30 minutes, or on mobile, where players prefer shorter, on-the-go experiences. Similarly, challenge levels must align with audience expectations 鈥� casual players gravitate toward accessible, narrative-driven, or light-strategy mechanics, while hardcore gamers seek skill-based challenges and competition.

To sustain engagement, it's essential to identify features that fit both thematically and gameplay-wise. Successful games strike this balance by refining popular mechanics and consistently introducing fresh content.

XR and LTE

In XR gaming, LTE becomes even more critical due to the immersive nature of the experience. Take Beat Saber, for example, which is designed for long-term engagement by combining intuitive interaction-driven gameplay, social elements, and smart retention strategies. Its rhythmic, hands-on mechanics keep players immersed, while responsive in-game objects encourage exploration and develop skills. Multiplayer modes and leaderboards create natural social hotspots, fostering competition and collaboration that incentivize players to return. Live events, custom song packs, and ongoing updates keep the experience fresh, while well-timed notifications remind players of challenges, rewards, and limited-time content. All this ensures that Beat Saber remains engaging long after the first playthrough. 

• Interaction-driven gameplay to keep players engaged with the environment and other players. 

• Live events & social spaces to encourage collaboration and/or competition and increase the incentive to return.

• Push notifications to remind players to return, whether for a special event, a daily reward, or a limited-time challenge.

Case study: The Office World, Meta Horizon

In partnership with Meta and NBC Universal, we brought the beloved IP The Office to life in VR on the Meta Quest 3 via the Meta Horizon platform. We were tasked with creating a virtual experience that captured the awkward, cringe-worthy humor and lovable chaos of The Office, while giving Horizon players a reason to keep coming back. We leveraged all of our knowledge on LTE to craft an experience that blended nostalgia and innovative gameplay to give fans plenty to discover over time. 

Role-playing as a core engagement driver

We designed The Office World to encourage roleplaying, drawing inspiration from games that let players create their own fun. In Meta Horizons, for example, a supermarket setting naturally leads to cashier and customer roles, similar to pretend play in childhood.

To foster this, we recreated iconic office locations with interactive props that encourage playful, immersive experiences. Sitting at reception provides an obvious role, enhanced by elements like sweets, a pretend-call phone, and a red button that broadcasts 鈥淵ou鈥檙e late!鈥� to all players. Michael鈥檚 office and other familiar spaces support similar engagement 鈥� one player even reenacted the iconic 鈥淚 DECLARE BANKRUPTCY鈥� scene.

Progression systems and currency-based rewards

Players earn in-game currency through activities and can spend it on vending machine items, encouraging repeat play. To deepen engagement, we introduced mini-games, leaderboards, and rank-based progression 鈥� from temp to general manager. Currency multipliers reward high performance, while desk customization serves as both a personal touch and a status symbol.

Continuous content updates

We provided seasonal updates for Christmas and Halloween, featuring visual changes and themed decorations, which kept active players engaged and helped rekindle interest with both long-time or inactive players.

A solid LiveOps (Live Operations) plan ensures a game stays fresh and engaging for years. These days, players expect games to deliver more valuable content over time. Depending on the success of a game, it may grow to multiples of its original scope as years of development continue. By employing the method of LiveOps development in many cases, teams can replace the need to create sequels and optimise the lifetime value of their existing audience.

A successful live ops strategy may include:

• Regular content updates: New characters, levels, storylines, and mechanics.

• Seasonal events & challenges: Time-limited modes that drive periodic spikes in engagement.

• Dynamic in-game economy & rewards: Games can use in-game currency earned through play to unlock exclusive items, cosmetic skins, or badges, creating an ongoing reward loop.

Competitive and replayable mini-games

Within the world, dynamic mini-games drive replayability and social interaction. Games like The Floor is Lava feature randomized start and end points, while PvP matches remain unpredictable. Other games, like Bats, Bats, Bats, and Angela鈥檚 Cats, randomize object spawns 鈥� leading to both competition and spontaneous roleplay.

Meta Portal integration

To further incentivize engagement, we integrated Meta Portal rewards, including stickers and exclusive outfits. These collectables encourage ongoing participation and provide tangible goals for players to pursue.

A note on measurement 

To design for long-term engagement, tracking key metrics and using A/B testing is essential. Retention rates (D1, D7, D30) help measure how well a game keeps players engaged over time, while churn rates and 鈥渟tickiness鈥� metrics like DAU-to-MAU ratios provide insight into player habits. A/B testing allows developers to experiment with different features and optimize gameplay based on real player behavior, while data segmentation helps tailor experiences to specific audiences. By continuously analyzing and refining based on these insights, developers can create games that keep players coming back.

Final thoughts

At the heart of designing for long-term engagement is the principle of creating a meaningful exchange between a player鈥檚 time and the value they receive. Games that succeed in retaining players do so by offering rich, rewarding, and socially engaging experiences that evolve over time.

By leveraging data, understanding different player types, and implementing proven retention mechanics, developers can craft experiences that stand the test of time, whether on mobile, console, or XR environments.

At 草莓视频在线, we apply these principles to design immersive XR experiences that keep players coming back. We hope this post provides valuable insights for your own projects and look forward to seeing how you create experiences that engage your audience for the long haul!

Bridging the gap between physical and digital spaces has always been a challenge. It requires technology that simplifies complexity rather than adding to it. Using insights from every mixed reality project we鈥檝e worked on, we鈥檝e built OKO, a suite of apps built on The Connected Spaces Platform (CSP). OKO breaks down the barriers between physical and digital spaces and enables creators to build things like digital twins. Typically, digital twins are pretty hard to create, tough to deliver, and even harder to anchor to their real-world counterparts. But this is something OKO does really well.

The OKO and CSP story began at the World Expo in Dubai, where we were challenged to share the experience of the physical site with a global audience in real-time. With no off-the-shelf tech available, we created a social, interactive digital twin of the 4.38km虏 site from the ground up to enable on-site and off-site visitors to experience the same digital layer in sync. We鈥檝e since evolved that technology into a flexible open-source platform that can easily build and maintain digital twins, allowing teams to explore and plan accurate digital environments before anything is built, streamlining workflows like previs for production.

Exploring the role of digital twins in live event pre-production

This capability was put to the test in a groundbreaking project: planning and executing a cross-reality concert in the heart of Times Square, synchronized with a virtual gaming world. By using OKO鈥檚 digital twin technology, we efficiently coordinated a complex event in a way that would have been impossible using traditional methods.

Key considerations unique to the project dictated the use of a digital twin:

1. No live rehearsal was possible: Times Square鈥檚 dense foot traffic made live rehearsals logistically unfeasible. The concert鈥檚 secrecy also meant we couldn鈥檛 rehearse with the talent.

2. Independently controlled screens in Times Square: Each screen in Times Square had unique technical specifications, requiring meticulous planning to synchronize content across them all while preserving creative integrity. Exporting to multiple endpoints added further complexity 鈥� challenges best solved with a digital twin.

3. Camera and stage setup previsualization: The team needed to previsualize camera placements, movements, and stage layouts to optimize the live broadcast.

Screen synchronization and content management

Managing Times Square鈥檚 vast array of independently controlled screens (we used 48 on this project, the most ever utilized for one event) was a complex challenge. Visualizing the digital twin in OKO streamlined this process by:

Screen Identification

Every screen in Times Square has different labels e.g. the vendor鈥檚 name, the screen鈥檚 name, and its technical details. The OKO space incorporated a feature to display this data on the screens, allowing the team to make informed decisions about which screens to activate while balancing the needs for the creative vision. The team efficiently coordinated content delivery and playback by visualizing which screens belonged to which vendors.

Content Previews

The digital twin displayed previews of the content that had been UV-mapped to each screen, enabling the team to ensure creative consistency across all displays.

Camera Placement and Zoning

Using the digital twin allowed the team to visualize equipment placement in relation to screen content and broadcast camera movements in Times Square

Key highlights included:

• Overhead Map Views

  The OKO Web client鈥檚 overhead view feature allowed the team to suggest placements using a map underlay of Times Square. This enabled the sharing of 3D plans that helped identify key camera and equipment positions.

• Interactive Control Panel

  A client-facing control panel brought everything together, letting stakeholders interact with LED screen content, overhead maps, and camera views to plan and adjust broadcasts.

Stage Layout Setup

Previsualizing the stage setup was essential for both creative and logistical purposes. OKO supported this effort. 

Stage Modeling

Using a Gaussian Splat model of the stage, the team replicated the stage鈥檚 dimensions as accurately as possible within the digital twin. This allowed the creative team to consider sizing and placement of stage elements like risers, lighting, and equipment. 

Iterative Design

The platform enabled iterative testing of stage designs, ensuring a balance between creative vision and functional needs. By previsualizing stage components, the team could explore and refine configurations that best supported the experience.

Immersive Stakeholder Collaboration

To ensure alignment among stakeholders, OKO鈥檚 Web Desktop client was employed for: 

Spatial Realism and Collaborative Reviews

The creative directors could immerse themselves in a realistic simulation of Times Square with the content mapped and placed spatially. The web-based experience made it easy for stakeholders in different locations to collaborate.

Informed Decision-Making

Viewing the setup allowed stakeholders to make informed decisions regarding screen content and selection, stage design, and camera placements.

How OKO improved the pre-production process

From content planning to on-site execution, OKO played a crucial role in streamlining pre-production. By enabling easy collaboration, real-time adjustments, and precise digital visualization, the platform helped teams work more efficiently and effectively, wherever they were. 

Key takeaways:

• Efficiency: By creating a detailed digital twin of Times Square, OKO streamlined coordination across creative and technical teams, allowing them to make quick, informed content decisions.

• Adaptability: Real-time iterations enabled the team to adjust plans dynamically, ensuring that the creative vision was preserved while accounting for any impact on the event's physical layout.

• Collaboration: The ability to collaborate effectively across distances 鈥� through OKO鈥檚 Web Desktop Client 鈥� resulted in a cohesive content vision and execution, regardless of team location.

Beyond Times Square

The success of the Times Square digital twin is just one example of how OKO can transform live event production. While this project presented unique challenges due to its high-profile location, the potential applications for OKO extend far beyond iconic spaces. Whether planning large-scale global events, designing complex installations, or managing production workflows remotely, creators can bring their visions to life more efficiently, effectively, and collaboratively through the use of a digital twin. 

Jo Murayama is a visual artist at 草莓视频在线. Since joining the team in June 2024, Jo has been crafting immersive visual experiences. Before 草莓视频在线, Jo built a diverse career as a video editor and growth marketer for e-commerce brands, and VFX artist/supervisor for commercials, music videos, and indie films. His passion for VFX and filmmaking began in junior high school, where his YouTube videos gained attention and kickstarted his career as a freelancer at just 15. Jo studied film and television direction at the Art Center College of Design in Pasadena before fully immersing himself in video production and VFX. Born in Tokyo, Jo has called Los Angeles home since the age of eight, drawing inspiration from both cultures to fuel his creativity.

Tell us more about your role at 草莓视频在线

As a visual artist at 草莓视频在线, I get to engage in some of the most exciting, cutting-edge technologies that bridge the gap between physical and digital experiences. These technologies complement my speciality in compositing, which means combining different visual elements 鈥� such as live-action footage, CGI, and effects 鈥� into a seamlessly combined image. Visual artists here are similar to generalists in the VFX industry, which involves multiple different skills within the field of visual effects. However, at 草莓视频在线, this role involves a lot more creative and technical depth as we鈥檙e constantly pushing the boundaries of what鈥檚 possible.

What attracted you to 草莓视频在线?

Having spent a lot of time in growth marketing and freelance visual effects, I found myself often wanting to work on projects that push my creative boundaries. Unlocking ways to do that through rigorously researching, experimenting, and collaborating has always been the way I鈥檝e grown as an artist. 草莓视频在线鈥� innovative ways of connecting physical and digital worlds aligned perfectly with my passion for creative innovation. Before joining 草莓视频在线, I primarily worked on personal projects and small independent ventures. So being part of an environment that鈥檚 more structured has been a significant career milestone for me. On top of that, the opportunity to work alongside some of the most forward-thinking, genuinely kind, and top-of-the-class artists and producers, who bring incredible experiences from AR/VR and VFX industries, was an offer I couldn鈥檛 pass up. I鈥檓 very glad to have joined 鈥� every day is exciting and rewarding.

Tell us how you got into VFX

Growing up, I was always fascinated by movies and constantly engaged in artistic expression. In my early childhood, I spent countless hours building cardboard houses, doodling in notebooks, and bringing new creations to life with LEGO. 

In middle school, I got into watching YouTubers making creative content. My friend and I decided to make videos inspired by them. Eventually, we met another group of friends doing the same thing, but they specialized in visual effects. Being fascinated by their skills, I eventually learned how to use the same tools they were using 鈥� Adobe After Effects. Since the 7th grade, I would spend my after school hours watching YouTube tutorials to learn new visual effects skills, so that I can make my videos more advanced. Inevitably, I fell in love with filmmaking, the process, and the vast history of cinema. Through high school, I experimented with 3D tools and learned how to create photorealistic assets, utilizing programs like and . It was the early exposure to both art and technology that ultimately led me to pursue a career in VFX, where I could bring the same kind of experiences to others that I felt as a child.

What鈥檚 your favorite thing to do when you鈥檙e not working?

When I鈥檓 not working, I like spending time with my friends; I often try new food spots, have group dinner/movie nights, and explore hiking trails with them. We also travel once or twice a year, either on a road trip to a nearby city we haven鈥檛 explored, or a new country. Around New Year, I fly back to Tokyo to spend the holidays with my parents, the rest of my family, and some of my childhood friends. I typically don鈥檛 work during this time of the year, so I can recharge and spend quality time with them. It鈥檚 one of my favorite things to look forward to throughout the year as I only get to see them once a year.

What's a unique tradition or ritual you have in your life that brings you joy or fulfillment?

One of my hobbies is making specialty coffee; sourcing whole beans from different roasters, and dialing in flavour profiles using different grind settings, brewing tools, and techniques. I got into this a few years ago when I came across videos about coffee on YouTube made by , an award-winning barista from the UK. Now, it鈥檚 a crucial part of my morning routine, making and drinking the perfect cup of coffee to start the day.

If you had unlimited resources and funding, what project or initiative would you launch?

I would like to pursue some of my personal passion projects in narrative films!

What is something you think everyone should do at least once in their lives?

Traveling and meeting new people in different parts of the world.

What would you consider to be your 鈥榮uperpower鈥�?

Being able to organize my dishwasher in the most optimal way possible, without missing a single dish.

As part of our ongoing R&D, we continuously explore ways to push the limits of real-time rendering on the web. In this article, I鈥檒l walk through how we use a customized build alongside our 草莓视频在线 Web Renderer to achieve high-fidelity visuals, focusing on one of our most exciting advancements: volumetric lighting.

PlayCanvas and the 草莓视频在线 Web Renderer

When you browse to a Connected Space via OKO, you are using our web engine, even if the space itself was built in Unreal Engine or Unity. To maintain visual fidelity across all platforms, we wrote a library called 草莓视频在线 Web Renderer (send catchy names on a postcard please!). This backend-neutral rendering framework reduces the large API scope of PlayCanvas and ensures that we are not too dependent on any particular engine. If we want, we could just as easily write another backend in another engine, e.g., Three.js, without having to rewrite the whole application.

Why PlayCanvas? 

We started out using , but found that the performance and features of PlayCanvas, particularly its lighting engine, were a better fit for our needs. PlayCanvas uses what's known as a 鈥淐lustered Forward鈥� rendering engine, which allows for lots of real-time lights in the scene without a heavy cost to performance, and without the downsides and complexity of deferred rendering.

Types of rendering

• Forward rendering: This is considered the most straightforward way to render an object. Each mesh and material is drawn to the screen in its final form, complete with textures, normals (direction light bounces off a surface), and lighting. However, this method doesn鈥檛 scale well on its own, as the complexity increases with the number of lights multiplied by the number of meshes.  

• Deferred rendering: The most commonly used technique in AAA game engines, deferred rendering leverages multiple render targets (MRTs) to separate different aspects of a mesh into individual buffers. These buffers store data such as normals, base color, and metallic roughness. Instead of calculating lighting per mesh, deferred rendering applies it per pixel after the initial rendering pass. This decouples the mesh complexity from the lighting complexity entirely, allowing for an unlimited number of lights in a scene at a more linear fixed cost.

• Clustered rendering: An optimisation technique that divides the scene into areas in 3D space called clusters. Lights are assigned only to the relevant clusters, eliminating the need to check every light against every mesh during rendering. This significantly improves performance, allowing for many more lights in a scene. If a light doesn鈥檛 affect a particular area, it鈥檚 simply ignored, reducing unnecessary calculations.

New features

Over the last couple of years, we鈥檝e added several new features over the baseline PlayCanvas engine:

• LOD (level of detail) System, including an automated LOD generation pipeline system

• LOD Animation System to reduce animations

• Reflection Probe Volumes, allow for the baking of reflections into a sphere or box for lighting and reflections

• Light Probes, capture directional lighting inside of a volume, allowing for dynamic objects in the scene to be lit correctly even when light maps are used

• It also allows for baked bounce lighting global illumination without a lightmap

• Jolt Physics Engine, for a more modern and performant physics engine

• Unreal Engine baked lightmaps

• Optimised per-pixel picking and selection for fast and accurate object selection

• Better tolerance for GLTF files that contain errors

• In progress experiments

• Hardware instanced Grass

• Navigation meshes for AI agents

• Ground fog

• Terrain

• Last but not least, the subject of today's blog, volumetric lighting!

Volumetric lighting, it鈥檚 just fog right?

Not quite, but they are similar. Let's start from the beginning.

Classic 鈥渇og鈥�

A classic technique to reduce draw distance and improve performance involves coloring every pixel on screen based on its distance from the camera 鈥� literally adding more "atmosphere" to a scene. Enhancements to this method include factoring in height, limiting fog to specific areas, applying different distance functions (e.g., exponential fog), or adjusting the color near light sources like the sun for expansive outdoor environments.

Ground fog

Building on exponential height fog, this technique adds noise 鈥� a key to making visuals more interesting and realistic by introducing imperfections. Noise refers to a set of algorithms that create natural, varied patterns. Instead of coloring based purely on distance, the fog is now defined by a "fog density" formula, which samples fBm (fractional Brownian motion) noise. This simulates the random movement of particles in a volume, causing some areas of the fog to appear denser than others, just like real fog.

Fog cards

"Fog cards" are a simple technique where a video or animated texture of real smoke is placed on a translucent surface. Using a specialized shader, you can enhance the effect with various techniques:

• Fresnel effect: Adjust visibility based on the viewing angle 鈥� fully visible when viewed head-on and gradually fading at sharper angles.

• Depth softening: Fade the effect where the plane intersects with geometry to avoid harsh clipping edges.

• Camera distance fade: Gradually fade the fog as the camera approaches to maintain a natural appearance.

Volumetric lighting

As the name suggests, this technique samples the "fog" based on the lighting in a scene. This allows for more precise control over fog placement and a more realistic depiction of how fog interacts with light. In reality, fog appears denser when illuminated by strong light sources, as the light scatters off particles in the air 鈥� an effect accurately reproduced with volumetric lighting.  

鈥淐heat鈥� method

This approach uses cone-shaped geometry matching the light source, combined with translucency and the Fresnel effect to soften the edges. It鈥檚 computationally inexpensive to render but comes with limitations: the effect disappears when the viewer is inside the volume, doesn鈥檛 hold up from all angles, and cannot account for light occlusion (e.g., shadows).

Volume rendering

To draw real volumetrics, we use a technique called volume rendering, which relies on raycasting. This involves tracing a ray from the camera into the scene and sampling points along its path. At each point, we determine how much light is present. The process continues until the ray either reaches its endpoint or hits an object.

By combining these samples, we can calculate the density of the volume along the ray. We can also factor in properties like light color, height falloff, and noise to enhance realism. Most excitingly, if the light source casts shadows, we can sample its shadow map to determine whether the light volume at our current world position is illuminated. We do this by converting our position from world space into the light's shadow space (This is the view of the world from the point of view of the light source). This gives us the corresponding location on the shadow texture, allowing us to multiply the light's power by the shadow map value to reduce or remove its contribution at that point.

This is run inside of a post processing effect after the scene is drawn into as small an image as we can get away with for performance reasons. This is then upsampled to the final resolution, smoothing out the final image.

What's next?

While the results and performance of the raycasting effect exceeded expectations, it remains a basic approach compared to cutting-edge techniques. Next, we鈥檙e experimenting with WebGPU Compute shaders to handle complex lighting calculations, storing the results in 鈥渇roxels鈥� 鈥� a frustum-aligned voxel grid. This approach enhances detail for nearby lights while reducing complexity for those farther away. By utilising compute shaders here we can also more efficiently utilise our GPU resources. 

By combining the power of the Connected Spaces Platform and PlayCanvas, we can make experiences accessible to everyone on any device, wherever they are. We鈥檙e committed to pushing the boundaries of web-based experiences and can鈥檛 wait to share more with you in the future.

Sarah Hudson-Semple is a lead artist at 草莓视频在线, and has worked on groundbreaking projects like Expo 2020, virtual production for Westworld and Fallout, and the Fortnite Remix cross-reality concert. With a Fine Art degree and experience studying sculpture in Florence, Italy, Sarah blends artistic finesse with technical expertise. Before joining 草莓视频在线, she honed her skills as a 3D generalist in the VR department at Method 草莓视频在线s.

Tell us more about your role at 草莓视频在线

I鈥檓 a lead artist on the US side of 草莓视频在线 where I鈥檓 currently working on a virtual production project. In addition to contributing art to the project, I also manage and mentor the artists on my team.

What has been your proudest/standout moment while working at 草莓视频在线?

Each time we successfully complete a project, it鈥檚 an incredibly rewarding moment for me. I truly enjoy seeing the reactions from my team and the rest of 草莓视频在线 to the experiences we鈥檝e created. It's such a supportive environment and everyone is super enthusiastic about the work we do. 

What attracted you to 草莓视频在线?

I had heard through colleagues and friends about the excellent culture at 草莓视频在线 and was also interested in the wide variety of projects worked on. I was and still am really inspired by the way that 草莓视频在线 tackles the intersection of art and cutting edge tech.

What made you decide to pursue a career in this field?

I have a background in Fine Art but struggled to find a way to support myself with my sculpture (the starving artist life is no joke!). My friend who works at ILM suggested that I could try to translate my physical sculpting skills into digital, so I rolled up my sleeves and taught myself how to 3D model. This decision has been one of the most fulfilling I've ever made!

What鈥檚 your favorite thing to do when you鈥檙e not working? 

I love to curl up with a cup of tea, good book and Dragon (my cat)! Or when the weather is warm here on the East Coast, I love to take my kids on new adventures outdoors!

What would be your specialist subject if you were on a game show?

My husband likes to joke that I have approximate knowledge of many things, which is very useful for pub trivia! But if I have to pick just one subject, I bet I would be pretty good at questions about sci-fi books/movies/tv. 

How do you want to leave a mark on the world 鈥� personally or professionally?

Professionally, I want to be able to pay it forward to the next-gen of artists. I got a good start in this industry because I had some really great mentors and friends and I really hope to be able to fill that role for others someday! 

How do you approach challenges and setbacks?

During tough times, I remind myself that everything is solvable. There鈥檚 always a way through or, sometimes, around any obstacle in production. I鈥檝e found it helpful to approach a project with the mindset that the trajectory may shift or something might break鈥攁nd that鈥檚 not always a bad thing!

If you had unlimited resources and funding, what project or initiative would you launch?

I would love to create a mentorship program for new 3D artists and introduce this amazing career to more people. If I had unlimited funding, I'd include free access to the tools of the trade, programs that we use daily, as sometimes they can have a steep price tag which may be a barrier to entry to many aspiring artists. 

What is something you think everyone should do at least once in their lives?

I think everyone should travel somewhere new, whether that is somewhere they've never been local to their home or to a far-away country. Exploration and curiosity are so important to being open minded and creative!

We鈥檙e fortunate to have an incredible team of engineers building the future of connected spaces through OKO and the Connected Spaces Platform (CSP). As a Product Manager, I鈥檓 responsible for defining and prioritising the features we want to build, and then overseeing their development to ensure we鈥檙e meeting users' needs.

Working closely with the engineering team every day, I get to see their passion for experimenting, problem-solving, and pushing boundaries firsthand. That鈥檚 why I鈥檓 a big advocate for creating opportunities for innovation 鈥� not just to advance our products, but to give our engineers the space to explore new ideas. We enable this continuous innovation through dedicated R&D time and annual Hackathons. In this article, I鈥檒l dive into why both are essential to our culture of creativity and share insights from the engineers who make it happen. 

Research & Development

R&D is the lifeblood of innovation 鈥� it's where we discover solutions to problems we didn't even know we had! This is why during each sprint - a two week cycle focused on completing a set amount of development tasks - we allocate 10% (equivalent to one day) for developers to tackle an R&D project of their choosing. The project must satisfy at least one of the following:

• It relates to the advancement of OKO or CSP in some way.

• It allows engineers to upskill on their own or other clients' codebase/tech stack (we have multiple client teams working on Unreal Engine, Unity, Web, and Cloud Hosted Services).

Each engineer has full control over the scope of their R&D work 鈥� there鈥檚 no product manager hovering over their shoulder asking for estimates or tracking deliverables. If they ever need inspiration, they can turn to our dedicated R&D Slack channel, where team members share project ideas, collaborate on challenges, and give behind-the-scenes sneak peeks of their progress. 

Hackathons 

Hackathons have been a driving force for innovation since their inception in the late 1990s. The first-ever hackathon, organized by the project in 1999, was designed to bring developers together to collaborate on technical challenges. What began as a niche event quickly grew in popularity, and today, hackathons have become a cornerstone of creative problem-solving and innovation across industries. 

These events offer a structured, yet flexible environment where participants can tackle problems without the constraints of day-to-day work. By breaking down silos, accelerating development, and sparking breakthroughs, hackathons encourage experimentation and the kind of creative thinking that traditional settings often can鈥檛 provide. 

Understanding just how valuable they are for innovation and engagement, we host an annual themed hackathon. Over five days, participants can team up or work solo to build a working prototype that aligns with the given theme. As I write, we鈥檙e gearing up for one of these hackathons, and I鈥檓 excited to see the fresh ideas and solutions that will emerge from it!

While we keep the atmosphere relaxed, we do have some key criteria for the projects. They must either extend an existing feature, demonstrate a new feature, or showcase how our current features can be used in innovative ways. This ensures that the outcomes remain relevant to our broader product goals, while still leaving plenty of room for experimentation.

Once the five days are up, participants present their project to the wider product organisation, discussing any setbacks they may have experienced, learnings they have made, and ideas for iterating on it in the future. Although hackathons are often seen as being engineer-focused, we make a point to include people from all disciplines at 草莓视频在线 鈥� from QA to the art team. This diversity of skills and perspectives creates a unique collaborative space, offering employees the chance to work with people they wouldn鈥檛 typically interact with on a day-to-day basis.

Initiatives like R&D and Hackathons cultivate a culture of innovation and creativity, giving our engineers the autonomy to own a piece of work from ideation to implementation while incorporating their own personal development goals. It might seem counterintuitive to reallocate time from scoped product work to experiment uncharted territory, but it鈥檚 an investment that continues to pay off. Learnings from these projects have led to faster and more confident feature execution and completed projects can also find their way into OKO once they鈥檝e been officially signed off. 

But don鈥檛 just take my word for it, let鈥檚 find out straight from the source!

Tom Yehya - Engineer, OKO Unreal Engine Client Team 

鈥淚 really enjoy my R&D time as it allows me to get creative and explore technologies I don't typically work with. My first complete R&D project was adding a simple right-click context menu in the Unreal Editor Extensions to speed up the creation of Space Entities 鈥� the core of OKO spaces. It had been a big pain point for me having to swap tabs, drag & drop actors, and then convert them.

Although the project was relatively simple, it had a big impact on the user experience. Seeing it be well-received and successfully integrated into OKO felt like a major accomplishment.鈥�  

As part of Tom's R&D time, he added a right-click context menu in the Unreal Editor Extensions, streamlining the creation of Space Entities in OKO and improving workflow efficiency.

Adrian Meredith - Specialist Engineer, OKO Web Client Team

鈥淪ome of the most pivotal features I鈥檝e worked on started as R&D experiments, with some taking up to five years to find their perfect application. For me, R&D is about pushing boundaries, challenging assumptions, and acting as insurance against stagnation. 

As a rendering lead, most of my projects have often been graphics-related, with my goal being to push the product in ways it鈥檚 never been done before in relation to 鈥済ameplay鈥�. During the first hackathon, my team created a game where users would have to roll through an obstacle course in a giant ball and get to the top first! It was such a success that the following year we took it to the next level. Players would navigate a creepy forest in the mountains, trying to avoid being caught by zombies hunting them down.

The key to successful R&D is creating an environment where 'bad' ideas are not just acceptable, but necessary. Every revolutionary product started as someone's 'terrible' idea and in my experience, the only true failure isn't in pursuing an idea that doesn't work; it's in being too afraid to try. When you free yourself from the pressure of immediate success and embrace the joy of exploration, that's where the real fun begins鈥�

Adrian鈥檚 2024 hackthon project 鈥楨scape The Dark Forest鈥� was a fan favourite!

Michelle Shyr - Engineer, OKO Unity Client Team

鈥淚n my last hackathon, I explored AR development and created an interactive experience where hand tracking and gesture recognition in AR could detect user gestures and translate them to display hand emojis in the OKO space. This year I hope to take it one step further by adding support for full-body pose recognition!

I enjoy R&D time and hackathons because they provide the freedom to explore new ideas without the usual constraints, often leading to innovative breakthroughs that wouldn鈥檛 emerge in a structured environment. The fast-paced nature of a hackathon makes problem-solving exhilarating, pushing me to think quickly, adapt, and iterate in real-time.鈥�

We know that the path to technical innovation isn鈥檛 linear and doesn鈥檛 happen by accident 鈥� it happens when people are given the freedom to explore and experiment. R&D and hackathons aren鈥檛 just side projects at 草莓视频在线; they鈥檙e a vital part of how we stay ahead, challenge assumptions, and unlock new possibilities. The true success of a project is measured by the learnings made along the way, irrespective of whether it leads to a working prototype. Whether it鈥檚 refining existing tools, discovering unexpected solutions, or chasing a 鈥渢errible鈥� idea that turns out to be brilliant, this culture of continuous learning fuels everything we do. 

We sat down with Jon Raftery, one of our lead engineers who鈥檚 recently joined us to find out why he took a leap from the AAA games industry to escape 鈥渕ore of the same鈥�.

You worked for many years in the games industry, why did you leave? 

Anybody who has paid any attention at all to the video games industry recently will be aware of the huge number of lay-offs that have taken place in the last few years. It鈥檚 been truly heartbreaking to hear about all these talented people losing their livelihoods. And it happened to me. 

After many years of happily working on a well-known strategy game franchise, and only two months after a promotion, I suddenly found my job disappearing from underneath me. To say I was surprised was an understatement, as I'd been a very loyal employee through good times and bad. But still, I found myself adding the dreaded green 鈥渙pen to work鈥� banner on my LinkedIn profile. The job market can be a harsh and brutal place if you haven鈥檛 dipped your toe in there in a while. I wanted a good job and fast, so naturally I started applying to roles that matched my skill set. Engineer. Team leader. Audio specialist.

As I started taking interviews I found myself getting a very strong feeling of deja vu. After working in a particular role for so long, and being interviewed by people in pretty much the exact position I was recently in, recruiting for the exact same role I鈥檇 just left, it all started feeling very familiar. The person sat opposite me... was me, just a few months ago. Did I really want to be doing the same thing in the same sized company for the same kind of game? The games, people, engine, etc, would be different. But I鈥檇 still be solving the same problems, fixing the same types of bugs and having the same conversations. Was I ready for 鈥渕ore of the same鈥�? 

The answer was no and I started widening my search. 

How did you hear about us? 

I saw an engineer role advertised and I was intrigued. I鈥檇 heard the name 鈥湶葺悠翟谙哜�� before from a game I'd played on my virtual reality headset; an immersive experience called where you explore the International Space Station. After some research, I found that the company was active in quite a few different areas. 

How would you describe what we do? 

草莓视频在线 is not a game developer, yet many AAA veterans work there and the team has developed hugely successful Fortnite events. It鈥檚 not a tech company, even though it has a tech platform 鈥� OKO 鈥� a suite of apps and plugins that allow users to create connected 3D spaces across realities. It鈥檚 not a media or production company, although the team has been involved with the recent Fallout television series. And it鈥檚 not a boutique XR studio, even though it鈥檚 created standout bespoke experiences for the likes of Disney, NBC, and Meta. The team has many talents, but its USP is helping others tackle unquantified challenges using technology and creativity. 

Was it easy for you to leave the games industry behind? 

I thought that I had allowed my mind to fossilise into an AAA mindset. It was a little scary to imagine myself potentially jumping between quite different projects. Different tools, products, engines. And of course, different people working on them. I asked myself; was it sensible to consider a sideways jump at this stage in my career or should I stick with the familiar?

When I was lucky enough to be offered the job, I had a big decision to make. This wasn鈥檛 just a new job, it was a new direction in my career. A big part of me advised sticking to the safe path 鈥� more of the same. But if I鈥檇 always listened to those voices, I wouldn鈥檛 have gone into game development in the first place. 

How does 草莓视频在线 differ from the work you were doing before? 

Everything I鈥檓 doing now is as far removed from the job I had, and the jobs I was initially heading towards, as you can get. 

A big part of the job that鈥檚 new for me is the number of relationships I manage. I find myself working with different companies, people, and processes, and learning something new from all of them. This is a big change from the traditional developer-publisher relationship I was so used to in my previous company, and looking back it's clear just how insular I'd become as a developer. I'm now exposed to so much more and, although the learning curve has been steep at times, it's refreshing always to be taking on new challenges. Within the company, inter-project collaboration is encouraged, and this time next year it's possible I could be using a different programming language, on a different engine, for a different game, for a different client. All while working at the same company.

Do you work in the office or remote? 

草莓视频在线 has offices in the US and UK and offers remote work. This is something that's becoming increasingly scarce in our post-lockdown world of work. It's not that I dislike working in an office. I just dislike the time and expense of commuting for a job I can do literally anywhere with a laptop and a decent internet connection. As a bonus, the UK office is always open for anyone who wants to use it, and I'll head in every so often to meet up with my colleagues face-to-face. Often in the pub afterwards as well.

What do you hope people get out of reading this? 

I hope this serves as a salutary lesson about risk and reward. I loved my old job, and I'll always be very proud of what I achieved, but sometimes life throws banana skins your way and it's important to reframe them as opportunities. If the axe falls on you, don't rush to automatically limit yourself to 鈥渕ore of the same鈥�. There are plenty of opportunities to thrive if you鈥檙e prepared to step outside your comfort zone.

What鈥檚 your favorite thing to do when you鈥檙e not working?

Aside from family time and playing video games, I try to counteract my screen time by keeping myself in touch with nature. Lots of long walks in the woods, gardening, gathering berries in the summer, and looking up at all the twinkling dots in the night sky. When you look at a distant star you are essentially looking at the past, as it takes the light so long to reach us. Due to light pollution, most people will live their lives having never seen the Milky Way. It's a sight to behold.

What are you reading/listening to right now?

In terms of reading, I always find myself coming back to the poetry of John "No man is an island" Donne. He has a fascinating life story; from scandalous rake and "visitor of ladies" in his youth, to preacher at St Paul's Cathedral in his later years. That's quite a story arc, and he lived during a fascinating period of history. To counteract this cultural intake I also read lots of trashy Science Fiction novels, the less scientifically plausible, the better.

How do you want to leave a mark on the world 鈥� personally or professionally?

I think generally you should try to leave things in a better state than when you found them. There's also a lot of negativity on social media, which I make an effort to avoid. We carry all the world's information around on our smartphones, yet people are mostly using it to watch cat videos or get into arguments with strangers. As a parent, I find this concerning.

What鈥檚 your life motto/ guiding principle you live your life by?

The golden rule; "Treat others as you'd like to be treated yourself".

Earlier this year, our Global Director of Innovation and I presented 鈥淏uilding Cross Platform, Cross Reality Social Experiences Using XR鈥� at the conference. If you missed the event, this article captures the highlights of our presentation and provides details on our Connected Spaces Platform (CSP) and how it can help developers create interoperable social experiences using XR.

What is Cross-Reality (XR)?

XR is the convergence of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), creating environments where virtual and physical spaces blend seamlessly. It allows users to step into shared digital worlds, interact with persistent virtual objects anchored to real-world locations, and collaborate in real-time.

What is Cross-Platform?

Cross-platform XR ensures these immersive experiences are accessible on a variety of devices, from mobile phones and desktops to VR headsets and AR glasses. Whether you鈥檙e wearing a high-fidelity Meta Quest 3 headset or joining on a smartphone, the content remains consistent and synchronized.

Why Multiplayer is Essential to XR

In a world that鈥檚 information-rich but experience-poor, the true magic of XR lies in bringing people together. Multiplayer XR experiences create shared environments where people can interact with each other in real-time 鈥� whether it鈥檚 a virtual concert where attendees in VR and AR see the same digital performers or a global team collaborating on a shared 3D prototype anchored in physical space. This kind of interaction opens up new possibilities for entertainment, collaboration, and community building.

A Multiplayer Example 

Prototypes like Cryptic Cabinet, a mixed-reality escape room we developed with Meta, showcase what鈥檚 possible. Using Meta鈥檚 Presence Platform, this experience enabled multiple users to interact with digital objects within a physical room while maintaining low-latency synchronization. Such prototypes highlight how XR can blend the physical and digital to create engaging, real-time multiplayer experiences.

Technologies Making Cross-Reality Social Experiences Possible

We鈥檝e built a powerful toolset to tackle the unique challenges of XR development. Central to this is our Connected Spaces Platform (CSP) 鈥� an open-source middleware that simplifies the creation of cross-reality, cross-platform experiences.

Key Features of CSP:

• Real-Time Multiplayer: Low-latency, synchronized interactions across platforms.

• Cross-Platform Compatibility: Operability with major engines like Unity, Unreal, and WebXR.

• Spatial Anchoring: Ensures objects and interactions remain persistent across sessions and devices.

• Scalable Infrastructure: CSP supports everything from intimate gatherings to large-scale events, adapting in real-time to user demand.

Using an alpha version of this tech, we created the world鈥檚 largest connected space for Expo 2020 Dubai. Over 39 months, we designed and developed a 4 km虏 digital twin of the Expo site, allowing millions of physical and virtual visitors to interact with a unified digital layer.

How It Worked:

• Real-Time Synchronization: On-site and remote visitors experienced events in perfect sync, thanks to our real-time multiplayer capabilities.

• Spatial Persistence: Digital objects and interactions remained anchored in the physical Expo site, allowing for a consistent and engaging experience across sessions.

• Cross-Platform Accessibility: Participants could join via mobile, web, or VR, ensuring inclusivity for a global audience.

The project demonstrated how XR can bridge the gap between physical and digital spaces, creating a shared experience for people "there" and "not there."

Overcoming Technical Challenges

Building cross-reality, cross-platform social experiences isn鈥檛 without its hurdles. Developers face several key challenges:

• Coordinate Systems: Every device uses its own coordinate system, requiring seamless transformations to ensure consistency.

• Interoperability: Ensuring a seamless experience across hardware ecosystems like Meta Quest, ARCore, and WebXR requires extensive testing and integration.

• Rendering Optimization: Different devices have varying performance capabilities, making it crucial to optimize rendering for smooth, immersive experiences.

We鈥檝e streamlined this process by developing several plugins for Web, Unity, and Unreal on top of the Connected Spaces Platform to handle these aspects for you. These are part of a set of cross-platform multiplayer applications that we call OKO.

Our APIs convert the coordinate system into the appropriate ones depending on the platform you are using them for and offer multiple functionalities to replicate transforms and related metadata to ensure that your player is correctly interoperable between different viewing modes (AR / VR / default), and different platforms (Unreal / Unity / Web).

Furthermore, our plugins adapt to the rendering performance of the targeted device (desktop or mobile) using performance optimization techniques such as LODs, culling, and simplified rendering pipelines for lower-end devices. 

To learn more about OKO, check out these resources:

• /oko 

•  

What鈥檚 Next for XR?

As technology advances, several trends are shaping the future of XR:

• Lightweight AR Glasses: Innovations like Meta鈥檚 Project Orion and Snap鈥檚 Spectacles are making AR more accessible and user-friendly.

• 5G and Cloud Rendering: Faster data speeds and cloud-based processing allow devices to offload computationally intensive tasks, enabling lighter, more mobile hardware.

• Gaussian Splats and Digital Twins: These technologies enhance realism in XR environments, improving spatial interactions and visual fidelity.

These trends will empower developers to build richer, more inclusive XR experiences, driving adoption across industries like healthcare, education, and entertainment. It鈥檚 an exciting time to explore what鈥檚 possible and to push the limits of how we connect in the digital age!

Eugenia is a Lead Producer at 草莓视频在线, working on projects like Metallica: Fuel FIre Fury, Karol G MSB Fortnite, the Fortnite Remix Concert in Times Square with Snoop Dogg and Ice Spice, and unannounced projects.

Previously, Eugenia has been a creative producer, creative director, and editor for various short and long-form projects including a Kyrgyz award-winning feature narrative film, 鈥淎fter The Rain (Jamgyrdan Kiin)鈥�; VR concerts with Megan Thee Stallion, T-Pain, and KPop artists;  film projects with UN Women, the President of Kyrgyzstan - Roza Otunbayeva, and the US Embassy of Kyrgyzstan; and media projects with K-pop artists.

Tell us more about your role at 草莓视频在线

I鈥檓 one of the producers at 草莓视频在线, currently leading the Fortnite Concert experiences. If you鈥檙e asking what I do every day, as producers, we put out fires and unblock roads, all the while moving projects forward with the best excellence the team can in the limits of time, and budget, while still keeping a smile and running toward the vision set for the project. Sounds impossible? It often feels that way, but that's what all the producers at 草莓视频在线 are doing every day - looking dead straight in the eyes of the impossible, and smug at it with optimistic anxiety. We鈥檙e the coxswains of the crew, steering the team鈥檚 morale, rhythm, and velocity to the direction we鈥檙e headed, all the while in the background paving out bumpy roads in advance for the team to have a smooth course as much as possible. We become everything and anything the team and project needs, pulling up our sleeves and wrestling with the mountain. I once was told by my former mentors that, 鈥渢he best form of a producer is one that is always in service of the team and vision of the project.鈥�

Personally, what keeps me going every day when I wake up, is when I remind myself and the team of the joy of creating projects that are technically groundbreaking and culturally paradigm-shifting, saying, 鈥淚f it were easy, anyone could do it. Alas, we are the chosen ones!鈥�

What has been your proudest/standout moment while working at 草莓视频在线?

Aside from the obvious moments of seeing projects launch and receiving a lot of praise, and seeing the team鈥檚 faces gleaming with pride -  personally, I would say when I see the spark of fun and glitter of passion in the eyes of the team, as they go forth and conquer impossible obstacles, trying to find a way to build and innovate around the blockage and see creative juices flowing elevating the excellence of the project to another level.

What attracted you to 草莓视频在线?

The MO of 鈥淲here physical & digital meet to create extraordinary experiences.鈥� - which is in line with my personal MO of wanting to be at the forefront of the 鈥渆volution of storytelling that bends reality.鈥�

What made you decide to pursue a career in this field?

I discovered that I had a passion for storytelling when I was in high school, working a summer job at a community art center making videos. I felt a spark and a drive to create stories that impact people, and eventually the way people think and feel, influencing culture. That passion kept fueling me to where I am today.

What鈥檚 your favorite thing to do when you鈥檙e not working? 

Searching for the perfect coffee shop on weekends (Good hand drip coffee, roasts their own beans, spacious high ceiling hip vibes, big windows with perfect nature views). Thrift hunting for vintage furniture and binge-playing the latest games.

What would you consider to be your 鈥榮uperpower鈥�?

If I really wanted to and my schedule allowed it, I could sleep more than 20 hours. Sleep is my zen. Or again, if I had free time, I could binge-play an entire game without stopping.

How do you want to leave a mark on the world 鈥� personally or professionally?

Make cool sh*t that impacts the culture, that the next generation will grow up in and experience. As someone in the realm of creativity, I have the privilege and responsibility that things I am involved in reach the masses, and have influence in small and big ways. By supporting and serving my team with excellence, generosity, and fun, it eventually influences the stories and projects the team creates, which impacts the audiences in ways that hopefully leave a mark of joy.

If you had unlimited resources and funding, what project or initiative would you launch?

HA! It鈥檚 top secret and confidential! 馃槈 But the gist is - when the technology and the fidelity come to a point, where you can鈥檛 differentiate between digital and reality, I have many ideas and stories I鈥檝e been brewing over the years that I want to create. Just waiting for the tech and times to catch up for what I鈥檓 envisioning to become possible.

What's a unique tradition or ritual you have in your life that brings you joy or fulfillment?

Turn up some music at the start of the day while brewing hand-drip coffee and lounge by the window with a blank state of mind - priceless.

If you could wake up in the body of another person (just for one day) who would it be and why?

If it's 鈥渁 person鈥�, probably myself in the far future - I want to see where all of this is headed. If it's anyone, then my cat - what a good life she has.

Using Pop Culture to Create Emotionally Resonant Virtual Worlds

Pop culture has an unmatched ability to unite people through shared interests, memories, and emotions. Whether it's iconic moments from movies, beloved TV characters, viral memes, or nostalgic video games, it serves as a cultural shorthand that instantly resonates across diverse groups. In virtual environments, users not only observe but actively engage with these references, creating new, personalized experiences around them.

For developers, integrating pop culture into virtual worlds goes beyond simply replicating familiar scenes or characters. It's about leveraging the emotional connections people have with these properties to create a connection. By embedding elements of pop culture, developers can create spaces where users feel more at home, sparking interactions based on shared fandoms that might not have happened otherwise. 

Pop culture references also act as a gateway to accessibility in VR. For new users, stepping into a virtual world filled with recognizable elements from movies, music, or games makes the experience less intimidating. It provides a familiar entry point, helping users feel instantly connected to the environment. 

When building The Office World, we were committed to faithfully recreating the iconic Dunder Mifflin office. Yes, that meant watching a lot of TV 鈥� all in the name of research, of course (and regular reviews with our IP stakeholders). From Pam's reception desk to Michael鈥檚 office, and even the wall where the show鈥檚 memorable 鈥榯alking head鈥� interviews were filmed, we made sure players would instantly recognize the key landmarks that bring the world of The Office to life.

Developing Immersive Pop Culture Experiences

Creating an engaging pop culture experience begins with understanding the audience. Developers need to know what fans love about a particular property and how they want to interact with it in a virtual space. But, it鈥檚 important to strike a balance between making the experience accessible to newcomers and rewarding hardcore fans with Easter eggs and references they will appreciate.

A deep understanding of how to harness the interactive nature of virtual reality is also required. Developers should focus on crafting experiences that allow users to go beyond passive consumption and actively participate in the world. For instance, building interactive challenges or storylines based on key moments from a show or movie can create a sense of agency for users, allowing them to become part of the narrative.

In The Office World, players can step into iconic moments from the series through engaging mini-games. These experiences allow users to actively participate in the narrative, such as navigating through the chaotic Kevin's Chili-Thon, where they must carry Kevin's infamous pot of chili while dodging office obstacles. Additionally, players can sort and deliver packages in 鈥淧ackage Pickle鈥�, where efficiency and speed earn them extra Shrute Bucks. By incorporating familiar scenarios and interactive challenges, The Office World enables fans to feel like they鈥檙e part of the show, enhancing their connection to the beloved characters and storylines.

Additionally, adding social elements 鈥� such as spaces for fans to collaborate, compete, or showcase their creativity 鈥� can further deepen engagement, transforming the experience from a simple virtual homage into a dynamic, evolving environment where the community plays an active role in shaping the world.

Designing for Retention and Replayability

One of the critical goals in developing virtual worlds is retention 鈥� encouraging players to return to the world time and time again. This requires a focus on more than just the initial "wow" factor of the experience; it鈥檚 about creating a world that evolves with the player. Drawing from game design principles, developers can implement progression systems, challenges that scale with player skills, and social mechanics that encourage collaborative play. For example, rather than just allowing users to explore a static environment, developers can introduce missions, unlockable content, or evolving narratives that provide players with reasons to continue their journey. 

In The Office World, players can fully immerse themselves in the quirky culture of Dunder Mifflin by climbing the ranks and earning promotions to various job titles featured in the show. The game fosters a dynamic social environment, encouraging roleplay through interactive features such as sitting at reception and pretending to be Pam, or pushing the 鈥淵ou Are Late鈥� button for comedic effect. 

When the office is overrun by bats, players can choose to channel their inner Dwight and bravely attempt to catch them, or panic and flee like the rest of the staff. Additional features like a stamp in the accountants' section allow players to approve any document in the office, while vending machines offer Dunderballs for a fun game reminiscent of classic moments from the series. With global leaderboards, unlockable customization items, and unique roleplaying scenarios, players have new objectives and social experiences every time they return, keeping the world dynamic and engaging.

Collaborating with Industry Partners

In the development process, collaboration with external partners or intellectual property (IP) owners can be a game-changer. Direct access to creators and producers allows developers to stay true to the original vision while adding layers of interactivity that make the experience fresh and exciting. 

These collaborations often provide access to a wealth of creative resources, from concept art and storyboards to behind-the-scenes insights into the creators' vision. This inside knowledge ensures that the virtual world not only reflects the original work but also adds new layers of interactivity and immersion that are possible only in VR. Moreover, working closely with IP owners opens the door to early feedback, which can be crucial in fine-tuning the experience for authenticity. For example, integrating the distinct tone, themes, and even fan expectations can elevate a project from a simple adaptation to a truly immersive extension of the brand. These partnerships not only foster creative innovation but also help developers navigate the complexities of intellectual property, ensuring that the final product aligns with both the brand's identity and the potential of virtual worlds.

During the development of The Office World, we worked closely with Meta, NBC, and Deedle Dee Productions. Regular check-ins and review meetings were essential to maintaining the integrity of the intellectual property and nailing the show鈥檚 signature humor. What made the process even more unique was that many of these meetings took place within VR itself. While these sessions were often fun and playful, they also played a crucial role in helping us fine-tune the world and its features. 

The Unique Advantages of Meta Horizon

Meta Horizon offers several advantages when it comes to building entertainment experiences. First and foremost, it is a social platform, where users can connect and interact with others in real-time. This communal aspect makes it an ideal environment for pop culture experiences, where fans can gather, share their passion, and even create their own narratives within the virtual space.

The platform's flexibility also allows developers to experiment with different types of interactivity, from narrative-driven adventures to freeform exploration and multiplayer challenges. By blending traditional game design with the open-ended nature of social VR, developers can create experiences that keep players coming back for more.

Best Practices and Looking Toward the Future

Developing successful pop culture worlds in Meta Horizon requires a blend of creativity, technical skill, and a deep understanding of the community. Here are some best practices for developers looking to enter this space:

1. Focus on the Community: Create experiences that allow users to engage with each other, not just the environment. Social interaction is key to keeping players invested.

2. Balance Accessibility and Depth: Make sure your world is enjoyable for both casual users and hardcore fans. Offer layers of content that cater to different levels of engagement.

3. Collaborate with Partners: Working with IP holders and creators can provide valuable insights and ensure the experience remains true to the original vision.

4. Design for Longevity: Implement game mechanics that encourage repeat visits, such as progression systems, evolving challenges, and unlockable content.

As VR continues to grow, the opportunities for integrating entertainment and pop culture into virtual worlds are expanding, giving developers the chance to create experiences that bring people together in new and exciting ways.

Si芒n Adeoye is a product manager with nearly six years of experience. She began her career in the insurance industry and transitioned to 草莓视频在线 in 2022. Based in London, but often traveling the world, she has a diverse background that includes roles as a Digital Product Analyst and actuarial intern at AEGIS London, as well as internships at Allianz and RBS.

Tell us more about your role at 草莓视频在线

I鈥檓 a product manager for two of our client teams (Unreal and Web), and my job is to define and prioritise the features that we want to build, and then oversee their development to ensure that our product is meeting the needs of our users. Our Technical Director recently wrote a great blog post introducing our product to the world, you can check it out here.

What has been your proudest/standout moment while working at 草莓视频在线?

My proudest moment was receiving a promotion! It validated all my hard work and growth that I experienced during my first year at the company and reassured me that recognition and career progression are genuinely valued here. 

What attracted you to 草莓视频在线?

When I first came across the job listing, OKO wasn鈥檛 a public product. But based on their previous projects 鈥� which were amazing 鈥� I knew whatever product was being cooked up behind the scenes at 草莓视频在线 was something I wanted to get involved in! The cherry on the top was when I saw the LinkedIn tagline, 鈥淣ot sucking since 2013鈥�.

What moment has had the most significant impact on your life?

Aged 5, I joined a chess club at school because other girls in my class said they gave out a Haribo at the end of each session and I had a killer sweet tooth. I ended up really enjoying it, and before I knew it I was spending most of my weekends for the next 7 years playing competitively in tournaments all over England. 

What made you decide to pursue a career in this field?

Honestly, I found my way into product management completely by chance! I graduated with a degree in chemical engineering at university but knew that I wanted to explore a new career path. During a summer internship, I got an unexpected job offer to join the company's newly formed product team and thus began my introduction to product management. I love the challenge that comes with driving product development as it鈥檚 the perfect blend of strategy and creativity, and it feels great to play a part in transforming ideas into reality. 

How do you approach challenges and setbacks?

I try to break a problem down into smaller chunks so it doesn鈥檛 feel overwhelming and tackle one issue at a time. My years playing chess taught me to stay calm under pressure, adapt my strategy when needed, and try not to take things personally!

If you were on a game show, what would be your specialist subject?

I feel like I鈥檓 a living library of random facts so I鈥檓 more suited to Richard Osmands House of Games than Mastermind. I鈥檇 also love to go on Taskmaster, the randomness of the tasks appeals to me and I love seeing how people think outside of the box to win.

What are you reading/listening to right now?

I鈥檓 currently reading 鈥榃eapons of Math Destruction鈥� by Cathy O鈥橬eil which explores how algorithms used in everyday life can perpetuate biases and the importance of critical thinking when working with data. 

My current album on repeat is HEIS by Rema - listening to this in the gym is a cheat code for a powerful workout. 

Where is one place you want to travel to in the world?

I love traveling and have been lucky enough to tick off most of my top bucket list destinations. I think next on my list would be Vietnam, I鈥檝e heard a lot of good things through friends and want to soak up the culture & history for myself 鈥� the food is also a big bonus! 

What鈥檚 your go-to karaoke song?

I recently did Karaoke for the first time while out in Japan and completely lost my voice attempting to sing 鈥楤reakeven鈥� by the Script (those high notes on the chorus are no joke!). 

Kevin Hanna, Creative Director, was instrumental in the concert鈥檚 creation. We ask him how the project came to life: "UEFN was crucial from the start. The ability to rapidly iterate was game-changing. We didn鈥檛 just want to create a passive event 鈥� we wanted it to feel like an interactive music video, where every moment meant something to the player. Whether creating special effects or ensuring that enemies moved in time with the beat, UEFN allowed us to play with those elements without a lot of downtime. That flexibility let us focus on crafting an experience that was as interactive as it was visually engaging."

One fan-favorite sequence was the waterslide, where players raced alongside Karol G while blasting away enemies. Hanna elaborates: "That waterslide sequence went through several iterations 鈥� five fully playable versions 鈥� before we finalized the one with the Joy Cannon. With UEFN, we could snap things together, test them in real-time, and see how they felt in context. The real-time adjustments meant we could experiment, fail fast, and move on, which ultimately helped us create something players really connected with."

Addison Herr Lead Designer, tells us about cross-discipline collaboration. 鈥淲ith the real-time feedback loop, we could prototype and test ideas immediately, whether it was gameplay or audiovisual elements. I worked closely with the art department as well as our other designers, Daryl Kimoto, Gladeline Rufo, and Thomas Wilson, and we were constantly learning from one another 鈥� they explored the intricacies of building complex mechanics in UEFN, while I gave input on audio-reactivity, shaders, and game feel. We pulled from our diverse backgrounds, ranging from rhythm game design to real-world concert visuals 鈥� to find a sweet spot of merging the spectacle of an arena show with satisfying interactivity.鈥�

We ask Lisa Barber, the environment artist responsible for much of the concert鈥檚 look, how UEFN assisted in building the environments. "UEFN鈥檚 asset library was a lifesaver. We needed a huge variety of very specific visuals 鈥� like glowing, iridescent mountains, beautiful pink foliage, and over five different unique settings 鈥� UEFN made it easy to find what we needed. It meant we could focus on perfecting the world rather than getting bogged down in technicalities."

Motion capture was part of the production process, Gabriela Montes, Rigger, explains its role in detail. 鈥淲e used motion capture for everything related to Karol G鈥檚 performance, from her dance moves to facial expressions. UEFN鈥檚 character device allowed us to play the motion capture on various character types, outside of Karol G herself,  with the click of a button! This flexibility meant that her character animations, as dynamic and lifelike as they were, could easily be leveraged for use throughout the project, like for the background dancers.鈥�

Montes continues, 鈥淭he amazing motion capture quality was all thanks to the people at Yoom 草莓视频在线s, Metastage, and Brazen Animation. Yoom captured data that consisted of facial data, body movement, and lipsync in one cohesive file, with production and facilitation effort from Metastage, while Brazen Animation added their animators' touch. All of this was critical for a project of this scale.鈥�

Creating custom arcade shooting mechanics 

On the technical side, Derrick Canfield, Engineering Lead, who was responsible for the arcade shooting mechanics discusses in detail how that came together, starting with the design.

鈥淥ne challenge was finding a way to implement custom arcade shooting mechanics that aligned with Fortnite鈥檚 fast-paced environment, while ensuring it wasn鈥檛 too difficult for casual players. We used UEFN to build custom devices that allowed us to control shooting mechanics and create unique physics for the targets. This way, players could get instant feedback without overwhelming them with too much complexity.

He continues, breaking down the core mechanics, 鈥淥ne of those mechanics was an arcade-style shooting section while players rode on a waterslide as Karol G performed. Here, we chose to have shots fired from the player instantly travel to the target, rather than simulating the arc of a projectile. This approach provided instant feedback, enhanced the arcade-like feel, and simplified our logic. Additionally, because all enemies were close to the player, the projectile's travel time would be negligible.鈥�

Canfield explains how the problem was further simplified, 鈥渨e opted to use spheres as the collision volumes for the targets. Raycasting against a sphere is fairly trivial, and we wanted the shooting to be forgiving in this experience. So, even if you aimed slightly above the shark's head, you'd still be rewarded as if you hit the target.鈥�

Using Verse, Epic Games' programming language for UEFN

Next, we hear how Verse was used, 鈥淲e created a module named 鈥淧hysics鈥� with a function for ray-sphere collision testing.

#Copyright (c) 草莓视频在线. All Rights Reserved.
using { /UnrealEngine.com/Temporary/SpatialMath }

Physics<public> := module:

    RaySphereTest<public>(RayOrigin:vector3, RayDirection:vector3, SphereCenter:vector3, SphereRadius:float)<transacts><decides>:float=
        M := RayOrigin - SphereCenter
        B := DotProduct(M, RayDirection)
        C := M.LengthSquared() - SphereRadius * SphereRadius

        # We know they do not intersect so we can exit early with a `false` value
        not (C > 0.0 and B > 0.0)

        Discriminant := B*B - C

        # When the discriminant is negative, the ray and sphere do not intersect
        not (Discriminant < 0.0)
        
        option{ Max(0.0, -B - Sqrt(Discriminant)) }?

The function accepts an origin and direction for a ray, as well as a sphere center and radius. If the ray passes through the sphere, the distance between the ray鈥檚 origin and the hit point is returned. If there is no intersection, the function fails using Verse鈥檚 <decides> specifier.鈥�

For those new to Verse, Derrick explains its unique specifiers, 鈥淯nique specifiers, such as <decides> and <transacts>, play a crucial role in controlling how functions behave and interact with the game state.

• <decides>: This specifier is used when a function needs to determine or "decide" a result that may not always produce a value. For example, in our ray-sphere collision test, <decides> is used because the function may or may not find a collision point, depending on whether the ray intersects the sphere.

• <transacts>: This specifier indicates that a function can modify or "transact" changes to the game state. It鈥檚 particularly important when you need to ensure that operations happen in a controlled, atomic manner, such as updating scores or player positions in real-time.

For those looking to deepen their understanding of these terms and other unique elements of Verse, we recommend checking out . This resource provides detailed explanations and examples, making it easier to get up to speed with the language鈥檚 capabilities and best practices.鈥�

Arcade Shooter Minigame Devices

Canfield, walks us through how the custom Verse devices for the arcade shooter minigame were developed, especially the mechanics behind the target system and how it all comes together during gameplay.

鈥淭he target system is pretty straightforward. We created a device that holds a reference to a prop of our choice, along with a collider radius. The internal logic manages the target's state鈥攚hether it's active or not鈥攁nd includes functions for spawning, despawning, and handling what happens when the target is hit. This code is barebones so a creator can extend it for their island鈥檚 needs.

using { /Fortnite.com/Devices }
using { /Verse.org/Simulation }

arcade_target_device := class(creative_device):
    @editable
    Prop<private>:creative_prop = creative_prop{}
    
    @editable
    ColliderRadius<public>:float = 150.0

    var Active<private>:logic = true

    OnBegin<override>()<suspends>:void=
        Despawn()

    IsActive<public>()<transacts><decides>:void=
        Active?
        
    Spawn<public>():void=
        Prop.Show()
        set Active = true

    Despawn<public>():void=
        Prop.Hide()
        set Active = false

    Hit<public>(Agent:agent):void=
        # Add particle effects, sound effects, etc
        Despawn()

The minigame kicks off when the player steps on a trigger and ends when a timer runs out. 

using { /Fortnite.com/Devices }
using { /Fortnite.com/Characters }
using { /Verse.org/Simulation }
using { /UnrealEngine.com/Temporary/SpatialMath }
using { Physics }

arcade_shooter_minigame_device := class(creative_device):

    @editable
    BeginMinigameTrigger:trigger_device = trigger_device{}

    @editable
    EndGameTimer:timer_device = timer_device{}

    @editable
    FireTrigger:input_trigger_device = input_trigger_device{}
    
    @editable
    WeaponGranter:item_granter_device = item_granter_device{}
    
    @editable
    Targets:[]arcade_target_device = array{}

    var FireTriggerCancellable:?cancelable = false

    OnBegin<override>()<suspends>:void=
        BeginMinigameTrigger.TriggeredEvent.Subscribe(BeginMinigame)
        EndGameTimer.SuccessEvent.Subscribe(EndMinigame)

    BeginMinigame<private>(MaybeAgent:?agent):void=
        BeginMinigameTrigger.Disable()

        # Give all players a gun to use
        for(Player:GetPlayspace().GetPlayers()):
            WeaponGranter.GrantItem(Player)

        # Enable the fire trigger and listen for fire events
        FireTrigger.Enable()
        set FireTriggerCancellable = option { FireTrigger.PressedEvent.Subscribe(OnFirePressed) }

        # Spawn all targets
        for (Target:Targets):
            Target.Spawn()

        # Start the timer
        EndGameTimer.Start()

    EndMinigame<private>(MaybeAgent:?agent):void=
        BeginMinigameTrigger.Enable()
        
        # Disable the fire trigger and remove the event callback
        FireTrigger.Disable()
        if (EventCancellable := FireTriggerCancellable?):
            EventCancellable.Cancel()
        
        # Despawn any targets that still remain
        for (Target:Targets):
            Target.Despawn()

        # Reset the timer for the next time the minigame is played
        EndGameTimer.Reset()

    BulletRaycast<private>(GunAimDirection:vector3, GunPosition:vector3, Target:arcade_target_device)<transacts><decides>:vector3=
        # Execute a ray-sphere cast to see if we hit the target
        TargetTranslation := Target.GetTransform().Translation
        HitDistance := RaySphereTest[GunPosition, GunAimDirection, TargetTranslation, Target.ColliderRadius]
    
        # If the target is hit, reconstruct the hit position that will be returned by this function
        HitPosition := GunAimDirection * HitDistance + GunPosition
        option{HitPosition}?

    OnFirePressed<private>(Agent:agent):void=
        if (FortChar := Agent.GetFortCharacter[]):
            PlayerTranslation := FortChar.GetTransform().Translation
            CameraRotation := FortChar.GetViewRotation()

            # Iterate through all active targets to see if any were hit
            for (Target : Targets, Target.IsActive[], HitPosition := BulletRaycast[CameraRotation.GetLocalForward(), PlayerTranslation, Target]):
                Target.Hit(Agent)

In the OnBegin function, we assign BeginMinigameTrigger to our logic to start the minigame as well as setting EndGameTimer鈥檚 completion event to the logic that ends it. 

Once it starts, we grant all players a gun and enable an input trigger that listens for when they fire. When they press 鈥渇ire鈥�, it runs the raycast logic we set up earlier to check if they hit any active targets. If a target is hit, we trigger the Hit(agent) function on the target. From there you can add any additional logic you need, such as custom scoring or integration with any other game mechanics.鈥�

Visualizing Logic with Debug Draws

Derrick dives into how debug draws can help when developing. 鈥淒ebug draws are incredibly valuable when it comes to visualizing the logic in your code, particularly for something as intricate as ray-sphere collisions. In our case, you could use them to visually trace the ray鈥檚 path and define the boundaries of the sphere within the game environment. This kind of visualization lets you see exactly where the ray intersects the sphere or if it misses entirely, making it much easier to debug and refine your collision logic.

For example, you can draw the ray with DrawDebugLine and the sphere with DrawDebugSphere. By doing so, you get instant visual feedback right in the game world, which helps confirm that your collision detection is functioning properly.鈥�

Bringing it all together 

Derrick tells us how to bring everything together in the level, especially when setting up player input and devices for the minigame, 鈥淲e set up all of the devices in the level and hooked them together.

The first item we set up was the input trigger for the player鈥檚 fire action. To do that, add an Input Trigger device to your scene. Set 鈥淚nput Type鈥� to 鈥淪tandard Action鈥�, and 鈥淪tandard Input鈥� to 鈥淔ire鈥�. This will make the input trigger when any player presses the fire button.

Next, you鈥檒l need an Item Granter to give the player a weapon to shoot the targets. I chose a rifle, but you can pick whatever fits your theme best. After that, you need to create a Trigger Device and a Timer Device. Both of these devices work well with their default settings, but you might want to adjust the timer duration to suit the experience you're crafting.

Derrick goes on to talk about the fun part - targets to shoot at! 

鈥淐reate an Arcade Target Device which you can find wherever your Arcade Target Device logic exists within your project. Disable 鈥淰isible in Game鈥� so the terminal mesh is not rendered. Also, find a prop to be the visual representation of your target and parent it underneath the Arcade Target Device. Adjust the prop so that the center point is aligned with the origin of the device鈥檚 mesh. In this example, I鈥檝e chosen a disco ball and you can see it aligned to the center of the bottom of the console mesh which is the pivot point. With the Arcade Target Device selected, set 鈥淧rop鈥� to the prop you created and set a collider radius to a value that would encapsulate the prop.

Then, select the prop and set the property 鈥淐an be Damaged鈥� to false. This ensures the default shooting logic in Fortnite doesn鈥檛 damage and destroy the prop. Our Verse logic is handling the destruction effects already! Duplicate the device and prop as often as you like and place them around your map as you see fit.鈥�

We ask Canfield about the final inclusion, 鈥淭hat鈥檚 the Arcade Shooter Minigame device. Create an instance by dragging it from the Content Browser wherever you saved the Verse file in your project. Again, set 鈥淰isible in Game鈥� to false to hide the default terminal mesh. Set BeginMinigameTrigger to the trigger created earlier, EndGameTimer to the timer created, FireTrigger to the input trigger, and WeaponGranter to the weapon granter we set up. Then for Targets, add every copied target to this array. You may want to click the lock icon in the top-right corner of the Details Panel so it doesn鈥檛 change to show the properties of the arcade targets if you drag and drop references into the Targets array.鈥�

He concludes, 鈥淵ou can now jump into a session and play through the minigame. Step on the trigger to equip yourself with a gun and show all of the targets. Pull out your weapon and shoot the targets before your time runs out! If you follow these instructions you鈥檒l have a fully functioning arcade-style shooter! The minigame is barebones, ready to be expanded upon with whatever fits the creator's needs 鈥� a custom scoring system, special effects, custom meshes for your targets, and more!鈥� 

Finally, we asked how the concert was received by both the audience and your team. Herr tells us, "We were unsure how people would react at first, but the feedback was incredible. It was especially meaningful to see the response from fans in a number of languages across social media platforms, particularly during the closing dance-off and in the epilogue. The climactic scenes of the experience celebrated unity and empowerment, and to see people across the world connect with all the love and care we poured into this concert was so rewarding."

Montes adds, "As a Latina artist, it was personally significant for me. Being able to share this project with my family, especially in a Spanish-speaking context, was really special. The emotional impact extended beyond the audience 鈥� it resonated with us as creators."

Permalink

]]>AJ Sciutto speaks to Meta about the creation of Alo Moves XR