I’m not big into VR. In fact, I can count on one hand the number of times I’ve done anything virtual reality–related. But this past summer, I interviewed the heads of Gugenka, a company dedicated to various forms of entertainment that blur the line between analog and digital. Thanks to that interaction, I recently received an invitation to participate in a Code Geass x FLOW virtual concert. It’s not a bad combo: I have generally fond memories of the Code Geass anime, and FLOW has done some of my favorite anime songs ever. So, even if my VR experience is limited to a demo booth in the 1990s and a Hololive Myth anniversary event, I decided to give it a shot.
Not only was it my first virtual concert, it was also my first time using VRChat—a program I only knew of through watching Virtual Youtubers. Getting things to work took a lot of coordinating, requiring me to link a variety of apps and accounts across various sites together. It might get easier with experience, but I was definitely confused, and I still doubt I fully grasped it all.
Because I don’t own a VR headset, I used VRChat on my desktop PC, and the experience was more like a first-person shooter (or like Incidentally, that’s also a form of entertainment that’s not really my cup of tea, but once I got the hang of it, I started to see the appeal of being able to navigate virtual spaces in a more naturalistic way. (It also let me understand how Gawr Gura navigated the aforementioned HoloMyth event.)
The actual concert consisted of five parts, each time starting with a speech from Lelouch (voiced by Fukuyama Jun), which was then followed by a song performance from FLOW. There were actually two ways to view the concert: VRChat and a Japanese streaming service called Showroom. Because of my own confusion, I actually watched the first song in Showroom, which acts more like Youtube or Twitch, but is aesthetically set up to resemble a simplistic movie theater. At first, I figured this was just the way the concert was, until I saw a bunch of 3D models run right in front of FLOW, clearly showing that Showroom wasn’t the only way to experience the event. That’s when I decided to switch and try VRChat after all, despite some earlier troubles.
One thing that complicated this process was that joining each part of the concert meant having to leave VRChat and click a link that would then send a message to VRChat with a special invite to the next “world.” The need to jump back and forth was a bit unintuitive, and I actually missed the 2nd part of the concert as a result before I figured out how the whole thing works. Once I got back on track, things finally fell into place.
This is when I finally understood exactly what Gugenka meant by having “instances” that allow their virtual events to have some flexibility for viewers. In the case of this Code Geass x FLOW concert, one could join in real time to mimic being part of a public concert, or one could join at a specific moment so that you can either coordinate with a smaller group or to make sure you didn’t miss anything. This isn’t permanent, however, as there were still specific overall time frames where the concert parts were available, and then they would go away. The difference is that if you missed something by, say, 10 minutes, you still had 45 minutes to watch from the beginning.
I don’t know if it’s because I started with Showroom and ended with VRChat, but in Part 1 of the concert, FLOW was being shown as video footage of the actual members, whereas after that, they were 3D models. In the VRChat experience, it was amusing to see people running up to the stage to get as close as they can to FLOW, while others would use the squat command to make viewing easier. The music was great (of course), but in some ways, the people-watching was better. I remember seeing one attendee in particular swaying and moving with a clear joy over getting to be there.
Tickets were 6,600 yen minimum, with a deluxe package that costs a great deal more. I don’t regularly attend concerts, virtual or otherwise, so I thought it was kind of steep. That said, understanding the kind of experience it’s supposed to be, and knowing that other virtual events cost similarly, I think I would pay for the right event. It also costs a lot less than actually flying to Japan to see a band in the flesh.
While the virtual experience can’t be a full replacement for a live performance, there’s a bit of joy in knowing you’re experiencing the same thing as people living in Japan. It also creates a great opportunity for those who don’t have the means to travel for concerts to do something more interactive. Especially in a time when COVID-19 is still affecting people around the world, it’s also a solid choice for those who are too afraid to travel to another country.
I want to end by talking about a funny incident that occurred. For one song, I hopped into the VRChat world, only to find myself somehow transported a great distance away from the waiting room. Confused, I tried getting closer and closer to the space, only to start hearing chatter from Japanese attendees who were discussing someone who looked to be stuck. Soon, I realized that the person they were talking about was me, and they were trying to help me get out of whatever weird glitch I was in. After resolving the issue, they asked if I was okay—to which I jumped up and down to show everything was fine. The way these random people looked out for me put a smile on my face, and it actually made the concert more enjoyable overall.
I’m not going to say that this is a universal experience for virtual spaces, but it reinforced the interpersonal connections these sorts of events can provide.
This interview was conducted at Otakon 2022 in Washington DC. Gugenka, Inc. is a company that describes itself as “Sales of digital goods and technical research on Japanese animation using advanced contents such as VR/AR/MR.” Mikami Masafumi is the CEO, and Kiral Poon is the CTO.
How did you get into the business of 3D, VR, AR—these sorts of subjects?
Poon: Do you mean academic side, or…?
Anything, really. If there’s an academic element, then definitely talk about it. If it’s sort of an amateur hobby, or…just what inspired you to take this up…
Mikami: We were originally making AR stuff for movie promotions, and we slowly moved that content into VR. That became for VR anime promotion, where we promote anime with our VR abilities.
Poon: One of the examples would be the Sailor Moon VR that we built. That is also, like, a movie for Sailor Moon Crystal promotion, and then we did the VR game for the event. And in the end now it’s on VIVEPORT for free, but it was originally promotional content.
I first became familiar with Gugenka through the HoloModels app.
Poon: Really?!
Yeah. I found out about it from a reader, actually.
Poon: When did you hear about it?
A couple of years ago, I think?
So what was the origin of the decision to make virtual AR models? Was the plan from the start to work with established properties?
Mikami: The way it started off was, you know how we have anime figures in real life, right? What if we could make digital versions of anime figures?
In physical figures, it’s probably not a real surprise to see on your figure rack a Dragon Ball character and a Re:ZERO characters side by side. But then in VR contents, it was an uphill battle to get the understanding to have two characters of different IPs sitting on the same app.
Poon: I’d like to actually convey my two cents on that. Japanese companies are really, really, really restrictive on the IP stuff. The more I work with them, the more I think that’s so crazy about it. Masafumi back in the day, he tried very hard. One problem is, how can you line up two different IPs together without any problem with the IP company. Cause they usually want to have their own world, right? When it’s in the real world, it’s just figures, so they don’t care about it. But in digital, they usually have control with their applications. So Masafumi was actually the producer. He is the one who actually produced it and persuaded those companies to do it together. The reason he can do that is because this company has a long history with these companies and these movies. He already knew these people, and that’s why he did it. It is not easy at all, in my opinion.
I’ve heard similar things about the game Super Smash Bros. When it comes to the director, Sakurai, it’s only because he has such a strong reputation in his industry that he’s able to convince all these different…
Poon: Yeah, exactly. In Japan, when you work, it’s not only about ability, but also about how people trust you. So that’s the case.
I also want to point out why this app exists, which is that there’s limited space around the world. We usually line up a lot of figures—I buy my own figures too—they take up a lot of space. In AR applications, you stick them into the space, and you can display them anytime. It’s just like an RPG item box, where you just bring it up anytime—exactly like that. That’s really convenient, and you can do more than with actual figures. You can move the pose and change the face. With real figures, you have to change the parts, and there’ll be lines on the face you don’t want. But with digital figures, you don’t have that problem. So that’s one of the main reason I can tell you why we created them.
You actually anticipated my next question!
[laughter]
So moving on, Gugenka has collaborated with virtual stars of all kinds. You have Vocaloids like Hatsune Miku. You have Virtual Youtubers like Kizuna A.I., Tokino Sora—and I remember that when I got the Gugenka HoloModels app, there was Shinonome Megu.
Poon: You did a good job. People don’t even know that one.
So what do you think about the fact that people are embracing virtual characters, and have you noticed any changes in the degree to which people are embracing them?
Mikami: Right now, everyone has their own different characteristics, and everyone would probably be able to express themselves differently in different bodies. I believe that our app would enable everyone to express themselves in their own body of their choice.
Around the world, the COVID-19 pandemic has brought the concept of virtual spaces to the forefront, as friends, families, and businesses try to gather online through various chat programs but also in creating virtual space. How has the pandemic affected your approach to virtual spaces? Has it cast a new light on them?
Poon: So with virtual space, during the pandemic, we have actually been trying to increase our opportunities with all these restrictive companies. In my point of view, usually Japanese companies tend not to do remote work and exhibitions lots online. But because of the pandemic, they start thinking about it, and they approach us. Of course, sometimes we approach those companies—“Hey, do you want to do it?” Usually, it’s the case that they will ask us about doing online exhibitions. Before, there were, like, no exhibitions, but because of the pandemic, we have those events hosted by Gugenka.
For example, the closest one would be the MF Bunko J one that we just did with Kadokawa—the light novel one. It’s a really popular real-world event in Japan. And there’s live concerts, like the Sanrio Virtual Fes. We tried to make it realistic, and we have a very good reputation on that event with VRChat. And of course, we also connect with all these partners from Japan—VRCast and other platforms. And I connect with all others across the world and have partnerships with them, like VRChat. After that, people started realizing what we’re trying to do to make people closer in distance, and display things in HoloModels, and also create your character through MakeAvatar.
You’ve worked on many events, and I know it’s hard to pick a favorite, I guess, but I’ll ask anyway: Do have any events you’re surprised that you managed to work on?
Poon: I guess I’ll let Masafumi answer first, and then I’ll answer.
Makami: I would say Kitty-chan’s Sanrio Virtual Fes is probably my favorite. I can take pride that I participated in it. The reason is that the people who are enjoying the real stuff Sanrio produces, as well as the artists and VTubers—they all come together in one event, so I thought it was a good event.
Poon: I’ll add two cents about the Sanrio virtual event because there’s not many people who know about the event in the US. After I did the panel, people were all surprised. This is a real diverse event that includes actual artists like AKB48—famous, real artists like HoneyWorks. There were also virtual artists like Kizuna A.I. and other VTuber characters. And then there’s also the system that we created so that we can bring this all in, and with synchronization with the timing we can make it feel like a live event.
What that means is, what if you join late but your friends have been in for five minutes? In a usual game, well, you start from zero, and you play it, right? But in real life, you’ll be seeing things five minutes late. What you’re seeing with friends is the same thing, and when you wave, and when you sing, you’re doing the same thing. This is what makes a huge difference with the Sanrio Virtual Fes event. We created a synchronization system to make sure everyone had the same experience, same timing to all these artists. And it’s a huge event because there’s more than a hundred songs. I didn’t sleep for two to three days just to check the songs, and it was crazy. I wish one day we could join us for the second one. So that’s the Sanrio Virtual Fes.
That actually makes me think: Sometimes, due to the pandemic and the increasing use of online spaces, people want to get together for a karaoke session, right? But due to the differences in synchronization, it can cause problems for people who want to sing together. Is there a possibility of creating a space like that for regular users?
Poon: So there’s a lot of problems in terms of people’s preferences. During the event, there’s a separate instance we created to adapt for each person. For example, if your computer is weak, but you still want to enjoy the event, there’s something called shitei [appointed] instance, which means you are pointed to that instance, and we make sure you can have the best experience in that instance. But because it’s a social VR, if you want to see the group, there’s something called jiyuu [free] instance. But if it crashes, or you just want to enjoy the experience with your weak computer, you can still go to the appointed instance.
But also, furthermore, for the paid content, there’s outfits using the MakeAvatar app. It’s like when you dress up as Mickey—or Marvel characters because Disney owns Marvel—and you go to Disneyland. So it’s the same thing as when you go to Sanrio Puroland; you dress as Mochipoly, as they call it. It’s really cute, and you can buy the hat from Kitty, or Keroppi.
There are two reasons for that. First, is because of the story. It’s best delivered that way. It feels like part of them, and it’s real good. When you look at pictures, it’s really nice. Second, is the performance. We want weak computers to also get in easily. So we think about all that and plan the event that way.
I know a lot of people around my age became anime fans in part or in whole due to Slayers.
Poon: Ooh!
So I was pleasantly surprised to see both a HoloModel of Lina Inverse—I saw the author, Kanzaki, constantly promoting it—
Poon: Man, I’m surprised you know Slayers, and that we did that?
And you created the Slayers 3D Live event. Were there any fun, creative challenges involved with working with this property?
Poon: Actually, Masafumi was talking about Sanrio, but my favorite event was Slayers. The reason for that is the equal system that I was trying to build. I’m actually the director of the concert for the live event in Tokorozawa Sakura Town.
So we actually have the live event, and then the sequences. We have HoloModels of Slayers, and usually it would be a waste that you can’t use them on other games. It’s really expensive to build a really high-quality CG model. We have the models, we have the live event in Sakura Town in real life, so it feels like Lina Inverse is there, and she does the song. Then, we host this event online in NeoChat so everyone can join. And then in the end, it’s sold on blu-ray. So this whole flow is the equal system that we try to tell those IP companies that we can balance promotion and also maybe earning a little money for the company. That’s why it was my favorite event, because all the systems of the Sanrio event are based on the Slayers event.
I’m surprised. Even though I didn’t do a lot of promotion for it, I’m surprised that you and—actually, one more person at the panel, he said he knew it, and he actually joined the event. I’m actually touched because I want North America—like myself from Canada, and there’s also the US—I want them to join us. But there’s limited resources, and there’s a problem that we didn’t have enough English translation at the time. So in the future, I’m trying to push everyone to do more English support. Good question. I want to talk more about Slayers!
Kiral, one of your interests, as written in the Otakon guidebook, is creating technology to help make life easier for animators. There’s an ongoing issue about animators being overworked and underpaid? Do you think your work can help deal with this issue?
Poon: Do you know Dragon Ball Super: Super Hero? That is one of the good examples of using good 3DCG in movies that doesn’t feel weird. Because sometimes it feels weird in 3DCG. There’s one good example, that you can have an efficient way of moving, and then you can have more stories to deliver to people.
So that is great, and it’s efficient, but to your question, can it solve the overtime and overwork problem? In some ways, but it comes down to the culture in Japan. Let’s say you need less people for the same project, right? Well, you just hire less people for the same work—there’s a chance like that. But it depends on the company. They could be, like, “Oh, there’s less work now, so now everyone doesn’t have to be doing overtime, and they can go home, right? So there’s two approaches, and it comes down to culture, I think, partially. But the reason I want to focus on technology and I keep improving that, like with Slayers, is to try io make it more toon-like with cel shaders—to make it more efficient. But can that solve the core problem is the question.
Are there any Gugenka products or services that get less attention that you think people should know more about?
Mikami: In regards to what I want people to know more about, it’s basically everything: HoloModels, MakeAvatar, virtual events. But I kind of believe there are two major points that I think we need to focus on to have a wider number of people know about us and these events.
One is that there is a time difference between Japan and the US. While we were able to get past the problem of distance because we could bring people together from vastly different parts of the world, we can’t really get past the problem of time at the moment. So, I believe what we want to do in the future is maybe have different events for different time zones to address the time issue.
The other thing I want to focus on is basically localization of the English versions for these apps that we have. We need to be able to disseminate this information about the apps and events, and it’s not as easy as Google or word-for-word translation because sometimes they miss the mark or have different words we would like to use. Also, the thing about media is basically that over here in the US, it’s not like we have a strong understanding of where the media pipelines are. So I believe what would be best is if we could get to know the media people here, and how to best communicate with our fans so that we could disseminate the information to everyone interested in an effective manner.
Poon: I actually have the same feeling. When we do the panels, people actually come to me and say, “I didn’t know Gugenka before—MakeAvatar, HoloModels—but now I’m going be a core fan and support it.” So I’m touched, and also really happy that people enjoy our service.
Like, with MakeAvatar—I don’t think you even know this much yet—but this allows us to use different parameters. The tech part is really hard—I tried so hard on this tech from January—and finally, you can use different kinds of morphs to different kinds of faces. The idea is, usually anime characters are a little bit more the shounen youngtype, but I want elderly people to also be cool. Even if you’re old, you should be really handsome because you do your own thing. You could be chubby, you could be muscular—allowing the diversity is human, and it should be possible in your own character. That’s why we have all these features.
Of course, we have the SD character, that is, the small one. The main reason we built this one is because we support Quest, cell phone, and browser, so we want the workload to be lower. But we have new, high-quality MakeAvatar that allows you to change clothing and stuff. And the business model would be to buy the cosplay—the costume—and the IP company gets paid, and the user is happy. So you can export to VRChat, VirtualCast, to see online on Vroid Hub, or browser games.
We think in the future maybe we can do more promotion in the US and more people could take advantage of this. You could become a VTuber easily. I could host a panel and just teach you to use the app, and you could be a VTuber for free. And we have the face-tracking support in this, which means it can detect your muscles and move. In normal characters, it only supports simple expressions like A-I-U-E-O and smlle, angry, crying—that’s it. But we support really detailed motions on this.
Augmented reality is a funny thing to me because its appeal feels somehow both obvious and yet elusive. Whether it was participating in Pokémon Goat the height of the craze or seeing people on Twitter post videos of their iDOLM@STER characters occupying “real” spaces, I end up thinking “that’s really cool” and “but do I really want to blur that line?” simultaneously.
I was asked this month, by Patreon request, to discuss HoloModels, which is an AR figures app by the company Gugenka. Essentially, rather than having physical PVC or resin kit models, you collect virtual ones that you can pose and “place” wherever you want. I had actually seen images of it without realizing what exactly I was looking at, thanks to retweets of the Lina Inverse HoloModel that have been filling my Twitter timeline. “Was it some video game? Maybe a fan project?” I thought.
Before trying out the app itself, my understanding of HoloModels led me to think that the advantage was basically like that of ebooks: the ability to keep a bunch of models without any of them taking up physical space. They can be placed and posed any way you want, so there’s also a certain degree of freedom for creativity. However, when I saw that HoloModels can be resized to pretty much any scale, I realized that the potential I had pictured was too limited.
The versatility of HoloModels means you can have life-size models, as if they’re less figures and more characters who have entered our world. Perhaps you can even pretend that they’re a friend or a lover. And even if you’re not into that sort of thing, you can still use them in a variety of different ways. You can use them in virtual dioramas or even as action figures after a fashion. What’s more, you can’t really “damage” them by accident. And of course, even this view is still probably a drop in the ocean of possibilities.
Because of the proximity of HoloModels to Virtual Youtubers—they’re essentially two ways of blurring fiction and reality together through anime aesthetics—I also had to see if there was any stronger connection between the two. It turns out that the default model you get when you first install HoloModels, Shinonome Megu, has since become a Virtual Youtuber with 40,000+ subscribers as of December 2020. I believe the HoloModels figure came first, based on comparing news articles announcing HoloModels with the oldest video on her channel, but if anyone has more information, feel free to share.
Am I interested in sticking with them? Not really. HoloModel figures are awfully pricey in my view, as less expensive characters run around 3,500 yen, and the Lina Inverse mentioned above is 5,000 yen. I might just be the wrong person to understand the true value of these AR characters—I’d still rather have a physical one, even if I can’t make it Godzilla-sized. That all said, if we compare HoloModels to another form of “virtual character collection,” i.e. mobile game gacha, the luck element is completely removed. That does make me wonder if that gambling high is part of why mobile game character lotteries work in the first place, but that’s another conversation for another day.
This post is sponsored by Ogiue Maniax patron Johnny Trovato. You can personally request topics through the Patreon or by tipping $30 via ko-fi.