Tech 🖥 : How to make Imish House

Have you ever thought about making an interactive 3d website? Check this out!

Last year, I was trying to make an interactive website using my virtual character called “Imish”.

This is still a buggy website however finally I am just finishing up ver1.0.

Imish Tokyo

Welcome to the Imish House

https://www.imish.tokyo/

1. Making 3D Model
2. Export the model for javascript
3. Designing Interactive script

First of all, I referred a lot from the amazing tutorial Three.js Journey and the websites below.

This a good tutorial for making avatar animation and exporting to react.

React Three Fiber Character Animation

Building 3D scenes with static objects and shapes is cool and all, but it's much cooler when you can bring a scene to life with animation. In the 3D world there is a process called character rigging which allows you to create special objects called armatures that act like a system of bones and joints.

https://codeworkshop.dev/blog/2021-01-20-react-three-fiber-character-animation/

React spline. you can make your 3d model, they even support deployment for javascript or react etc, and are pretty easy to handle.

Spline - Design tool for 3D web browser experiences

Open App Drag with two fingers to orbit Create 3D scenes, edit materials, and model 3D objects.Control the outcome of your design work. 3D opens a new door for creativity. A solid set of features that will help you create your best designs. Spline is constantly growing just like the possibilities of what you can achieve.

https://spline.design/

Obviously, I know I am not good 3d modeling but I start learning blender also this model is based on where I used to live because I would like to make closer my environment as much as possible.

My friends(GADARA) are also helping me to make 3D model and a character in my room.

3d modeling of my room

My Virtual Character, sooo adorable. designed by Kazuya YANAGIHARA

He even tries to export from 3d printer.

After finishing really fundamental models, I started using react-spline

Original Model

Exporting react-spline to design materials.

Since react-spline is supporting the export function, you can export as a code like Three.js,React and react-three-fiber.

I exported it as a react-three-fiber for my website.

If you want to make an animation for your character I recommend using mixamo.

Imish animation made by mixiamo

react-three-fiber can only accept the JSX component you need to export your gltf file to import JSX and there is a good library here!

https://github.com/pmndrs/gltfjsx

I will not dive into this so much because there are so many ways to make it interactive but as I am a huge fan of tone.js, I used tone.js to respond to my website with music.

I made a kawaii future bass song for this site and each midi sequence is connecting the trigger with some parameter of react-three-fiber.

My music MIDI

So basic idea is, If you export your model from react-spline you can get code like this.

import useSpline from '@splinetool/r3f-spline'

export default function Scene({ ...props }) {
  const { nodes, materials } = useSpline('https://prod.spline.design/***.splinecode')
  return (
    <>
      <color attach="background" args={['#667489']} />
      <group {...props} dispose={null} >
        <pointLight
          name="ParttLight 2"
          castShadow
          intensity={1.5}
          decay={2}
          distance={1000}
          shadow-mapSize-width={128}
          shadow-mapSize-height={128}
          shadow-camera-near={100}
          shadow-camera-far={2000}
          color="#8795fe"
          position={[140.29, 270.71, -146.58]}
        />

And I just set the parameter according to the midi note number, for example you can change the color parter depending on what midi number is currently playing.

useEffect(()=>{
    if(signal === 'D#2') {
      setIntens(0.2)
      setColor("#9fb5fe")
    }
    if(signal === 'C#2') {
      setIntens(0.3)
      setColor("#c0e6a4")
    }
    if(signal === 'D2') {
      setIntens(0.4)
      setColor("#eaeaea")
    }
  },[signal])

Finally combining those scripts I made this webstie.

There is also some glsl script I tried to include this site for water simulation.

You might get also something from below.

Water Floor

GitHub - dli/waves: Ocean Wave Simulation - http://david.li/waves

Ocean Wave Simulation - http://david.li/waves. Contribute to dli/waves development by creating an account on GitHub.

https://github.com/dli/waves

three-stdlib examples - CodeSandbox

Learn how to use three-stdlib by viewing and forking three-stdlib example apps on CodeSandbox

https://codesandbox.io/examples/package/three-stdlib