언리얼엔진 + 유즈3D + OpenAI API

3D로 테스트해보고 싶은게 몇개 있어서 만들어보는중임,,

언리얼(프롬 x 게임엔진 o)에 유즈 대충 모델링해서 넣고 애니 대충 구해서 넣고

배경 있는거 대충 꺼낸다음에

OpenAI API 플러그인 받아서 적용시켜봄

(테스트 애니메이션 출처 https://www.mixamo.com/)

3D에서 캐릭터의 표정을 만들때는 '모프 타겟'이라는 방법을 쓰기도 함

이게 뭐냐면

이런식으로 각 부위별 움직임을 미리 만들어놓고 섞어 쓸 수 있는거라고 생각하면 됨

'기본 상태'가 눈을 둘다 뜨고 있는 상태라고 할 때,

왼쪽 눈 감음 / 오른쪽 눈 감음 각각의 상태를 미리 만들어 준 뒤

필요할 때 왼쪽 눈감음 활성화 / 오른쪽 눈감음 활성화 이런식으로 지정해줘서

각 부분의 움직임에 가중치를 주는 방식임

값은 0이 최소(비활성화), 1이 최대(활성화) 인데, 둘다 0.5씩 주는 방식으로 눈을 게슴츠레하게 뜬 것 같이 만들 수도 있음.

아무튼,

어? 그럼 이 값을 GPT한테 써달라해도 되는거 아닌가? 싶은 마음에

바로 테스트 해 봄

[ 나 ] (대충 언리얼 엔진으로 모프 타겟 쓰려고 하는데 대사에 맞는 표정 값좀 달란 얘기)

I'm animating a character's face expression with morph targets in Unreal Engine.

Morph key list=left eye closed, right eye closed, left eyebrow raised, right eyebrow raised, mouth opened, mouth smile, cheek pouted.

0 is the minimum value and 1 is the maximum value.

You have to give me the values of the each target to make face expression in Unreal Engine.

Target Dialogue 1="Oh, hi! Nice to meet you."

Target Dialogue 2="Ugh, I hate this."

Target Dialogue 3="Let me give you a wink."

Now, give me the values of the morph targets for each of the dialogues.

[ GPT ] (대충 값 던져줌. 근데 꽤 괜찮아 보임)

To animate facial expressions using morph targets in Unreal Engine for the given dialogues, I'll provide the values for each morph target. These values range from 0 (minimum) to 1 (maximum) and control the degree of each facial movement. Here's a breakdown for each dialogue:

Target Dialogue 1: "Oh, hi! Nice to meet you."

This expression should be friendly and welcoming.

• Left Eye Closed: 0
• Right Eye Closed: 0
• Left Eyebrow Raised: 0.5 (slight raise for a friendly look)
• Right Eyebrow Raised: 0.5 (slight raise for a friendly look)
• Mouth Opened: 0.4 (slightly open mouth for speaking)
• Mouth Smile: 0.7 (smiling to show friendliness)
• Cheek Pouted: 0 (not pouted)

Target Dialogue 2: "Ugh, I hate this."

This expression should convey dislike or annoyance.

• Left Eye Closed: 0.1 (slight squinting to show irritation)
• Right Eye Closed: 0.1 (slight squinting to show irritation)
• Left Eyebrow Raised: 0.3 (one eyebrow slightly raised in annoyance)
• Right Eyebrow Raised: 0.1 (the other eyebrow slightly lowered)
• Mouth Opened: 0.2 (mouth slightly open as if muttering)
• Mouth Smile: 0 (no smile)
• Cheek Pouted: 0.6 (cheeks pouted to emphasize dislike)

Target Dialogue 3: "Let me give you a wink."

This expression should include a playful wink.

• Left Eye Closed: 1 (left eye fully closed for the wink)
• Right Eye Closed: 0 (right eye open)
• Left Eyebrow Raised: 0.4 (slight raise for a playful look)
• Right Eyebrow Raised: 0 (neutral)
• Mouth Opened: 0.3 (slightly open mouth)
• Mouth Smile: 0.8 (big smile to show playfulness)
• Cheek Pouted: 0.2 (slightly pouted cheek for added playfulness)

These values should give you a good starting point for animating each expression in Unreal Engine. Adjustments can be made based on the specific nuances you want to capture in your character's facial expressions.

간단히 정리하자면, 꽤 그럴듯하게 준다.

양쪽 눈썹 높이까지 달리해가면서, 윙크같이 한쪽 눈만 감아야 하는 표정도 잘 써줌.

이거 괜찮은데? 싶어서 다른 것도 가능하냐고 물어봄.

[ 나 ] (만약에 다른 값들도 준다면 뭘줄수 있냐 물어봄)

What other values in animation (similar functions like morph target) can you give me? Please give me some examples.

[ GPT ] (뭔가 많음)

In addition to morph targets, there are several other values and parameters in animation that can be used to control various aspects of character movement and expression. Here are some common examples:

1. Bone Transformations (Skeletal Animation)

• Translation: Moving bones along the X, Y, or Z axis. Example: Moving the hand forward.
• Rotation: Rotating bones around the X, Y, or Z axis. Example: Rotating the head to look left.
• Scale: Scaling bones along the X, Y, or Z axis. Example: Enlarging or shrinking the hand.

2. Blend Shapes

• Blend Shape Weights: Similar to morph targets, but often used in different software. Controls the influence of different shapes on the final mesh.

3. Animation Curves

• Float Curves: Control variables over time, such as the intensity of a light or the volume of a sound.
• Vector Curves: Control vectors (e.g., color changes over time) over time.

4. Inverse Kinematics (IK)

• Effector Position: Position of the end effector in an IK chain (e.g., where the hand should be).
• Pole Vector: Control the direction of the elbow or knee bend in an IK setup.

5. Material Parameters

• Opacity: Control the transparency of a material.
• Color: Control the color of a material.
• Emissive Intensity: Control the glow effect of a material.

6. Physics Properties

• Mass: Control the mass of a physical object.
• Damping: Control the reduction of motion over time for physical simulations.
• Spring Stiffness: Control the stiffness of a spring in a physical simulation.

7. Facial Control Rig Parameters

• Jaw Open: Control how open the jaw is.
• Eye Blink: Control the blinking of eyes.
• Lip Corner Puller: Control the movement of the lip corners for smiles or frowns.

8. Procedural Animation Parameters

• Noise: Add procedural noise to create natural, randomized motion.
• Amplitude: Control the height of waves or oscillations.
• Frequency: Control the speed of waves or oscillations.

9. Animation States and Transitions

• Blend Weights: Control the blending between different animation states (e.g., blending between walking and running).
• Transition Duration: Control the time it takes to transition between animation states.

Example Values for Specific Use Cases:

Facial Rig Example:

• Jaw Open: 0.5 (half-open jaw)
• Left Eye Blink: 0.3 (slightly closed left eye)
• Right Eye Blink: 0.3 (slightly closed right eye)
• Smile Left: 0.6 (left side of the mouth smiling)
• Smile Right: 0.6 (right side of the mouth smiling)

Hand IK Example:

• Effector Position: (x: 50, y: 10, z: 30) (hand reaching forward and slightly to the side)
• Pole Vector Position: (x: 5, y: 10, z: 0) (elbow pointing outward)

Material Parameter Example:

• Color: (R: 1.0, G: 0.5, B: 0.0) (orange color)
• Emissive Intensity: 2.0 (glowing effect)

By using these parameters along with morph targets, you can create detailed and nuanced animations in Unreal Engine or any other animation software.

대충 정리하자면 표정뿐만이 아니라 '움직일' 때 필요한 값들도 줄 수 있다는 건데...

이거 잘 쓰면 그냥 3D 좌표상에서 캐릭터 움직일 수 있는 거 아님?

→ 이 생각 들자마자 바로 언리얼 킴 ㅋㅋㅋ

로어북마냥 오브젝트들 위치 좌표로 저장해놓고,

상태창 출력하듯이

포맷:

[대사]

[위치 좌표] (x, y)

[뛰어갈지 아닌지 여부]

[표정 값]

[재생할 애니메이션] (단순히 방방 뛰거나 한바퀴 도는 등 미리 만들어둔 애니메이션 재생이 필요할때)

[손을 뻗는 위치] (필요할 경우 오브젝트를 집어든다거나 하게 만들 수 있을 거 같음... 아마도)

[실행할 함수] (00으로 옷 갈아입기 / 배경음악 00 틀기 등등,,, 알잘딱,,)

이런식으로 몇가지 받아오면 뭔가 만들어볼 수 있을 거 같음,,,

추가로 현실 시간에 맞춰서 라이팅 변경하는거나

날씨 api 연결해서 실시간 날씨 구현하는것도 해보고싶다

이불 던지는것처럼 특정 오브젝트에 맞았을때 해당 상황에 맞는 프롬으로 ai에 요청가도록 만들어보고도 싶은데

내가 프로그래머가 아니라 아직 가능한지, 어떻게 해야 할 지는 모르겠음

아마 기능들 구현하는데 다 성공하면 모델링 퀄리티도 높여볼 수 있을 듯

근데 애니 느낌 내는건 유니티가 개인적으로 더 쉬워서 유니티로 할까 싶기도 함..

계속 고민중읾,,

유니티로 해야되나,,,,,

일단 뭘로 만들던 갈길은 존나멀듯

내일 출근해야돼서 오늘은 여기까지만 해야겠다,,