1. Setting Up SynthDa
This section provides a step-by-step guide to setting up the SynthDa pipeline. It covers installing dependencies, placing pretrained models in the correct directories, and testing each component individually. Additionally, common errors and troubleshooting tips are included to help you quickly resolve any setup issues.
📦 Step 1: Install Required Packages
Use pip to install the required packages and libraries:
Install all necessary dependencies with:
pip install -r requirements.txt
📁 Step 2: Clone Repositories SynthDa relies on the following repositories. Make sure you clone and place them correctly in your project structure:
⚙️ Step 3: Configuration Setup Create a .env file in the root directory and add the following paths and keys:
- Your OpenAI GPT API key
- Folder paths to each of the cloned repositories
💾 Step 4: Download Pretrained Models Each repository requires specific pretrained models or checkpoints. Download all required files from the link below and place them according to the instructions in each repo's section: 👉 Download Pretrained Files
The StridedTransformer is used to generate Human3.6M (H36M) 3D pose estimations from a real-world video input.
- Place the 2
.pthfiles in the following directory:
./checkpoint/pretrained/
- Place another 2
.pthfiles in the following directory:
./demo/lib/checkpoint/
How to Test the Repository Run the following command:
python demo/vis.py --video sample_video.mp4
Video Path:
StridedTransformer-Pose3D/demo/video/sample_video.mp4
Output Folder Created:
StridedTransformer-Pose3D/demo/output/sample_video/
❌ Error: model_path referenced before assignment
This error might occur due to one or more of the following reasons:
-
📹 Video Quality Issues
- The video resolution is too low
- The person is not clearly visible or does not appear in the video
-
🎬 Video Transition
- A scene change or transition disrupts the tracking
-
📁 Model Path Error
- The pretrained model is missing or not placed in the correct folder
📷 Screenshot of Error
✅ Suggested Solution
To resolve this issue, ensure the video shows a single continuous action with the full body of the person visible throughout. You may need to crop or split videos that contain transitions or multiple angles.
✂️ **Example Fix: **
❌ Original Video (with angle transition):
The original video contains a shift in camera angle, which can disrupt pose tracking.
W_jgJzb8IxM_000013_000023.mp4
✅ Fixed Videos (split by angle):
Splitting the video into segments with consistent camera angles can significantly improve pose tracking accuracy.
-
First angle:
W_jgJzb8IxM_000013_000023_part1.mp4
-
Second angle:
W_jgJzb8IxM_000013_000023_part2.mp4
The text-to-motion is used to generate a 3D pose estimations from a text input.
- Replace the existing
checkpointsfile with a folder namedcheckpoints. - Inside that folder, place the two pre-trained models:
./checkpoints/kit/ and ./checkpoints/t2m/
How to Test the Repository Run the following command:
python gen_motion_script.py --name Comp_v6_KLD01 --text_file input.txt --repeat_time 1
The expected output should create a folder called C000 containing a .npy file in the eval_results folder:
text-to-motion/eval_results/t2m/Comp_v6_KLD01/default/animations/C000/gen_motion_00_L044_00_a.npy
📌 Note: The folder name (e.g., C000) and filename may differ.
❌ Error: Deprecated NumPy version This error occurs due to the use of deprecated attributes in newer versions of NumPy.
📷 Screenshot of Error
✅ Suggested Solution
Replace both instances of np.float in ./common/quaternion.py — use float on Line 11, and np.float64 on Line 13.
❌ Error: Error faced with ax.lines
This error occurs because ax.lines and ax.collections are read-only properties and cannot be directly assigned.
📷 Screenshot of Error
✅ Suggested Solution
Adjust the ax.lines assignment lines inside the update(index) function located in ./utils/plot_script.py.
❌ Error: Unable to Recognize spacy Package
This error occurs when the required SpaCy language model is not installed. It will state that it is "unable to find model 'en_core_web_sm'. It doesn't seem to be a Python package or a valid path to a data directory. "
✅ Suggested Solution python -m spacy download en_core_web_sm
❌ Error: Unable to Recognize ffmpeg Package
This error occurs when the required ffmpeg package is not installed
✅ Suggested Solution conda install -c conda-forge ffmpeg
The joints2smpl is used to attach the generated joints to the SMPL character body.
- SMPL models must be placed inside the following path:
./smpl_models/smpl/
How to Test the Repository
Run the following command:
python fit_seq.py --files test_motion2.npy
Input .npy File Location:
joints2smpl/demo/demo_data/test_motion2.npy
Output Folder with .obj Files:
joints2smpl/demo/demo_results/test_motion2/
❌ Error: ImportError: cannot import name 'bool' from 'numpy'
This occurs due to deprecated usage of np.bool, which has been removed in recent versions of NumPy.
✅ Suggested Solution
Navigate to the location where your chumpy package is installed by typing:
pip show chumpy
Go to the __init__.py file inside the chumpy directory and replace line 11 with lines 13 to 19, as shown in the image below:
The Blender library is used to generate a video from the .ply objects.
Download Blender version 3.0.0 from the official Blender website or use the command below to download it directly:
wget https://download.blender.org/release/Blender3.0/blender-3.0.0-linux-x64.tar.xz
After downloading, unzip the folder using:
tar -xf blender-3.0.0-linux-x64.tar.xz
Blender Setup Files
-
Place your animation_pose.py script into the Blender directory (where ./blender is run).
-
Create a angleInput.txt file in the same directory. Inside this file, specify the desired camera angle (e.g., "front"). This determines the camera viewpoint used when rendering the video.
How to Test the Blender Tool
Navigate into the extracted folder and launch Blender using ./blender. If Blender launches successfully, the installation is complete.
Alternatively, if the .ply object folder has already been created, you can convert it into a video directly using the following command:
./blender -b -P animation_pose.py -- --name <name_of_folder_containing_ply_files>
📌 For detailed model download instructions, refer to the documentation pages of each repository.
❌ Error: File path of the different repos are not found
This error occurs due to the codebase's inability to retrieve the variables from the .env file.
✅ Suggested Solution
Add the variables directly into the codebase or create a separate Python file to store the folder path variables.
❌ Error: Python not recognized
This error occurs when the environment is set up to run python files using python3 test.py instead of python test.py. The repository expects Python files to be run with the python command. If python is not mapped to python3, the script may fail to execute correctly.
✅ Suggested Solution
Reassign the python command to point to python3 using the following command:
sudo ln -s $(which python3) /usr/local/bin/python