Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

USD release 22.08 compiles, but any CLI invocation crashes on an M1 Mac #1996

Closed
heckj opened this issue Aug 16, 2022 · 6 comments
Closed

USD release 22.08 compiles, but any CLI invocation crashes on an M1 Mac #1996

heckj opened this issue Aug 16, 2022 · 6 comments

Comments

@heckj
Copy link
Contributor

heckj commented Aug 16, 2022

Description of Issue

I've tried multiple variations of build_scripts/build_usd.py with python3 to get USD compiled and installed. In each of the cases, when I try to test the installation (either usdcat or usdview, depending on if I enabled USDView to be built), the result matches #1620:

python crashed. FATAL ERROR: Failed axiom: ' Py_IsInitialized() '
in operator() at line 149 of /Users/heckj/src/USD/pxr/base/tf/pyTracing.cpp
writing crash report to [ Sparrow.local:/var/folders/8t/k6nw7pyx2qq77g8qq_g429080000gn/T//st_python.29962 ] ... done.

Steps to Reproduce

  1. installed Python3 with mini forge:
    sh ~/Downloads/Miniforge3-MacOSX-arm64.sh
    source ~/miniforge3/bin/activate
    conda update conda -y
    conda install pyopengl
  1. installed PySide6: pip install PySide6
  2. Cloned, switched to release branch, got latest, and built:
    cd ~/src
    # git clone https://github.com/PixarAnimationStudios/USD
    cd USD
    git checkout release
    git pull
    # clean earlier install attempts:
    rm -rf /opt/local/USD/*
    export PXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON
    python build_scripts/build_usd.py /opt/local/USD

build completes without issue: https://gist.github.com/heckj/12fd07859b5e8f80babb62281ef98cb6

Then attempting to use either usdview or usdcat on a the Kitchen Set sample USD results in the crash:

    export PATH=/opt/local/USD/bin:$PATH
    export PYTHONPATH=$PYTHONPATH:/opt/local/USD/lib/python
    /opt/local/USD/bin/usdcat ~/Dice.usdz

System Information (OS, Hardware)

macOS Monterey, 12.5. M1 chip, 16GB ram.

uname -a:

Darwin Sparrow.local 21.6.0 Darwin Kernel Version 21.6.0: Sat Jun 18 17:05:47 PDT 2022; root:xnu-8020.140.41~1/RELEASE_ARM64_T8101 arm64

Package Versions

Details above in setup, and git head reported as:

commit 3abc46452b1271df7650e9948fef9f0ce602e3b2 (HEAD -> release, tag: v22.08, origin/release, origin/HEAD)
Merge: e0e515614 48460d145
Author: pixar-oss <pixar-oss@users.noreply.github.com>
Date:   Wed Jul 20 16:14:16 2022 -0700

    Merge release v22.08

Building with settings:
  USD source directory          /Users/heckj/src/USD
  USD install directory         /opt/local/USD
  3rd-party source directory    /opt/local/USD/src
  3rd-party install directory   /opt/local/USD
  Build directory               /opt/local/USD/build
  CMake generator               Default
  CMake toolset                 Default
  Downloader                    curl

  Building                      Shared libraries
    Variant                     Release
    Imaging                     On
      Ptex support:             Off
      OpenVDB support:          Off
      OpenImageIO support:      Off
      OpenColorIO support:      Off
      PRMan support:            Off
    UsdImaging                  On
      usdview:                  On
    Python support              On
      Python Debug:             Off
      Python 3:                 On
    Documentation               Off
    Tests                       Off
    Examples                    On
    Tutorials                   On
    Tools                       On
    Alembic Plugin              Off
      HDF5 support:             Off
    Draco Plugin                Off
    MaterialX Plugin            Off

  Dependencies                  zlib, boost, TBB, OpenSubdiv

Build Flags

I tried exactly as above, and I've tried with the following environment variable set - with the same result:
export PXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON
@sunyab
Copy link
Contributor

sunyab commented Aug 16, 2022

PXR_PY_UNDEFINED_DYNAMIC_LOOKUP is a setting that must be passed to CMake, I don't believe it'll get picked up from the environment. Try:

python build_scripts/build_usd.py /opt/local/USD --build-args USD,"-DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON"

@meshula
Copy link
Member

meshula commented Aug 16, 2022

I just did a test of using export as a means of communicating values to Cmake, the value was not detected.

@heckj
Copy link
Contributor Author

heckj commented Aug 16, 2022

Thank you, I'll give it a shot. is that additional element needed for the M1 build? It wasn't explicitly stated, but i saw it referenced in other issues, and tried it blindly without knowing even what it was meant to do.

@sunyab
Copy link
Contributor

sunyab commented Aug 16, 2022

No, this isn't a requirement for M1 builds. This flag wouldn't be needed if USD was built using the Python that ships with macOS, for example.

My understanding from #1620 is that this is specific to building USD with the Python interpreter provided by conda, which statically links against Python and causes duplicate symbol issues since USD also links against Python.

@meshula
Copy link
Member

meshula commented Aug 16, 2022

To add to what Sunya said, the flag is specific to statically linked Python distributions, it's independent of processor architecture and operating system - the same problem also occurs with conda on linux and windows on both aarch64 and x64. Conda distributed Pythons are statically linked as a way to reduce unexpected system and installation coupling, but Python isn't, to my knowledge, built in such a way that an external build system can robustly discover whether you are targeting a static or dynamically linked Python so we are forced to deal with it manually.

@heckj
Copy link
Contributor Author

heckj commented Aug 16, 2022

@sunyab @meshula Thank you both, this helps my understanding tremendously. @sunyab Part of the issue, and why I'm even using the conda/miniforge version of Python3, is that Apple is no longer including an updated Python as an "installed thing" within it's operating systems. There's a version of python3 that's now embedded within Xcode, but with even less header access, and really looks to be for Xcode's (Apple Developer Tools) own internal use.

A fresh install of macOS 12.5 (Monterey) trying to use Python3 results:

> python3 --version
xcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.

So if you install Xcode, it'll drop a Python 3.8.9 into place, but I'm uncertain if that has all the pieces (headers, specifically, as well as library additions) to support USD. I don't expect USD to provide the guidance there, but having a very clear way for a developer w/ macOS to get the USD tooling up and running on their current hardware would be super helpful in the docs or README, including a tidbit about this additional CLI option.

I'll post something on a my blog to at least get it out there. The only reference prior to this version's release that I found related to the M1 was https://simo.virokannas.fi/2022/04/compiling-pixars-usd-natively-on-apple-m1/ from April 2021.

And thank you again - getting the PXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON passed through correctly is what made all the difference, and I have a functional setup now working on my laptop!

Screen Shot 2022-08-16 at 1 58 14 PM

@heckj heckj closed this as completed Aug 16, 2022
@meshula meshula mentioned this issue Aug 21, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@heckj @meshula @sunyab